JP5391637B2 - Data similarity calculation system, data similarity calculation method, and data similarity calculation program - Google Patents

Description

  The present invention relates to a data similarity calculation system, a data similarity calculation method, and a data similarity calculation program for calculating a similarity between data.

  In a retail store, a POS (Point of Sales) system is used, and the sales data of the collected products is used in various ways. A system has been proposed in which customers are classified based on customer characteristics and various analyzes are performed using sales data. For example, Patent Literature 1 describes a system that can classify customers according to the will of an analyst and check the purchase history of the customer.

  Patent Document 2 describes a system for grasping a consumer's tendency to sell simultaneously using POS data.

  Patent Document 3 describes a system that includes a database having a plurality of virtual BOXes for associating and registering a plurality of products in various categories and presenting a recommended product group to a customer.

  Further, Patent Document 4 discloses an apparatus for obtaining data similarity. The apparatus described in Patent Literature 4 calculates an inter-case similarity, which is a similarity between a case stored in advance and a given case (query). In addition, a class (for example, weather three hours after the time of measurement of meteorological data) is defined for the stored case (for example, meteorological data at a certain point). The apparatus described in Patent Literature 4 derives an association rule from a case that is held. The association rule is a rule for deriving a conclusion (class) according to a condition (example). Then, the total similarity is calculated from the similarity between cases using the association rule.

Patent Document 4 describes an association rule expressed in a format of “A ₁ , A ₂ ,..., A _k → B”. “A ₁ , A ₂ ,..., A _k , B” are referred to as item sets. Further, the ratio of the number of records including the item sets A ₁ , A ₂ ,..., A _k , B to the total number of records is called the support degree of the association rule, and the item sets A ₁ , A ₂ ,. A ratio of records including item B among records including _k is referred to as an association rule confidence factor. The apparatus described in Patent Document 4 extracts a correlation rule that allows the support level and the correlation level to be equal to or higher than the predetermined lower limit values.

  An apparatus for automatically classifying data is described in Patent Document 5. The data classification device described in Patent Literature 5 extracts a plurality of pattern data in descending order of similarity to the target pattern data, and determines neighboring clusters.

  Moreover, there exists an apparatus described in patent document 6 as an apparatus which estimates using a rule. The device described in Patent Document 6 generates a logical rule from an event that occurs in a certain region (first region), applies the rule to another region (second region), and generates a second region. To predict the tendency of the event to occur.

Japanese Patent Laid-Open No. 9-101984 (paragraphs 0014-0050) JP 2007-94592 A (paragraph 0024) JP 2003-337886 (paragraphs 0010, 0015, 0016) JP 2002-149697 A (paragraphs 0018-0065) JP 2003-256443 A (paragraphs 0090-0093) JP 2004-126757 A (paragraphs 0033, 0051)

  In the system described in Patent Document 1, an analyst who is a user of the system needs to manually classify customers. For this reason, it has not been possible to automatically classify customers based on their purchasing behavior. Also in the apparatus described in Patent Document 3, it is necessary for a product observer such as a customer or a store clerk to associate a plurality of products in various categories.

  In order to classify data, it is conceivable to calculate the similarity between the data and classify the data using the similarity. The apparatus described in Patent Literature 4 derives an association rule and calculates a total similarity. However, when the association rule is derived as in the device described in Patent Document 4, if there are many related attributes (items in Patent Document 4) in the data, the influence of those related attributes is strongly affected. I can't calculate the appropriate similarity.

  For example, it is assumed that ten pieces of data with data numbers 1 to 10 shown in FIG. 12 are given. In the data shown in FIG. 12, the data numbers 1 to 4 and the data number 6 have the same attributes b, c, d, and e, and the classes representing the classification of the data. Therefore, when extracting an association rule having a data attribute value as a condition part and a class as a conclusion part, the influence of attributes b, c, d, and e, which have a common combination of attribute values among many data, is strongly affected. I will receive it. As a result, the similarity is also affected by a specific attribute related to the similarity. That is, the similarity between data that matches an attribute value of a certain attribute is relatively high, and the similarity between data that does not match tends to be relatively low.

  Therefore, according to the present invention, when determining the similarity between data for classifying data, even if there are many attributes related to each other in the data, they are similar without being strongly influenced by those related attributes. It is an object to provide a data similarity calculation system, a data similarity calculation method, and a data similarity calculation program capable of obtaining a degree.

  A data similarity calculation system according to the present invention is a subset generation for generating a subset of a plurality of times by allowing data duplication from a set of data including an attribute indicating data characteristics and a class indicating data classification. A classifier that is a rule for determining a class from an attribute each time a subset is generated, a classifier generating means that generates a classifier based on the subset, and a classifier that is generated each time a classifier is generated. A class determination unit that determines a class of individual data belonging to the subset, and a subset of the data set is generated, and when the class determination unit determines a class of individual data that belongs to the subset, It is characterized by comprising similarity calculation means for adding a value to the similarity between data determined to be the same class.

In the data similarity calculation method according to the present invention, the subset generation means allows a plurality of subsets of the set by allowing data duplication from a set of data including an attribute indicating data characteristics and a class indicating data classification. A sub-set generation step, and a classifier generation unit that generates a classifier that is a rule for determining a class from an attribute each time a sub-set is generated, based on the sub-set, class determination means, each time the classifier is generated, by using the classifier, and each class determining step of determining the class of data belonging to the subset, the similarity calculation means, a subset of the set of data A similarity calculation that adds a value to the similarity between data determined to be the same class when the class of individual data belonging to the subset is determined in the class determination step Characterized in that it comprises a step.

  A data similarity calculation program according to the present invention allows a computer to generate a plurality of subsets of a set of data by allowing duplication of data from a set of data including an attribute indicating data characteristics and a class indicating data classification. Each time a subset is generated, a classifier that is a rule for determining a class from an attribute every time a subset is generated, a classifier generation process that generates a classifier based on the subset, and every time a classifier is generated, a classifier Class determination processing that determines the class of individual data that belongs to the subset using, and a subset of the data set is generated, and the class determination processing determines the class of individual data that belongs to the subset Further, a similarity calculation process for adding a value to the similarity between data determined to be the same class is executed.

  According to the present invention, when determining the similarity between data in order to classify the data, even if there are many attributes related to each other in the data, they are similar without being strongly influenced by these related attributes. Degree can be obtained.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Here, the embodiment of the present invention will be described by taking as an example the case of obtaining the similarity between data related to customers of products and services. In addition, not only those who use products and services for a fee, but also those who use products and services for free, and those who will use the products and services in the future are called customers. In the case of free, the selling price is 0. The concept of goods or services includes not only a finished product such as a product but also a minimum unit for maintaining the function of the product. Furthermore, a certain product or service category can be considered as one product and applied.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing an example of the first embodiment of the present invention. The data similarity calculation system 1 according to the present invention includes a subset generation unit 11, a classifier generation unit 12, a self-evaluation unit 13, and a similarity calculation unit 14.

  The subset generation unit 11 generates a subset of the set from the data set. For example, the data set may be input via an input device (not shown) such as a keyboard provided in the data similarity calculation system 1, but the input mode of the data set is not particularly limited. Hereinafter, a data set means a set of all data.

  Each piece of data included in the data set includes an attribute indicating data characteristics and a class indicating data classification. Here, a case where at least customer characteristics or sales conditions are data characteristics and customer behavior is a class will be described as an example. FIG. 2 is an explanatory diagram illustrating an example of data. In the example illustrated in FIG. 2, “weather”, “temperature”, “humidity”, and “whether wind is strong” are used as attributes. These attributes are sales conditions, but customer characteristics such as “age” and “gender” may be used as attributes, and both customer characteristics and sales conditions may be attributes. In the example shown in FIG. 2, there are two types of customer behavior, “Purchase” and “Do not purchase”, and the data is also classified into two classes, “Purchase” and “Do not purchase”. For example, The data illustrated in FIG. 2 represents whether a customer has purchased a certain product or service under the condition indicated by the attribute. For example, the first data shown in FIG. 1 represents that the customer did not purchase under the conditions of “sunny”, temperature “29 ° C.”, humidity “85%”, and “not strong wind”. ing.

  The subset generation unit 11 generates a subset of the data set by selecting some data from such a data set. In addition, the subset generation unit 11 generates a subset multiple times. At this time, a subset is generated while allowing duplication of data. That is, the same data may belong to a subset different from a certain subset generated by the subset generation unit 11.

  Each time a subset of a data set is generated, the classifier generator 12 generates a classifier based on the subset. The classifier is a rule for determining a class from an attribute. A subset corresponds to a classifier generated based on the subset.

  Each time a classifier is generated, the self-evaluation unit 13 uses the generated classifier to determine the class of individual data belonging to the subset corresponding to the classifier. This determination can be said to be a process of predicting a class from attributes included in individual data using a classifier. Note that the class determined by the self-evaluation unit 13 (in other words, the predicted class) does not always match the class predetermined in the data.

  The similarity calculation unit 14 calculates the similarity between individual data belonging to the subset based on the class determined by the self-evaluation unit 13. An initial value of similarity 0 is set in advance for combinations of arbitrary data belonging to a data set. The similarity calculation unit 14 generates a subset by the subset generation unit 11, and when the self-evaluation unit 13 determines a class of individual data belonging to the subset, between the data determined as the same class The similarity between each data is calculated by adding a value to the similarity.

  The subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, and the similarity calculation unit 14 are realized by, for example, a CPU that operates according to a program (data similarity calculation program). In this case, the program is stored in a program storage device (not shown) provided in the data similarity calculation system 1, for example, and the CPU reads the program, and the subset generation unit 11, the classifier generation unit 12, the self-evaluation according to the program. The unit 13 and the similarity calculation unit 14 may be operated.

Next, the operation will be described.
FIG. 3 is a flowchart illustrating an example of processing progress of the data similarity calculation system according to the first embodiment. For example, when a set of data is input via an input device (not shown) such as a keyboard provided in the data similarity calculation system 1, the data similarity calculation system operates as follows. However, the number of individual data (referred to as individual data) belonging to the data set is N. Also, T is the number of times a subset of the data set is generated. T is, for example, a number such as 100, 500, or 1000, but T is not limited to these values. Also, let M be the number of individual data that the subset generation unit 11 selects from the data set as individual data belonging to one subset. M may be a value such as 1%, 5%, or 10% of N, but M is not limited to these values. T, which is the number of iterations for repeating the generation of the subset, and M, which is the number of individual data included in the subset, are each input by the user of the data similarity calculation system via an input device (not shown) such as a keyboard. May be. Alternatively, T and M may be specified in other manners.

  Further, the first variable for designating the individual data belonging to the data set in turn is i, and the second variable is j. i is an integer that satisfies 1 ≦ i ≦ N. Similarly, j is an integer that satisfies 1 ≦ j ≦ N. Individual data can be designated by designating i and j, and a pair of individual data can be designated by a combination of i and j. The similarity of a pair of individual data designated by i, j is denoted as Sim (i, j).

  First, the subset generation unit 11 sequentially determines combinations of i and j determined by i in the range of 1 ≦ i ≦ N and j in the range of 1 ≦ j ≦ N. The similarity Sim (i, j) is initialized to 0 (step A1). That is, Sim (i, j) that is the similarity between the i-th individual data and the j-th individual data is initialized to 0 for each combination of i and j. In other words, the subset generation unit 11 initializes the similarity between the individual data to 0 for each pair of a pair of individual data obtained from all the individual data belonging to the data set.

  Next, the subset generation unit 11 initializes a variable t indicating the number of times the subset is generated from the data set to 0 (step A2).

  Subsequently, the subset generation unit 11 compares the variable t with T and determines whether t is less than T (step A3). If the variable t is greater than or equal to T (No in step A3), the process ends. If the variable t is less than T (Yes in step A3), the subset generation unit 11 generates a subset by selecting M individual data from the data set (step A4). That is, M pieces of individual data that are elements of the subset may be selected from the data set. The process of step A4 is performed every time the value of t is incremented, but at the time of generating the second and subsequent subsets, the individual data belonging to the newly generated subset and the individual data belonging to the already generated subset Data may be duplicated. In this manner, the subset generation unit 11 generates a subset by allowing duplication of individual data between the subsets.

  In step A4, when selecting individual data from a set of data, for example, the individual data may be randomly sampled. That is, the subset generation unit 11 may select M pieces of individual data by repeating the random selection of one piece of individual data from the data set M times.

  However, the selection method of M pieces of individual data is not limited to random sampling. For example, the subset generation unit 11 selects the first to Mth individual data when executing step A4 for the first time, and when executing step A4 next time, the M + 1th to 2 · Mth individual data. As in the case of selecting data, M pieces of individual data may be selected for each step A4 according to a predetermined order for the individual data.

  The individual data included in the subset may be data obtained by extracting a part of the attributes included in the individual data belonging to the data set and the class. For example, in the example shown in FIG. 2, the individual data includes four attributes “weather”, “weather”, “humidity”, and “whether the wind is strong”, but the individual data included in the subset Is selected, a part of the four individual data (for example, “weather” and “weather” only) and a class may be extracted, and the data including the attribute and the class may be used as a subset element. . When there are A attributes included in the individual data belonging to the data set, how many attributes are included in the individual data belonging to the subset, and which attributes are included in the individual data belonging to the subset? For example, it may be input by the user of the data similarity calculation system via an input device (not shown) such as a keyboard, or may be specified in another manner.

  After step A4, the classifier generation unit 12 generates a classifier based on the subset generated in the immediately preceding step A4 (step A5). For example, a determiner or a support vector machine may be generated as a classifier. The type of classifier to be used may be input by the user of the data similarity calculation system via an input device (not shown) such as a keyboard or may be specified in another manner.

  Hereinafter, a specific example of the classifier will be described by taking a case where a decision tree is used as the classifier as an example. In this example, a case where “purchased or not purchased” is a class and “weather”, “temperature”, “humidity”, and “whether or not wind is strong” is described as an example. Further, here, a case where a subset composed of 14 pieces of individual data shown in FIG. 2 is generated is taken as an example. FIG. 4 is an explanatory diagram showing an example of a classifier. Here, a decision tree is used as a classifier. If there is a combination of known attributes and classes as shown in FIG. 2, a decision tree can be generated from the known attributes and classes. In the following description, branching a node in a decision tree according to an attribute value (that is, an attribute value) is called division.

  In the example illustrated in FIG. 2, each row corresponds to individual data to which a class “purchase” or “do not purchase” is attached. In order to simplify the description, a class “purchase” may be represented as a positive (+) class, and a class “not purchased” may be represented as a negative (−) class. In the decision tree, information that summarizes the number of individual data for each class (for example, “purchase (positive)”, “not purchase (negative)” for each class) is used as a node. For example, in the node of the route shown in FIG. 2, the information “Yes: 9, No: 5” is used as the node.

  Given the subset generated in step A4, the classifier generator 12 determines which attribute is used to divide the root node first. Further, when the number of attributes included in the individual data is R, each attribute is defined as attributes 1 to R. At this time, the classifier generation unit 12 calculates an evaluation value at the time of division for each of the attributes 1 to R, and selects an attribute having the largest evaluation value as an attribute most suitable for division. Here, although the case where the difference between the entropy of the node before the division and the entropy after the division is used as an evaluation value is taken as an example, the evaluation value may be obtained by another calculation method. The entropy of a node is -qlogq- (1-q) log (1-), where q is the ratio of individual data with a positive (+) class and 1-q is the ratio of individual data with a negative (-) class. q). The entropy of the node after the division is a weighted average of the entropy of each node after the division.

  For example, in the subset generated in step A4, if “positive” has 9 data and “negative” has 5 data, the root node is “positive: 9, negative: 5”. In this case, since there are nine positive (+) data and five negative (-) data, the entropy of the root node is-(9/14) * log (9/14)-(5 / 14) × log (5/14) = 0.940. However, in this example, the bottom of the log is 2.

  The classifier generator 12 obtains a node obtained by dividing the root node by one attribute. That is, information (node) representing the number of positive and negative individual data is generated for each attribute value of the attribute. For example, when the possible value of attribute 1 is “0” or “1” and the value of attribute 1 is “0”, it is assumed that there are 5 positive data and 2 negative data, and the value of attribute 1 is “ In the case of “1”, it is assumed that there is 0 data for positive and 7 data for negative. In this case, as nodes that branch depending on whether the value of attribute 1 is “0” or “1”, a node “positive: 5, negative: 2” and a node “positive: 0, negative: 7” are generated. . The classifier generation unit 12 calculates the entropy of each node after division, and obtains a weighted average of entropy of each node according to the number of individual data counted as positive or negative in each node after division. In the above example, since the total number of individual data is 7 in both the nodes “positive: 5, negative: 2” and “positive: 0, negative: 7”, the weighting coefficient when performing the weighted average is Each node is (7/14). Therefore, in this example, the classifier generation unit 12 calculates the entropy after the division as follows.

(7/14) × {− (5/7) × log (5/7) − (2/7) × log (2/7)}
+ (7/14) × {− (0/7) × log (0/7) − (7/7) × log (7/7)}
= 0.432

  However, when the log coefficient is 0, the value of the term is 0. In the above example, the value of − (0/7) × log (0/7) is 0.

  Therefore, in the case of this example, the classifier generation unit 12 calculates the evaluation value when divided by the attribute 1 as 0.940−0.432 = 0.508.

  The classifier generation unit 12 calculates not only the attribute 1 but also other attributes in the same manner, calculates an evaluation value when the attribute value is divided, and determines that the attribute value is maximized. In this way, the attribute for dividing the root node is determined.

  In the example of the attribute 1, the case where the value that the attribute 1 can take is only “0” or “1” is shown. If the attribute value is age and the value is a continuous value such as 20, 21, 22, it is also determined which attribute value is used for division. In this case, the classifier generation unit 12 uses an intermediate value between the attribute values as a threshold value, and divides each threshold value into “below the threshold value” and “greater than the threshold value”. The evaluation value is obtained when Then, by selecting the case where the evaluation value is maximized, it is also determined which attribute value is used for division. For example, in the case where the attribute values are continuous with 20, 21, 22,..., The evaluation values when divided by “20.5 or less” and “greater than 20.5”, “21.5 or less” and “ An evaluation value or the like when dividing by “greater than 21.5” may be calculated and divided so that the evaluation value becomes the highest.

  The classifier generation unit 12 performs the same process as described above for each node after division, and sequentially repeats the process of determining which attribute is to be divided next. Further, the classifier generation unit 12 stops the node division when a predetermined condition is satisfied. The predetermined condition is, for example, a condition that “the classes of individual data in the node are all the same” or “the number of individual data counted as positive or negative in the node is equal to or less than a predetermined number (for example, 2)”. Conditions may be used. When the former condition is adopted, when all the individual data in the node becomes positive or negative, the division of the node is not continued. In this way, the classifier generation unit 12 generates a tree-structured decision tree by repeating division sequentially from the root node.

  Further, as described above, the classifier generation unit 12 generates a tree-structured decision tree, and then prunes the decision tree. In the decision tree, the final node generated by dividing is called a leaf. Assume that the number of data classified into a certain leaf is D (that is, the number of individual data counted as positive or negative is D). It is assumed that E data is erroneous in D data classified into these leaves. Under this assumption, it can be considered that the event of error is observed E times during D trials, and it can be considered as a binomial distribution in which the probability of the event of error occurring in a sample of size D is r. If the upper limit of r is expressed as U_CF (E, D) with respect to the reliability CF given in advance, the expected value at which an error occurs in the D data is D × U_CF (E, D). The classifier generation unit 12 calculates, with respect to a parent node whose child nodes are all leaves, an expected value of errors in the parent (an expected value at which an error occurs) and a sum of expected values of error of the child leaves. Compare. If the sum of the expected values at the child is larger than the expected value of the parent error, the classifier generation unit 12 degenerates the leaf and sets the parent as the leaf. The classifier generator 12 prunes the leaves of the entire decision tree by sequentially repeating this process.

  When degenerating a leaf, the classifier generation unit 12 may delete the leaf and use the parent node of the deleted leaf as the leaf. For example, in the decision tree illustrated in FIG. 4, when the node divided according to the attribute value “humidity” is degenerated, the classifier generation unit 12 uses the individual data whose humidity attribute value is 70% or less. The node “Yes: 2, No: 0” representing the number and the node “Yes: 0, No: 3” representing the number of individual data whose humidity attribute value is higher than 70% are deleted. The parent node “Yes: 2, No: 3” of the two nodes may be set as a leaf.

  When generating a decision tree as a classifier, for example, a decision tree may be generated by determining and pruning a decision tree as described above.

  The classifier generation unit 12 may generate the decision tree after heavily weighting data of a specific class among the data belonging to the subset generated in step A4. Here, weighting means setting a weight for data. For example, weighting may be performed by creating a copy of the individual data of a particular class so that the number of individual data of a specific class determined in advance is several times out of the data belonging to the subset. Note that it is sufficient to determine in advance how many times the number of individual data of a specific class is to be increased. Then, the classifier may be generated after copying the individual data of the class so as to increase the number of individual data of the specific class. The class to be weighted and the weight (for example, how many times the data is increased) may be input by the user of the data similarity calculation system via an input device (not shown) such as a keyboard. , May be specified in other manners.

  When a classifier is generated in step A5, the self-evaluation unit 13 determines a class of individual data belonging to the subset generated in step A3 using the classifier (step A6). Here again, the case where the classifier is a decision tree will be described as an example. When a decision tree to be a classifier is generated, a class is predicted from the decision tree and known attributes (attributes of individual data belonging to a subset). At this time, the self-evaluation unit 13 refers to the attribute value of the individual data regarding the attribute used when dividing the node, starting from the node of the root of the decision tree, and traces the child node according to the attribute value. If the self-evaluation unit 13 traces the child node and traces to the leaf node, the class having the larger count number in the leaf node may be determined as a result of the determination using the classifier.

  FIG. 5 is an explanatory diagram illustrating an example of classes determined from the classifier. The attribute values of the attributes shown in FIG. 5 are the same as the attribute values shown in FIG. FIG. 5 shows the result of determining a class from its attribute value using the decision tree (classifier) illustrated in FIG. For example, it is assumed that the attribute value of the attribute “weather” in the individual data is “rain”. Then, the child node “Yes: 3, No: 2” is traced from the root node (see FIG. 4). Since the node is a leaf, the self-evaluation unit 13 determines that the class is “not purchased” based on the count number “Yes: 3, No: 2”. As a result of the determination, the determination results of the sixth and last individual data classes shown in FIG. 5 are different from the original class shown in FIG. 2 (see FIGS. 2 and 5).

  After the class of each individual data belonging to the subset is determined in step A6, the similarity calculation unit 14 initializes a first variable i for sequentially specifying the individual data belonging to the data set to 1. (Step A7). Subsequently, the similarity calculation unit 14 determines whether or not the value of the variable i is equal to or less than the total number N of individual data (step A8). If the value of the variable i exceeds N (No in step A8), the variable t is incremented by 1 (step A8), and then the data similarity calculation system repeats the processing after step A3 again.

  If the value of the variable i is N or less (Yes in step A8), the similarity calculation unit 14 includes the i-th individual data determined by the variable i in the subset generated in the latest step A4. It is determined whether or not (step A10). If the i-th data is not included in the subset (No in Step A10), the similarity calculation unit 14 increments the variable i by 1 (Step A11), and repeats the processes after Step A8 again.

  If the i-th data is included in the subset (Yes in step A10), the value of the second variable j for sequentially specifying individual data belonging to the data set is set to i + 1 (step A12). ). That is, i + 1 is substituted for j.

  After step A12, the similarity calculation unit 14 determines whether the value of the variable j is N or less (step A13). If the value of the variable j exceeds N (No in step A13), the similarity calculation unit 14 increments the variable i by 1 (step A11), and repeats the processing after step A8 again.

  If the value of the variable j is N or less (Yes in step A13), the similarity calculation unit 14 includes the j-th individual data determined by the variable j in the subset generated in the latest step A4. It is determined whether or not (step A14). If the j-th individual data is not included in the subset (No in Step A14), the similarity calculation unit 14 increments the variable j by 1 (Step A17), and repeats the processes after Step A13.

If the j-th individual data is included in the subset (Yes in Step A14), the similarity calculation unit 14 determines that the class (denoted as PredC _i ) determined from the attribute of the i-th individual data, j The class (denoted as PredC _j ) determined from the attribute of the th individual data is compared with each other to determine whether or not both are the same class (step A15). PredC _i and PredC _j are classes determined from attributes for the i-th and j-th individual data in the most recent step A6 using a classifier. The fact that PredC _i and PredC _j are the same class means that the i-th individual data and the j-th individual data are data determined to be the same class.

If PredC _i and PredC _j are different (No in Step A15), the similarity calculation unit 14 increments the variable j by 1 (Step A17), and repeats the processes after Step A13.

If PredC _i and PredC _j are the same class (Yes in step A15), a predetermined number is added to the similarity Sim (i, j) between the i-th individual data and the j-th individual data (step A16). In this example, the predetermined number is set to 1, and 1 is added to Sim (i, j) in step A16. After step A16, the similarity calculation unit 14 increments the variable j by 1 (step A17), and repeats the processing after step A13.

As described above, when one subset is generated, a pair of individual data belonging to the subset, and the degree of similarity between individual data in which PredC _i and PredC _j are in the same class The value of Sim (i, j) is increased by 1. Further, in the case of individual data that does not belong to a subset, one or both of them does not move to step A16, so the similarity between the individual data does not increase. Even if a pair of individual data belonging to the subset is different in class determined from the attribute, the process does not proceed to step A16, and the similarity between the individual data does not increase.

  Then, the data similarity calculation system increments the variable t (step A8), generates a subset (step A4), and executes the processing after step A5. At this time, the processing of step A16 is not uniformly executed for each set of individual data. When a subset is generated a plurality of times while allowing duplication of individual data, the value of Sim (i, j) is also increased for a set of individual data for which step A16 has been performed many times. On the other hand, the increment from the initial value of Sim (i, j) is small for the set of individual data in which step A16 is performed a small number of times. Sim (i, j) for each pair of data when the number of generations t of the subset reaches the upper limit value T is the similarity between the paired individual data.

  In the first embodiment, when two data belong to a subset and the conditions determined for the two data are the same, the two data are satisfied. A predetermined value is added to the data similarity Sim (i, j). In addition to this condition, two data similarities Sim (i, i, i) are satisfied when the condition determined for the two data is a specific class (for example, “purchase”). A predetermined value is added to j), and in other cases, addition to Sim (i, j) may not be performed. In this case, even if the class determination results for the two data are the same, if the class does not come out of a specific class, the addition to the similarity is not performed.

When calculating the similarity as described above, the similarity calculation unit 14 determines that PredC _i and PredC _j are the same in Step A15, and PredC _i and PredC _j are specific classes (for example, “Purchase”). ) Or not. If this condition is satisfied, step A16 is performed. If not, the process proceeds to step A17.

Next, the effect of the first embodiment will be described.
According to the first embodiment, an initial value of similarity is determined for each set of data belonging to a given data set. Then, a subset is generated from the set of data, and a classifier is generated from attributes (for example, customer characteristics and sales conditions) and classes defined for each data included in the subset. Furthermore, using a classifier, a class is determined from the attribute, and a predetermined value is added to the similarity of the data that are included in the subset and have the same determined class. The degree of similarity between the data is determined by repeating the processing after the above subset generation a plurality of times. In the present invention, the degree of similarity between each data can be automatically calculated by such processing.

  Further, in the present embodiment, at the time of generating a subset (step A4), it is only necessary to select a predetermined number M of individual data, and individual data belonging to the already generated subset is selected to allow duplication. To do. For example, random sampling may be performed. As described above, the selection of M pieces of individual data is performed a plurality of times to generate the similarity. If the similarity is obtained, the individual data can be classified based on the similarity. When an analyst manually classifies a customer as in the system described in Patent Document 1, special skills for customer classification are required, or analysis is performed through trial and error. However, if the similarity is obtained as described above, the similarity for use in classification can be calculated without taking the user's effort even if the user of the system does not have special skills. .

  In addition, if each data corresponds to an individual customer, the attributes of the data are customer characteristics and sales conditions, and the data is given a class of “buy” or “do not buy”, the customer for the product or service Based on the behavior (whether or not purchased), the similarity of data can be obtained. In addition, since the data corresponds to the customer, it can be said that the degree of similarity between the customers.

  In the following explanation of the effect, when the attribute of the data is a characteristic of the customer, the data corresponds to the customer, and the data class is the purchase behavior of the customer (for example, “buy” or “do not buy”) Will be described as an example.

  When a subset is generated by random sampling that allows duplication, individual data (in this example, an individual customer) is selected as a subset with a probability of M / N. That is, it is included in the subset with a probability of M / N. Other data (other customers) are also included in the subset with similar probabilities. Each data set obtained by combining data belongs to a subset with the same probability. Therefore, if the number of times T for creating the subset is sufficiently large and the original class and the class determined in step A6 are all the same, the similarity between the data is almost the same, and the data with the same class is finely divided. It cannot be classified. For example, it is not possible to classify the data of the class “Purchase” or to classify the data of the class “Do not purchase”. Also, the similarity between the class data “purchase” (customers who have purchased products) and the data class “do not purchase” (customers who did not purchase products) is zero. On the other hand, in the first embodiment of the present invention, a classifier is generated from each individual data belonging to the subset, and the class is determined from the attribute of the individual data using the classifier. And an opportunity to create a case where the class after the judgment is different. Then, the similarity Sim (i, j) can be determined for each set of the i-th data and the j-th data by obtaining the similarity of the individual data using the determined class. it can.

  In the present invention, even if there are many attributes that are related to each other, the similarity can be calculated without being strongly influenced by the related attributes. For example, it is assumed that ten pieces of data with data numbers 1 to 10 shown in FIG. 12 are given. If the data of data numbers {1, 2, 7, 9, 10} shown in FIG. 12 is a subset, for example, the classifier is “If the value of attribute a is 1, the class is“ purchase ”and attribute a If the value of 2 is 2, a decision tree “class is“ not purchased ”” is generated, and the similarity between data of data numbers {1, 2} and between data of data numbers {7, 9, 10} Are added together. Also, for example, data with data numbers {1, 2, 6, 7, 9, 10} is a subset. In this case, for example, a decision tree is generated as a classifier “if the value of attribute b is 1, the class is“ purchase ”, and if the value of attribute b is 2, the class is“ do not purchase ””, The similarity between the data of data numbers {1, 2, 6} and the similarity between the data of data numbers {7, 9, 10} are added.

  In general, a plurality of attributes appear in the decision tree, but when attributes are related to each other, the number of attributes appearing in the decision tree is reduced. Taking the case where the above data numbers {1, 2, 6, 7, 9, 10} are a subset, for example, when the individual data is classified into two using the attribute b, the attributes c, d, or The classification of the individual data is exactly the same when the individual data is classified into two using e. Therefore, after dividing by attribute b, it is not divided by attributes c, d, or e. As a specific example, when the data with the data number {1, 2, 6, 7, 9, 10} is divided by the attribute b, the node with the value of the attribute b of 1 has the data number {1, 2, 6}. A node whose data is included and the value of attribute b is 2 includes data number {7, 9, 10}. When the node with the data number {1, 2, 6} is divided by c, all the data with the data number {1, 2, 6} has the attribute c value of 1 and the data with the attribute c value of 2 Absent. Even in a node where the value of attribute b is 2, the value of attribute c is biased to one side. The same applies when dividing by attribute d or e after division by attribute b.

  In this way, when attributes are related to each other, the number of attributes appearing in the decision tree is reduced, and the attribute is not strongly influenced by the related attributes. In the example in which the data numbers {1, 2, 6, 7, 9, 10} are a subset, for example, the attribute b appears in the decision tree, but the other attributes c, d, and e are simultaneous with the attribute b. It does not appear in the decision tree and is not affected by the attributes c, d, and e related to the attribute b. Therefore, even when there are many attributes related to the data, the similarity can be obtained without being strongly influenced by the related attributes.

  Moreover, not only is the condition that the two data belong to a subset and the classes determined for the two data are the same, but the class determined for the two data is a specific class. When the addition (step A16) of the two data to the similarity Sim (i, j) is performed only when the condition of class (for example, “purchase”) is satisfied, the following effects are obtained. It is done. That is, the similarity can be calculated with emphasis on the difference in characteristics of data of a specific class. Here, the case where data and a customer correspond is described as an example. If a condition that the class determined for the two data is a specific class is also added as a condition for shifting to step A16, the similarity can be calculated with emphasis on the difference in characteristics of customers of the specific class. For example, a customer who purchases a product or service has one of several features and has not yet purchased the product or service, but a customer who has a feature similar to an already purchased customer (a customer who is likely to purchase) ) May have one of several features. In such a case, an analyst may think that the similarity with a customer who has not purchased is not emphasized but the similarity with a purchased customer is important. In such a case, the data of the customer determined to be the specific class “purchase” in step A6 and the data of “customer who is expected to purchase” determined to be “purchase” in step A6. By adding the similarities, it is possible to calculate the similarities with an emphasis on the difference in characteristics of customers of the class “buy”.

  Further, when creating the classifier, the classifier generation unit 12 applies a weight to the data belonging to the subset generated in step A4 and having a predetermined class as a specific class. Once done, the classifier may be created. For example, among the data belonging to the subset, a copy of the data is created so that the number of data of the class for which the class given in advance is “purchase” increases by the specified amount, The duplicated data may also be used to generate a classifier. If similarity is calculated using such a classifier, when data is classified based on the similarity, it is classified into clusters (groups) with emphasis on the difference in characteristics of data of classes other than a specific class. can do. For example, it is assumed that the number of customers (data) in the class “Purchase” is the same as the number of customers in the class “Do not purchase”. When data is not weighted, a cluster in which data of any class is mixed tends to be generated. When the data of a specific class is heavily weighted, the similarity between the data of the specific class becomes high in any combination, thereby forming one cluster. On the other hand, data of classes other than a specific class do not have such a high degree of similarity, and can be classified into a plurality of clusters without being combined into one cluster. Note that processing for classifying data into clusters (groups) using the similarity will be described in the second embodiment.

Embodiment 2. FIG.
FIG. 6 is a block diagram showing an example of the second embodiment of the present invention. The data similarity calculation system 20 of the second embodiment includes a subset generation unit 11, a classifier generation unit 12, a self-evaluation unit 13, a similarity calculation unit 14, and a similarity clustering unit 21. Constituent elements similar to those in the first embodiment are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof is omitted. As in the first embodiment, the data similarity calculation system 20 of the second embodiment calculates the similarity between individual data belonging to a set of data, and then classifies the individual data using the similarity. . Therefore, the data similarity calculation system 20 of the second embodiment can be called a classification system. Hereinafter, in this embodiment, a group of individual data is referred to as a cluster.

  The similarity clustering unit 21 classifies the individual data belonging to the data set into a plurality of clusters based on the similarity between the individual data obtained by the similarity calculation unit 14. The similarity clustering unit 21 receives the target number of clusters, and the similarity clustering unit 21 classifies the individual data so that the number of clusters is the same. The target number of clusters may be input by a user of the data similarity calculation system via an input device (not shown) such as a keyboard provided in the data similarity calculation system 20. Alternatively, the number of clusters may be specified in another manner.

  The subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, the similarity calculation unit 14, and the similarity clustering unit 21 are realized by a CPU that operates according to a program (data similarity calculation program), for example. In that case, the CPU may operate as the subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, the similarity calculation unit 14, and the similarity clustering unit 21 according to the program.

Next, the operation of the second embodiment will be described.
The process for obtaining the similarity between the individual data belonging to the data set is the same as in the first embodiment. The subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, and the similarity calculation unit 14 operate in the same manner as in the first embodiment. For example, the processing shown in FIG. What is necessary is just to obtain | require similarity Sim (i, j) between separate data for every group.

  After the similarity Sim (i, j) between the individual data is calculated, the similarity clustering unit 21 clusters (i.e. classifies) the individual data using the similarity in each set of individual data.

  There are various clustering methods when the similarity is given. For example, there are a shortest distance method, a longest distance method, a county average method, a Ward method and the like as a hierarchical clustering method, and a K average method and the like as a non-hierarchical clustering method. The similarity clustering unit 21 may classify the individual data by any method. In addition, designation of which method to perform clustering may be input by a user of the data similarity calculation system via an input device such as a keyboard. Or you may specify in another aspect.

  Hereinafter, the clustering process of individual data will be described by taking as an example the case of clustering individual data by the shortest distance method. FIG. 7 is a flowchart illustrating an example of a process progress in which individual data is clustered by the shortest distance method. In the following description, the target number of clusters is K. The target cluster number K is input to the similarity clustering unit 21 in advance via an input device such as a keyboard, for example.

  The similarity clustering unit 21 determines, for each individual data, a cluster that includes only one individual data belonging to the data set (step B1). Accordingly, the similarity clustering unit 21 determines N clusters equal to the total number N of individual data belonging to the data set in step B1. The individual data belonging to the N clusters is different for each cluster. After step B1, the process proceeds to step B2.

In step B2, the similarity clustering unit 21 obtains the similarity between the clusters for each set of clusters composed of two clusters, specifies the two clusters having the highest similarity between the clusters, and determines the two clusters. Are merged into one cluster (step B2). Assuming that the number of clusters at the time of shifting to step B2 is L, the similarity between the clusters is obtained for each of the _{two L} C groups that take out two clusters from the _L clusters, and the similarity between the clusters is the highest. Two clusters can be merged into one cluster. Note that L = N when the process first proceeds from step B1 to step B2.

An example of a method for determining the similarity between two clusters in a set of two clusters is shown below. One of the two clusters for which the similarity is obtained is denoted as C ₁ and the other is denoted as C ₂ . The similarity between the two clusters C ₁ and C ₂ is denoted as S (C ₁ , C ₂ ). Similarity clustering unit 21, for example, the individual data belonging to C _1, of the similarity between the individual data for each combination of the individual data belonging to C _2, the similarity between the maximum value C _1, C ₂ S _(C 1, C ₂₎ and it may be determined. That is, when one of the individual data retrieved from the _{C 1} and _{x 1,} the one individual data retrieved from the _{C 2} and _{x 2,} the similarity of x 1, _{x 2} and _{S (x} 1, _{x 2),} The similarity clustering unit 21 may determine the maximum value of S (x ₁ , x ₂ ) as S (C ₁ , C ₂ ) as shown in Expression (1).

  Further, merging two clusters into one cluster is to combine individual data belonging to the two clusters into one cluster. By merging two clusters into one cluster, the total number of clusters is reduced by one.

  After merging two clusters into one cluster in step B2, the similarity clustering unit 21 determines whether or not the number of clusters has reached the target number K (step B3). If the number of clusters has not decreased to the target number K (No in step B3), the processing after step B2 is repeated. If the number of clusters is the target number K (Yes in step B3), K clusters are obtained, and the individual data is classified into K pieces, so the processing is terminated.

  According to the second embodiment, individual data included in a data set can be classified into a target number of clusters.

  For example, if a data attribute is a characteristic of a customer, the data corresponds to the customer, and the data class represents the purchase behavior of the customer (such as “buy” or “do not buy”), the product or service The customers can be automatically classified according to the purchase behavior of the customer.

  In addition, as illustrated in FIG. 7, first, individual data is allocated to separate clusters, and if the clusters are merged until the number of clusters reaches the target number, the individual data is classified into the target number of clusters. be able to. That is, it is possible to prevent individual data from being collected in a specific cluster and the number of clusters of individual data from reaching the target number.

Embodiment 3. FIG.
FIG. 8 is a block diagram showing an example of the third embodiment of the present invention. The data similarity calculation system 30 according to the third embodiment includes a subset generation unit 11, a classifier generation unit 12, a self-evaluation unit 13, a similarity calculation unit 14, a similarity clustering unit 21, and attribute data. A classification unit 31 and a relevance calculation unit 32 are provided. The same components as those in the first embodiment and the second embodiment are denoted by the same reference numerals as those in FIGS. 1 and 6, and detailed description thereof is omitted. Since the data similarity calculation system 30 of the third embodiment classifies individual data using the similarity as in the second embodiment, it can be called a classification system. Also in the present embodiment, the group of individual data classified by the similarity clustering unit 21 is referred to as a cluster. In addition, the data similarity calculation system 30 according to the third embodiment calculates the relevance of the attribute to the similarity calculation. The attribute relevance to similarity calculation is a numerical value indicating the degree of attribute relevance to similarity calculation (in other words, the degree to which an attribute affects similarity calculation).

  Information that specifies an attribute and a classification method based on the attribute value (hereinafter referred to as classification method designation information) is input to the attribute data classification unit 31, and the attribute data classification unit 31 performs individual processing according to the designated classification method. Classify the data. The number of individual data classifications by the attribute data classification unit 31 is determined by the classification method designation information. As an example of the classification method designation information, “individual data whose attribute value of attribute name A is equal to or greater than Thres is classified into group 1, and individual data whose attribute value of attribute name A is less than Thres is classified as group 2.” Information. In this case, the individual data is classified into two groups 1 and 2. The classification method designation information is input to the attribute data classification unit 31 via an input device (not shown) such as a keyboard, for example, but may be input in another manner.

  Based on the relationship between the cluster (group) classified by the similarity clustering unit 21 and the group classified by the attribute data classifying unit 31, the relevance calculating unit 32 determines the degree of attribute relevance (hereinafter, Simply referred to as attribute relevance). This attribute is an attribute designated by the classification method designation information.

  The subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, the similarity calculation unit 14, the similarity clustering unit 21, the attribute data classification unit 31, and the relationship calculation unit 32 are, for example, a program (data similarity calculation This is realized by a CPU that operates according to a program. In that case, the CPU operates as the subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, the similarity calculation unit 14, the similarity clustering unit 21, the attribute data classification unit 31, and the relevance calculation unit 32 according to the program. To do.

Next, the operation of the third embodiment will be described.
The operation until the similarity clustering unit 21 classifies the individual data is the same as that in the second embodiment. The subset generation unit 11, the classifier generation unit 12, the self-evaluation unit 13, and the similarity calculation unit 14 operate in the same manner as in the first and second embodiments. For example, the processing illustrated in FIG. , J, the similarity Sim (i, j) between the individual data may be obtained. After that, the similarity clustering unit 21 classifies the individual data into the designated number of clusters as in the second embodiment.

  FIG. 9 is a flowchart illustrating an example of an operation for obtaining the degree of association of attributes after classification by the similarity clustering unit 21. Here, a case where the number of clusters grouped by the similarity clustering unit 21 and the number of groups grouped by the attribute data classification unit 31 are equal will be described as an example. In this example, the similarity clustering unit 21 classifies individual data belonging to a data set into two clusters. Further, the attribute data classification unit 31 may, for example, “classify individual data whose attribute value of attribute name A is equal to or greater than Thres into group 1 and group individual data whose attribute value of attribute name A is less than Thres into group 2. It is assumed that the classification method designation information “Classify” is input, and the attribute data classification unit 31 classifies the individual data into two groups.

  In the following description, the variable k is a variable for designating a cluster grouped by the similarity clustering unit 21. In this example, k can be 1 or 2. The variable l is a variable for designating groups grouped by the attribute data classifying unit 31. In this example, l can be 1 or 2.

  First, the attribute data classification unit 31 sequentially determines a combination of k and l determined by k = 1, 2 and l = 1, 2, and a variable N [k] [l] corresponding to the combination of k and l one-to-one. Are initialized to 0 (step C1). N [k] [l] represents the number of individual data belonging to the kth cluster and belonging to the lth group.

  The attribute data classification unit 31 initializes a variable i for designating individual data to 1 (step C2).

  Next, the attribute data classification unit 31 sets the value of the variable k as the cluster number of the cluster to which the individual data i (that is, the i-th individual data) belongs (step C3). For example, the similarity clustering unit 21 assigns numbers 1 to K for identifying clusters to K clusters obtained as a result of classification. The number assigned to each cluster is the cluster number. In step C3, the cluster number of the cluster to which the data i belongs may be substituted for the variable k. In this example, since the number of clusters is 2, the cluster number is 1 or 2.

  Next, the attribute data classifying unit 31 determines in which group the individual data i is included according to the attribute value of the individual data i (the attribute value of the attribute designated by the classification method designation information) (step C4). . That is, the classification for the individual data i is performed according to the classification method designation information. In this example, a classification method designation of “individual data whose attribute value of attribute name A is equal to or greater than Thres is classified into group 1 and individual data whose attribute value of attribute name A is less than Thres is classified as group 2” is specified. According to the information, it is determined whether or not the attribute value of the attribute A of the individual data i is equal to or greater than Thres. If the attribute value of attribute A of individual data i is equal to or greater than Thres (that is, if it is determined that individual data i is classified into group 1), the value of variable l is set to 1 (step C5). If the attribute value of attribute A of individual data i is less than Thres (that is, if it is determined that individual data i is classified into group 2), the value of variable l is set to 2 (step C6).

  After step C5 or step C6, the value of N [k] [l] corresponding to the combination of k defined in step C3 and l defined in step C5 or step C6 is incremented by 1 (step C7). . For example, when the process moves from step C5 to step C7 and N [k] [1] is incremented by 1, one piece of individual data belonging to the kth cluster and belonging to the first group is obtained. The count value is incremented by one. Further, when the process moves from step C6 to step C7 and N [k] [2] is incremented by 1, individual data belonging to the kth cluster and belonging to the second group is one. The count value is incremented by one.

  Next, the attribute data classification unit 31 increments the value of the variable i by 1 (step C8). Then, the attribute data classification unit 31 determines whether or not the value of the variable i is less than or equal to the total number N of individual data (step C9), and if the value of i is less than or equal to N (Yes in step C9), step The process after C3 is performed again. Accordingly, the processing from step C3 is performed on the individual data from the first to the Nth, and the individual data belonging to the kth cluster and also belonging to the lth group The number of data N [k] [l] is obtained for each combination of k and l.

  If the value of i exceeds N (No in step C9), the process proceeds to step C10.

  In step C10, the relevance calculation unit 32 uses N [k] [l] to test the independence between the classification by the similarity clustering unit and the classification by the attribute data classification unit, and calculates the p value (step S10). C10). This p value can be used as the degree of association of the attribute designated by the classification method designation information.

  For example, the relevancy calculation unit 32 obtains the p value based on a hypothesis that the classification performed by the similarity clustering unit 21 and the classification performed by the attribute data classification unit 31 are independent. In this case, the relevancy calculation unit 32 first calculates an expected value of N [k] [l] (denoted as E [k] [l]) for each combination of k and l. Under the above hypothesis, the relevancy calculation unit 32 may obtain the expected value E [k] [l] of each N [k] [l] by the calculation of the following equation (2).

Then, using the N [k] [l] and E [k] [l], when computing the chi ₀ ² by calculation of equation (3) below, chi ₀ ² follows the chi ² distribution.

Therefore, the relevancy calculation unit 32 may obtain χ ₀ ² by calculation of Equation (3) and determine the p value with reference to the χ ² distribution table. This p value is the degree of association of the attribute designated by the classification method designation information. Note that the χ ² distribution table may be stored in advance in a storage device (not shown) included in the data similarity calculation system 30, for example.

The degree of freedom used when determining the p value from the χ ² distribution table is the number of clusters −1 classified by the similarity clustering unit 21. Therefore, in this example, since the number of clusters is 2, the degree of freedom may be 2-1 = 1.

FIG. 10 is an explanatory diagram showing an example of the χ ² distribution table. In FIG. 10, the degree of freedom is represented by ν, and the case of ν = 1, 2, 3 is illustrated, but the case of ν = 4 or more is also included. The uppermost values “0.7”, “0.5”, “0.3”, “0.2”, “0.1”, etc. in the χ ² distribution table shown in FIG. 10 are p values. For example, as a result of calculating χ ₀ ² by the expression (3), it is assumed that the relevancy calculation unit 32 satisfies χ ₀ ² ≈1.07. In this example, since χ ₀ ² follows the χ ² distribution with 1 degree of freedom, the p value “0.3” corresponding to χ ₀ ² ≈1.07 at the degree of freedom 1 is specified from the χ ² distribution table, The p value “0.3” may be set as the relevance level of the attribute A to the similarity calculation.

  As described above, the relevance level of the attribute A can be obtained. The degree of association of other attributes can also be obtained by inputting classification method designation information.

In addition, the relevance calculation unit 32 calculates χ ₀ ² and then calculates Equation (4) shown below to obtain a Cramer's relevance coefficient, and calculates the relativity of the Cramer's relevance coefficient. It is good also as the relevance degree of the attribute to

  The number of relations of Kramer is a numerical value from 0 to 1, and the closer to 1, the stronger the relation.

  Further, after the relevance calculation unit 32 obtains the relevance level of the attribute to the similarity calculation, the relevance level of the attribute is output to an output device (not shown) such as a display device or a printer device, and then the process ends. May be. The user may input classification method designation information that designates an attribute via an input device (not shown), and the data similarity calculation system 30 may obtain the association of the attribute. The relevance calculation unit 32 may output the degree of relevance of each attribute thus obtained in an output device (not shown) such as a display device or a printer device in a table format or a graph format. Good.

Next, effects of the third embodiment will be described.
According to the present invention, the degree of association between clustering based on similarity and attribute values can be examined. For example, the relationship between the clustering based on the similarity calculated according to the customer's behavior for the product or service and the attribute value can be examined. For example, for example, it is possible to examine whether the clustering by similarity is highly related to gender (gender), and when classifying customers according to customer behavior for goods or services, It is possible to extract the characteristics and sales conditions of a certain customer. As a result, it is possible to analyze a customer or the like to be a future target of the product or service.

  Next, the outline of the present invention will be described. FIG. 11 is a block diagram showing an outline of the present invention. The data similarity calculation system 80 of the present invention includes a subset generation unit 81, a classifier generation unit 82, a class determination unit 83, and a similarity calculation unit 84.

  The subset generation unit 81 (for example, the subset generation unit 11) allows a plurality of subsets of the set by allowing data duplication from a set of data including an attribute indicating data characteristics and a class indicating data classification. Generate times.

  The classifier generation unit 82 (for example, the classifier generation unit 12) generates a classifier (for example, a decision tree) that is a rule for determining a class from an attribute every time a subset is generated based on the subset. To do.

  The class determination unit 83 (for example, the self-evaluation unit 13) determines a class of individual data belonging to the subset using the classifier every time a classifier is generated.

  The similarity calculation unit 84 (for example, the similarity calculation unit 14) generates a subset of data sets, and when the class determination unit determines the classes of individual data belonging to the subset, A value is added to the similarity between the determined data.

  According to the present invention, data duplication is permitted, a subset is generated a plurality of times from a set of data, a classifier is generated for each subset, and a class of data belonging to the subset is determined. Since the similarity is calculated using this determination result, even if there are many attributes that are related to each other in the data, the similarity of the data can be obtained without being strongly influenced by those related attributes. Can do.

  Further, the above embodiment discloses a configuration including data grouping means (for example, the similarity clustering unit 21) that classifies data belonging to a data set into a plurality of groups based on the similarity between the data. ing. According to such a configuration, the individual data included in the data set can be classified.

  In the above embodiment, the data grouping means classifies the individual data belonging to the set of data into different groups, obtains the similarity between the data belonging to two different groups, and the similarity is The structure which repeats merging two largest groups and reduces the total number of groups to a target number is disclosed. According to such a configuration, the individual data included in the data set can be classified into a target number of groups.

  In the above embodiment, the data belonging to the data set is classified by the attribute data classification unit (attribute data classification unit 31) that classifies the data into a group according to the attribute value of the specific attribute, and the data grouping unit. Relevance calculating means (relevance calculating unit 32) for calculating the relevance of a specific attribute to the similarity calculation based on the relationship between the group of data and the data group classified by the attribute data classifying means. An arrangement comprising the above is disclosed. According to such a configuration, it is possible to check the degree of association between the result of classification using the similarity and the attribute value.

  Further, the above embodiment discloses a configuration in which the subset generation unit 81 generates a subset of a set by randomly sampling data from the data set.

  Further, the above embodiment discloses a configuration in which the similarity calculation unit 84 adds a value only to the similarity between data determined as a specific class by the class determination unit 83. According to such a configuration, the similarity can be calculated with emphasis on the difference in characteristics of data of a specific class.

  In the above-described embodiment, a configuration is disclosed in which the classifier generation unit 82 generates a classifier by weighting data in which a given class is a specific class among data belonging to a subset. According to such a configuration, when classifying data based on the obtained similarity, it is possible to classify the data into groups with emphasis on the difference in characteristics of data of classes other than the specific class.

  In the above embodiment, a configuration is disclosed in which the subset generation unit 81 generates a subset of the set from a set of data having at least customer characteristics or sales conditions as attributes and customer behavior as a class. ing.

  The present invention is suitably applied to a data similarity calculation system for obtaining a similarity between data belonging to a data set and a classification system for classifying data by calculating a similarity between data.

It is a block diagram which shows the example of the 1st Embodiment of this invention. It is explanatory drawing which shows the example of data. It is a flowchart which shows the example of process progress of the data similarity calculation system of 1st Embodiment. It is explanatory drawing which shows the example of a classifier. It is explanatory drawing which shows the example of the class determined from the classifier. It is a block diagram which shows the example of the 2nd Embodiment of this invention. It is a flowchart which shows the example of the process progress which clusters individual data by the shortest distance method. It is a block diagram which shows the example of the 3rd Embodiment of this invention. It is a flowchart which shows the example of the operation | movement which calculates | requires the relevance degree of an attribute. It is explanatory drawing which shows the example of (chi) ² distribution table. It is a block diagram which shows the outline | summary of this invention. It is explanatory drawing which shows the example of data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Subset production | generation part 12 Classifier production | generation part 13 Self-evaluation part 14 Similarity calculation part 21 Similarity clustering part 31 Attribute data classification part 32 Relevance calculation part

Claims (24)

A subset generation means for generating a subset of the set a plurality of times by allowing data duplication from a set of data including an attribute indicating data characteristics and a class indicating data classification;
Classifier generation means for generating a classifier, which is a rule for determining a class from an attribute each time a subset is generated, based on the subset;
Class determination means for determining a class of individual data belonging to the subset using the classifier each time a classifier is generated;
A similarity calculation that adds a value to the similarity between data determined to be the same class when a subset of the data set is generated and the class determination means determines the class of each data belonging to the subset And a data similarity calculation system. The data similarity calculation system according to claim 1, further comprising data grouping means for classifying data belonging to the data set into a plurality of groups based on the similarity between the data. The data grouping means classifies the individual data belonging to the data set into separate groups, calculates the similarity between the data belonging to two different groups, and merges the two groups having the maximum similarity The data similarity calculation system according to claim 2, wherein the total number of groups is reduced to a target number. Attribute data classification means for classifying data belonging to a data set into groups according to attribute values of specific attributes;
Relevance calculating means for calculating the relevance of the specific attribute to similarity calculation based on the relationship between the group of data classified by the data grouping means and the group of data classified by the attribute data classifying means; The data similarity calculation system according to claim 2 or 3. The data similarity calculation system according to any one of claims 1 to 4, wherein the subset generation unit generates a subset of the set by randomly sampling data from the set of data. The data similarity according to any one of claims 1 to 5, wherein the similarity calculation means adds a value only to the similarity between the data determined as the specific class by the class determination means. Calculation system. 7. The classifier generation unit generates a classifier by weighting data in which a given class is a specific class among data belonging to the subset. 8. Data similarity calculation system. The subset generation means generates a subset of the set from a set of data having at least customer characteristics or sales conditions as attributes and customer behavior as a class. The data similarity calculation system described in 1. A subset generation step for generating a subset of the set a plurality of times by allowing duplication of data from a set of data including an attribute indicating the characteristics of the data and a class indicating the classification of the data;
A classifier generating step for generating a classifier as a rule for determining a class from an attribute each time a subset is generated based on the subset;
A class determination step for determining a class of individual data belonging to the subset using the classifier each time a classifier is generated;
When the similarity calculation means generates a subset of a set of data and determines the class of individual data belonging to the subset in the class determination step, the similarity is calculated for the data determined to be the same class. A data similarity calculation method comprising: a similarity calculation step of adding. Data grouping means, on the basis of the similarity of data among the data similarity calculation method according to claim 9 comprising a data grouping step of classifying the data belonging to the set of data into a plurality of groups. In the data grouping step , the data grouping means classifies the individual data belonging to the data set into separate groups, obtains the similarity between the data belonging to two different groups, and the similarity is the maximum. The data similarity calculation method according to claim 10, wherein the two groups are repeatedly merged to reduce the total number of groups to a target number. An attribute data classification means for classifying data belonging to the data set into groups according to attribute values of specific attributes;
The relevance calculation means calculates the relevance of the specific attribute to the similarity calculation based on the relationship between the data group classified in the data grouping step and the data group classified in the attribute data classification step. The data similarity calculation method according to claim 10, further comprising: a relevance level calculating step. The data according to any one of claims 9 to 12, wherein the subset generation means generates a subset of the set by randomly sampling data from the set of data in the subset generation step. Similarity calculation method. The similarity calculation means adds a value only to the similarity between the data determined to be a specific class in the class determination step in the similarity calculation step. The data similarity calculation method described in 1. The classifier generation means generates a classifier by weighting data in which a given class is a specific class among data belonging to the subset in the classifier generation step. The data similarity calculation method according to claim 1. The subset generation means generates a subset of the set from a set of data in which at least the feature or sales condition of the customer is an attribute and the behavior of the customer is a class in the subset generation step. The data similarity calculation method according to any one of the above. On the computer,
A subset generation process for generating a subset of the set multiple times by allowing data duplication from a set of data including an attribute indicating data characteristics and a class indicating data classification;
A classifier generation process for generating a classifier that is a rule for determining a class from an attribute each time a subset is generated, based on the subset,
A class determination process for determining a class of individual data belonging to the subset using the classifier each time a classifier is generated; and
A similarity calculation that adds a value to the similarity between data determined to be the same class when a subset of the data set is generated and the class of each data belonging to the subset is determined in the class determination process Data similarity calculation program to execute processing. On the computer,
The data similarity calculation program according to claim 17, wherein a data grouping process for classifying data belonging to a set of data into a plurality of groups based on the similarity between the data is executed. On the computer,
In the data grouping process, the individual data belonging to the data set is classified into different groups, the similarity between the data belonging to two different groups is obtained, and the two groups having the maximum similarity are determined. The data similarity calculation program according to claim 18, wherein the merging is repeated to reduce the total number of groups to a target number. On the computer,
Attribute data classification processing for classifying data belonging to a data set into groups according to attribute values of specific attributes; and
Relevance calculation processing for calculating the relevance of the specific attribute to the similarity calculation based on the relationship between the data group classified by the data grouping processing and the data group classified by the attribute data classification processing. The data similarity calculation program according to claim 18 or 19 to be executed. On the computer,
The data similarity calculation program according to any one of claims 17 to 20, wherein a subset of the set is generated by randomly sampling data from the set of data in the subset generation process. On the computer,
The data similarity according to any one of claims 17 to 21, wherein a value is added only to the similarity between data determined to be a specific class in the class determination processing in the similarity calculation processing. Calculation program. On the computer,
The classifier generation process generates a classifier by weighting data in which a given class is a specific class among the data belonging to the subset. Data similarity calculation program. On the computer,
24. The subset generation processing generates a subset of the set from a set of data having at least customer characteristics or sales conditions as attributes and customer behavior as a class. The data similarity calculation program described in 1.

Images