JP6066089B2 - Data relationship determination system, data relationship determination method, and program - Google Patents

Description

  The present invention relates to a system for determining relevance between data in information processing, and more particularly to a system suitable for determining relevance between specifications in system and software development.

In the relevance determination between data, each data to be compared is converted into a character string, and the similarity between the character strings is calculated, so that it is possible to estimate that there is a relevance between data having a high similarity. For example, as described in Non-Patent Document 1, the similarity between data is calculated for data such as text, images, and time-series data, and the similarity between the data is estimated as the relevance between the data. It can be processed.
In system and software development, there is a case where the relevance is estimated using the similarity between data. For example, in Non-Patent Document 2, the relevance between business flows is calculated using the similarity between data.
However, in these techniques, when the data belongs to different concepts but the character strings are similar, there is a case where the relevance is high.
Here, the concept is not an accidental property belonging only to individual data, but the same essential characteristic of a plurality of data. For example, when the data is a specification (text) in system development, the concept corresponds to a system component or business classification.
Specifically, the two specifications to be compared for the similarity calculation process are “the order management system corresponds to telephone, FAX (facsimile), EDI (electronic data interchange)” and “order management system In the case of "corresponding to telephone, FAX, EDI", the two specifications belong to "order management system" and "order management system" which are different components (concepts). For this reason, there is no direct relevance, and there is no problem in system and software development even if one of these two specifications is changed to “Not compatible with telephones”.
However, since the original two specifications are identical except for “order” and “ordering”, using the conventional technology, the original two specifications tend to be calculated as highly related. For this reason, when one of them is changed to “not compatible with telephones”, the system for determining the relationship between data erroneously detects an inconsistency such as a contradiction in the development specification.
In order to deal with such a problem, a method for automatically configuring a concept and its weight is described in Non-Patent Document 3, for example. In this method, a large number of documents such as newspaper articles are collected, and the concept of the document is configured by a multidimensional vector. The reason for using a multidimensional vector is that the concept generally does not have a simple hierarchical structure, and there is an overlapping portion between the concepts.
Patent Document 1 describes an example of a method for obtaining a concept by referring to a concept database and calculating a similarity between a plurality of data using the concept. Patent Document 1 describes that related similar data is extracted using the concept of data input for search, and information search is performed using these data.

JP 2006-106970 A

Ming, “The Similarity Metrics,” published in 2004, IEEE Transactions on Information Theory, Vol. 50, No. 12, pages 3250-3264 Keigo et al., “Compression-based data mining of sequential data”, published in 2007, Journal Data Mining and Knowledge Discovery Vol. 129 pages Proceedings of supercomputing, pp. 787-796, published by H. Schutze, “Dimensions of Meaning”, 1992

However, for example, in many cases, the relationship between the specifications of system and software development cannot be analyzed using many documents.
This is because many projects use terminology specific to a project as a conceptual component or business flow. For this reason, only documents within a project (including similar projects and parallel projects as the case may be) are to be compared. This makes it difficult to create an accurate and large amount of concept database.
As described above, in system and software development, etc., analysis cannot be performed using many documents, so it is difficult to automatically construct a concept using the existing method as described above. On the other hand, a document used for system development often has a limited terminology compared to a general document, and has a hierarchical structure from a plurality of viewpoints.
On the other hand, a document used in system development is easier if it is incomplete concept information in which there are two concept elements or a lack of concept elements compared to general documents. Can be configured.
The present invention provides a relevance determination system between data that accurately determines relevance between data based on incomplete concept information in which all information is not registered.

The relevance determination system between data according to the present invention is based on a concept set having a plurality of concepts as elements with respect to a concept having one or more generated character strings as elements as pre-processing of similarity calculation. If the case where the data to be compared contains the string belonging to the same concept both, other data which one has a character string containing Mikatsu the comparison belonging to the concept of the data to be the contrasts that do not contain the string belonging to the concept If both of the data to be compared do not include a character string belonging to the concept, a candidate selection unit that selects the data to be compared as a candidate for similarity calculation, and the candidate selection unit On the other hand, the similarity is calculated for the selection candidate selected in step 4, while the similarity is set to a predetermined small value for the candidate not selected by the candidate selection unit, and the similarity of the data to be compared Characterized in that the calculated and a similarity calculation unit to output.
In the method for determining the relationship between data by the information processing apparatus according to the present invention, the data to be compared is based on a concept set having a plurality of concepts as elements with respect to the generated concept having one or more character strings as elements. and If both contain the string belonging to the same concept, and when one of the data to be the contrast is free of strings belonging to the other concept of data strings containing Mikatsu the contrasting belonging to the concept, the comparison If both of the data to be included do not include a character string belonging to the concept, if it was either , a candidate selection step corresponding to pre-processing of similarity calculation that selects the data to be compared as a candidate for similarity calculation, and While calculating the similarity for the selection candidate selected in the candidate selection step, the similarity is set to a predetermined small value for the candidate not selected by the candidate selection unit. And having a similarity calculation step of enabling outputs the similarity of the data to be the comparison processing to.
The program for determining relevance between data according to the present invention is based on a concept set having a plurality of concepts as elements with respect to the concept having one or more character strings generated as elements in the control unit of the information processing device. If the case where the data to be compared contains the string belonging to the same concept both, other data which one has a character string containing Mikatsu the comparison belonging to the concept of the data to be the contrasts that do not contain the string belonging to the concept If both of the data to be compared do not include a character string belonging to the concept , the candidate selection corresponding to the pre-processing of the similarity calculation that selects the data to be compared as a candidate for similarity calculation Processing and calculating the similarity for the candidate selected in the candidate selection process, while setting the similarity to a predetermined small value for a candidate not selected by the candidate selection unit Characterized in that to execute the similarity calculating process of calculating processes the similarity of the data to be the comparison.

  ADVANTAGE OF THE INVENTION According to this invention, the relationship determination system between the data which determines the relationship between data correctly based on the incomplete concept information in which all the information is not registered can be provided.

FIG. 1 is a block diagram illustrating a configuration example of a first embodiment of a system for determining relevance between data.
FIG. 2 is an explanatory diagram illustrating an example of concept information stored in the concept storage unit 100.
FIG. 3 is an explanatory diagram illustrating an example of data to be determined stored in the data storage unit 101.
FIG. 4 is an explanatory diagram illustrating an example of candidates stored in the candidate storage unit 102.
FIG. 5 is a flowchart illustrating an example of processing progress of the first embodiment of the relationship determination system between data.
FIG. 6 is a flowchart showing an example of processing progress of the candidate selection unit 103 that checks whether data p and q are candidates for similarity calculation in the concept set i of the first embodiment of the relationship determination system between data. .
FIG. 7 is a block diagram illustrating a configuration example of the second embodiment of the relationship determination system between data.
FIG. 8 is an explanatory diagram illustrating an example of a glossary stored in the glossary storage unit 200.
FIG. 9 is an explanatory diagram illustrating another example of a glossary stored in the glossary storage unit 200.
FIG. 10 is a flowchart illustrating an example of a process progress of the conceptual configuration unit 201 of the second embodiment of the relationship determination system between data.
FIG. 11 is a block diagram illustrating a configuration example of the third embodiment of the relationship determination system between data.
FIG. 12 is an explanatory diagram illustrating an example of the structure data stored in the structure data storage unit 300.
FIG. 13 is a flowchart illustrating an example of processing progress of the conceptual configuration unit 301 of the third embodiment of the relationship determination system between data.
FIG. 14 is an explanatory diagram illustrating an example of a concept configured by the conceptual configuration unit 301.
FIG. 15 is a flowchart illustrating an example of processing progress of the data generation unit 302 of the third embodiment of the relationship determination system between data.
FIG. 16 is an explanatory diagram illustrating an example of data generated by the data generation unit 302.
FIG. 17 is a block diagram showing an example of realization of the present invention.
FIG. 18 is a block diagram showing an example of another embodiment of the present invention.

(First embodiment)
Next, a first embodiment of the data relationship determination system according to the present invention will be described in detail with reference to the drawings. The relationship determination system between data which concerns on 1st Embodiment calculates the similarity which shows the relationship between data based on the concept and data which were stored beforehand. Here, a case where a natural language is used as a character string or data character string that is a concept element will be described as an example. However, the character string to be used is any character string indicating an image, time-series data, or the like. It does not matter.
Referring to FIG. 1, the system for determining relevance between data according to the present embodiment includes a storage unit 11 that stores information and a calculation unit 12 that operates under program control.
The storage unit 11 includes a concept storage unit 100, a data storage unit 101, and a candidate storage unit 102.
The concept storage unit 100 stores one or more character strings indicating concepts for a plurality of concepts. The concept may be specified by the user of the system by inputting via a keyboard or the like, may be stored as a default in the system, or may be specified in other manners.
FIG. 2 is an explanatory diagram illustrating an example of a phrase indicating a concept stored in the concept storage unit 100. In the figure, one row represents one concept set, one concept for each “,”, and within one concept, character strings separated by “/” represent variations of character strings belonging to the same concept. Show.
It is desirable that any two concepts out of a plurality of concepts included in one concept set should be disjoint (that is, the elements of one concept and the elements of the other concept do not match). It is not limited to being disjoint. Moreover, when the arbitrary two concepts are different concept sets, they may not be disjoint.
For example, in the figure, the character string “order receiving management system” and the character string “order receiving system” are elements of the same concept. In addition, the character string “order management system” and the character string “order system” indicate elements of a concept different from the concepts indicated by the “order management system” and the “order system”. Further, “telephone” and “FAX” are elements of different concepts. It should be noted that “order management system” and “telephone” do not indicate that they belong to the same concept or different concepts.
By using the concept set having the plurality of concepts as elements, the candidate selection unit 103 selects whether or not the two pieces of data to be compared should be selected as candidates for similarity calculation.
The data storage unit 101 stores a plurality of data for determining relevance. Individual data is composed of character strings, which may be specified by the user of the system via a keyboard, stored as default in the system, or specified in other ways. May be. FIG. 3 is an explanatory diagram illustrating an example of data stored in the data storage unit 101. In the figure, the first row shows the contents of each column, the first column shows the data ID number, and the second column shows the data content. That is, in the drawing, the first data is “the order management system corresponds to telephone, FAX, EDI”, and the second data is “the order management system corresponds to telephone, FAX, EDI”. The third data is “the order management system supports telephone, FAX, EDI”, and the fourth data is “the order receiving system supports telephone, FAX, EDI”. Is shown. Here, the individual data is Japanese text, but it may be any character string indicating an image or time-series data. Further, hereinafter, each data is referred to as first data (ID = 1) as data 1 and second data (ID = 2) as data 2.
The candidate storage unit 102 stores an ID number of certain data and the ID number of the data to be subjected to similarity calculation as a candidate in association with each other. FIG. 4 is an explanatory diagram illustrating an example of candidates stored in the candidate storage unit 102. Here, the similarity between the data 1 and the data 2 is regarded as the same as the similarity between the data 2 and the data 1, and only a data having a larger number than that data is selected as a candidate for a certain data. If so. Each row indicates that the subsequent numbers are candidates for similarity calculation for the first number delimited by “,”. That is, the figure shows that data 1 calculates similarity with data 3 and data 4 but does not calculate similarity with data 2. Data 2 indicates that the similarity is calculated with data 3 but is not calculated with data 4. Furthermore, data 3 indicates that the similarity with data 4 is not calculated.
The calculation unit 12 includes a candidate selection unit 103 and a similarity calculation unit 104.
The candidate selection unit 103 selects a candidate for calculating the similarity based on the concept information stored in the concept storage unit 100 and the data to be determined stored in the data storage unit 101, and selects the selected candidate. It memorize | stores in the candidate memory | storage part 102. FIG.
The similarity calculation unit 104 calculates the similarity between the data based on the candidates stored in the candidate storage unit 102 and the data to be determined stored in the data storage unit 101. At this time, the similarity calculation unit 104 sets the similarity to a predetermined small value for candidates not selected by the candidate selection unit 103 (that is, elements not stored in the candidate storage unit 102).
Next, the overall operation of the first embodiment will be described in detail with reference to the flowchart of FIG.
First, the candidate selection unit 103 initializes the value of a variable p indicating a data number (ID number illustrated in FIG. 3) to 1 (step A1).
Next, the candidate selection unit 103 compares p with the number of data M indicating the total number of data (step A2). If p is M or less, the process proceeds to the next step. If p is greater than M, the process proceeds to step A12 performed by the similarity calculation unit 104.
Next, the candidate selection unit 103 initializes the value of the variable q indicating the number of data to be determined as a candidate together with p to p + 1 (step A3).
Next, the candidate selection unit 103 compares q with the number of data M (step A4). If q is less than or equal to M, the process proceeds to the next step. If it is larger than M, the process proceeds to step A11.
Next, the candidate selection unit 103 initializes the value of the variable i indicating the concept set to 1 (step A5). Hereinafter, the i-th concept set is referred to as concept set i.
Next, the candidate selection unit 103 compares i with the number of concept sets I indicating the total number of concept sets (step A6). If i is equal to or less than I, the process proceeds to the next step. If i is larger than I, it is determined that the data p and data q specified by the variable p and variable q are all concept sets and the data p and data q are candidates, and the process proceeds to step A9.
Next, the candidate selection unit 103 performs processing to check whether both data p and data q can be candidates for calculating similarity when the concept set i is used as a reference (step A7). Details of the processing will be described later. If it is not a candidate, the process proceeds to step A10 (No in step A7). If it is a candidate, the process proceeds to the next step (Yes in step A7).
Next, the candidate selection unit 103 increments i to use the next concept set as a reference (step A8). And it transfers to step A6.
If i is greater than I in step A6, the candidate selection unit 103 stores in the candidate storage unit 102 that p and q are similarity calculation candidates (step A9).
Next, the candidate selection part 103 increments q (step A10). Then, the process proceeds to step A4.
If q is larger than the number of data M in step A4, the candidate selection unit 103 increments p (step A11). Then, the process proceeds to step A2.
If p is larger than M in step A2, the similarity calculation unit 104 calculates the similarity between the data among the candidates stored in the candidate storage unit 102 (step A12). Here, the similarity between non-candidate data is set to zero. Then, the operation ends. Similarity between data can be calculated, for example, using an estimate of Kolmogorov complexity. The calculated similarity may be output immediately via a display device or a printing device, or may be stored and output in response to a request from a user of the system. May be output.
Next, the operation of the candidate selection unit 103 that checks whether the data p and q are candidates for similarity calculation in the concept set i of the first embodiment will be described in detail with reference to the flowchart of FIG.
First, the candidate selecting unit 103 sets a candidate flag indicating whether data p and data q are candidates for similarity calculation to true (step A13).
Next, the candidate selection unit 103 sets a variable n1 indicating whether the data p includes one or more character strings indicating a concept to false, and indicates whether the data q includes one or more character strings indicating a concept. n2 is set to false (step A14).
Next, the candidate selection unit 103 initializes a variable j indicating the concept number in the concept set i stored in the concept storage unit 100 to 1 (step A15). Hereinafter, the jth concept is referred to as concept j.
Next, the candidate selection unit 103 compares the value of the concept j with the concept number J indicating the total number of concepts included in the concept set i (step A16). If j is less than or equal to J, the process proceeds to the next step. If j is larger than J, the process proceeds to step A26.
Next, the candidate selection unit 103 sets m1 [j] indicating whether the data p includes the concept j to false, and sets m2 [j] indicating whether the data q includes the concept j to false1 (step 1). A17).
Next, the candidate selection unit 103 initializes a variable k indicating the number of the character string indicating the concept j of the concept set i to 1 (step A18). Hereinafter, the k-th character string is referred to as a character string k.
Next, the candidate selection unit 103 compares k with the number K of character strings indicating the total number of character strings included in the concept j of the concept set i (step A19). If k is K or less, the process proceeds to the next step. If k is larger than K, the process proceeds to step A25.
Next, the candidate selection unit 103 checks whether the data p includes the character string k of the concept j of the concept set i (step A20). If so, move on to the next step. If not included, the process proceeds to step A22.
Next, the candidate selection unit 103 sets n1 to true and sets m1 [j] to true (step A21). For example, let us consider a case where the conceptual information recorded in the conceptual storage unit 100 is FIG. 2, the data to be determined recorded in the data storage unit 101 is FIG. 3, and p is 1. Under this condition, when i = 1, j = 1, and k = 1, the “order receiving management system” that is the character string 1 of the concept 1 of the concept set 1 is changed to the data 1 “order receiving management system is telephone, FAX, EDI. "Corresponding" is included as a character string. Therefore, the process proceeds to this step, where n1 is set to true and m1 [1] is set to true. On the other hand, when i = 1, j = 1, and k = 2, the “order receiving system” that is the character string 2 of the concept 1 of the concept set 1 does not include the data 1, and therefore, the process does not proceed to this step. Similarly, both the “ordering management system” when i = 1, j = 2, and k = 1 and the “ordering system” when i = 1, j = 2, and k = 2 are both data 1 Because it is not included in this step, it does not move to this step. Therefore, in the contents illustrated in FIGS. 2 and 3, for i = 1, n1 = true, m1 [1] = true, and m1 [2] = false.
Next, the candidate selection unit 103 checks whether the data q includes the character string k of the concept j of the concept set i (step A22). If so, move on to the next step. If not included, the process proceeds to step A24.
Next, the candidate selection unit 103 sets n2 to true and sets m2 [j] to true (step A23). For example, consider the case where the conceptual information recorded in the concept storage unit 100 is FIG. 2, the data to be determined recorded in the data storage unit 101 is FIG. 3, and q is 2. Under this condition, when i = 1, j = 1, and k = 1, the “order management system” that is the character string 1 of the concept 1 of the concept set 1 is changed to the data 2 “order management system is telephone, FAX, EDI. Is not included as a character string. For this reason, it does not shift to this step. Similarly, even if i = 1, j = 1, and k = 2, the “order receiving system” is not included in the data 2, and thus the process does not proceed to this step. On the other hand, when i = 1, j = 2, and k = 1, the “ordering management system” is included in the data 2, and therefore, the process proceeds to this step and n2 = true and m2 [2] = true are set. Therefore, in the contents illustrated in FIGS. 2 and 3, for q = 2 and i = 1, n2 = true, m2 [1] = false, and m2 [2] = true are set. In addition, when q is 3, when i = 1, the data 3 does not include the corresponding character string for any j and k, so the process does not proceed to this step. Therefore, n2 = false, m2 [1] = false, and m2 [2] = false. When q is 4, the process proceeds to this step only when i = 1, j = 1, and k = 2. Therefore, n2 = true, m2 [1] = true, and m2 [2] = false.
Next, the candidate selection unit 103 increments k (step A24). Then, the process proceeds to step A19.
When k becomes larger than K in step A19, the candidate selection unit 103 increments j (step A25). Then, the process proceeds to step A16.
When j becomes larger than J in step A16, the candidate selection unit 103 checks whether both n1 and n2 are true (step A26). If both are true, the process proceeds to the next step. If one or both are false, the candidate flag remains true, ie, the data p and data q are candidates in the concept set i, and the operation ends. For example, when the concept information recorded in the concept storage unit 100 is FIG. 2 and the data to be determined recorded in the data storage unit 101 is FIG. 3 and p is 1, n1 = true. When q = 2, since n2 = true, the process proceeds to the next step. When q = 3, n2 = false, so that data 1 and data 3 are candidates and the operation is terminated. When q = 3, since n2 = true, the process proceeds to the next step. Thus, since data 1 and data 3 are one data including the character string belonging to the concept in the concept set 1, the operation is terminated as a candidate.
Next, the candidate selection unit 103 temporarily sets the candidate flag to false (step A27).
Next, the candidate selection unit 103 initializes a variable j indicating the concept number of the concept set i to 1 (step A28).
Next, the candidate selection unit 103 compares j with the concept number J of the concept set i (step A29). If j is less than or equal to J, the process proceeds to the next step. If j is greater than J, the operation is terminated assuming that the candidate flag remains false, that is, data p and data q are not candidates in concept set i. For example, when the concept information recorded in the concept storage unit 100 is FIG. 2 and the data to be determined recorded in the data storage unit 101 is FIG. 3 and p is 1, when q = 2, Since m2 [1] = false when m1 [1] = true and m2 [2] = true when m1 [1] = false, both are not true in the next step, and the process proceeds to this step. The operation is terminated assuming that data 1 and data 2 are not candidates in concept set 1. Thus, since data 1 and data 2 do not include character strings of the same concept and there are two data including character strings belonging to the concept, the operation is terminated as not being a candidate.
Next, the candidate selection unit 103 checks whether both m1 [j] and m2 [j] are true (step A30). If both are true, the process proceeds to step A32. If either one is false, the process proceeds to the next step.
Next, the candidate selection unit 103 increments j (step A31). Then, the process proceeds to step A29.
If both m1 [j] and m2 [j] are true in step A30, the candidate selection unit 103 sets the candidate flag to true (step A32). Then, the operation is finished assuming that the candidate flag remains true, that is, the data p and the data q are candidates in the concept set i. For example, when the concept information recorded in the concept storage unit 100 is FIG. 2 and the data to be determined recorded in the data storage unit 101 is FIG. 3 and p is 1, when q = 4, Since m1 [1] = true and m2 [1] = true, the process proceeds to this step, and the operation is terminated assuming that data 1 and data 4 are candidates in the concept set 1. As described above, the data 1 and the data 4 include character strings of the same concept in the concept set 1, so that the operation is terminated as being candidates.
As described above, according to the present embodiment, the relationship between data can be accurately determined even based on an incomplete concept set in which two concept elements overlap or there is a lack of concept elements. Can be determined.
(Second Embodiment)
Next, a second embodiment of the data relevance determination system according to the present invention will be described in detail with reference to the drawings. The relevance determination system between data according to the second embodiment constructs a concept from a glossary stored in advance, and based on the data to be compared with the configured concept, a similarity indicating the relevance between the data is obtained. calculate. Here, a case where a character string or data indicating a concept is a natural language will be described as an example.
In system and software development, in order to eliminate ambiguity, a glossary is often created by organizing terms used in a project. In the present embodiment, the concept is configured using the glossary arranged in this way, and then the similarity indicating the relationship between the data is calculated as in the first embodiment. In addition, about the component similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.
Referring to FIG. 7, the system for determining relevance between data according to the present embodiment includes a storage unit 21 that stores information and a calculation unit 22 that operates under program control.
The storage unit 21 includes a glossary storage unit 200, a concept storage unit 100, a data storage unit 101, and a candidate storage unit 102.
The glossary storage unit 200 stores a glossary used in system and software development. The glossary is a collection of terms that are character strings, and preferably includes character strings that are related phrases of the terms. Here, the related phrases are synonyms, synonyms, related words, and the like. FIG. 8 is an explanatory diagram illustrating an example of a glossary stored in the glossary storage unit 200. In the figure, the first row shows the description of the contents of each column, the first column shows the term, the second column shows the meaning of the term, and the third column shows the related word of the term. For example, the figure shows that the meaning of the term “order received” is “receiving an order” and that there is a term “special order” used in the case of a special order related to “order received”. The figure also shows that the term “ordering” means “putting an order” and there is a term “special issue” used in a special case instead of “ordering”. FIG. 9 is an explanatory diagram illustrating another example of a glossary stored in the glossary storage unit 200. In FIG. 9, the first row shows the contents of each column, the first column shows the component name, and the second column shows the abbreviation of the component. For example, in FIG. 9, there are two components, “order management system” and “order management system”. The abbreviation of “order management system” is “order system”, and the abbreviation of “order management system” is “order system”. It shows that there is.
The other components of the storage unit 21, which are the concept storage unit 100, the data storage unit 101, and the candidate storage unit 102, are the same as those in the first embodiment.
The calculation unit 22 includes a conceptual configuration unit 201, a candidate selection unit 103, and a similarity calculation unit 104.
Based on the glossary stored in the glossary storage unit 200, the conceptual configuration unit 201 configures a character string indicating the concept and stores it in the concept storage unit 100.
The candidate selection unit 103 and the similarity calculation unit 104 are the same as those in the first embodiment.
Next, the operation of the conceptual configuration unit 201 according to the second embodiment will be described in detail with reference to the flowchart in FIG.
First, the conceptual configuration unit 201 extracts a term at a designated place from the glossary (step A33). The designated location may be designated by a user of the system through a keyboard or the like, may be stored in the system as a default, or may be designated in other manners. For example, in FIG. 8, what is necessary is just to designate the 1st column except the 1st line as a place with a term. Also in FIG. 9, the first column except the first row may be designated as the place where the term exists.
Next, the concept constructing unit 201 extracts a related phrase at a designated location from the glossary (step A34). The designated location may be designated by a user of the system through a keyboard or the like, may be stored in the system as a default, or may be designated in other manners. For example, in FIG. 8, the third column excluding the first row may be designated as the location where the related phrase is located. In FIG. 9, the second column excluding the first row may be designated as a place where there is a related phrase.
Next, the concept constructing unit 201 collects the extracted terms and related phrases, and constructs a character string indicating the concept (step A35). For example, in order to configure the format shown in FIG. 2, the concept may be configured by separating character strings of extracted terms and related terms by separating them with “/”.
Next, the concept configuration unit 201 configures a plurality of individual configured concepts as a concept set (step A36). For example, in order to configure the format shown in FIG. 2, the concept set may be configured by separating the configured concepts with “,”. The configured concept set is stored in the concept storage unit 100. Then, the operation of the conceptual configuration unit 201 ends. For example, in FIG. 9, the configured concept set is the first line of the concept information in FIG.
Since the calculation processing of the similarity indicating the relevance between the data after the concept constructing unit 201 registers the concept set is the same as in the first embodiment, the description thereof is omitted.
As described above, according to the present embodiment, it is possible to automatically construct a concept from a glossary and calculate the similarity indicating the relationship between data using the concept. In this example, the case where terms and related phrases are extracted from a glossary has been described as an example. However, for example, when there is an explanation of terms in the data, the data is regarded as a glossary and terms are extracted. Also good.
(Third embodiment)
Next, a third embodiment of the system for determining relevance between data according to the present invention will be described in detail with reference to the drawings. In addition, about the component similar to 1st and 2nd embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.
Referring to FIG. 11, the data relevance determination system according to the present embodiment includes a storage unit 31 that stores information and a calculation unit 32 that operates under program control.
The storage unit 31 includes a structural data storage unit 300, a concept storage unit 100, a data storage unit 101, and a candidate storage unit 102.
The structure data storage unit 300 stores structure data having a hierarchical structure and given hierarchical item names and contents. The structure data may be designated by the user of the system through a keyboard or the like, may be stored in the system as a default, or may be designated in other manners. FIG. 12 is an explanatory diagram illustrating an example of the structure data stored in the structure data storage unit 300. In the figure, the first row shows the contents of each column, the first column shows the item names in the large classification, the second column shows the item names in the small classification, and the third column shows the contents. It should be noted that chapter / section information is automatically extracted as item names from general documents with chapters / sections, and structure data with major category item names as chapter titles and minor category item names as section titles. May be stored in the structure data storage unit 300.
The other components of the storage unit 31, which are the concept storage unit 100, the data storage unit 101, and the candidate storage unit 102, are the same as described above.
The calculation unit 32 includes a conceptual configuration unit 301, a data generation unit 302, a candidate selection unit 103, and a similarity calculation unit 104.
The concept configuration unit 301 configures a concept based on the structure data stored in the structure data storage unit 300 and stores the concept in the concept storage unit 100.
The data generation unit 302 generates data based on the structure data stored in the structure data storage unit 300 and stores the data in the data storage unit 101.
The candidate selection unit 103 and the similarity calculation unit 104 are the same as described above.
Next, the operation of the conceptual configuration unit 301 according to the third embodiment will be described in detail with reference to the flowchart of FIG.
First, the conceptual configuration unit 301 extracts a character string serving as an item name from the structure data stored in the structure data storage unit 300 (step A37). For example, in FIG. 12, the character strings “functional specifications” and “screen specifications” of the large classification and the character strings “order management system”, “order management system”, “setting screen”, and “display screen” of the small classification are extracted. .
Next, the concept constructing unit 301 constructs a concept from the extracted item names (step A38). For example, in FIG. 12, one concept set is configured using a large classification character string, and another concept set is configured using a small classification character string. FIG. 14 is an explanatory diagram illustrating an example of a concept configured by the conceptual configuration unit 301.
Next, the concept configuration unit 301 stores the configured concept in the concept storage unit 100 (step A39). And the process of the conceptual structure part 301 is complete | finished.
Next, the operation of the data generation unit 302 of the third embodiment will be described in detail with reference to the flowchart of FIG.
First, the data generation unit 302 extracts a character string serving as an item name from the structure data stored in the structure data storage unit 300 (step A40). This step is the same as the operation of extracting the item name of the conceptual component 301.
Next, the data generation unit 302 extracts a character string indicating the content from the structure data stored in the structure data storage unit 300 (step A41).
Next, the data generation unit 302 creates data by arranging item names and contents (step A42). FIG. 16 is an explanatory diagram illustrating an example of data generated by the data generation unit 302. The figure shows an example of data generated by the data generation unit 302 when the structural data storage unit 300 is shown in FIG. Here, the data is generated by arranging the item name and contents by separating them with “.”.
Next, the data generation unit 302 stores the generated data in the data storage unit 101 (step A43).
Here, the structure data stored in the structure data storage unit and the data stored in the data storage unit have a one-to-one correspondence. Therefore, when the structural data is input, the first and second data are generated using the concept configured by the concept configuration unit 301 and stored in the concept storage unit 100 and the data generated by the data generation unit 302 and stored in the data storage unit 101. The relationship between the structural data can be determined by the same processing as that of the second embodiment.
As described above, according to the present embodiment, it is possible to automatically construct a concept using the structure information of structured data and calculate the similarity indicating the relationship between the data.
In addition, what is necessary is just to implement | achieve each part of the relationship determination system between data using the combination of hardware and software. In the form of a combination of hardware and software, a program for determining the relationship between data is expanded in the RAM, and each unit is realized as various means by operating hardware such as a control unit (CPU) based on the program. . The program for determining the relevance between data may be configured such that each unit is constructed by causing an operating system, other general software, or the like to execute each process.
Further, this program may be fixedly recorded on a storage medium and distributed. The program recorded on the recording medium is read into a memory via a wired, wireless, or recording medium itself, and operates a control unit or the like. Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
Further, as shown in FIG. 17 and FIG. 18, the system for determining the relationship between data may be constructed as a single computer or a server-client system.
To explain the above embodiment in another expression, an information processing device that operates as a data relevance determination system is based on a data relevance determination program developed in a RAM, a candidate selection unit, a similarity calculation It can be realized by operating a control unit as a unit. In addition, it can be realized by operating a control unit as a conceptual configuration unit and a data generation unit.
As described above, according to the system for determining relevance between data according to the present invention, it is possible to accurately determine the relevance between data based on incomplete concept information in which all information is not registered.
In addition, the specific configuration of the present invention is not limited to the above-described embodiment, and changes within a range not departing from the gist of the present invention are included in the present invention. Further, a desired effect can be obtained by appropriately combining a plurality of components. For example, some components of all the components shown in the embodiment may be integrated or deleted.
In addition, a part or all of the above-described embodiments can be described as follows. Note that the following supplementary notes do not limit the present invention.
[Appendix 1]
The data to be compared is the same based on one or a plurality of concept sets having a plurality of elements each of which is a concept including one or more character strings in which characteristics of the data to be compared are indicated. A candidate selection unit for selecting as a candidate for similarity calculation when the character string of the concept is included or when there is one of the data including the character string belonging to the concept;
The similarity is calculated for the candidates selected by the candidate selection unit, while the similarity is set to a predetermined small value for the candidates not selected by the candidate selection unit, and the comparison data A similarity calculator that outputs the similarity of
A system for determining relevance between data, comprising:
[Appendix 2]
The above-mentioned supplementary note, wherein the candidate selection unit selects a candidate for similarity calculation using an incomplete concept set due to an overlap of two concept elements or a lack of concept elements as the concept set. A system for determining relevance between listed data.
[Appendix 3]
The candidate selection unit has a plurality of concept sets, and in all of the concept sets, the two data to be compared include a character string of the same concept, or data including a character string belonging to the concept In this case, the relevance determination system between data described in the above supplementary notes is selected as a candidate for similarity calculation.
[Appendix 4]
The similarity calculation unit according to the above supplementary note, wherein the similarity calculation unit calculates the similarity between the data to be compared using an approximation of Kolmogorov complexity.
[Appendix 5]
The system for determining relevance between data as described in the above supplementary note, comprising a conceptual component that constitutes the conceptual set.
[Appendix 6]
The conceptual component is
Based on a glossary describing terms that are strings and their related phrases, each concept is composed of terms and their related phrases as elements,
Construct one concept set with each individual concept as an element
A system for determining relevance between data described in the above supplementary notes.
[Appendix 7]
The conceptual component is
Based on the structure data given multiple item names and contents, each item name is used as an element to construct each concept,
Construct a concept set with each configured concept as an element
A system for determining relevance between data described in the above supplementary notes.
[Appendix 8]
A data generation unit that generates, as data, a character string in which item names and contents are concatenated based on structure data to which a plurality of item names and contents are given. Sex determination system.
[Appendix 9]
The data to be compared is the same based on one or a plurality of concept sets having a plurality of elements each of which is a concept including one or more character strings in which characteristics of the data to be compared are indicated. A candidate selection step of selecting as a candidate for similarity calculation when the data includes a character string of a concept or data including a character string belonging to a concept;
The similarity is calculated for the candidates selected in the candidate selection step, while the similarity is set to a predetermined small value for the candidates not selected by the candidate selection unit, and the comparison data A degree-of-similarity calculation step for enabling the output of the degree of similarity.
[Appendix 10]
The above-mentioned candidate selection step, wherein the candidate selection step selects a candidate for similarity calculation by using, as the concept set, an incomplete concept set due to an overlap of two concept elements or a lack of concept elements. A method for determining relevance between described data.
[Appendix 11]
The candidate selection step uses a plurality of concept sets, and in all the concept sets, if the two data to be compared contain character strings of the same concept, or data containing a character string belonging to the concept, In some cases, the relevance determination method between data described in the above supplementary notes is selected as a candidate for similarity calculation.
[Appendix 12]
In the similarity calculation step, the similarity between data to be compared is calculated using an estimate of Kolmogorov complexity.
[Appendix 13]
The method of determining relevance between the data described in the above supplementary notes, comprising a concept configuration step for configuring the concept set.
[Appendix 14]
The conceptual configuration step includes:
Based on a glossary describing terms that are strings and their related phrases, each concept is composed of terms and their related phrases as elements,
Construct one concept set with each individual concept as an element
A method for determining the relationship between data described in the above supplementary notes.
[Appendix 15]
The conceptual configuration step includes:
Based on the structure data given multiple item names and contents, each item name is used as an element to construct each concept,
Construct a concept set with each configured concept as an element
A method for determining the relationship between data described in the above supplementary notes.
[Appendix 16]
A data generation step of generating, as data, a character string in which item names and contents are concatenated based on structure data to which a plurality of item names and contents are given. Sex determination method.
[Appendix 17]
In the control unit of the information processing device,
The data to be compared is the same based on one or a plurality of concept sets having a plurality of elements each of which is a concept including one or more character strings in which characteristics of the data to be compared are indicated. A candidate selection process for selecting as a candidate for similarity calculation when a character string of a concept is included, or when data including a character string belonging to a concept is one side;
The similarity is calculated for the candidates selected in the candidate selection process, while the similarity is set to a predetermined small value for the candidates not selected by the candidate selection unit, and the comparison data Similarity calculation processing to calculate the similarity of
A program for determining relevance between data, characterized in that
[Appendix 18]
In the candidate selection process, as the concept set, a candidate for similarity calculation is selected by using an incomplete concept set due to an overlap of two concept elements or a lack of concept elements as the concept set. Relevance judgment program between listed data.
[Appendix 19]
In the candidate selection process, a plurality of concept sets are used, and in all of the concept sets, whether the two data to be compared include character strings of the same concept or data including a character string belonging to a concept In this case, the relevance determination program between data described in the above supplementary notes is selected as a candidate for similarity calculation.
[Appendix 20]
In the similarity calculation process, the degree of similarity between data to be compared is calculated using an estimate of Kolmogorov complexity.
[Appendix 21]
The program for determining the relevance between the data described in the above supplementary notes, characterized in that the concept composition processing that constitutes the concept set is performed.
[Appendix 22]
In the conceptual configuration process,
Based on a glossary describing terms that are strings and their related phrases, each concept is composed of terms and their related phrases as elements,
Construct a concept set with each individual concept as an element
A program for determining relevance between data described in the above supplementary notes.
[Appendix 23]
In the conceptual configuration process,
Based on the structure data given multiple item names and contents, each item name is used as an element to construct each concept,
Have the process of constructing a concept set with each constructed concept as an element
A program for determining relevance between data described in the above supplementary notes.
[Appendix 24]
Based on the structure data to which a plurality of item names and contents are given, a data generation process for generating a character string in which the item names and contents are concatenated as data is performed. Relevance judgment program.
[Appendix 25]
The recording medium which recorded the relationship determination program between the data of the said additional remarks description.
The present invention can be used in many systems in which the similarity between data is digitized, for example, a system for confirming specifications, a system for confirming procedure manuals, and an increased number of keywords for extracting information from a database. It is also possible to improve the accuracy by setting incomplete conceptual information in the system.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-172924 for which it applied on August 8, 2011, and takes in those the indications of all here.

11, 21, 31 Storage unit 12, 22, 32 Calculation unit 100 Concept storage unit (concept storage means)
101 Data storage unit (data storage means)
102 Candidate storage unit (candidate storage means)
103 Candidate selection unit (candidate selection means)
104 similarity calculation unit (similarity calculation means)
200 Glossary storage unit (glossary storage means)
201 Conceptual configuration unit (conceptual configuration means)
300 Structure data storage unit (structure data storage means)
301 Concept component (concept component)
302 Data generation unit (data generation means)

Claims (10)

As a pre-processing of similarity calculation, for a concept that has one or more generated character strings as elements , based on a concept set having a plurality of concepts as elements, both of the data to be compared belong to the same concept. If a, and when one of the data to be the contrast is free of strings other belonging to the concept of data strings to free Mikatsu the comparison belonging to the concept, the characters both of the data to be the comparison belong to the concept including If without the columns, if was either a candidate selection unit that selects the data to be the comparison as a candidate of the similarity calculation,
While calculating the similarity for the selection candidate selected by the candidate selection unit, the similarity is set to a predetermined small value for the candidate not selected by the candidate selection unit, and the comparison is performed. A system for determining relevance between data, comprising: a similarity calculation unit that calculates and outputs the similarity of data. The candidate selection unit selects a candidate for similarity calculation using an incomplete concept set due to an overlap of two concept elements or a lack of concept elements as the concept set. Item 4. The system for determining the relationship between data according to Item 1. The candidate selecting unit includes a plurality of concepts set, in all of its concepts set, two data to be compared are identical concept of string including cases or the, or data including the character string belonging to the concept is one If it is, it selects as a candidate of similarity calculation, The relationship determination system between the data of Claim 1 or Claim 2 characterized by the above-mentioned.   The relevance between data according to any one of claims 1 to 3, wherein the similarity calculation unit calculates the similarity between the data to be compared by using an estimate of Kolmogorov complexity. Judgment system.   The system according to claim 1, further comprising a concept configuration unit that configures the concept set. The conceptual component is
Based on a glossary describing terms that are strings and their related phrases, each concept is composed of terms and their related phrases as elements,
6. The system for determining relevance between data according to claim 5, wherein one concept set is configured by using each configured concept as an element. The conceptual component is
Based on the structure data given multiple item names and contents, each item name is used as an element to construct each concept,
7. The system for determining relevance between data according to claim 5 or 6, wherein a concept set is configured by using each configured concept as an element.   8. A data generation unit for generating, as data, a character string obtained by concatenating item names and contents based on structure data to which a plurality of item names and contents are given. The relationship determination system between the data of any one of Claims 1. And if the relative concept and that generated 1 or more strings elements, based on the concept set to a plurality of concepts and elements, including a character string belonging to the same concept data to be compared are both data the comparison If one other data containing Mikatsu the contrasts strings belonging to the concept of the free and if they do not contain the string belonging to the concept, the character string belonging to both the concept of the data to be the contrast, any if was either a candidate selecting step corresponding to a previous treatment of the similarity calculation to select the data to the comparison as a candidate of the similarity calculation,
While calculating the similarity for the selection candidate selected in the candidate selection step, the similarity is set to a predetermined small value for the candidate not selected by the candidate selection unit, and the comparison is performed. A method for determining the relationship between data by an information processing apparatus, comprising: a similarity calculation step for calculating and outputting data similarity. In the control unit of the information processing device,
And if the relative concept and that generated 1 or more strings elements, based on the concept set to a plurality of concepts and elements, including a character string belonging to the same concept data to be compared are both data the comparison If one other data containing Mikatsu the contrasts strings belonging to the concept of the free and if they do not contain the string belonging to the concept, the character string belonging to both the concept of the data to be the contrast, any if it was either the candidate selection process corresponding to a previous treatment of the similarity calculation to select the data to the comparison as a candidate of the similarity calculation,
The similarity is calculated for the candidates selected in the candidate selection process, while the similarity is set to a predetermined small value for the candidates not selected by the candidate selection unit, and the comparison data A program for determining relevance between data, characterized in that a similarity calculation process for calculating a similarity is executed.

Images