JP4933741B2 - Information processing apparatus, synonym pair generation method, synonym pair generation program, and recording medium recording synonym pair generation program - Google Patents

Description

  The present invention relates to an information processing apparatus that generates a synonym database, a synonym pair generation method, a synonym pair generation program, and a recording medium on which a synonym pair generation program is recorded.

  In recent years, with the development of Internet technologies such as the World Wide Web (WWW), it has become possible to easily access large amounts of document data, and there are various technologies for searching only interesting document data from a large amount of document data. Document retrieval techniques have been proposed. A typical example of such a search system is google (http://www.google.com), and the search performance of google is very high.

  However, the search system has an essential problem that an appropriate search result cannot be returned unless the query word input by the user is appropriate, no matter how high the system itself is. As a technique for dealing with such problems, there is a related word expansion technique for improving the search accuracy by extracting the related words of the query word input by the user and including the obtained related words in the query word. It is implemented in commercial search systems such as CepceptBase search (Just System Co., Ltd.).

  A so-called thesaurus dictionary such as a related word dictionary is used for this related word expansion technique. A thesaurus is a system that organizes words into meanings and organizes them. In addition to being used for the development of related words, it is used for association support, terminology control, and for the ambiguity of surface expression in machine translation systems. This dictionary is useful for advanced processing of text written in natural language.

  Typical general-purpose thesauruses include "Classified vocabulary table" and "Japanese vocabulary system". However, the construction of a general-purpose thesaurus requires a great deal of expertise and manual assignment of meaning codes to words. New construction is difficult.

  For this reason, in a document processing system using a thesaurus, the thesaurus is treated as an existing one, and there are few proposals regarding the construction method of the thesaurus itself. "Automatic generation of semantic code system" by Sugimura et al. Is an attempt to automatically generate a semantic code system, but it is an automatic generation from examples collected manually. Issues remain. Also, when dealing with highly specialized documents and materials, the general-purpose thesaurus cannot reflect its speciality, and it may be difficult to obtain effective processing results. However, it is difficult to prepare a technical term thesaurus for each specialized field.

  Therefore, as an attempt to automatically create a thesaurus from text without human intervention, a “thesaurus creation device” in Patent Literature 1 and a “word thesaurus construction method and computer for causing a computer system to construct a word thesaurus” in Patent Literature 1. Software program products "have been proposed.

  The above method is basically based on the statistical property of co-occurrence information between word data, although there are differences in grammatical constraints on the extracted word data. Assuming a correlation in semantic relevance is considered problematic. For example, even if the word pair “document” and “search”, which are highly likely to have a dependency relationship, and the word pair “document” and “process” have high co-occurrence, “search” and “process” It is dangerous to assume that there is a semantic connection.

On the other hand, in “Synonym pair extraction apparatus and method and storage medium” in Patent Document 3, attention is paid to a pair of parentheses in a sentence, and a character string (character string 2) in the pair of parentheses is immediately before the opening parenthesis. A synonym pair is extracted as a synonym of a character string (character string 2).
JP-A-9-120401 JP 2001-331515 A JP 11-328205 A

  However, since the character string 1 and the character string 2 having the above relationship do not always have a synonym relationship, whether or not they have a synonym relationship based on the heuristic rules regarding the character string 1 and the character string 2 is determined. Judgment is made. Although this heuristic rule is effective in a limited field, it is extremely versatile. Therefore, in a field where new expressions are always created, such as language, it is always necessary to modify or renew existing rules. It is necessary to generate a simple rule.

  In view of such problems, the present invention provides an information processing apparatus that generates a simple and highly accurate synonym database, a synonym pair generation method, a synonym pair generation program, and a recording medium on which a synonym pair generation program is recorded. The purpose is to provide.

  In order to solve the above-mentioned problem, the present invention provides a predetermined part consisting of a part of speech of a sentence constituting a sentence, dependency analysis means for analyzing the dependency of the phrase, and a first symbol and a second symbol from the sentence. A target sentence detection unit that detects a target sentence that is a sentence including a pair of symbols, and a dependency analysis result by the dependency analysis unit, from a predetermined continuous phrase immediately before the first symbol in the target sentence A character string pair extracting means for extracting a first character string and a second character string consisting of a predetermined continuous phrase in the symbol pair, and both the first character string and the second character string A relationship determination unit that determines whether or not a predetermined phrase having a relationship exists, and when the relationship exists by the relationship determination unit, the first character string and the second character string Character strings are used as synonym pairs.

  In order to solve the above problems, the present invention provides a dependency character string database for storing data in a dictionary format and a dependency analysis result by the dependency analysis means. Detecting a dependency pair, and generating, from the detected dependency pair, a predetermined dependency character string including at least a dependency clause morpheme and a predetermined reception character string including at least a dependency clause morpheme, A dependency character string generation unit that registers a character string as a value and the received character string as a key in the dependency character string database; and the relationship relationship determination unit extracts the character string pair by the character string pair extraction unit Determining whether or not there is a cooperating relationship between the target character string pair depending on whether or not there is one or more records in the dependency character string database in which the value of the character string pair is a value. And butterflies.

  In order to solve the above-mentioned problem, the present invention has a synonym candidate database in which ordered character string pairs are stored, and the character string pair extraction unit uses the extracted character string pairs as the synonym candidates. Stored in the database, the co-relationship presence determination means is subject to whether there is one or more records in the dependency character string database whose value is a character string pair acquired from the synonym candidate database. It is characterized by determining the existence of the relationship between character string pairs.

  In order to solve the above problem, the present invention provides a synonym database in which a synonym set is stored, and a character string pair and a synonym that are determined to have a cooperating relationship by the cooperating relationship existence determining unit. When the synonym set determining means for determining whether or not the synonym set of the database satisfies a predetermined condition, and when the predetermined condition is satisfied by the synonym set determining means, the corresponding synonym set of the synonym database is A synonym set synthesizing unit that replaces the union of the corresponding synonym set and the extracted character string pair, and if the synonym set determining unit does not satisfy the predetermined condition, a character string pair that has a cooperating relationship is newly created. Synonym set adding means for adding to the synonym database as a synonym set.

  In order to solve the above problem, the present invention is that the predetermined condition is true when a product set of a synonym set in a synonym database and a character string pair determined to have a cooperating relationship is not an empty set. It is characterized by.

  In order to solve the above-mentioned problem, the present invention has unnecessary synonym candidate deletion means for deleting information on synonym candidates that satisfy a predetermined time after being added to the synonym candidate database. It is characterized by.

  Further, in order to solve the above-mentioned problem, the present invention provides a character string pair generated by the dependency character string generation means, in which the part of speech of the independent phrase of the dependency phrase is a noun and the part of speech of the independent word of the reception phrase is a verb. The character string that is the value is constructed based on a noun phrase that is a self-supporting word recursively dependent on the dependency phrase, and the key character string is based on the self-supporting word of the receiving phrase. It is characterized by comprising.

  In order to solve the above-mentioned problem, the present invention provides that each character string extracted by the character string pair extraction unit is a morpheme in which the independent word of the phrase has a noun or a part of speech similar to the noun, and any pair of There is a dependency relationship between phrase pairs, and it consists of consecutive phrases.

  In order to solve the above problems, the present invention provides an automatic page collecting means for automatically collecting web pages on the Internet, and a sentence extraction for extracting the sentence from the web pages collected by the automatic page collecting means. Means.

  In order to solve the above-mentioned problem, the present invention includes a part of speech of a phrase constituting a sentence, a dependency analysis stage for analyzing the dependency of the phrase, the sentence, a first symbol, and a second symbol. A target sentence detection stage that detects a target sentence that is a sentence including a predetermined symbol pair and a dependency analysis result obtained by the dependency analysis stage, and a predetermined sequence immediately before the first symbol in the target sentence. A character string pair extraction step of extracting a first character string composed of a phrase and a second character string composed of a predetermined continuous phrase in the symbol pair; and the first character string and the second character string A relationship determination step for determining whether or not a predetermined phrase having both a relationship exists, and when a relationship exists in the relationship determination step, the first character string and the second character string Characterized by making the second character string a synonym pair

  Further, in order to solve the above-mentioned problem, the present invention detects a dependency pair in which a self-supporting word of each clause has a predetermined part of speech from the dependency analysis result in the dependency analysis stage, and detects the detected dependency. From the pair, a predetermined character string including at least a morpheme of a dependency clause and a predetermined character string including at least a morpheme of the reception clause are generated, and the relationship character string is used as a value and the reception character string is used as a key. A dependency character string generation stage to be registered in the reception character string database, and in the co-relationship presence determination step, a record whose value is the character string pair extracted by the character string pair extraction step is the dependency character The existence of a co-relationship between target character string pairs is determined based on whether one or more exist in the column database.

  In order to solve the above-mentioned problem, the present invention stores the extracted character string pair in the synonym candidate database in which the ordered character string pairs are stored in the character string pair extraction stage. The relationship existence determination step determines whether or not the relationship of the target character string pair depends on whether or not there is one or more records whose value is the character string pair acquired from the synonym candidate database in the dependency character string database. It is characterized by determining existence.

In order to solve the above-described problem, the present invention is based on whether the synonym set in the synonym database and the character string pair determined to have a cooperating relationship by the cooperating relationship existence determining step satisfies a predetermined condition. A synonym set determining step for determining whether or not, and when the predetermined condition is satisfied in the synonym set determining step, the corresponding synonym set in the synonym database storing the synonym set is extracted as the corresponding synonym set If the predetermined condition is not satisfied in the synonym set synthesizing step and the synonym set determining step to replace the union set with the character string pair, the character string pair having a cooperating relationship is defined as a new synonym set. And a synonym set addition stage to be added to the synonym database.

  In order to solve the above problem, the present invention is that the predetermined condition is true when a product set of a synonym set in a synonym database and a character string pair determined to have a cooperating relationship is not an empty set. It is characterized by.

  In order to solve the above-mentioned problem, the present invention has an unnecessary synonym candidate deletion step of deleting information on synonym candidates that satisfy a predetermined storage time after being added to the synonym candidate database. It is characterized by.

  In order to solve the above-mentioned problem, the present invention relates to the character string pair generated in the dependency character string generation stage, in which the part of speech of the independent phrase of the dependency clause is a noun, and the part of speech of the independent word of the reception clause is a verb. The character string that is the value is constructed based on a noun phrase that is a self-supporting word recursively dependent on the dependency phrase, and the key character string is based on the self-supporting word of the receiving phrase. It is characterized by comprising.

  Further, in order to solve the above-mentioned problem, in the present invention, each character string extracted in the character string pair extraction stage is a morpheme in which a self-supporting word of a phrase has a noun or a part of speech similar to it, and any pair of words There is a dependency relationship between phrase pairs, and it consists of consecutive phrases.

  To solve the above problems, the present invention provides an automatic page collection stage for automatically collecting web pages on the Internet, and a sentence extraction for extracting the sentence from the web pages collected by the automatic page collection stage. A stage.

  Moreover, in order to solve the said subject, this invention is a synonym pair production | generation program for making a computer perform the synonym pair production | generation method of any one of Claim 10 to 18.

  In order to solve the above problem, the present invention provides a computer-readable recording medium on which the synonym pair generation program according to claim 19 is recorded.

  As described above, according to the present invention, an information processing apparatus that generates a simple and highly accurate synonym database, a synonym pair generation method, a synonym pair generation program, and a recording medium on which a synonym pair generation program is recorded. Can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration example of a computer that realizes an information processing apparatus according to an embodiment of the present invention. In FIG. 1, a keyboard 12, a communication I / O 13, a display 14, a printer 15, a memory 16, a CPU (Central Processing Unit) 17, a hard disk 18, a bus 10 for connecting them, and a network 11 are shown. Is shown.

  The keyboard 12 is input means for registering a document. The communication I / O 13 is a communication unit that receives a signal from the outside or transmits a signal from the apparatus. The CPU 17 executes the program according to the procedure recorded in the memory 16. The memory 16 is a storage unit. There are usually volatile or non-volatile ones. The hard disk is a non-volatile storage means.

  The display 14 and the printer 15 are output means. The communication I / O 13 can be a modem or a terminal adapter, and can receive data from a server connected to an intranet or the Internet via a communication line.

  Next, a module and a database mounted on the information processing apparatus will be described with reference to FIG. The modules include a dependency analysis processing unit 51, a character string pair extraction processing unit 52, a dependency character string generation processing unit 53, a synonym set addition processing unit 54, a target sentence detection processing unit 55, and synonyms. There are a set determination processing unit 56, a synonym set synthesis processing unit 57, a synonym set addition processing unit 58, an unnecessary synonym candidate deletion processing unit 59, and a cooperative relationship existence determination processing unit 60.

  These modules are stored in the hard disk or in a removable disk (not shown), and the information processing apparatus operates by reading them.

  The databases include a synonym candidate database 71, a synonym database 72, and a dependency character string database 73.

  First, an outline of the module will be described. Note that these detailed descriptions will be made when a flowchart described later is described.

  The dependency analysis processing unit 51 analyzes the part of speech of a phrase constituting the sentence and the dependency of the phrase. The dependency analysis system in the present embodiment may be any one that can appropriately output notation and part-of-speech information, and therefore, an existing dependency analysis system may be used.

  The character string pair extraction processing unit 52 uses the dependency analysis result of the dependency analysis processing unit to generate a first character string and a symbol pair consisting of a predetermined continuous phrase immediately before the symbol “(” in the target sentence. A second character string consisting of a predetermined continuous phrase in “()” is extracted.

  The dependency character string generation processing unit 53 detects a dependency pair in which the independent word of each phrase has a predetermined part of speech from the dependency analysis result by the dependency analysis processing unit 51, and from the detected dependency pair, A predetermined character string including at least a morpheme of a dependency phrase and a predetermined character string including at least a morpheme of the reception clause are generated, the dependency character string is a value, and the dependency character string database is a reception character string as a key. Register with.

  The target sentence detection processing unit 55 detects a target sentence that is a sentence including a predetermined symbol pair including the first symbol “(” and the second symbol “)”.

  The synonym set determination processing unit 56 determines whether or not a character string pair determined to have a cooperating relationship and a synonym set in the synonym database 72 satisfy a predetermined condition.

  The synonym set synthesis processing unit 57 replaces the corresponding synonym set in the synonym database with the union of the corresponding synonym set and the extracted character string pair.

  The synonym set addition processing unit 58 adds a character string pair having a cooperating relationship to the synonym database as a new synonym set.

  The unnecessary synonym candidate deletion processing unit 59 deletes synonym candidates that satisfy a predetermined time after being added to the synonym candidate database.

  The co-relationship presence determination processing unit 60 determines whether there is a predetermined phrase in which both the first character string and the second character string are related.

  Next, each database will be described. First, the dependency character string database 73 will be described. First, the dependency character string includes a predetermined character string including at least a dependency word independent word of a dependency pair in which each word independent word (morpheme) has a predetermined part of speech, and at least a dependency word independent word. A character string pair composed of a predetermined character string. In the dependency character string, the receiving character string is a key character string, and the dependency character string is a value character string, which are managed by a dependency character string database having a dictionary structure. Furthermore, the independent word of the phrase that forms the key character string is the key independent word, and the independent word of the phrase that forms the value character string is the value independent word.

  In the following operation example, a hash is used as a database structure for simplicity, but any structure may be used as long as an equivalent function can be realized. In the present invention, since a database does not require special specifications, an existing database may be used.

  FIG. 3 shows an example of the data structure of the dependency character string database when the part of speech of the key independent word is a verb and the part of speech of the value independent word is a noun in the dependency character string. Although not clearly shown in the present embodiment, the part of speech of the key independent word may be a noun and the part of speech of the value independent word may be an adjective, or a combination thereof may be used.

  In FIG. 3, five dependency independent word data in which the first column is a key character string and the second and subsequent columns are value character strings are managed. In this example, in order to absorb the change in the use of parts of speech, all terminal free words constituting the key character string are managed in a terminal form. For example, the dependency character string of identification number 1 is that the key character string is “announce”, and the value character string is “International Atomic Energy Agency”, “IAEA”, “Ministry of Internal Affairs and Communications”, and “IOC”. Show.

  Next, the synonym candidate database 71 will be described. The synonym candidate pairs are character string pairs having an order, and are managed in the synonym candidate database 71. A character string pair having an order is expressed as “A (B)” in the two character strings A and B, but is not determined whether it is expressed as “B (A)”. Say. For example, in the character string pair of Tokyo Tower and Minato Ward, it is expressed as “Tokyo Tower (Minato Ward)”, but the phrase “Minato Ward (Tokyo Tower)” with the phrase replaced is not usually used. That is, the synonym candidate pair is a character string pair that may be a character string pair made up of synonym character strings. In the present embodiment, the synonym candidate database does not require special specifications, and an existing database may be used.

  FIG. 4 shows an example of the data structure of the synonym pair database. In FIG. 4, each row is information on each synonym candidate, the first column is the first character string, the second column is the second character string, and the third column is the date when the synonym candidate is added. An identification number is assigned to identify each row. For example, the synonym candidate pair with the identification number 1 is “Jiro Matsushita Outfielder” and “NY Bears”, which was added on February 29, 2004.

  Moreover, you may update the additional date information of a synonym candidate pair whenever the same synonym candidate pair is observed.

  Next, the synonym database 72 will be described. First, a set of character strings having synonymous expressions is called a synonym set, and the synonym set is managed in the synonym database 72. In the present embodiment, the synonym database 72 does not require special specifications, so an existing database may be used.

  FIG. 5 shows an example of the data structure of the synonym database. In FIG. 5, each row is a synonym set, and each cell is a character string constituting a synonym. An identification number is assigned to identify each row. For example, the synonym set of the identification number 2 is “North Atlantic Treaty Organization”, “NATO”, and “North Atlantic Treaty Organization”, respectively.

  This completes the description of the module and database. Next, the overall processing flow will be described with reference to the flowchart of FIG. After the description of this flowchart, each step will be described in detail.

  In step S101, sentence dependency analysis is performed. In the next step S102, dependency character string generation processing is performed. Next, in step S103, a target sentence detection process is performed. In step S104, character string pair extraction processing is performed.

  In the next step S105, a cooperative relationship existence determination process is performed. If the determination result is false, the process ends. If the determination result is true, a synonym set determination process is performed in step S106. If the determination result here is true, a synonym set synthesis process is performed in step S108, and the process ends. When the determination result is false, a synonym set addition process is performed in step S107, and the process ends.

  Step S101 described above corresponds to the dependency analysis stage. Step S102 corresponds to a dependency character string generation stage. Step S103 corresponds to the target sentence detection stage. Step S104 corresponds to the character string pair extraction stage. Step S105 corresponds to the relationship determination step. Step S106 corresponds to a synonym set determination stage. Step S107 corresponds to a synonym set addition stage. Step S108 corresponds to a synonym set synthesis stage.

  Next, each step will be described in detail. First, the dependency analysis process in step S101 will be described with reference to FIGS.

  FIG. 8 shows the result of executing the dependency analysis process for the three sentences shown in FIG. In FIG. 8, the first column is the phrase identification number, the second column is the related phrase identification number, the third column is the morpheme notation constituting the phrase, the fourth column is the ending morpheme notation, and the fifth column Indicates the part of speech of the morpheme. In this embodiment, the phrase is based on a general generation rule that is composed of one independent word and zero or more attached words.

Further, in the second column of FIG. 8, the dependency clause identification number “−1” indicates that there is no dependency clause, the fourth column describes the ending form only for the morpheme having the part of speech to be used, In the fifth column, nouns, verbs, conjunctions, and adjectives are not specified, but adjectives and adjective verbs are parts of speech that indicate independent words. (Strictly speaking, the part of speech of “explanation”, which is an independent word in the phrase 21 of the sentence C, should be a sub-variable, but in this embodiment, the sub-variable is also a verb for simplicity. The part of speech that is treated as a word or unknown word is also a noun for simplicity.)
In addition, specific symbols are assigned to specific symbols. For example, an open parenthesis symbol, a closing parenthesis symbol, a punctuation symbol, and a punctuation symbol have parts of speech of “open parenthesis”, “closed parenthesis”, “reading mark”, and “punctuation mark”, respectively. Furthermore, in this example, symbols are treated as independent words, but punctuation symbols and punctuation symbols are attached words.

  Next, the dependency character string generation processing in step S102 will be described. In the dependency character string generation process, a dependency pair having a predetermined part-of-speech is detected from the dependency analysis result, and a predetermined character including at least a dependency clause morpheme is detected from the detected dependency pair. A predetermined character string including a string and at least a morpheme of the receiving clause is generated, and each character string is registered in the dependency character string database as a value and a key.

  In this embodiment, from the dependency analysis result shown in FIG. 8, the dependency character is based on a dependency independent word pair in which the part of speech of the dependency phrase is a noun and the part of speech of the dependency phrase is a verb. A column is generated.

  The value character string is generated by combining the character string of the noun phrase with the part of speech of the self-supporting word, which has recursive dependency relation with the dependency phrase for the independent word of the dependency phrase. The key character string is generated as an end form of the self-supporting word of the receiving clause. As shown in FIG. 9, the recursive dependency relationship indicates a chain relationship in which the independent word on the dependency side of the dependency pair becomes the character string on the reception side of another dependency pair.

  The processing of the dependency character string generation processing unit 53 will be described with reference to the flowchart of FIG. In step S201, the dependency character string generation processing unit 53 detects a phrase pair in which the part-of-speech of the independent phrase of the dependency phrase is a noun and the part-of-speech of the independent word of the reception phrase is a verb. In step S202, the dependency character string generation processing unit 53 detects all the clauses in which the part of speech of the independent word recursively depends on the dependency clause, and combines the phrase representations in the order in which the clauses appear, Generate a column. In step S <b> 203, the dependency character string generation processing unit 53 acquires the end form of the self-supporting word of the receiving phrase, and generates a key character string. In step S204, the dependency character string generation processing unit 53 registers the generated key character string and value character string in the dependency character string database as keys and values, respectively.

  For example, in the sentence C of FIG. 7, it can be seen that the phrase pairs satisfying the condition of the dependency character string are the phrase pairs having the identification numbers 18 and 22, and 20 and 21. In addition, since the clauses having the identification numbers 18 and 20 have no relation from the clause whose part of speech of the independent word is a noun, “describe”-“JASS”, “explain”- Get a “problem”.

  Next, since the dependency character string: “describe”-“JASS” key character string: “describe” is the same as the key of the identification number 3 in FIG. 3, the value character string: “JASS” is the identification number 3 Added as record value.

  On the other hand, since the dependency character string: “explain”-“problem” key character string: “explain” is a new key character string, it is registered as a new record in FIG. Similarly, FIG. 11 shows the result of generating dependency character string pairs from sentence A and sentence B and registering them in the dependency pair database of FIG.

  Next, the target sentence detection process in step S103 will be described. In the target sentence detection process, a sentence including a predetermined symbol pair composed of a first symbol and a second symbol is detected. In the present embodiment, the method disclosed in Patent Document 3 may be used. For example, in the case of a pair of parentheses, a regular expression “\ ([^ \)] + \)” or the like is used as a sentence. It can be easily detected by applying.

  Note that \ indicates a backslash, [] indicates a class, and + indicates one or more occurrences. By using this regular expression, for example, in the sentence shown in FIG. 7 or FIG. 8, it can be easily detected that the sentence including the pair of parentheses is the sentence A.

  Next, the character string pair extraction process in step S104 will be described. In the character string pair extraction process, a predetermined character string immediately before and inside the specific symbol pair is extracted based on the dependency result of the sentence detected in the target sentence detection process.

  In the present embodiment, from the dependency analysis results shown in FIG. 8, the specific symbol pair is a parenthesis, the part of speech of the independent phrase in the first and last phrases is a noun, and the independent words in the other phrases are nouns or symbols FIG. 12 is a flowchart illustrating an example of processing to be acquired as a character string generated from successive phrases. In this embodiment, for simplicity, it is assumed that there is no symbol pair nesting.

In step S301, the character string pair extraction processing unit detects the appearance position of the symbol “(” and assigns it to the variable P1. Next, in step S302, the character string pair extraction processing unit first appears from the position P1 + 1. The position of the symbol “)” to be detected is detected and assigned to the variable P2. In the next step S303, the character string pair extraction processing unit extracts a phrase string satisfying the regular expression of the part of speech shown in the following expression for the phrase string in the range of P1 + 1 to P2-1, To do.
(Noun) ((noun | unregistered word) * (noun))?
In this expression, (|) is a selection,? Indicates at most one occurrence.

  In step S304, the character string pair extraction processing unit extracts a phrase string that satisfies the part-of-speech regular expression shown in the expression from the phrase string starting from P1-1, sets the first character string, and ends the process.

  For example, in the sentence A in FIG. 7, first, 5 (see FIG. 8) is obtained as the position of the phrase including the symbol “(”, and 7 is obtained as the position of the phrase including the symbol “)”. Next, when a column satisfying the regular expression of the part of speech of the above formula is detected in the morpheme sequence from position 6 to position 6, a noun is obtained, so the second character string is “JSAA”. Next, when a column satisfying the regular expression of Formula 1 is detected from position 4 to the beginning, it becomes a noun / noun / noun / noun, so the first character string becomes “Japan Sports Arbitration Agency”, and the character string pair: “Japan Sports Extract "Arbitration Agency"-"JSAA".

  The character string pairs extracted in this way are registered in the synonym candidate database. For example, when the extracted character string pair: “Japan Sports Arbitration Agency”-“JSAA” is registered in the synonym candidate database shown in FIG. 5, the synonym candidate database becomes as shown in FIG. The registered time is assumed to be June 7, 2004.

  Next, the cooperative relationship existence determination process in step S105 will be described. In the cooperative relationship existence determination process, it is determined whether or not there is a predetermined phrase in which the first character string and the second character string are both related.

  In this embodiment, it is assumed that all the determination target character strings are registered in the synonym candidate database, and the determination condition is true when both the target character string pairs are included in the value character string of the dependency character string database. Acquires the identification number of the synonym candidate character string pair.

  The process of this co-relationship presence determination processing unit will be described with reference to FIG. In step S401, the cooperative relationship existence determination processing unit creates an identification number list holding identification numbers. In step S402, the cooperative relationship existence determination processing unit extracts one synonym candidate from the synonym candidate database.

  In step S403, the cooperative relationship existence determination processing unit determines whether there is a synonym candidate. If there is no synonym candidate, the cooperative relationship existence determination processing unit returns the identification number list in step S404 and ends the process.

  If there is a synonym candidate, the co-relationship presence determination processing unit searches the dependency character string database for a record that includes both character string pairs of synonym candidates as value character strings in step S405, and exists. In this case, the identification number of the target synonym candidate is added to the identification number list. Then, the process returns to step S402.

  For example, consider obtaining an identification number list of a synonym candidate database satisfying the co-relationship existence existence condition under the dependency character string database shown in FIG. 11 and the synonym candidate database shown in FIG.

  First, the character string pair: “Jiro Matsushita Outfielder”-“NY Bears” is acquired from the synonym candidate database. When a record including the character string pair as a value character string is searched in the dependency character string database, it is found that there is no corresponding one.

  Next, the character string pair: “International Atomic Energy Agency”-“IAEA” is acquired from the synonym candidate database. When a record including the character string pair as a value character string is searched in the dependency character string database, the record with the identification number 1 corresponds, and therefore the identification number 2 of the target character string pair is added to the identification number list.

  Next, the character string pair: “Japan Sports Arbitration Period”-“JASS” is acquired from the synonym candidate database. When a record including the character string pair as a value character string is searched in the dependency character string database, the record with the identification number 3 is applicable, so that the identification number 3 of the target character string pair is added to the identification number list.

  Since there are no more entries in the synonym candidate database, an identification number list having identification numbers 2 and 3 as values is obtained as a result, and it can be seen that the synonym candidates corresponding to this identification number list are synonym pairs.

  Next, the synonym set determination process in step S106 will be described. In the synonym set determination process, it is determined whether or not the generated synonym pair and the synonym set in the synonym database satisfy a predetermined condition.

  Specifically, in the synonym set determination process, the determination condition is “true when the product set of two sets is not an empty set”, and the synonym database is recognized as a synonym pair in the relationship existence determination process. The processing using the identification number of the character string pair in the synonym candidate database is performed.

  The processing of the synonym set determination processing unit will be described with reference to FIG. In step S501, the synonym set determination processing unit determines whether there is a synonym set in the synonym database. If not, -1 is returned in step S505, and the process ends. In the next step S502, the synonym set determination processing unit extracts one synonym set from the synonym database.

  Next, in step S503, the synonym set determination processing unit determines whether the product set of the generated synonym set and the corresponding synonym set is an empty set. If it is not an empty set, the synonym set determination processing unit returns the identification number of the corresponding synonym set in step S504 and ends the process. In the case of an empty set, the synonym set determination processing unit returns to the process of step S501.

  For example, consider a case where the operation example up to the above-described cooperative relationship existence determination process is inherited and a synonym database is given as FIG. First, it can be seen from the previous example that the character string pair of identification numbers 2 and 3 is a synonym pair in the synonym candidate database of FIG. 15. First, the character string pair (set) of identification number 2: (International Atomic Energy Agency) , IAEA). Next, when the product set of this character string set and each record (synonym set) of the synonym database in FIG. 5 is taken, it can be seen that the product set is not an empty set in the case of identification number 1. Number 1 is returned, and the process is terminated.

  Similarly, if a similar determination is made with respect to the character string pair of identification number 2 in the synonym candidate database: (Japan Sports Arbitration Agency, JASS), in this case, it is found that all the product sets are empty sets. 1 is returned and the process ends.

  Next, the synonym set synthesizing process in step S108 will be described. In the synonym set synthesis process, when the generated synonym string pair satisfies the determination condition in the synonym set determination process, the synonym set and the corresponding synonym set in the synonym database are combined, and the corresponding synonym set Update.

  Here, a processing example of synthesizing a synonym character string pair (set) satisfying a determination condition and a corresponding synonym set in the synonym database will be described with reference to a flowchart of FIG. In step S601, the synonym set synthesis processing unit generates a difference set between the generated synonym set and the corresponding synonym set in the synonym database. In the next step S602, the synonym set synthesis processing unit adds all elements of the generated difference set to the synonym set.

  For example, if the operation example up to the above-described synonym set determination process is inherited, the synonym set determination process is true in the synonym character string set: (International Atomic Energy Agency, IAEA), and the corresponding synonym set is Since the entry of ID number 1 in the synonym database is taken, the sum of the two character string sets becomes (IAEA, International Atomic Energy Agency), and the entry of ID number 1 in the synonym database is shown in FIG. become that way.

  Next, the synonym set addition process in step S107 will be described. In the synonym set addition process, when the generated synonym character string pair does not satisfy the determination condition in the synonym set determination process, the synonym set is added as a new synonym set in the synonym database.

  Here, a processing example is shown in which a synonym character string pair that does not satisfy the determination condition is added to the synonym database and the corresponding synonym database is updated.

  For example, if the operation example up to the above-described synonym set determination process is inherited, the synonym character set: (Japan Sports Arbitration Agency, JASS) is true in the synonym set determination process. What is registered in the synonym database as a synonym set is the record with the identification number 3 in FIG.

  The record of the synonym candidate database processed by the synonym set synthesis process and the synonym set addition process, that is, the synonym character string pair may be deleted because the same information is registered in the synonym database. .

  In this manner, a high-accuracy synonym database can be easily generated by repeatedly executing the above-described processing on a plurality of documents. In the above processing example, there are cases where the identification numbers are discontinuous in both the synonym database and the synonym candidate database, but if the identification numbers are consistent in units of one extracted character string pair Therefore, the identification number may be corrected to a continuous value after the processing for any extracted character string pair is completed.

  Next, unnecessary synonym candidate deletion processing will be described. In the unnecessary synonym candidate deletion process, synonym candidates registered in the synonym candidate database that have been added for a certain period of time are regarded as not constituting synonyms and are deleted from the synonym candidate database.

  This process does not need to be synchronized with the process described above, and may be executed independently periodically.

  Here, a processing example in which synonym candidates exceeding the specified maximum elapsed days are deleted from the synonym candidate database shown in FIG.

  A flowchart showing this unnecessary synonym candidate deletion processing is shown in FIG. In step S701, the unnecessary synonym candidate deletion processing unit acquires the maximum number of elapsed days and the current date. In step S702, the unnecessary synonym candidate deletion process determines whether there is a synonym candidate database. If not, the process ends. In step S703, the unnecessary synonym candidate deletion processing unit obtains one synonym candidate from the synonym candidate database, and calculates the number of elapsed days from the addition date of the corresponding synonym candidate and the current date.

  Next, the unnecessary synonym candidate deletion processing unit determines whether or not the number of elapsed days exceeds the maximum number of elapsed days in step S704. If exceeded, the unnecessary synonym candidate deletion processing unit deletes the entry of the corresponding synonym candidate from the synonym candidate database in step S705. If not, the unnecessary synonym candidate deletion processing unit returns to the process of step S701.

  For example, if the current date and time is July 4, 2004 and the maximum number of elapsed days is 120 days, the synonym candidate database with identification numbers 1 and 2 in FIG. 15 has an additional date of February 29, 2004. Therefore, the number of elapsed days is 126 days, and since it exceeds 120 days, it is deleted. Similarly, the synonym candidate with the identification number 3 is not deleted because the number of elapsed days is within 120 days.

  Next, automatic page collection processing and sentence extraction processing will be described. In automatic page collection processing and sentence extraction processing, text data on the Internet is automatically collected, and sentences including specific symbol pairs are extracted from the collected documents. Then, the plurality of extracted sentences are repeatedly applied to a series of processes from the above-described character string pair extraction process. In addition, by executing unnecessary synonym deletion processing periodically, a synonym database with high accuracy and high freshness can be easily generated.

  The text collection process may be performed using an existing automatic document collection agent such as a so-called search robot, and since the sentence extraction including the specific symbol pair has been described above, an example of the operation is not specified here.

  According to the present embodiment, it is possible to automatically generate a synonym dictionary, which has been conventionally performed manually.

  Furthermore, since a synonym set is generated using more detailed grammatical rules as compared with a conventionally proposed automatic synonym dictionary generation method, a more accurate synonym dictionary can be automatically generated.

  In addition, because it targets Internet resources that are updated every day, a synonym dictionary with high freshness can be constructed.

It is a figure which shows the structural example of a computer. It is a figure which shows a module and a database. It is a figure which shows the example of a data structure of a dependency character string database (the 1). It is a figure which shows the example of a data structure of a synonym candidate database (the 1). It is a figure which shows the example of a data structure of a synonym database (the 1). It is a flowchart which shows the flow of the whole process. It is a figure which shows an example sentence. It is a figure which shows a dependency analysis result. It is a figure which shows recursive dependency relationship. It is a flowchart which shows the process of a dependency character string production | generation process part. It is a figure which shows the example of a data structure of a dependency character string database (the 2). It is a flowchart which shows a character string pair extraction process. It is a figure which shows the example of a data structure of a synonym database (the 2). It is a flowchart which shows a cooperative relationship presence determination process. It is a figure which shows the example of a data structure of a synonym candidate database (the 2). It is a flowchart which shows a synonym set determination process. It is a flowchart which shows a synonym set synthetic | combination process. It is a flowchart which shows an unnecessary synonym candidate deletion process.

Explanation of symbols

10 bus 11 network 12 keyboard 13 communication I / O
14 Display 15 Printer 16 Memory 17 CPU
18 Hard Disk 51 Dependency Analysis Processing Unit 52 Character String Pair Extraction Processing Unit 53 Dependent Character String Generation Processing Unit 55 Target Sentence Detection Processing Unit 56 Synonym Set Determination Processing Unit 57 Synonym Set Synthesis Processing Unit 58 Synonym Set Addition Processing Unit 59 Unnecessary synonym candidate deletion processing unit 60 Co-relationship existence determination processing unit 71 Synonym candidate database 72 Synonym database 73 Dependent character string database

Claims (14)

Part-of-speech of a clause constituting a sentence; dependency analysis means for analyzing the dependency of the clause;
From the sentence, a target sentence detection means for detecting a target sentence that is a sentence including a symbol pair consisting of a first symbol that is an open parenthesis and a second symbol that is a close parenthesis;
A first character string consisting of a noun immediately before the first symbol and a noun consecutive before the noun in the target sentence, using the part of speech of the analyzed clause among the dependency analysis results by the dependency analysis means A character string pair extracting means for extracting a noun in the symbol pair and a second character string consisting of a noun continuous before or after the noun,
A dependency string database for storing dictionary-format data;
From the dependency analysis result by the dependency analysis means, a dependency pair having a predetermined part of speech as an independent word of each clause is detected, and from the detected dependency pair, a dependency character string including at least a dependency clause morpheme and Generating a received character string including at least a morpheme of the received clause, the modified character string as a value, and a received character string generating unit that registers the received character string as a key in the received character string database;
A co-relationship presence determination means for determining whether or not there is an end form of a phrase in which the first character string and the second character string are both related;
The co-relationship presence determination means has one key in the dependency character string database in which the character string pair of the first character string and the second character string extracted by the character string pair extraction means is a value. The existence of the co-relationship between the target character string pair is determined based on whether or not it exists,
An information processing apparatus comprising: a first character string and a second character string as synonym pairs when a relationship is present by the relationship relationship determination unit. A synonym candidate database in which character string pairs are stored;
The character string pair extraction means stores the extracted character string pair in the synonym candidate database,
The co-relationship presence determination means determines whether the target character string pair is shared depending on whether or not there is one or more records whose values are character string pairs acquired from the synonym candidate database in the dependency character string database. The information processing apparatus according to claim 1 , wherein presence of a relationship is determined. A synonym database that stores one or more synonym sets that collect synonymous words using words as elements;
A synonym set for determining whether a synonym set stored in the synonym database and a character string pair determined to have a cooperating relationship satisfies the predetermined condition by the cooperating relationship existence determining unit A determination means;
When the predetermined condition is satisfied by the synonym set determination unit, a synonym set synthesizing unit that replaces the corresponding synonym set in the synonym database with a union of the corresponding synonym set and the extracted character string pair;
If the predetermined condition is not satisfied by the synonym set determination means, the synonym set addition means for adding a character string pair having a cooperating relationship as a new synonym set to the synonym database,
The predetermined condition is true when the intersection of the character string pair determined to have a cooperating relationship and the synonym set of the synonym database is not an empty set;
The information processing apparatus according to claim 2 . The unnecessary synonym candidate deletion means for deleting information on synonym candidates whose storage time has elapsed after being specified in the synonym candidate database, according to claim 2 or 3 . Information processing device. The character string pair generated by the dependency character string generation means is such that the part-of-speech of the independent phrase of the dependency clause is a noun, the part-of-speech of the independent word of the dependency clause is a verb, and the character string serving as the value corresponds to the dependency phrase claim 1 independent word with recursive dependency relationship is constituted by combining the string of noun phrase, and wherein the character string to be the keys are independent word termination shaped receiving clauses 5. The information processing apparatus according to any one of items 1 to 4 . Automatic page collection means for automatically collecting web pages on the Internet;
The information processing apparatus according to any one of claims 1 to 5, characterized in that the web pages collected by the automatic page collection means and a sentence extraction means for extracting the sentence. Information processing device
A part of speech of a clause constituting the sentence, a dependency analysis stage for analyzing the dependency of the clause,
A target sentence detection step of detecting a target sentence that is a sentence including a symbol pair consisting of a first symbol that is an open parenthesis and a second symbol that is a close parenthesis from the sentence;
A first character string consisting of a noun immediately before the first symbol and a noun consecutive before the noun in the target sentence, using the part of speech of the analyzed clause among the dependency analysis results by the dependency analysis means A string pair extraction step of extracting a noun in the symbol pair and a second string of nouns consecutive before or after the noun;
From the dependency analysis result of the dependency analysis stage, a dependency pair having a predetermined part of speech as an independent word of each clause is detected, and from the detected dependency pair, a dependency character string including at least a dependency clause morpheme and Generating a received character string including at least a morpheme of the received clause, setting the modified character string as a value, and registering the received character string in a modified character string database using the received character string as a key; and
A co-relationship presence determination step for determining whether or not there is an end form of a phrase in which both the first character string and the second character string are related.
In the co-relationship presence determination step, one key in the dependency character string database is a key whose value is the character string pair of the first character string and the second character string extracted in the character string pair extraction step. The existence of the co-relationship between the target character string pair is determined based on whether or not it exists,
A synonym pair generation method characterized in that, when a relationship exists at the step of determining the presence of a relationship, the first character string and the second character string are used as a synonym pair. In the character string pair extraction step, the extracted character string pair is stored in a synonym candidate database in which the character string pair is stored,
The co-relationship existence determination step determines whether the target character string pair is shared depending on whether or not there is one or more records whose values are character string pairs acquired from the synonym candidate database in the dependency character string database. The synonym pair generation method according to claim 7 , wherein presence of a relationship is determined. Information processing device
Any one of the synonym databases in which one or more synonym sets in which synonyms are collected by using a character string pair and a word determined as having a cooperating relationship by the cooperating relationship existence determining step are stored. A synonym set determination stage for determining whether or not a synonym set of
When the predetermined condition is satisfied in the synonym set determination step, a synonym set synthesis step of replacing the corresponding synonym set in the synonym database with a union of the corresponding synonym set and the extracted character string pair;
If the predetermined condition is not satisfied in the synonym set determination step, a synonym set addition step of adding a character string pair having a cooperating relationship to the synonym database as a new synonym set,
The predetermined condition is true when the intersection of the character string pair determined to have a cooperating relationship and the synonym set of the synonym database is not an empty set;
The synonym pair production | generation method of Claim 8 characterized by these. Information processing device
The unnecessary synonym candidate deletion step of deleting information on synonym candidates that have passed a specified time after being added to the synonym candidate database, according to claim 8 or 9 . Synonym pair generation method. The character string pair generated in the dependency character string generation step is such that the part-of-speech of the independent phrase of the dependency clause is a noun, the part-of-speech of the independent word of the dependency clause is a verb, and the character string serving as the value corresponds to the dependency phrase claim 7 independent word with recursive dependency relationship is constituted by combining the string of noun phrase, and wherein the character string to be the keys are independent word termination shaped receiving clauses synonyms pairing method according to any one of 10. Information processing device
An automatic page collection stage that automatically collects web pages on the Internet,
The synonym pair generation method according to claim 7 , further comprising: a sentence extraction step of extracting the sentence from the web page collected by the automatic page collection step. Synonyms pair generation program for executing the synonym pair generation method according to the computer in any one of claims 7 to 12. A computer-readable recording medium on which the synonym pair generation program according to claim 13 is recorded.

Images