JP6499537B2 - Connection expression structure analysis apparatus, method, and program - Google Patents

Description

  The present invention relates to a connection expression term structure analysis apparatus, method, and program, and more particularly, to a connection expression term structure analysis apparatus, method, and program for extracting a term connected by a connection expression from a given document.

  Conventionally, there is a technique for extracting a connection span, a semantic class thereof, and a text span (part of text in the document) which is two terms connected by the connection representation from a given document. This technique is called a connected expression-term structure analysis technique. For example, in the following sentence, the term 1: “He caught a cold” and the term 2: “he got soaked in the rain” are connected by a connection expression: “because” in a meaning: causal relationship.

  He caught a cold because he got soaked in the rain.

  By collecting a large number of such semantically linked sets of text spans for each semantic class and using it as a knowledge source, various tasks of natural language processing (entailment recognition, document summarization, machine translation, etc.) The quality can be improved.

  The relationship between connection expressions and terms is broadly divided into explicit cases and implicit cases. An explicit case is when the connection representation itself appears. For example, two spans are connected by “because” representing a causal relationship, “after” representing a transition of time, and the like. On the other hand, even if the connection expression does not appear, there exists a sentence pair that semantically indicates a causal relationship or a transition of time. For example, a causal relationship is established between the following two sentences.

It has been raining since morning.
The baseball game was also canceled.

  Conventional connection expression-term structure analysis technology (Non-patent Document 1) extracts connection expressions, semantic classes, and terms from a document with the configuration of the connection expression term structure analysis apparatus shown in FIG.

  In the conventional connection expression term structure analysis apparatus, the explicit connection expression-term structure is extracted by the following procedures (1) to (4).

(1) All connection expressions stored in the connection expression candidate dictionary are extracted from the input document, and it is determined whether or not these expressions are connection expressions having terms (connection expression extraction unit). (2) When it is determined that the connection expression has a term, its semantic class is determined using a classifier. (3) Further, it is determined whether two terms (term 1 and term 2) of the connection expression appear in the same sentence (SS) or whether term 1 appears in the preceding sentence (PS). (4) Depending on each case, two terms corresponding to the connection expression are extracted using the intra-sentence term extraction unit and the inter-sentence term extraction unit.

  The extraction of the implicit connection expression-term structure was performed in the following procedures (1) and (2).

  (1) A pair of adjacent sentences in the document (extracting item 1 from the previous sentence and item 2 from the subsequent sentence) is extracted, and a semantic class is assigned using a semantic class classifying unit. It is assumed that a plurality of classifications are defined in advance for the semantic class. Non-Patent Document 1 uses four semantic classes: Expansion, Contingency, Temporal, and Comparison. The semantic class classification unit outputs some semantic class when there is a connection relationship between two sentences, and outputs that there is no connection relationship when there is no connection relationship. That is, the semantic class classifying unit classifies the relationship between two sentences into classes having the number of semantic classes + 1. (2) Using the inter-sentence term extraction unit, terms are extracted from two sentences to which semantic classes are assigned. As described above, the connection expression term structure analysis apparatus extracts an explicit connection expression-term structure and an implicit connection expression-term structure from a document.

Lin Ziheng, Ng Hwee Tou, and Kan Min-Yen. 2014.A PDTB-styled End-to-End Discourse Parser.Natural Language Engineering, 20: 151-184.

  The conventional connection representation-term structure extraction method extracts a term without considering the discourse structure of a document or sentence. In the case of implicit connection expression-term extraction, only two adjacent sentences are subject to term extraction. Even if explicit connection representation-term extraction is used, if terms 1 and 2 do not appear simultaneously in a statement in which connection representation appears, term 1 is extracted from the statement immediately preceding the statement in which the connection representation appears. To do. However, since the connection relationship over one sentence is not limited to two adjacent sentences, a connection expression-term structure that should be originally extracted is missed.

  The present invention has been made to solve the above-described problems, and a connected expression term structure analysis apparatus, method, and program capable of extracting terms connected by connection expressions even between non-adjacent sentences. The purpose is to provide.

  In order to achieve the above object, the connected expression term structure analyzing apparatus according to the first aspect of the present invention relates to each of the sentences based on the rhetorical structure of each of the sentences included in the document based on the input document. A discourse structure analysis unit that generates a discourse structure tree represented by a node; a syntax analysis unit that generates a syntax tree by performing syntax analysis for each sentence included in the document; and the syntax analysis unit that generates the syntax tree. A connection expression extraction unit that extracts a connection expression having a term based on a syntax tree for each sentence, and the connection expression extracted by the connection expression extraction unit includes the connection expression in a sentence including the connection expression. A term positional relationship determination unit that determines whether or not two terms connected by an expression appear, and two terms connected by the connection expression in a sentence including the connection expression by the term positional relationship determination unit But If it is determined to be present, a sentence in-sentence extraction unit that extracts two terms connected by the connection expression from the sentence including the connection expression, and a sentence including the connection expression by the term positional relationship determination unit When it is determined that the two terms connected by the connection expression do not appear, either one of the two terms connected by the connection expression is extracted from the sentence including the connection expression, and the discourse structure analysis is performed. In the discourse structure tree generated by the section, an inter-sentence term extraction that extracts one of the two terms connected by the connection expression from a sentence corresponding to a parent node or a sibling node of the sentence including the connection expression And a semantic class classifying unit that classifies the semantic classes of the connection representation based on the connection representation extracted by the connection representation extraction unit.

  Further, the connection expression term structure analyzing apparatus according to the second invention is based on an input document and based on a rhetorical structure of each sentence included in the document, and a discourse structure in which each of the sentences is represented by each node. Based on the discourse structure analysis unit for generating a tree and the discourse structure tree generated by the discourse structure analysis unit, a sentence pair corresponding to a parent-child node and a sentence pair corresponding to a sibling node are connected to each other. A related sentence pair extraction unit that determines whether or not there is a connection relationship for each of the sentence pair candidates having the connection relationship, and a connection relationship by the related sentence pair extraction unit. An inter-sentence term extraction unit that extracts, from each of the sentence pair candidates having the connection relationship, two terms connected by an implicit connection expression for each of the sentence pair candidates having the connection relationship determined as , The related sentence For each of the sentence pair candidates having the connection relationship determined by the extraction unit to be connected, the semantic class of the implicit connection expression is determined based on the sentence pair candidates having the connection relationship. And a semantic class classification unit for classification.

  In the connected expression term structure analysis method according to the third invention, the discourse structure analysis unit, based on the input document, based on the rhetorical structure of each sentence included in the document, each of the sentences at each node. Generating a represented discourse structure tree, a syntax analysis unit generating a syntax tree by performing syntax analysis for each sentence included in the document, and a connection expression extracting unit by the syntax analysis unit A step of extracting a connection expression having a term based on a syntax tree for each of the generated sentences; and the connection expression for the connection expression extracted by the connection expression extraction unit by a term positional relationship determination unit. Determining whether or not two terms connected by the connection expression appear in the sentence including , When it is determined that two terms connected by the connection expression appear, a step of extracting two terms connected by the connection expression from a sentence including the connection expression, and an inter-sentence term extraction unit, When it is determined by the term positional relationship determination unit that two terms connected by the connection expression do not appear in the sentence including the connection expression, 2 connected by the connection expression from the sentence including the connection expression. One of the two terms is extracted, and in the discourse structure tree generated by the discourse structure analysis unit, the sentence is connected by the connection expression from the sentence corresponding to the parent node or sibling node of the sentence including the connection expression. A step of extracting one of the two terms, and a semantic class classification unit, based on the connection representation extracted by the connection representation extraction unit, determines a semantic class of the connection representation. And executes includes a step similar, the.

  In the connected expression term structure analyzing method according to the fourth invention, the discourse structure analyzing unit, based on the input document, based on the rhetorical structure of each sentence included in the document, each of the sentences at each node. The step of generating the expressed discourse structure tree and the related sentence pair extraction unit correspond to the sentence pair corresponding to the parent-child node and the sibling node based on the discourse structure tree generated by the discourse structure analysis unit. The sentence pair is a sentence pair candidate having a connection relation, and for each of the sentence pair candidates having the connection relation, determining whether or not there is a connection relation; Each of the sentence pair candidates having the connection relation determined to have a connection relation by the related sentence pair extraction unit is connected from the sentence pair candidates having the connection relation by an implicit connection expression 2. Two terms And a sentence pair candidate having the connection relationship for each of the sentence pair candidates having the connection relationship determined by the related sentence pair extraction unit by the semantic class classification unit. And classifying the semantic class of the implicit connection representation based on:

  A program according to a fifth aspect of the invention is a program for causing each section to constitute the connected expression term structure analyzing apparatus according to the first or second aspect of the invention.

  According to the connection expression term structure analysis apparatus, method, and program of the present invention, it is possible to extract the terms connected by the connection expression even between non-adjacent sentences.

It is a block diagram which shows the structure of the connection expression term structure analysis apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the structure tree used as extraction object in the case of subordinate connection in sentence internal term extraction. It is a figure which shows an example of the structure tree used as extraction object in the case of equipotential connection in sentence internal term extraction. It is a figure which shows an example of the structure tree used as extraction object in the case of equipotential connection in sentence internal term extraction. It is a flowchart which shows the connection expression term structure analysis processing routine in the connection expression term structure analysis apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the connection expression term structure analysis apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the connection expression term structure analysis processing routine in the connection expression term structure analysis apparatus which concerns on the 2nd Embodiment of this invention. It is an example of the block diagram which shows the structure of the conventional connection expression term structure analysis apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Configuration of connection expression term structure analysis apparatus according to first embodiment of the present invention>

  First, the configuration of the connection expression term structure analysis apparatus according to the first embodiment of the present invention will be described. The connection expression term structure analysis apparatus according to the first embodiment extracts connection expressions, terms, and semantic labels relating to explicit connection expressions from a document.

  As shown in FIG. 1, the connection expression term structure analysis apparatus 100 according to the first embodiment of the present invention includes a CPU, a RAM, a program for executing a connection expression term structure analysis processing routine to be described later, and various programs. It can be constituted by a computer including a ROM storing data. Functionally, the connection expression term structure analysis apparatus 100 includes an input unit 10, an arithmetic unit 20, and an output unit 50 as shown in FIG.

  The input unit 10 receives a document to be analyzed.

  The computing unit 20 includes a sentence dividing unit 30, a discourse structure analyzing unit 32, a syntax analyzing unit 34, a connection expression extracting unit 36, a term positional relationship determining unit 38, an intra-sentence term extracting unit 40, and an inter-sentence term. An extraction unit 42 and a semantic class classification unit 44 are included.

  The sentence division unit 30 acquires the document received by the input unit 10 and gives a sentence break to the document. Sentence delimiters are identified using existing sentence dividers. Alternatively, it may be just a clue. Note that a document that has been divided into sentences in advance may be received by the input unit 10 and the processing of the sentence dividing unit 30 may be omitted.

  The discourse structure analysis unit 32 generates a discourse structure tree in which each sentence is represented by each node based on the rhetorical structure of each sentence included in the document, based on the document given the sentence break by the sentence division unit 30. To do. The discourse structure tree represents a parent-child relationship between nodes of sentences. The discourse structure tree generates an RST tree using a rhetorical structure analyzer such as Non-Patent Document 2, and then determines the parent-child relationship between nodes of sentences by applying the rules described in Non-Patent Document 3. can do. Moreover, it is not always necessary to generate an RST tree, and a parent-child relationship between nodes of sentences is obtained by using an analyzer that has been learned using data of a rhetorical structure tree that represents parent-child relationships between nodes of sentences. It is also possible.

[Non-Patent Document 2]: duVerle, D. and Prendinger, H. 'A Novel Discourse Parser Based on Support Vector Machine Classi_cation'. Proc of the 47th ACL, pp. 665 {675 (2009).

 [Non-Patent Document 3]: Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, Norihito Yasuda and Masaaki Nagata. 'Single-Document Summarization as a Tree Knapsack Problem'. Proc. Of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1515 {1520, (2013).

  The syntax analysis unit 34 performs syntax analysis on each of the sentences included in the document to which the sentence delimiter 30 has been given a sentence break to generate a syntax tree. Since various software has been developed for parsing, it is only necessary to generate a syntax tree for each sentence using existing software.

  The connection expression extraction unit 36 extracts a connection expression having a term based on the syntax tree for each sentence generated by the syntax analysis unit 34. Specifically, the connection expression extraction unit 36 first refers to a dictionary entry expression of a connection expression candidate dictionary (not shown) prepared manually in advance for a word that appears in a document, and selects a word that matches the dictionary entry expression. Extract. Then, binary classification such as SVM and logistic regression trained using learning data in which whether or not a word matching the dictionary entry expression is a connected expression that takes a term is annotated. A connection expression that takes a term is extracted. The following features (1) to (5) may be used as features used to determine whether a word appearing in a document is a connected expression that takes a term.

(1) Dictionary entry expression and its part of speech (2) Five words before and after dictionary entry expression and their part of speech (3) Depth of dictionary entry expression in syntax tree (4) Parent of dictionary entry expression in syntax tree, left brother , Right sibling (5) Path from dictionary entry representation to root in syntax tree

  The term positional relationship determination unit 38 determines whether or not two terms connected by the connection expression appear in the sentence including the connection expression for the connection expression extracted by the connection expression extraction unit 36. Specifically, the term positional relationship determination unit 38 uses a binary classifier such as SVM or logistic regression previously trained using learning data in the same manner as the connection representation extraction unit 36, and uses the binary classifier such as SVM and logistic regression in the sentence including the connection representation. Then, it is determined whether or not two terms connected by the connection expression appear. In addition to the features (1) to (5) used for the connection expression extraction unit 36, the characteristics used for the determination use the appearance position of the connection expression (the first half of the sentence, the middle, the second half, etc.).

  When it is determined that the two terms connected by the connection expression do not appear in the sentence including the connection expression, the term positional relationship determination unit 38 selects the sentence including the connection expression from the two terms connected by the connection expression. In the discourse structure tree generated by the discourse structure analysis unit 32, the sentence corresponding to the parent node or sibling node of the sentence including the connection expression is connected by the connection expression. Of the two terms, term 1 is determined as a sentence for extracting.

  When the term positional relationship determination unit 38 determines that two terms connected by the connection expression appear in the sentence including the connection expression, the in-sentence term extraction unit 40 uses the connection expression from the sentence including the connection expression. The connected terms 1 and 2 are extracted and output to the output unit 50.

  Specifically, the sentence internal term extraction unit 40 receives a syntax tree of a sentence including a connection expression among the syntax trees of the sentence generated by the syntax analysis unit 34, and the connection expression is a dependent connection or a coordinate connection. In this case, the following rules are applied to extract the terms 1 and 2 respectively. The correspondence relationship between the connection expression and the subordinate connection or the equipotential connection is previously given manually.

  First, the method for extracting terms 1 and 2 when the connection representation is dependent connection will be described.

  When the connection expression is a subordinate connection, the term 2 is extracted by the following procedures (1) and (2).

(1) A node representing the last word of the target connection expression is assigned to a node variable x representing a node in the syntax tree.
(2) Substitute the parent node of x into x. This operation is repeated until the node assigned to x takes the label of SBAR or S, and the text span dominated by x at the time of taking either label is term 2.

  FIG. 2 shows an example of extraction. In the example of FIG. 2, first, because is substituted for x. Since “because” is neither S nor SBAR, the parent node of “because” is substituted for x. Since IN is neither S nor SBAR, SBAR which is the parent node of IN is substituted for x. Since x becomes SBAR, the process ends, and the span “because he is honest” controlled by the SBAR assigned to x is term 2.

  Next, when the connection expression is a subordinate connection, the term 1 is extracted by the following procedures (1) and (2). Note that x takes over the value at the time point when the procedure of item 2 is completed.

(1) Substitute the parent node of x into x.
(2) Repeat until the node assigned to x takes the SBAR or S label, extract the text span dominated by x at the time of taking either label, and remove the span of term 2 from it Is term 1.

  In the example of FIG. 2, when the term 2 is determined, since the SBAR governing “because he is honest” is substituted for x, the parent node VP is substituted for x. Since VP is neither S nor SBAR, the parent node S is further substituted for x. Since x becomes S, the process ends, and the span “I like him because he is honest” that S controls is taken out, and the span “I like him” is removed from the span “because he is honest” in item 2. This is term 1.

  Next, the extraction method of term 1 and term 2 when the connection representation is equipotential connection will be described.

  When the connection expression is equipotential connection, the term 2 is extracted by the following procedures (1) to (3).

(1) The node representing the last word of the target connection expression is assigned to the node variable x, and the parent node of x is assigned to the node variable y.
(2) Substitute the respective parent nodes for x and y.
(3) Repeat (2) until the leftmost words of span (x) and span (y), which are spans governed by x and y, do not match. The term from the word immediately after the connection expression to the last word of the span among the spans dominated by y at the time when they do not coincide with each other is termed item 2.

  FIG. 3 shows a first example of a structural tree to be extracted. In the example of FIG. 3, first, and is substituted for x, and CC is substituted for y. Since the leftmost word of span (x) and span (y) matches with and, CC is substituted for x and S is substituted for y. The leftmost word of span (x) is “and”, and the leftmost word of span (y) is “He”. Then, span (y), that is, the span “he received a grant” immediately after the “He became a student and he received a grant” is term 2.

  FIG. 4 shows a second example of the structural tree to be extracted. In the example of FIG. 4, first, but is substituted for x and CC is substituted for y. Since the leftmost word of span (x) and span (y) matches with but, CC is substituted for x and VP is substituted for y. Since the leftmost words of span (x) and span (y) do not match in but and were, respectively, the process ends. The span “were not adjusted for ination” immediately after but of the spans controlled by y is term 2.

  Next, when the connection representation is equipotential connection, the term 1 is extracted by the following procedures (1) and (2). Note that x and y take on the values at the time point when the procedure of item 2 is completed.

(1) If S or SBAR exists to the left of x among the child nodes of y (the rightmost is selected when there are a plurality of child nodes), the span controlled by the node is term 1.
(2) If the above (1) does not apply, the parent is substituted for y and the syntax tree is traced until SBAR or S label is obtained. The span obtained by removing the connection expression and the term 2 from the span dominated by y at the time when SBAR or S is taken is defined as term 1.

  In the example extracted from the structural tree in FIG. 3, x is CC and y is S when the term 2 is determined. Here, since S is in the child node of y to the left of x, the span “He became a student” governed by S is term 1.

  In the example extracted from the structural tree in FIG. 4, x is CC and y is VP when the term 2 is determined. Here, since neither S nor SBAR exists in the child node of y to the left of x, its parent is substituted for y. Then, since y becomes S, the process is terminated. “The _gures were adjusted for deation”, which is obtained by removing “but ware not adjusted for ination” from the span “y gures were adjusted for deation, but were not adjusted for ination”, is defined as item 1.

  The inter-sentence term extraction unit 42 determines that the term positional relationship determination unit 38 determines that the two terms connected by the connection representation do not appear in the sentence including the connection representation. The term 2 is extracted from the sentence including the connection expression determined as the sentence for extracting the term 2 out of the two terms connected by the expression, and is connected by the connection expression by the term positional relationship determination unit 38. The term 1 is extracted from the sentence corresponding to the parent node or the sibling node of the sentence including the connection expression determined as the sentence for extracting the term 1 of the two terms, and the two extracted terms are output. Output to 50.

  The semantic class classification unit 44 classifies the semantic class of the connection representation based on the connection representation extracted by the connection representation extraction unit 36, and outputs the connection representation and the semantic class of the connection representation to the output unit 50. Specifically, the semantic class classification unit 44 inputs the connection expression extracted by the connection expression extraction unit 36 and words around the connection expression, and solves a multi-class classification problem learned in advance from learning data. To classify semantic classes for connection expressions. Since this is a multi-class classification problem, the data is resampled from the learning data so that the class distribution in the learning data is as uniform as possible.

<Operation of the connection expression term structure analyzing apparatus according to the first embodiment of the present invention>

  Next, the operation of the connection expression term structure analysis apparatus 100 according to the first embodiment of the present invention will be described. When the input unit 10 accepts a document, the connection expression term structure analysis apparatus 100 executes a connection expression term structure analysis processing routine shown in FIG.

  First, in step S100, a document accepted by the input unit 10 is acquired, and sentence breaks are given to the document.

  Next, in step S102, a discourse structure tree in which each sentence is represented by each node is generated based on the rhetorical structure of each sentence included in the document, based on the document given the sentence break in step S100.

  In step S104, a syntax tree is generated by performing syntax analysis on each of the sentences included in the document given the sentence break in step S100.

  In step S106, a connection expression having a term is extracted based on the syntax tree for each of the sentences generated in step S104.

  In step S108, it is determined whether or not two terms connected by the connection expression appear in the sentence including the connection expression for the connection expression extracted in step S106. In step S108, when it is determined that two terms connected by the connection expression do not appear in the sentence including the connection expression, the sentence including the connection expression is replaced with a term of the two terms connected by the connection expression. 2 is extracted as a sentence for extracting, and in the discourse structure tree generated in step S102, a sentence corresponding to the parent node or sibling node of the sentence including the connection expression is selected from the two terms connected by the connection expression. Is determined as a sentence for extracting the first term.

  In step S110, when it is determined in step S108 that two terms connected by the connection expression appear in the sentence including the connection expression, the term 1 and the term connected by the connection expression from the sentence including the connection expression are included. 2 is extracted and output to the output unit 50.

  In step S112, if it is determined in step S108 that the two terms connected by the connection expression do not appear in the sentence including the connection expression, the connection determined as the sentence for extracting the term 2 in step S108. The term 2 is extracted from the sentence including the expression, and the term 1 is extracted from the sentence corresponding to the parent node or the sibling node of the sentence including the connection expression determined as the sentence for extracting the term 1 in step S108. The two extracted terms are output to the output unit 50.

  In step S114, the semantic class of the connection representation is classified based on the connection representation extracted in step S106, the connection representation and the semantic class of the connection representation are output to the output unit 50, and the connection representation term structure analysis processing routine is executed. finish.

  As described above, according to the connection expression term structure analysis apparatus according to the first embodiment, a discourse structure tree is generated based on the rhetorical structure of each sentence included in the document based on the document, and the syntax Parse to generate a syntax tree, extract a connection expression with terms, determine whether two terms connected by the connection expression appear in the sentence containing the connection expression, and connect by the connection expression When it is determined that the two terms appear, the two terms connected by the connection expression are extracted from the sentence including the connection expression, and it is determined that the two terms connected by the connection expression do not appear. The term 2 is extracted from the sentence including the connection expression, and the term 1 is extracted from the sentence corresponding to the parent node or the sibling node of the sentence including the connection expression in the discourse structure tree, and the semantic class of the connection expression is classified. , Even between non-adjacent sentences Can be extracted term tied by a connection represented.

<Configuration of connection expression term structure analysis device according to second embodiment of the present invention>

  Next, the configuration of the connection expression term structure analysis apparatus according to the second embodiment of the present invention will be described. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted. In the connection expression term structure analysis apparatus according to the second embodiment, a term relating to an implicit connection expression and a semantic label are extracted from a document.

  As shown in FIG. 6, the connection expression term structure analysis apparatus 200 according to the second embodiment of the present invention includes a CPU, a RAM, a program for executing a connection expression term structure analysis processing routine to be described later, and various programs. It can be constituted by a computer including a ROM storing data. Functionally, the connection expression term structure analysis apparatus 200 includes an input unit 10, a calculation unit 220, and an output unit 50 as shown in FIG.

  The computing unit 220 includes a sentence dividing unit 30, a discourse structure analyzing unit 32, a related sentence pair extracting unit 238, an inter-sentence term extracting unit 242, and a semantic class classifying unit 244.

  The discourse structure analysis unit 32 performs the same process as in the first embodiment based on the rhetorical structure of each sentence included in the document based on the document given the sentence break by the sentence division unit 30. A discourse structure tree in which each node is represented is generated.

  Based on the discourse structure tree generated by the discourse structure analysis unit 32, the related sentence pair extraction unit 238 converts a sentence pair corresponding to a parent-child node and a sentence pair corresponding to a sibling node to a sentence having a connection relationship. It is determined whether or not there is a connection relationship for each candidate pair of sentences having a connection relationship as a pair candidate.

Specifically, the related sentence pair extraction unit 238 receives a discourse structure tree as an input, sets a sentence pair that becomes a parent-child node and a sibling node of the tree as candidates for a sentence pair having a connection relationship, and sets these sentence pairs. For each of the candidates, it is determined whether or not the sentence pair has a connection relationship by using a binary classifier learned in advance. The binary classifier prepares sentences S _i and S _j as sentence pairs of learning data, and learns the binary classifier using the following features (1) to (5).

(1) sentence _{S i,} and sentence _S beginning word of _j (2) statements _{S i,} and sentence _S last word (3) of the _j statement _{S i,} and sentence _S 3 words (4) of the head of the _j sentence All pairs of words included in S _i and words included in sentence S _j (5) All pairs of meaning classes of words included in sentence S _i and meaning classes of words included in sentence S _j

Note that the word semantic class, which is the feature (5) above, can be obtained from the results of existing thesauruses and word clustering. Further, the related sentence pair extraction unit 238 extracts the term 1 for each of the sentence pair candidates determined to have the connection relation by using the modification and the modified relation represented by the discourse structure tree. The sentence and the sentence for extracting the term 2 are determined. For example, if the child node of the statement S _i the sentence S _j, the statement S _i, a statement for extracting the second aspect, the sentence S _j, a sentence for extracting claim 1. If the sentence S _i and the sentence S _j are sibling nodes, a sentence having a small sentence number is a sentence for extracting the term 1, and a sentence having a large sentence number is a sentence for extracting the term 2.

  The inter-sentence term extraction unit 242 implicitly determines, from each of the sentence pair candidates having a connection relationship determined by the related sentence pair extraction unit 238, from the sentence pair candidates having the connection relationship. Two terms connected by a simple connection expression are extracted. In addition, since the related sentence pair extraction unit 238 determines which sentence the terms 1 and 2 are extracted from, only the terms are extracted by the following operations (1) and (2). .

(1) Delete the sentence end expressions of “.”, “!”, “?” Among the symbols included in the sentence.
(2) Delete parentheses such as ““ ”at the beginning and end of the sentence.

  The inter-sentence term extraction unit 242 repeats the operations (1) and (2) until there is no change, and outputs two terms having an implicit connection relationship to the output unit 50.

  The semantic class classifying unit 244 implicitly determines, based on the sentence pair candidates having the connection relationship, each of the sentence pair candidates having the connection relationship determined by the related sentence pair extracting unit 238 to have the connection relationship. The semantic classes of connection expressions are classified and output to the output unit 50. The semantic class classification unit 244 receives each sentence pair candidate as an input and solves a multi-class classification problem learned in advance from learning data, thereby determining a semantic class of a connection relationship that connects the sentences of each sentence pair candidate. decide. As features used for learning and classification, the features (1) to (5) used in the related sentence pair extraction unit 238 are used. Furthermore, since it is a multi-class classification problem, the data is resampled from the learning data so that the class distribution in the learning data is as uniform as possible.

<Operation of the connection expression term structure analyzing apparatus according to the second embodiment of the present invention>

  Next, the operation of the connection expression term structure analysis apparatus 200 according to the second embodiment of the present invention will be described. When the input unit 10 accepts a document, the connection expression term structure analysis apparatus 200 executes a connection expression term structure analysis processing routine shown in FIG. In addition, about the location which becomes the effect | action similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In step S200, based on the discourse structure tree generated in step S102, the sentence pair corresponding to the parent-child node and the sentence pair corresponding to the sibling node are used as candidate sentence pairs having a connection relation, and the connection relation It is determined whether or not there is a connection relationship for each of the sentence pair candidates having. Further, in step 200, for each sentence pair candidate determined to have a connection relationship, a sentence for extracting term 2 and a term using the modification and modified relationships expressed by the discourse structure tree The sentence for extracting 1 is determined.

  Next, in step S202, each of the sentence pair candidates having the connection relationship determined to have the connection relationship in step S200 is connected from the sentence pair candidates having the connection relationship by an implicit connection expression. Are extracted and output to the output unit 50.

  Then, in step S204, for each of the sentence pair candidates having the connection relation determined to have the connection relation in step S200, the meaning of the implicit connection expression based on the sentence pair candidates having the connection relation. The class is classified and output to the output unit 50, and the connection expression term structure analysis processing routine is terminated.

  In addition, about the other structure and effect | action of the connection expression term structure analysis apparatus 200 which concern on 2nd Embodiment, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

  As described above, according to the connection expression term structure analysis device according to the second embodiment, a discourse structure tree in which each sentence is represented by each node based on a rhetorical structure is generated based on a document, Based on the discourse structure tree, a sentence pair corresponding to a parent-child node and a sentence pair corresponding to a sibling node are used as a sentence pair candidate having a connection relation, and each of a sentence pair candidate having a connection relation is selected. Determine whether there is a connection relationship, extract two terms connected by an implicit connection expression from candidate sentence pairs with a connection relationship, and classify the semantic class of the implicit connection expression Thus, it is possible to extract semantically connected terms having a connection relationship even between non-adjacent sentences.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, in the above-described embodiment, the connection expression, the term, and the semantic label related to the explicit connection expression are extracted from the document by the connection expression term structure analysis apparatus according to the first embodiment, and the second embodiment. In the above description, the connection expression, the term, and the semantic label related to the implicit connection expression are extracted from the document by the connection expression term structure analysis apparatus according to the present invention. However, the present invention is not limited to this. You may make it extract the connection expression regarding an explicit connection expression, a term, and a semantic label, and the term regarding an implicit connection expression, and a semantic label from a document by a structure analysis apparatus.

DESCRIPTION OF SYMBOLS 10 Input part 20,220 Operation part 30 Sentence division part 32 Discourse structure analysis part 34 Syntax analysis part 36 Connection expression extraction part 38 Term positional relationship determination part 40 Sentence term extraction part 42,242 Inter sentence sentence extraction part 44,244 Meaning Class classification unit 46 Inter-sentence term extraction unit 50 Output unit 100, 200 Connection expression term structure analysis device 238 Related sentence pair extraction unit

Claims (5)

Based on the input document, based on the rhetorical structure of each of the sentences included in the document, a discourse structure analyzing unit that generates a discourse structure tree representing each of the sentences by each node;
For each of the sentences included in the document, a syntax analysis unit that performs syntax analysis and generates a syntax tree;
Based on a predetermined connection expression candidate dictionary and the syntax tree, a connection expression candidate from each of the sentences included in the document, and a feature amount corresponding to the connection expression candidate, and an appearance position of the connection expression candidate A first classification that has been learned in advance so as to extract a feature quantity that includes the extracted feature quantity and output a first determination result indicating whether or not the connection expression corresponding to the feature quantity has a term. A connection expression extraction unit that extracts a connection expression having a term from the connection expression candidates using a container ;
Learning in advance to output the second determination result indicating whether or not two terms connected by a connection expression appear in a sentence including the connection expression corresponding to the feature quantity, using the extracted feature quantity as an input. A second determination result corresponding to the connection representation for each connection representation extracted by the connection representation extraction unit using the second classifier
For the connection expression for which the second determination result is obtained, when the second determination result indicates that two terms do not appear, a sentence including the connection expression is expressed as one of the two terms. And a sentence corresponding to the parent node or sibling node of the sentence including the connection expression in the discourse structure tree, and a sentence for extracting the other of the two terms A term position relationship determining unit to determine;
An inter-sentence term extraction unit that extracts two terms connected by a connection expression based on the determination by the term positional relationship determination unit ;
Connecting representation obtained the second determination result, which indicates that the second determination result is two terms emerge from sentences including the connection representation, the two terms that are connected by the connection represented An internal sentence extraction unit to extract;
A semantic class classification unit that classifies semantic classes of the connection representation based on the connection representation extracted by the connection representation extraction unit;
Connection expression term structure analysis device including Based on the input document, based on the rhetorical structure of each of the sentences included in the document, a discourse structure analyzing unit that generates a discourse structure tree representing each of the sentences by each node;
Based on the discourse structure tree generated by the discourse structure analysis unit, a sentence pair corresponding to a parent-child node and a sentence pair corresponding to a sibling node are used as candidate sentence pairs having a connection relationship, and the sentence The feature amount corresponding to the pair candidate is extracted, and the extracted feature amount is input, and learning is performed in advance to output a determination result indicating whether or not there is a connection relationship for the sentence pair corresponding to the feature amount. A related sentence pair extraction unit for determining whether or not there is a connection relation for each of the sentence pair candidates having the connection relation,
Each of the sentence pair candidates having the connection relation determined to have a connection relation by the related sentence pair extraction unit is connected from the sentence pair candidates having the connection relation by an implicit connection expression 2. An inter-sentence term extraction unit for extracting two terms;
For each of the sentence pair candidates having the connection relationship determined to have a connection relationship by the related sentence pair extraction unit, based on the sentence pair candidates having the connection relationship, the implicit connection expression A semantic class classification unit for classifying semantic classes;
Connection expression term structure analysis device including A discourse structure analysis unit, based on the input document, based on the rhetorical structure of each sentence included in the document, generating a discourse structure tree in which each of the sentences is represented by each node;
A syntax analysis unit that parses each sentence included in the document to generate a syntax tree;
The connection expression extraction unit is a connection expression candidate from each of the sentences included in the document based on a predetermined connection expression candidate dictionary and the syntax tree, and a feature amount corresponding to the connection expression candidate, the connection expression Learning in advance to extract a feature quantity including the appearance position of an expression candidate, and using the extracted feature quantity as an input, output a first determination result indicating whether or not a connected expression corresponding to the feature quantity has a term Extracting a connection expression having a term from the connection expression candidates using the first classifier ,
A second determination indicating whether or not two terms connected by a connection expression appear in a sentence including the connection expression corresponding to the feature quantity, the term positional relationship determination unit receiving the extracted feature quantity as an input; Using a second classifier previously learned to output a result, a second determination result corresponding to the connection expression is obtained for each connection expression extracted by the connection expression extraction unit,
For the connection expression for which the second determination result is obtained, when the second determination result indicates that two terms do not appear, a sentence including the connection expression is expressed as one of the two terms. And a sentence corresponding to the parent node or sibling node of the sentence including the connection expression in the discourse structure tree, and a sentence for extracting the other of the two terms A step to determine ;
Bunkanko extraction unit, the connection representation obtained the second determination result, which indicates that the second determination result is two terms emerge from sentences including the connection represented by the connection expressed Extracting the connected two terms;
A semantic class classification unit classifying a semantic class of the connection representation based on the connection representation extracted by the connection representation extraction unit;
Connection expression term structure analysis method including A discourse structure analysis unit, based on the input document, based on the rhetorical structure of each sentence included in the document, generating a discourse structure tree in which each of the sentences is represented by each node;
Based on the discourse structure tree generated by the discourse structure analysis unit, a related sentence pair extraction unit converts a sentence pair corresponding to a parent-child node and a sentence pair corresponding to a sibling node to a sentence having a connection relationship. A determination result indicating whether or not there is a connection relationship with respect to a sentence pair corresponding to the feature amount, by extracting the feature amount corresponding to the sentence pair candidate as a pair candidate, and using the extracted feature amount as an input Determining whether or not there is a connection relationship for each of the sentence pair candidates having the connection relationship using a classifier previously learned to output
An inter-sentence term extraction unit implicitly determines, from each of the sentence pair candidates having the connection relationship, for each of the sentence pair candidates having the connection relationship determined by the related sentence pair extraction unit. Extracting two terms connected by a simple connection expression;
For each of the sentence pair candidates having the connection relationship determined by the related sentence pair extraction unit, the semantic class classification unit, based on the sentence pair candidates having the connection relationship, Categorizing semantic classes of implicit connection expressions;
Connection expression term structure analysis method including   The program for functioning a computer as each part which comprises the connection expression term structure analysis apparatus of Claim 1 or 2.