JPH077418B2 - Natural language context processor - Google Patents

Description

The present invention relates to a context processing device capable of effectively determining the context structure of a text (sentence) in natural language processing.

(Prior Art) Recently, various researches on natural language processing have been advanced, and as a means for extracting the contents of a text (text) such as a summary, it is possible to examine the context structure of the text and analyze its meaning. It began to take place. Conventionally, this kind of context structure is exclusively determined by examining the nouns and verbs that appear in the text, and referring to a dictionary that stores information such as dependency conditions and part-of-speech connection relations for these words It is being done while examining the connection. That is, the nouns and verbs appearing in the text are used as clues to obtain the context structure.

However, it is practically almost impossible to register in advance a dictionary with information about all nouns, verbs, etc. that may appear in a text. Therefore, it cannot be denied that the determination of the context structure becomes very difficult when a word that is not registered in the dictionary appears in the text. Further, even if the context structure is determined using such dictionary information, there is a problem that a huge amount of knowledge is required and the processing load is very large.

On the other hand, we also try to find the context structure by looking up the series permitted as a connection relation between multiple sentences from this connection expression by referring to the connection rules, using the connection expression such as the conjunction that appears in the sentence as a clue. Has been. However, since multiple sentences that make up a text do not necessarily have a connective expression such as a connective, there are still some problems in accurately determining the context structure even if the connective expression is used. .

(Problems to be solved by the invention) As described above, conventionally, the nouns and verbs appearing in the text are exclusively used as clues to examine the context structure, and even if the connection expressions are used as clues, the connection relations are used. Since there are many sentences that do not have expressions, there is a problem that it is very difficult to uniquely determine the context structure. Further, there is a problem that enormous amount of knowledge is required to go into the semantic content of a sentence and analyze the context structure, and advanced knowledge processing such as inference processing is required.

The present invention has been made in consideration of such circumstances, and an object thereof is to properly determine the context structure of a text even when information about individual nouns or verbs is insufficient. It is to provide a natural language context processing device which is excellent in practicality.

[Configuration of Invention] (Means for Solving the Problems) A natural language context processing device according to the present invention is a context processing device that analyzes a natural language sentence and extracts the entire structure of the sentence. Detects rhetorical expressions in sentences using sentence analysis means that performs morphological analysis for each sentence that composes the sentence, and morphological analysis results for each sentence that are obtained by this sentence analysis means. And a connection relation extracting means for obtaining a connection relation series connecting the and in a relation corresponding to the rhetorical expression, and each sentence including the connection relation series obtained by the connection relation extracting means can be taken as a rhetorical structure. Rhetoric relationship determination means for finding candidates and excluding candidates that are inappropriate as sentence structures from these, and topic presentation in sentences with the particle as a condition for the morphological analysis result of each sentence obtained by the sentence analysis means The word as the present is extracted, and it is checked whether or not the word as the topic presentation expression appears in the sentence preceding the extracted sentence, and when it appears, the extracted sentence and the preceding sentence Sentence relevance determining means for outputting as information on the relevance between the sentences by associating with each other, for each candidate of the rhetorical structure in which the inappropriate candidate is excluded from the rhetorical relation determining means, Each of the extracted sentences according to the information, in which the rhetorical expression is not detected, is connected to the preceding sentence to obtain connection structure candidates, and the extracted sentence and the preceding sentence are connected to these connection structure candidates. And a context structure creating means for excluding candidates whose inter-sentence structure with the sentence is inappropriate.

(Operation) According to the present invention, paying attention to the rhetorical expression in the text, the rhetorical structure is determined by examining the connection structure of the connection relation between the sentence parts from the rhetorical expression, and the sentence in which the rhetorical expression is not clearly presented Also, since we focus on the topic presentation expression in the sentence and investigate the connection relation between sentences according to this topic presentation expression, if the information about nouns and verbs is insufficient and the sentence structure cannot be analyzed sufficiently, , It is possible to effectively determine the context structure of a sentence even if there is no rhetorical expression indicating the connection relation between sentences. Moreover, since the context structure is examined by utilizing the rhetorical structure between the sentence parts and the topic expression, it is possible to perform appropriate context processing and improve the accuracy of structure extraction of the entire sentence.

(Embodiment) Hereinafter, a context processing device according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic configuration diagram of a main part of the embodiment apparatus, and 1 is a text input unit for inputting text such as document data or recorded data to be analyzed. A natural language text (sentence) input via the text input unit 1 is given to the sentence analysis unit 2 and subjected to format structure analysis and morpheme analysis. The format structure analysis section 2a constituting the sentence analysis section 2 analyzes the format structure of the entire text from the format information such as chapters and sections included in the text input from the text input section 1 and determines the structure. To do. Further, the morpheme analysis unit 2b constituting the sentence analysis unit 2 is the format structure analysis unit 2a.
The morphological analysis of the sentence contained in the paragraph is performed on the text, the formal structure of which is determined by the analysis by the paragraph. By this morphological analysis, morphemes such as nouns and verbs in the text are individually obtained.

The processing function of the sentence analysis unit 2 described above has the same function as that of the conventional context processing device, and is realized by using various conventionally proposed methods.

Here, the feature of the present apparatus is that, according to the result of the formal structure analysis of the text obtained by the sentence analysis unit 2 and the result of the morpheme analysis, a plurality of sentences that compose the text with the rhetorical expression in the text as a clue. The point is that a connection relation extraction unit 3 for obtaining a connection relation between parts (for example, paragraphs) and a rhetorical relation judgment unit 4 are provided, and a sentence relevance degree judgment unit 5 described later is also provided.

The connection relation extraction unit 3 refers to the relation name table storage unit 6 that stores a relation name table showing connection relations and rhetorical representations such as connectives as shown in FIG. This is to find the connection relation between paragraphs etc. linked by the rhetorical expression. For example, when a rhetorical expression of "But" is requested, a connection relation indicating that the paragraph existing before the rhetorical expression and the paragraph with the above rhetorical expression have a "capture" relationship Looking for. The connection relation extracting unit 3 sequentially finds all rhetorical expressions that appear in the text, and individually finds the connection relations between the text parts indicated by the rhetorical expressions.

Therefore, the rhetorical structure determination unit 4 refers to the prohibition rule storage unit 7 which stores, as a relational table, the prohibition rule relating to the connection of the connection relation shown by the rhetorical expression, for example, as shown in FIG. The rhetorical structure within and between paragraphs is obtained from the series of connection relations between sentence parts obtained in.

The relation table stored in the prohibition rule storage unit 7 is the third
Information showing structural connection prohibition rules of connection relations expressed by rhetorical expressions as shown in FIG. 3A, and FIG. 3B.
And the information indicating the negative connection rule of the above-mentioned connection relation are registered, and the structure between the connection relations which is inappropriate as a rhetorical structure of the text is shown as a rule. In particular, the connection prohibition rule shown in FIG. 3 (a) shows a combination structure that does not actually appear in the connection relation between paragraphs connected by rhetorical expressions. The negative connection rule shown in FIG. 3 (b) shows a combination structure that is rarely used in general, if not as much as the above connection prohibition rule.

The rhetorical relation determination unit 4 refers to the prohibition rule storage unit 7 having such information, and the rhetorical structure of the text is extracted from the series indicating the relation of connection between paragraphs connected by the rhetorical expression obtained by the relational relation extraction unit 3. As a result, the rhetorical structure of the text is determined by finding the combination structure while eliminating inappropriate combinations.

When the rhetorical structure of the text is not uniquely determined and a plurality of rhetorical structure candidates are obtained, the decision is entrusted to the next context structure creation unit 8.

On the other hand, the sentence relevance determining unit 5 checks words appearing in the text (sentence) according to the text analysis result obtained by the sentence analyzing unit 2, and detects a plurality of sentences based on the relationship between the words appearing in each sentence. Relevance is judged. That is, the sentence relevance determination unit 5
Examines words appearing in each sentence of a plurality of sentences forming an input sentence (text), and the topic expression extracting unit 5a particularly seeks a word as a topic presentation expression in the sentence. The word as the topic presentation expression is extracted, for example, on the condition that the particle attached to the word is “ha” or “mo”. Specifically, when the expression "Xwa" exists, the word "X" is extracted as a word as a topic presentation expression. When the expression “X of Y” is present, both “X” and “Y” are extracted as words that present a topic.

It should be noted that not all of the words extracted in this way represent the topic presentation. For example, among the words extracted using the particle as a clue, for example, "recently"
It cannot be said that it is a topic presentation expression such as "Fig. X is" or "in the case of ...", and there is something that does not serve as a clue for checking the connection relation between sentences. For such words, for example, these are predetermined as unnecessary words and are stored in the unnecessary word table storage unit 5b, and the unnecessary word table storage unit 5b is referred to exclude them from words as topic presentation expressions. You can leave it as it is.

The word repetition extraction unit 5c in the sentence relevance determination unit 5 basically checks whether or not the same word is repeatedly used among a plurality of sentences. When a word as a topic presentation expression is sought in a certain sentence, it is checked whether the word is also used in another sentence, and if the word is used in another sentence, the sentence is identified. The information (sentence management number) is requested.

The sentence relevance determination processing unit 5d is the topic expression extraction unit 5a as described above.
According to the information of the word as the topic presentation expression obtained in, and the information of the appearing word obtained in the word repetition extraction unit 5c, the relationship between a plurality of sentences forming the input text is determined according to the word information. There is. Specifically, when a word as a topic presentation expression is extracted from a sentence by the topic expression extraction unit 5a, the information of the appearance word in each sentence obtained by the word repetition extraction unit 5c for this word is checked, and A sentence in which the same word as the word appears in a sentence preceding the sentence in which the word as the topic presentation expression is extracted is obtained, and this sentence is associated with the sentence in which the topic presentation expression is extracted. If the word extracted as the topic presentation expression is a demonstrative or a pronoun, the sentence relevance determination processing unit 5d causes the sentence presentation expression in the sentence preceding the sentence including the topic presentation expression composed of the demonstrative or the pronoun. Look up the sentence with the word as
The sentence after this sentence is associated with the sentence for which the above-mentioned demonstrative or pronoun is sought.

By such a determination process, it is possible to find out how the plurality of sentences forming the input text are related to each other by using the word appearing in each sentence as a clue.

As described above, the context structure creation unit 8 uses the sentence relevance determination unit 5
Of the sentence structure stored in the knowledge storage unit 9 from the information on the relevance between the sentences obtained in step S1 and the rhetorical structure of the input text obtained by the connection relation extraction unit 3 and the rhetorical relation determination unit 4. While referring to the knowledge, I investigate the sentence structure and perform context analysis. The information of the context structure of the input text obtained in this way is given to the application unit 10 and is used for, for example, a summary sentence creation process.

That is, the context structure creation unit 8 stores the knowledge stored in the knowledge storage unit 9, for example, information on the field to which the content referred to by the text belongs, knowledge on the connection relation of parts of speech appearing in the text, knowledge on the dependency relation, etc. To determine the contextual structure of the text, or by determining the rhetorical structure of the text. In other words, this context structure is basically determined in the same manner as the conventional context processing device, but at this time, the sentence obtained according to the rhetorical structure analyzed from the rhetorical expression described above and the word as the topic presentation expression. The contextual structure is determined according to the information about the relationships between them.

The above is the configuration and function of the present apparatus.

First, the rhetorical structure determination process based on the rhetorical expression described above will be described in more detail with reference to FIGS. 4 and 5.

FIG. 4 is a diagram showing an example of text to be subjected to context processing. When such text is input, the format structure analysis unit 2a divides the text part constituting the text into texts or paragraphs based on the textual information such as indentation, punctuation marks, commas, periods, etc. Is to seek. Here, the input text shown in FIG. 4 is divided into a plurality of sentences, and here, for example, sentences, ... Are respectively obtained. The morphological analysis unit 2b executes the morphological analysis on each sentence thus obtained.

For this morphological analysis, the analysis method shown in, for example, "Makoto Nagao," Japanese Information Processing "published by The Institute of Electronics, Information and Communication Engineers, (1984)" may be used. According to such a Japanese sentence analysis method, for example, when a sentence “The following refers to drawings.” Is given, “below (noun) + (particle) + is (particle) + drawing (noun ) + To (particle) + reference (noun) + to (particle) + (verb). The result of morphological analysis will be required.

Then, the connection relation extraction unit 3 refers to the relation name table storage unit 6 to find the rhetorical expression in each sentence based on the morpheme analysis result obtained for each sentence constituting the input text, and detects the rhetorical expression detected in each sentence. Information about the connection relation that the rhetorical expression means is obtained from the relation name table storage unit 6.

In the example shown in FIG. 4, the rhetorical expression "like this" is obtained from the sentence and the connection relation is "forward" as shown by the hatching in the figure. Also, from the sentence
Rhetorical expression and its connection relation "contrast" are obtained.

From such a connection relation, the connection relation extracting unit 3 determines the fourth relation.
As shown in Fig. 5A, the text shown in the figure asks that a plurality of sentences, ... are sequentially connected in a connection relation [-forward-contrast].

However, such a connection relation series merely shows individual connection relations between a plurality of sentences, and does not show how the rhetorical structures are combined.

Then, considering the combination of the connection relation series forming the rhetorical structure, if these five paragraphs are present, theoretically, there are 14 possible combinations as shown in FIG. 5B. When there are two sentences, the combination structure can take one form, and when there are three sentences, the combination structure can take two forms. In the case of four sentences, there are five types, and the number of forms that the combination structure can take is theoretically determined by the number of sentences.

Note that the combinational structure taken by this sequence of connection relationships indicates where and in what relationship each sentence in the text is connected. For example, the rhetorical formula is [-[[ordering]-[contrast]]]. The structure (combination structure) indicates that the first sentence has some relation to the whole of the second sentence to the fifth sentence. And for the second sentence to the fifth sentence, the second sentence and the third sentence are "junction"
It is shown that there is a relationship, and it is further shown that the fourth sentence and the fifth sentence have a "contrast" relationship. Then, it is shown that the sentences in the "junction" relationship and the sentences have some connection relationship with the sentences in the "contrast" relationship.

The expression of such rhetorical structure shows the connection structure relation between a plurality of sentences. However, the above-mentioned combination structure taken by the series of connection relations shown in B merely represents the structural form of the combination, and some of them are unsuitable as rhetorical structures in the text structure. Therefore, the rhetorical relationship determination unit 4
Refers to the prohibition rule storage unit 7 described above and finds an improper rhetorical structure in the combination structure, and excludes it from the rhetorical structure candidates.

For example, from the connection prohibition rule "junction [... [X order"X; statement] shown in the relation table as shown in FIG. If there is something that includes it, it is considered inappropriate as a rhetorical structure and is excluded from the candidates for that rhetorical structure.

However, as is clear from the rhetorical structure candidates shown in FIG. 5, in the example of the input text shown in FIG. 4, no rhetorical expression can be found from the sentence, and the connection relation to these sentences becomes unknown. There is. Therefore, the appropriate rhetorical structure cannot be narrowed down only from the candidates of the rhetorical structure of the sentence obtained as described above, and the narrowing down process is ultimately entrusted to the context structure creating unit 8.

Therefore, in this apparatus, the context structure of the input text is obtained as follows by using the information on the relation between the sentences obtained by the sentence relation determining unit 5.

That is, as described above, the sentence relevance determining unit 5 extracts the words of the topic presentation expression and the repeated occurrences of the same words that appear in the sentence, and the input text shown in FIG. 4 is underlined in the figure. As shown in the figure, a topic presentation expression "rule composition method" is obtained from the sentence, a topic presentation expression "segment" is extracted from the sentence, and a topic presentation expression "segment" is extracted from the sentence. ing. Regarding the extraction processing of words as this topic presentation expression, for example, in the input text, a topic presentation expression as shown in "Ken Nagano" Review of writing theory [grammatical consideration] "published by Asakura Shoten (1986) It should be extracted from.

Then, the word repetition extraction unit 5c examines whether or not the word of the subject presentation expression extracted from each sentence in this way repeatedly appears in other sentences,
For example, the information shown in FIG. 6 is requested. The information obtained by the word repetition extraction unit 5c indicates the word of the topic presentation expression and the information of the repetition expression for each sentence. Specifically, the sentence of the above-mentioned text has a topic presentation expression of "rule composition method", and the sentence has a topic presentation expression of "segment", and the word appears in the sentence. In addition, there is a topic presentation expression that is also a "slice" from the sentence, and it is required that the word appears in the sentence.

Then, the sentence relevance determination processing unit 5d checks from the words of the topic presentation expression obtained as described above whether or not the same word appears in the preceding sentence, and when the same word exists in the preceding sentence, the preceding And the sentence from which the topic presentation expression is extracted are associated with each other. Specifically, for the word “element” of the topic presentation expression obtained from the sentence, it is examined whether or not the same word appears in the sentence (here, sentence) preceding the sentence. From the information shown in FIG. 6, it is found that the same word appears in the sentence, and as a result, it is found that the sentence and the sentence have a "supplementary" relationship.
Similarly, for sentences, the topic presentation expression "elements"
It is found that the sentence has a "supplementary" relationship with the sentence that precedes the sentence because the sentence exists in the sentence that precedes.

The context structure creating unit 8 refers to the information about the relevance between the sentences, so that the plurality of sentences constituting the input text described above are displayed as shown in C of FIG. Supplementary comparison] It is required to be sequentially connected by the connection relation.

When such a connection relation is obtained, the context structure creating unit 8 obtains the connection structure candidate as shown in D of FIG. Here, referring to the knowledge storage unit 9, since the “supplementary” relationship indicating the connection relation of the above sentences indicates that the preceding sentence is supplemented by the succeeding sentence, the context structure creating unit 8 For example, "... [X supplement [Y r ...] X, Y; sentence r; find out that other related connection structures are not allowed. By following the knowledge information of connection prohibition between sentences as described above, the context structure creating unit 8 can obtain the [supplemental [ordering [supplemental] [compare]] as shown in 1) of the connection structure candidates shown in D of FIG. ]] Is determined to be inappropriate as a sentence structure, and is excluded from connection structure candidates. As a result of such a determination process, as shown in E of FIG. 7, the connection structure candidates are [[supplemental] ordering [[supplementary] contrast]] [[[supplemental] ordering [supplementary] contrast]] [[[[[ Supplement] Ordering] Supplement] Contrast]. Further focusing on these connection structure candidates, the sentence supplemented by the sentence is the sentence in which the word of the topic presentation expression described above appears, and does not supplement the sentence. It can be seen that no other connection structure is allowed between the sentences. As a result, connection structure candidates such as [... supplementation ... [... supplementation ...] that have a structure "[" between them and are excluded, and finally, as shown in F of FIG. [Supplement] Ordering] Supplement] Contrast] Only the sentence connection structure candidate is obtained as the context structure of the input text.

In this way, the context structure creating unit 8 uses the sequence of connection relations obtained by the connection relation extracting unit 3 and the sequence of connection relations obtained by the sentence relation determining unit 5 as shown in FIG. A connection structure candidate of a proper sentence is found, and from the combination of rhetorical structures that the sequence can theoretically take, an inappropriate one as a rhetorical structure is detected / excluded by referring to the prohibition rule storage unit 7 or the knowledge storage unit 9. Then, narrow down the candidates as a rhetorical structure. Then, when the connection structure is uniquely obtained in narrowing down the candidates, this is determined as the connection structure, and when a plurality of candidates are left, the determination is performed by the selection process in the application unit 10. Entrust. Then, the final structure of the context is determined by taking into account the rhetorical structure and the connection structure of the text determined in this way, and determining the structure of the text content and the like.

Thus, according to the device thus configured, not only the nouns and verbs of the text are analyzed, but also the rhetorical structure of the text indicated by the rhetorical expressions appearing in the text is analyzed, and this rhetorical structure is taken into consideration. Since the context structure of the above text is determined and context processing is performed, even if the information about the noun or verb appearing in the text is not registered in the dictionary, an appropriate context is not affected by the unregistered word. It becomes possible to perform processing. Moreover, since the context processing is performed by using the rhetorical structure analyzed from the rhetorical expression, the context processing can be performed by appropriately grasping the rhetorical structure such as premise, conclusion, judgment, and illustration. It is possible to execute the language processing that is accurately captured.

Even if there is a sentence in which the rhetorical expression does not exist, it is possible to analyze the sentence connection structure effectively by determining the connection relation between sentences using the words of the topic presentation expression that appear in the sentence as clues. It is possible to effectively narrow down the connection structure and perform highly accurate context processing.

It should be noted that the present invention is not limited to the above-described embodiment, and the analysis of the format structure, the morphological analysis, etc. may be performed by appropriately using various algorithms conventionally proposed. Further, the information registered in the relation name table and the prohibition rule table is not limited to the above example.

In addition, here, we focused on words as topic presentation expressions to find the relevance between sentences. For example, "these" and "this xx"
It is also possible to find the relevance between sentences by paying attention to pronouns and demonstratives such as. In this case, check the sentence that has the topic presentation expression in the sentence that precedes the sentence in which the pronoun or the demonstrative is extracted, and extract the pronoun or the verb in any of the sentences after this sentence. It is only necessary to determine that the given sentence has relevance. In addition, without extracting the topic presentation expression, we will examine the repeated appearance frequency of the words that appear in the sentence and determine that the sentence with the same repeated occurrence frequency of the same word is highly related to the sentence being processed. It is also possible.

Furthermore, although the Japanese processing has been described here, the context processing for other languages such as English can be similarly applied. In addition, the present invention can be variously modified and implemented without departing from the scope of the invention.

[Effects of the Invention] As described above, according to the present invention, the rhetorical structure appearing in a text is focused on to obtain its rhetorical structure, and the word as a topic presentation expression is focused on to determine the relevance between sentences. Since the context structure of the above text is analyzed using this information, there are cases where unregistered words are included in the text or when rhetorical expressions (connective expressions) are not clearly detected from the sentence. However, the practically great effect such as the ability to determine the context structure effectively and accurately can be obtained.

[Brief description of drawings]

FIG. 1 shows a context processing device according to an embodiment of the present invention. FIG. 1 is a schematic configuration diagram of a main part of the embodiment device, FIG. 2 is a diagram showing a configuration example of a relation name table, and FIG. FIG. 4 is a diagram showing a configuration example of a prohibition rule table, FIG. 4 is a diagram showing an example of input text subjected to context processing, FIG. 5 is a diagram schematically showing a form of text rhetorical structure analysis processing, and FIG. FIG. 7 is a diagram showing a topic presentation expression obtained by the sentence relevance determination unit and information on its repeated appearance, and FIG. 7 is a diagram schematically showing a form of connection structure analysis processing using sentence relevance information. 1 ... Text input part, 2 ... Sentence analysis part, 2a ... Form structure analysis part, 2b ... Morphological analysis part, 3 ... Connection relation extraction part, 4 ... Rhetorical relation determination part, 5 ... Sentence relation Degree determining unit, 5a
…… Topic expression extraction unit, 5b …… Unnecessary word table storage unit, 5c
...... Word repetition extraction unit, 5d ...... Sentence relevance determination processing unit, 6 ...
... relation name table storage unit, 7 ... prohibition rule storage unit, 8 ...
... Context structure creation unit, 9 ... Knowledge storage unit, 10 ... Application unit.

Claims (1)

[Claims] Claim: What is claimed is: 1. In a context processing device for analyzing a sentence of a natural language to extract the structure of the entire sentence, sentence analysis means for performing morphological analysis on each sentence constituting the input sentence, and a sentence analysis means for obtaining the sentence. The relational relation extraction for detecting the rhetorical expression in the sentence using the morpheme analysis result of each sentence, and for obtaining the connection relation series connecting the detected sentence and the sentence before it by the connection relation corresponding to the rhetorical expression And a rhetorical relationship determining means for finding possible candidates as a rhetorical structure for each of the sentences including the connection relation series obtained by the connection relation extracting means, and excluding candidates that are inappropriate as a sentence structure from the candidates. And a word as a topic presentation expression in the sentence is extracted from the morphological analysis result of each sentence obtained by the sentence analysis unit, using a particle as a condition, and the word as the topic presentation expression is the extracted sentence. A sentence relevance determining means for checking whether or not the sentence appears in a preceding sentence, and, when it appears, correlating the extracted sentence with the preceding sentence and outputting as information on the relation between the sentences; For each candidate of the rhetorical structure in which the inappropriate candidate is excluded by the rhetorical relationship determining means, the extracted sentence according to the information on the relation between the sentences and the rhetorical expression is not detected Context structure creating means for respectively finding connection structure candidates connected to the preceding sentence, and excluding candidates for which the inter-sentence structure between the extracted sentence and the preceding sentence is inappropriate for these connection structure candidates, A natural language context processing device comprising: