JP3043038B2 - Linguistic expression feature determination device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a linguistic expression feature determination device that automatically determines the characteristics of a linguistic expression of a sentence.

In the "prior art" sentences, the linguistic expressions may be different even if they have the same meaning. For example, in a male expression, a sentence that uses the expression "you do your best" is used, and in a female expression, an expression such as "you also do your best" is used. Also, a sentence that means "There is no such thing" is such as "No such thing" in the Tokyo dialect, and "Such a thing" in the Kansai dialect.
The Kyushu dialect uses expressions such as "Sogenkotsunaka".

In this way, since the expression of the sentence expressing the same meaning differs from person to person, when selecting a favorite sentence from a message collection such as a telegram example sentence, whether the expression is a male expression, a female expression, or any dialect It is desirable to give guidelines to users, such as

"Problem to be Solved by the Invention" The message collection database search device and the database construction device used heretofore have the configuration shown in FIG. 15, and when trying to display the type of expression with this message collection search device, It is necessary to input the type of expression in the message collection database in advance, and in this case, conventionally, it was necessary to determine the type of linguistic expression by obtaining it and input it to the message collection database.

SUMMARY OF THE INVENTION An object of the present invention is to provide a linguistic expression feature determining apparatus for automatically determining the characteristics of a linguistic expression in order to reduce the labor for manually determining the characteristics of the linguistic expression.

"Means for Solving the Problems" In order to achieve the above object, a linguistic expression feature determination apparatus according to the present invention includes a sequence of morphemes including a notation, a standard form, a part of speech, and an inflected form which constitute a characteristic linguistic expression , And a feature expression dictionary that divides arbitrary sentences into morphemes,
A morphological analysis means to be a sequence of standard forms, parts of speech and inflected forms;
A feature that matches the strings of notation, standard form, part of speech, and inflected form of each morpheme that is the result of the morphological analysis with the feature expression dictionary, extracts the expression part characterizing the sentence, and obtains the sum of the weights of the characteristic expressions. An expression extraction unit, and a characteristic determination unit that determines the characteristic of the sentence from the difference between the total values of the weights of the plurality of characteristic expressions obtained from the characteristic expression extraction unit.

[Operation] In the linguistic expression feature determination apparatus of the present invention, first, an input sentence is morphologically analyzed, and the notation, standard form, part of speech, and inflected form of each morpheme are obtained. Next, the feature expression dictionary is searched based on the result of the morphological analysis, the characteristic expressions in the sentence and their weights are extracted, and the weights of each expression are summed to obtain a total value of the feature expression weights. This is repeated for a plurality of feature expression dictionaries, and the total value of each feature expression weight is determined. Finally, the difference between the largest total value of the feature expression weights and the total value of the other feature expression weights is calculated, and those having a difference within a certain value are determined as features of the input sentence.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 shows the configuration of an embodiment of a linguistic expression feature determining apparatus according to the present invention. The language expression feature determination device body 6 is composed of a CPU, a memory, and the like in terms of hardware, but functionally includes a morphological analysis unit 1, a feature expression extraction unit 2, a feature determination unit 3, a control unit 4, and a switching unit. switch 5, n pieces characterized composed expression dictionary 10 ₁ to 10 _n of. Here, n is an integer greater than 1 and is the number of features of the linguistic expression to be determined. For example, when determining whether the expression is a male expression or a female expression, n = 2.

Figure 2 is an example of the configuration of a record of the feature representation dictionary 10 ₁ to 10 _n. In the figure, reference numeral 7 denotes a notation indicating the number m of morphemes constituting the feature expression and indicating the character face of the morpheme itself.
101, standard form 102, part of speech 103, and inflected form 104 are sequentially recorded. The standard form 102 is a character string obtained by removing the fluctuation of the notation from the morpheme. For example, “Katakana”, “Katakana”, and “Katakana” have different notations, but the standard form is the same morpheme. The inflected form 104 is an inflected form when the morpheme is a verb or an auxiliary verb. One morpheme is these notations 101, standard form 10
2, expressed by part of speech 103 and inflected form 104. Notation 101, standard form 102, part-of-speech 103, inflected form 104 may be blank, but for one morpheme, notation 101, standard form 102, part-of-speech 10
3. Not all items in the inflected form 104 are blank. From the first 20 ₁ in the m morphemes constituting the feature representation to the m-th morpheme 20m Similarly contained in one record. At the end of the record, a weight w of the feature expression is provided. w is a positive number. In the feature expression dictionaries 10 ₁ to 10 _n in FIG. ₁ , these records are arranged in descending order of the weight w, and those having the same weight w are arranged in descending order of the morpheme number m.

Next, the operation will be described with reference to the apparatus configuration diagram of FIG. 1, a configuration example of one record of the feature expression dictionary of FIG. 2, and the flowchart of FIG.

In FIG. 3, step 1 corresponds to the operation of the morphological analysis unit 1 in FIG. Similarly, step 4,
5, 6, 7, 8 and 9 are operations of the feature expression extraction unit 2, steps 12, 13, 14, 15 and 16 are operations of the feature determination unit 3, and steps 2, 3, 10, and 11 are operations of the control unit 4. Is equivalent to

The morphological analysis unit 1 (step 1) performs a morphological analysis on the input sentence, divides the input sentence into each morpheme, and gives notation, standard form, part of speech, and inflected form information (step 1). Assuming that the input message is composed of the morpheme number l, the morphological analysis result is a sequence of l notations, standard form, part of speech, and inflected form. Also, let P be the number of phrases in the input sentence that has undergone morphological analysis.

Next, 1 is set to a counter i (step 2),
The changeover switch 5 is operated to connect the feature expression extraction unit 2 to the feature expression dictionary 10i (step 3).

Feature expression extraction unit 2 (steps 4, 5, 6, 7, 8,
In 9), the feature expression dictionary 10i is searched with the result of the morphological analysis, a matching feature expression is extracted, and the total value of the weights is calculated.

First, as an initial setting of this processing, the counter j is set to 1
Is substituted into the sum sum (i) of the feature expression weights (step 4). Next, the records of the feature expression dictionary 10i and the morpheme strings after the j-th morpheme analysis result are compared in order from the top of the dictionary (step 5). If there is a match, step 7 is performed until the end of the dictionary. If no match is found even after the comparison, step 8 is executed (step 6). Here, when the dictionary record to be compared is composed of the morpheme number m constituting the feature expression, the j-th to (j + m-1) -th morpheme notations, the standard form, the part of speech, and the inflected form of the morphological analysis result If all of the notation, standard form, part of speech, and inflected form of the m morphemes written in the dictionary match, it is regarded as a match. If any of the morpheme notation, standard form, part of speech, and inflected form items in the record of the feature expression dictionary are blank, the blank items of the morpheme are unconditionally matched.

Step 7 is a process in the case where there is a match in step 5, where m is added to the counter j, and the morpheme sequence to be compared with the dictionary is advanced. Also, the weight w of the dictionary record that matches the feature expression weight sum value sum (i)
Add.

Step 8 is a process performed when there is no match in step 5, and the counter j is advanced by one.

Then, if j is equal to or less than 1, the processing is repeated from step 5, and if j is greater than 1, the processing of the feature expression extraction unit 2 ends, and the processing proceeds to step 10 (step 9).

In step 10, 1 is added to the counter i, and in step 11
Then, if i is equal to or smaller than the number n of the feature expression dictionaries, steps 3, 4, 5, 6, 7, 8, 9, and 10 are repeated, and if i is larger than n, the process proceeds to step 12.

Feature determination unit 3 (steps 12, 13, 14, 15, 16, 17, 1
8) determines the features of the input sentence from the n feature expression weight sums sum (1) to sum (n) obtained in the processing of steps 1 to 11.

First, all n feature expression weight composite values are divided by the number of clauses P and normalized (step 12). As a result, the feature expression weight sums sum (1) to sum (n) are numerical values that are not affected by the sentence length.

Next, the normalized n feature expression weight sums are sorted in descending order of value (step 13). The sorted result is the sum of the largest feature expression weight sum (x ₁ ),
The sum of the second largest feature expression weights is sum (x ₂ ), and similarly, sum (x ₃ ),... Sum (x _n ). Here, x ₁ , x ₂ , x ₃ ,..., X _n are integers from 1 to n, respectively, and are feature expression numbers.

Next, the largest sum of the feature expression weights sum (x ₁ ) and the feature expression threshold value s are compared (step 14). As a result, if sum (x ₁ ) <s, it is determined that determination is impossible, that is, the input sentence is a sentence having no feature.

In step 15, 1 is substituted for the counter k. If k is equal to or smaller than n and sum (x _k )> {sum (x ₁ ) s} (step 16), 1 is added to the counter k (step 16).
17), repeat step 16. When step 16 is not satisfied, the process proceeds to step 18. However, the feature expression threshold value s is a predetermined positive number.

In step 18 expression x _1, x _2, judges ..., and x _k-1 as a feature of the input sentence.

FIG. 4 is a schematic diagram showing the operation of steps 14, 15, 16, 17, and 18. When the number of feature expression dictionaries is n = 3, normalized normalized feature expression weight sums sum (1 to 3) are calculated. This is plotted on a number line. (1) and (2) in FIG.
If m (x ₁ )> s, and the sum of normalized feature expression weights is greater than {sum (x ₁ ) −s} and equal to or less than sum (x ₁ ), the feature of the input sentence judge. Also,
(3) is a case where sum (x ₁ ) ≦ s. In this case, it is assumed that the feature of the linguistic expression cannot be determined.

 Next, a specific example will be described.

Specific Example 1 A case will be described in which it is determined whether the linguistic expression of the sentence "You should do your best" is a male expression or a female expression using this device. However, the feature expression dictionary number n = 2, characterized representation 10 ₁ 5 view of male representation, wherein expression dictionary 1 female representation
0 ₂ and is as Figure 6, of FIG. 5 and in FIG. 6, "-" indicates that the item is blank. Hereinafter, the feature representation dictionary 10 ₁ of the male representation male representation dictionary, referred to as a feature representation dictionary 10 ₂ female surface and the female representation dictionary. Similarly, the morpheme numbers constituting the characteristic expression of the male (female) expression are the male (female) expression morpheme number, the characteristic expression weight of the male (female) expression is the male (female) expression weight, and the characteristic expression of the male (female) expression The total weight is referred to as a male (female) expression weight total. Also,
It is assumed that the feature expression threshold value s = 0.5.

First, the morphological analysis unit 1 morphologically analyzes the input sentence,
Notation, standard form, part of speech, and inflected form information are added to each morpheme in the input sentence. In this specific example, the message is divided into morphemes with the morpheme positive l = 5 and the number of phrases P = 2 represented by the notation "you", "mo", "do your best", "yo" and "." (Step 1). FIG. 7 shows an example of the morphological analysis result.

Next, 1 is set to the counter i (step 2), by operating the changeover switch 5 connects the characteristic expression extraction unit 2 and the male expression dictionary 10 ₁ (step 3).

The feature expression extraction unit 2 substitutes 1 for the counter j and 0 for the male expression weight sum value sum (1) (step 4).
Then, j = 1 th and subsequent morpheme string, ie, "You", "be", "Do your best,""good","." And to search match the representation from male representation dictionary 10 _1. When male expression dictionary 10 ₁ and is as Figure 5, since the fourth record is matched against "you" (step 5,6), the male representation morphemes number m = 1 in addition to j, sum ( The male expression weight w = 1.0 is added to 1) (step 7).

Next, a search is made from the male expression dictionary of FIG. 5 for expressions that match j = second and subsequent morpheme strings, that is, “mo”, “ganbare”, “yo”, and “.” (Step 5). Since (Step 6), 1 is added to j (Step 8).

Next, a search is made from the male expression dictionary of FIG. 5 for expressions matching j = third and subsequent morpheme sequences, that is, “Ganbare”, “Yo”, and “.” (Step 5). 6) So j is a male morphological morpheme m
= 2, and a male expression weight w = 1.0 is added to sum (1) (step 7).

Next, the morpheme sequence j = fifth or later, that is, an expression that matches "." Is searched from the male expression dictionary of FIG. 5 (step 5), and there is no match (step 6). 1 is added (step 8). Then j
= 6> l, the processing of the feature expression extraction unit 2 is terminated (step 9), and as a result, the male expression weight sum sum
(1) = 2.0 is obtained.

In step 10, 1 is added to the counter i. Then Therefore, the process jumps to step 3 (step 11) and operates the changeover switch 5 to operate the feature expression extraction unit 2 and the female expression dictionary 10.
₂ is connected (step 3).

The feature expression extraction unit 2, the same processing as performed targeting male expression dictionary 10 ₁ carried out to women expression dictionary 10 ₂ (step 4,5,6,7,8,9). Woman expression dictionary
If 12 is as shown in FIG. 6, there is no match with the morphological analysis result, and as a result, the female expression weight sum value sum (2) = 0 is obtained.

Then, when 1 is added to the counter i (step 10),
Since i = 3> n, the processing of the feature determination unit 3 is performed.

First, the feature determining unit 3 divides the total sum of all feature expression weights by the number of phrases P (= 2) to perform normalization (step 1).
2). Then, the normalized feature expression weight sum is sum
(1) = 1.0 and sum (2) = 0.

Next, n = 2 feature expression weight sums are sorted in descending order of value (step 13). In this specific example, sum
Since (1)> sum (2), the expression x ₁ = 1 having the largest total value of the feature expression weights and the expression x ₂ = 2 having the second largest total value of the feature expression weights are obtained.

Next, the largest feature expression weight sum value sum (x ₁ ) is compared with the feature expression threshold value s (step 14). In this specific example, since sum (1) ≧ s, the process proceeds to step 15.

Next, 1 is substituted for the counter k (step 15), and k ≦
n and sum (x ₁ )> {sum (x ₁ ) −s},
1 is added to the counter k (steps 16 and 17). As a result, since the conditional expression is not satisfied by k = 2, as a result, an input sentence "You also Ganbareyo.", The expression x ₁ =
It is determined that the sentence has the feature 1, that is, the feature of the male expression (step 18).

Specific Example 2 A case will be described in which the linguistic expression of the sentence "Please do your best also" is determined to be a male expression or a female expression using the present device. However, as with embodiment 1, wherein expression dictionary number n = 2, male expression dictionary 10 ₁ and FIG. 5, the female expression dictionary 10 ₂ is as Figure 6, wherein expression threshold s = 0.5 And

First, the morphological analysis unit 1 performs a morphological analysis on the input message. Then, the number of morphemes l = 6, the number of segments P = 2
Is obtained. FIG. 8 shows the results of the morphological analysis.

Then, in the same manner as the processing of feature expression extraction section 2 In Example 1, the fifth searched male expression dictionary 10 ₁ represented in the figure, but find a match, there is no match in the specific example Therefore, sum (1) = 0 is obtained as a result.

Subsequently, the feature expression extraction unit 2, similarly to the processing in Example 1, to find the female expression dictionary 10 ₂ represented by Figure 6 examines a match. In the case of this specific example, since the morpheme “you” only matches the sixth record of the female expression dictionary of FIG. 6, sum (2) = 0.3 is obtained as a result.

In the feature determination unit 3, first, the feature expression weight sum sum
Normalize (1) to sum (2) with the number of clauses P, and sum (1) =
0, sum (2) = 0.15 is obtained. Next, sum (1)-sum
(2) is sorted, and normalized feature expression weight sum su
The maximum value of m (1) to sum (2), that is, sum (2) = 0.1
The feature is determined from 5 and the feature expression threshold value s = 0.5. Since sum (2) <s, the input sentence "Please do your best," indicates that it is impossible to determine whether it is a male expression or a female expression. Become.

Example 3 A case will be described in which the linguistic expression of the sentence "Such a thing is not true" is determined using this device, whether it is a Tokyo dialect, a Kansai dialect, or a Kyushu dialect. However, the number of feature expression dictionaries n = 3
In, FIG. 9 is characterized expression dictionary 10 ₁ Tokyo valve, characterized expression dictionary 10 ₂ 10 view of Kansai dialect, the feature expression dictionary 10 ₃ Kyushu valve 11
As shown in the figure, "-" in FIGS. 9, 10, and 11 indicates that the item is blank. Hereinafter, similarly to the specific example 1, the feature expression dictionary of the Tokyo dialect is the Tokyo dialect dictionary,
The Kansai dialect feature expression dictionary is called the Kansai dialect dictionary, and the Kyushu dialect feature expression dictionary is called the Kyushu dialect dictionary. Similarly, the morpheme numbers that make up the feature expression of the Tokyo dialect (Kansai dialect, Kyushu dialect) are the Tokyo dialect (Kansai dialect, Kyushu dialect) morpheme number, and the characteristic expression weight of the Tokyo dialect (Kansai dialect, Kyushu dialect) is also the Tokyo dialect ( The sum of the characteristic expression weights of the Kansai dialect and Kyushu dialect and the weight of the characteristic expression of the Tokyo dialect (Kansai dialect and Kyushu dialect) are referred to as the Tokyo dialect (Kansai dialect and Kyushu dialect) weight. Also, the feature expression threshold value s is set to 0.5.

First, the morphological analysis unit 1 performs a morphological analysis on the input message. Then, a morpheme sequence with the number of morphemes l = 6 and P = 2 is obtained. Fig. 12 shows the results of the morphological analysis.

Next, the feature expression extraction unit 2, similarly to the processing in embodiment 1, those searching Tokyo valve dictionary 10 ₁ represented by FIG. 9, but find a match, a match in this example As a result, the sum of Tokyo valve weights sum (1) = 0 is obtained as a result.

Subsequently, the feature expression extraction unit 2, similarly to the processing in embodiment 1, searches the Kansai dictionary 10 ₂ represented by FIG. 10, checks a match. In this specific example, "Oh""He"
Since the morpheme string of "I" is only matched in the second record of the Kansai dialect dictionary 10 _2, as a result Kansai dialect weighted sum sum (2) = 1.0 is obtained.

Next, the feature expression extraction unit 2, similarly to the processing in embodiment 1, those 11 searches the Kyushu valve dictionary 10 ₃ represented in the figure, but find a match, a match in this example As a result, the Kyushu valve weight sum value sum (3) = 0 is obtained as a result.

In the feature determination unit 3, first, the feature expression weight sum sum
Normalize (1) to sum (3) with clause P, and sum (1) =
0, sum (2) = 0.5, and sum (3) = 0 are obtained. Next, the normalized feature expression weight sums sum (1) to sum (3) are sorted. Then, sum (2) = 0.5 is obtained as the maximum value. Since sum (2) ≧ s, sum (2) −s (=
0) When an expression having a larger normalized feature expression weight total value is examined, only the expression 2, that is, the Kansai dialect, is applied. Therefore, the input sentence “such a thing” is a sentence having the characteristics of the Kansai dialect. Is determined.

[Effects of the Invention] As described above, according to the present invention, it is possible to determine the characteristics of linguistic expressions independently of humans, and it is possible to reduce the labor required to determine the characteristics manually in the past. it can. Accordingly, in the message collection database search device shown in FIG. 15, the message searched as shown in FIG. 13 is expressed by the linguistic expression feature determination device of the present invention, for example, whether it is a male expression or a female expression. If the type of the language expression is automatically determined and displayed, it is not necessary to determine and store the type of the language expression in the message collection database in advance. As shown in FIG. 14, when a message is registered in the message collection database, the feature of the linguistic expression is automatically determined by the linguistic expression feature determining apparatus of the present invention, and the determination result is also registered. If you do
There is no need to manually determine the type of linguistic expression when registering a message in the database.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of a linguistic expression feature determination apparatus according to the present invention, FIG. 2 is a diagram showing an example of the configuration of one record of the feature expression dictionary, and FIG. FIG. 4 is a flowchart showing the operation of the linguistic expression feature determination apparatus. FIG. 4 is a diagram schematically illustrating the operation of the feature expression determination unit 3 when the number of feature expression dictionaries is n = 3. FIG. 6 is a diagram showing an example of a feature expression dictionary relating to female expressions, FIG. 7 is a diagram showing an example of a morphological analysis result of the sentence "You also do my best," and FIG. FIG. 9 shows an example of a feature expression dictionary for the Tokyo dialect, FIG. 10 shows an example of a feature expression dictionary for the Kansai dialect, and FIG. Fig. 11 shows an example of a feature expression dictionary for the Kyushu dialect. FIG. 13 is a block diagram showing an example of the result of morphological analysis of the sentence "Nanakoto Arahen." FIG. 13 is a block diagram showing a message collection database search device using the linguistic expression feature determination device according to the present invention, and FIG. FIG. 15 is a block diagram showing an example in which a message collection database construction apparatus is applied to the invention apparatus, and FIG. 15 is a block diagram showing a conventional message collection database search apparatus and construction apparatus. 1 ... morphological analysis unit, 2 ... feature expression extraction unit, 3 ... feature determination unit, 4 ... control unit, 5 ... changeover switch, 6 ...
Feature determining device for linguistic expression, 7: morpheme number constituting feature expression, 8: feature expression weight, 10 ₁ ... feature expression dictionary 1, 10 ₂ ... feature expression dictionary 2, 10 _n ... feature expression dictionary n,
20 ₁ … morpheme information 1, 20 _m … morpheme information m, 101…
Notation of morpheme, 102: Standard form of morpheme, 103: Part-of-speech of coarse form, 104: Utilization form of coarse form.

────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Masahiro Oku 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Noriyuki Horii 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation (56) References JP-A-63-278174 (JP, A) JP-A-63-68973 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 17/20-17/28

Claims (1)

(57) [Claims] 1. A plurality of feature expression dictionaries, each of which stores a sequence of morphemes including notations, standard forms, and parts of speech, and weights of the feature expressions, constituting a characteristic linguistic expression for each type of feature expression. A morphological analysis unit that divides the input sentence into morphemes and forms a sequence of the notation, standard form, part of speech, and inflected form of each morpheme. A feature expression extraction unit that matches the sequence with the feature expression dictionary, extracts an expression part characterizing the sentence, and obtains a total value of the weights of the feature expressions, and a plurality of feature expressions obtained from the feature expression extraction unit. A feature determination unit that determines a feature of the sentence from a difference in a total value of expression weights;