JP2579304B2 - Contracted word processor - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a language analyzer, and more particularly, to a contracted word processing device in language analysis such as a machine translation device.

Prior art When dealing with reduced forms (it'll, who'd, what's, etc.) of pronouns, relatives, interrogatives, etc. and auxiliary verbs, if they are handled as they are, Grammar rules are created only for the purpose, a new grammar category is also required, and it tends to lack consistency. Also, since it does not follow the grammar system, it is likely that humans will have greater difficulty tracking the analysis. In addition, as described above, the pronouns, relatives, interrogatives, etc., and the contracted forms (it's, what '
s) would require new grammatical symbol rules for analysis, which would require not only a major rework, but also a loss of uniformity of the grammar system, The likelihood of performing an analysis increases.

Objective The present invention has been made in view of the above-mentioned circumstances, and in treating a contracted word, it is not a contracted form, but by returning to the two words before the contracted form,
It is an object of the present invention to provide a contracted word processing device that eliminates grammatical modification and reduces the possibility of the above-described erroneous analysis.

A target language sentence is input, a dictionary search is performed for a word of the input target language sentence, and a dictionary search unit that determines whether or not there is an entry in the dictionary, and when the dictionary search unit determines that there is an entry, A dictionary registered word processing unit that extracts a pre-reduced form of a contracted word from a dictionary and processes the contracted word; and if the dictionary search unit determines that there is no entry, discriminates the type of the contracted word, and And a dictionary unregistered word processing unit that performs individual processing to return the contracted word to its pre-reduced form and performs the processing. Hereinafter, a description will be given based on examples of the present invention.

FIG. 14 is a diagram for explaining the entire structure of an English-Japanese automatic translation apparatus to which the contracted word processing method according to the present invention is applied. It goes without saying that the present invention can be effectively applied not only to a device but also to an analyzer of any language for analyzing a sentence of an input language when translating a certain language into another language.

In FIG. 14, 1 is an input section, 2 is an English text, 3
Is a pre-editing section, 4 is a morphological analysis section, 5 is a parsing I section, 6
Is a syntax analysis II unit, 7 is an operation display unit, 8 is a word dictionary, 9 is an analysis rule file, 10 is a control unit, 11 is a structure conversion unit, 12
Is a translated sentence generating unit, 13 is a post-editing unit, 14 is an output unit, 15 is a Japanese sentence, and the translator has an input unit 1 as shown in the figure.
Thus, the English text 2 to be translated into Japanese is input. The input unit 1 includes, for example, a keyboard having character keys such as alphanumeric keys and function keys, an optical character reading device (OCR) for reading English text recorded on paper,
And / or a file storage device for reading English text recorded on a recording medium such as a magnetic disk.

The English text input by the input unit 1 is read by the pre-editing unit 3 and pre-translation processing is performed. Here, recognition of sentences and processing of unknown words are mainly performed. This functions as part of the morphological analysis.

The pre-edited English sentence data is transferred to the morphological analysis unit 4 together with the information obtained by the pre-editing. The morphological analysis unit 4 indexes the word dictionary 8 and divides the sentence into sentences, analyzes the morphemes of the English sentence, performs various processing such as processing of unknown words, proper nouns, the surface of time, and expressions of numbers. , And performs the processing of the entire sentence such as recognition of the same rank. The morphological analysis rules are stored in the analysis rule file 9.

The English sentence data subjected to the morphological analysis is transferred to the syntax analysis unit 5 together with the dictionary information obtained by the morphological analysis. The syntactic analysis unit 5 is a functional unit that applies a grammar rule to English sentence data, analyzes the surface structure of the sentence, and finds all syntactic possibilities.

The English sentence data parsed by the parsing I unit 5 is sent to the parsing II unit 6 together with the parsing information. Here, a solution is selected by applying a structural description from the surface analysis result obtained by the analysis I. This creates a probable parse tree of the English description, and its structure. These structural analysis rules are also stored in the analysis rule file 9.

The parsed English sentence data is transferred to the structure conversion unit 11 as parse tree data. The structure conversion unit 11 creates a corresponding syntax tree of a Japanese sentence from a syntax tree that is an intermediate structure of the English sentence, and converts the Japanese sentence into a Japanese base structure that is easy to translate.

The syntax data indicating the Japanese base structure whose structure has been converted in this way is sent to the translated sentence generating unit 12, and the latter generates a translated sentence. This is a functional unit that generates a Japanese sentence from the tree structure of the Japanese syntax tree.

The translated sentence data, that is, the translated sentence data, is sent to the post-editing unit 13. The post-editing unit 13 uses the information used for the translation process to index the dictionary 8 to correct the translated sentence data, thereby completing a more natural Japanese sentence. The Japanese sentence data is transferred to the output word 14, and the translated Japanese sentence data is transferred to the output unit 14 and output from the output unit 14 as a translated Japanese sentence 15. The output unit 14 includes a file storage device such as a printer, a display, and / or a magnetic disk.

The flow of these series of translation processes is controlled by the control unit 10 which controls the entire control of the present apparatus. In the case of the illustrated example, the word dictionary 8 stores dictionary data for English and Japanese words, and describes not only vocabulary but also various kinds of information such as relations, that is, co-occurrence relations, meanings, singular words, parts of speech, and the like. Have been. The analysis rule file 9 stores rule data for morphological analysis and English sentence analysis.

The operation display unit 7 is connected to the control unit 10. The operation display unit 7 gives various instructions from the operator to the apparatus. For example, it has operation keys such as a translation instruction key and a cursor key, a display and an indicator for visually displaying input English sentence text, translation result Japanese sentence, intermediate data such as dictionary information, various instructions to an operator, and the like.
Many of these operation display functions may be configured to be included in the keyboard when the input unit 1 includes a keyboard, and included in the display when the output unit 14 includes a display.

According to the present invention, in the above-described machine translation apparatus, if there is a contracted word in the English text 2, it is processed without altering the grammar and without erroneous analysis. Table 1 shows an example of the correspondence between the contracted form (I) and the form before the contraction.

Table 1 In Table 1, 1 is a contraction of the first type (reduction of one word), 2 is a contraction of the second type, and 3 is a contraction of the third type (reduction of two words into words). ).

FIG. 1 is an overall block diagram for explaining an embodiment of contracted word processing according to the present invention, and FIG. 2 is a flowchart for explaining the operation of the circuit of FIG. Is an input processing unit, 21 is a unit cutout unit, 22 is a delimiter (de
limit) table, 23 is a dictionary search unit, 24 is a reference dictionary, 25
Is a dictionary registered word processing unit, 26 is a dictionary unregistered word processing unit, and 27 is a dictionary information storage table. When an English sentence text is input to the input processing unit 20 (step 1), the English sentence is cut out by the word cutout unit 21 in dictionary lookup units with reference to the delimit table 22 (step 2), and the end (en)
If not d) (step 3), the dictionary search unit 23 performs a dictionary search using the reference dictionary 24 (step 4). If there is an entry (step 5), the dictionary registered word processing unit 25 processes the dictionary registered word (step 3). step6), if there is no entry (s
tep5), the dictionary unregistered word processing unit 26 processes the dictionary unregistered words (step 7). If it is the end in step 3, the output is made (step 8).

Table 2 shows reference dictionaries from which, for example,
ain't is a contraction of am and not.

Table 3 shows that in the case of I ain't a worm., Ain't is composed of am and not, the start position of am is 3 (position of a), the end position is 4 (position of m), and not Indicates that the start position is 5 (the position of n) and the end position is 7 (the position of t). The start position and the end position are determined according to the calculation formula during the processing of the dictionary registration word ( However, if the value is not an integer, it is truncated.)

Table 4 shows an example of the case of I'm a worm.I would't ..., and from Table 1, I'm is the first type contracted form and wouldn't is the second type contracted form. It turns out that.

In the fourth, I of I'm is position 1-1 and 'm = a
m represents the position of 2-3, and wouldn't
Indicates that the start position (w) is 15, the end position (d) is 19, and n't = not indicates that the start position (n) is 20 and the end position (t) is 22. I have.

FIG. 3 is a block diagram of the dictionary registered word processing unit in FIG. 1, and FIG. 4 is a flowchart for explaining the operation of FIG. Each word form reading processing unit before contraction, 32 is a dictionary search unit for the word form before contraction, 33 is a position information adding unit, and 34 is a dictionary information storage table. If the contracted word is determined by the contracted word determination unit 30 to be a contracted word (step 1), and if the contracted word is registered in the dictionary, the word form reading processing unit 31 uses the dictionary information to process The word form is read (step 2, dictionary search is performed for each word form before contraction by the dictionary search unit 32 (step 2).
3). Next, the position information adding unit 33 adds position information to each word form so as not to overlap (step 4), and records each word form and its dictionary information in the dictionary information storage table 34 (st
ep5). If it is determined in step 1 that the word is not a contracted word, it is recorded in the dictionary information storage table 27 (step
6). In Note 1 of FIG. 4, the dictionary does not show the registration of the contracted form, and may all be registered in the contracted form table of the third type. Also, when performing a dictionary search for each word form before contraction, in Note 2, only constituent word forms are allowed in the dictionary, but each word form may have the position of each word form in the dictionary. Alternatively, it may have (copy of) the dictionary information of each word form. The processing in the dotted line in FIG. 4 is based on the assumption that the starting position is a, the ending position is b, and the number of constituent words is c. End position It is processed as.

FIG. 5 is a block diagram for explaining an example of a dictionary unregistered word processing section in FIG. 1, and FIG. 6 is a flowchart for explaining the operation of FIG. Matching unit, 41 is a contraction table of the third type, 42 is a matching unit with the tail, 43 is a contraction table of the second type, 44 is a matching unit with the tail, and 45 is a contraction of the first type Shape table, 46 is the first type processing unit, 47 is the second type
A type processing unit, 48 is a third type processing unit, and 49 is a dictionary information storage table. The first to third type contracted tables correspond to contracted forms (I) 1 to 3 shown in Table 1, respectively.
It corresponds to. The whole matching unit 40 performs matching with the third type reduced form table 41 as a whole (st
ep1), determine if there is any (step
2) If there is a corresponding one, the third type processing unit 48 performs the third type of contracted processing (step 3). Said s
In tep2, if there is no corresponding one, in the matching 42 with the tail part, the tail part is matched with the second type reduced form table 43 (step 4), and if there is a corresponding one (step 5), By the two-type processing unit 47,
A second type of contraction processing is performed (step 6). Step5
If there is no corresponding item in the above, then, in matching 44 with the end portion, matching is performed between the end portion and the first type contraction table 45 (step 7), and if there is a corresponding one (step 8), By the first type processing unit 46,
The first type contraction processing is performed (step 9). Step8
If there is no corresponding word in, it is recorded in the dictionary information storage table 49 as a dictionary unregistered word.

FIG. 7 is a block diagram for explaining an example of a processing section for the first type of contracted words, and FIG. 8 is a flowchart for explaining the operation of FIG. A search unit for searching in a non-shaped form, 51 is a contractible word table (the first type of supplementary table in Table 1), 52 is a search unit for performing a dictionary search without a reduced part, and 53 is a contracted part. An extraction unit for extracting a shape before contraction, 54 is a shape table before contraction (a first type (II) table of the first type in Table 1), and 55 is a retrieval unit for searching a dictionary in a form before contraction. , 56 are dictionary information storage tables, and this processing is applied to a type of contraction in which words that can be attached to the contracted portion are limited to a certain range of words. First, the search unit 50 removes the contracted part,
The contractable word table 51 is searched (step 1). next,
It is determined whether the word is a contractible word (step 2). If No, the entire word is recorded in the dictionary information storage table 56 as an unregistered word (step 3). If yes in step 2 above,
The search unit 52 performs a dictionary search with the reduced part removed (step 4), and records the part with the reduced part removed in the dictionary information storage table 56 (step 5). Next, the extraction unit 53
By using the table 54, the shape (s) of the contracted portion before contraction can be
(Step 6), and the search unit 55 performs a dictionary search in each of the forms before the contraction (step 7). Next, the shapes before reduction are recorded in the dictionary information storage table 56 (step
8). Note that, in Note 1 in FIG. 8, words registered in the dictionary are not allowed to be placed in these tables,
Naturally, these tables themselves can store the position of information in the dictionary or (copy of) the dictionary information itself. In Note 2, Apostrophe's
May be performed. In Note 3, the position information is the starting position and the ending position of the part where the contracted form is removed, and in Note 4, the positional information (all (even if there are plural)) is the starting position of the contracted part, End position.

FIG. 9 is a block diagram for explaining an example of processing of the second type of contracted words, and FIG. 10 is a flowchart for explaining the operation of FIG. A search unit that performs a dictionary search without the shape part, 61 is a dictionary information storage table, 62 is an extraction unit that extracts a shape before contraction, 63 is a shape table before contraction (the second type in Table 1). Form table before contraction), 64 is a search unit that performs dictionary search in the form before contraction. This processing is applied when processing unlimited words that can be attached to the contracted part. It is. First, search unit 60
To search the dictionary without the contracted part (step
1), it is determined whether there is an entry (step 2). No
If it is, the contracted part is removed and recorded in the dictionary information storage table 61 as an unregistered word (step 3). If Yes, the form with the contracted part removed is recorded in the dictionary information storage table 61. (Step4). Next, the extraction unit 62 extracts the contracted form (pluralization) of the contracted part from the table 63 (st
ep5), the search unit 64 performs a dictionary search in the form before each reduction (step 6). Next, the data is recorded in the dictionary information storage table 61 in each form (step 7). Note that, in Note 1 in FIG. 10, it may be unregistered as a whole, but in that case, go directly to end. In Note 2, the position information is the start position and the end position of the part in which the contracted form is removed, and in Note 3, the position information is the start position and the end position of the contracted part (in any case). Note 4 is the same as Note 1 in the first type contracted word processing described with reference to FIG.

FIG. 11 is a block diagram for explaining an example of a processing unit for the third type of contracted words, and FIG. 12 is a flowchart for explaining the operation of FIG. Extraction unit for extracting each word form before the previous, 71 is a contracted table (third type of contracted table in Table 1), 72 is a dictionary search unit, 73 is a position information adding unit, and 74 is a dictionary information storage table In this third
The type contraction is a special contraction method, and the entire contraction must be registered in a table. However, in the present invention, the contraction processing is the same as the processing of contracted words registered in a dictionary. Therefore, there is no essential difference between the contraction of the third type and the registration in the dictionary, and only one of them is allowed.
It is also possible to omit the processing which is not permitted. First, each word form before reduction is extracted from the contracted form table 71 by the extraction unit 70 (step 1), and a dictionary search is performed on each word form by the dictionary search unit 72 (step 2). Next, the position information giving unit 73 gives position information to each word form (step 3), and records each word form in the dictionary information storage table 74 (step 4). Note that, in Note 1 in Fig. 12, each word form is only allowed to be registered in the dictionary.
Are the same as those in the contracted form registered in the dictionary.

FIG. 13 is a detailed block diagram showing an embodiment of a contracted word processing unit according to the present invention. Since each block in the figure has already been described in FIGS. The same reference numerals as those used in FIGS. 1 to 12 denote the same reference numerals, and a detailed description thereof will be omitted.

Effect As is apparent from the above description, according to the present invention, in processing a contracted word, not only the contracted word but also the two words before the contracted word form are processed. , Grammar and the like need not be modified, and processing can be performed with a reduced possibility of erroneous analysis.

[Brief description of the drawings]

FIG. 1 is an overall block diagram for explaining an embodiment of a contracted word processing apparatus according to the present invention, FIG. 2 is a flowchart for explaining the operation of FIG. 1, and FIG. Block diagram for explaining an example of the registered word processing unit, FIG.
The figure is a flowchart for explaining the operation of FIG. 3,
FIG. 5 is a block diagram for explaining an example of a dictionary unregistered word processing section, FIG. 6 is a flowchart for explaining the operation of FIG. 5, and FIG. 7 is a first type contraction. FIG. 8 is a block diagram for explaining an example of a word processing unit.
7 is a flow chart for explaining the operation of FIG. 7, FIG. 9 is a block diagram for explaining the processing unit of the second type contracted word, and FIG. 10 is for explaining the operation of FIG. FIG. 11 is a block diagram for explaining an example of a processing unit of the third type contracted word, and FIG.
FIG. 13 is a flowchart for explaining the operation of FIG.
FIG. 14 is a detailed block diagram of the contracted word processing unit according to the present invention, and FIG. 14 is a diagram showing the entire configuration of an automatic translation apparatus to which the contracted word processing according to the present invention is applied. 20 input processing unit, 21 unit cutout unit, 22 delimiter table, 23 dictionary search unit, 24 reference dictionary, 25
… Dictionary registered word processing unit, 26 …… Dictionary unregistered word processing unit, 27 ……
Dictionary information storage table, 30... Contracted word determination unit, 31... Each word form reading processing unit before contraction, 32... Dictionary search unit for word form before contraction, 33. ... Dictionary information storage table, 40 ... Overall matching unit, 41 ... Third type contraction type table, 42 ... Matching unit with end part, 43 ...
… The second type contracted table, 44… matching part with the end part, 45… the first type contracted table, 46…
First type processing unit, 47 ... Second type processing unit, 48 ...
Processing unit of the third type, 4 ... Dictionary information retrieval table, 50 ...
… Retrieval part for searching in a form without contraction, 51 …… Reducible word table, 52… Search part for dictionary search in a form without contraction part, 53 …… Retrieval form before contraction Extraction unit to extract, 54 ...
… A shape table before contraction, 55… a search unit that searches the dictionary in a form before contraction, 56 …… a dictionary information storage table, 60 …… a search unit that performs a dictionary search without the contraction part, 61 …… Dictionary storage table, 62 …… Extraction unit that extracts the shape before contraction, 63 ……
Shape table before contraction, 64 Search unit for dictionary search in form before contraction, 70 Extraction unit for extracting each word form before contraction, 71 ...
Reduced form table, 72: Dictionary search unit, 73: Position information adding unit, 74: Dictionary information storage table.

Claims (1)

(57) [Claims] 1. A dictionary search unit for inputting a target language sentence, performing a dictionary search for words of the input target language sentence, and determining whether or not there is an entry in the dictionary. When it is determined, each word form before the contraction is read, a dictionary search is performed on the read word form before the contraction, and a dictionary is attached with position information so as not to overlap each word form before the contraction. A dictionary registration word processing unit for recording in a storage table, and a plurality of contraction type tables having different contraction types, wherein when the dictionary search unit determines that there is no entry, a contraction of a dictionary unregistered word Compare the whole or the end part of the contracted form with the contracted form of the contracted form table for each type,
The matched type is discriminated as the type of the given contracted form, the type is returned to the pre-reduced word form for each type, and location information is added so as not to overlap the pre-reduced word form, and the dictionary storage table And a dictionary unregistered word processing unit for recording in a dictionary.