JPH0827804B2 - Japanese dictionary data management method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Field of Industrial Application) The present invention is a Japanese dictionary data management system for managing data of a Japanese dictionary for extracting words from Japanese sentences in Japanese processing by a computer. More specifically, the present invention relates to a Japanese dictionary data management method for reducing the search time for words by classifying the Japanese dictionary into a plurality of dictionaries according to the length of the words.

(Prior Art) A conventional method of classifying Japanese sentences into words is to register all the words in one Japanese dictionary and divide the Japanese sentences into words using this dictionary.

FIG. 4 is an explanatory diagram of a conventional method for extracting a word having two or more characters from a Japanese sentence. In the figure, 1 is a target character string for word candidate extraction, 2 is a word candidate string exhaustively extracted from this target character string, and 3 is a Japanese word having the longest word length as a field length. The index key part of the dictionary (the length is n characters), and 4 is the data part.

FIG. 5 is a flow chart showing a schematic process of word extraction in the method shown in FIG. As shown in FIG. 4 and FIG. 5, the word extraction is performed comprehensively by matching the heading with the target character string 1 for word candidate extraction, increasing one character from two characters. In this conventional method, at the time of this matching, a character string of any length has to be adjusted to n characters of the length of the index key portion 3 of the dictionary, so a blank is added to the part that is insufficient for n characters. Processing such as filling is required. Further, generally, the longer the key length, the longer the time required for matching.

In FIG. 4, “patent” in the word candidate character string 2
When searching the Japanese dictionary with the key as a key, a blank of (n-2) characters is padded after the character string "patent" to match the heading key length of the Japanese dictionary of n characters, and this "patent" A Japanese dictionary is searched using a character string of n characters including "," and "patent" is not found as a word until the search is successful. Next, in order to check whether or not the character string “patented”, which is one character longer than “patent”, is a word, as in the case of “patent”, a blank of (n−3) characters is filled,
Search Japanese dictionary. Since the character string "patented" is not registered in the Japanese dictionary, it turns out that it is not a word. Hereinafter, the same processing is performed for the character strings “grant”, “application”, and “applicant”. As a result, it is found that the three word candidate strings of “patent”, “application”, and “applicant” are registered in the Japanese dictionary, and the word is recognized as a word.

(Problem to be Solved by the Invention) As described above, in the conventional method, it is necessary to match the key length of the heading with the longest heading key length of the Japanese dictionary even for a short word candidate string. There is a problem that it takes a long search time corresponding to the longest heading key length even for a short word candidate string, and there is a lot of useless empty area in the key portion, which is inefficient.

The present invention has been made in view of the above, and an object of the present invention is to provide a Japanese dictionary data management method in which there is little wasted empty area and the search time is short.

[Structure of Invention]

(Means for Solving the Problem) In order to achieve the above object, the Japanese dictionary data management method of the present invention is a classification registration in which each word is classified and registered in a plurality of word dictionaries according to the number of characters forming the word. Means and, in the word dictionary with a short number of characters in the word dictionary with a long number of characters in the word dictionary having the same word as the first character that does not exist in the word dictionary with a short number of characters in the word dictionary, Additional registration means for additionally registering the word that does not exist in the word dictionary having a shorter number of characters, and a word that includes the same word as the first character in the word dictionary having a shorter number of words in the plurality of word dictionaries as the first character When present in a word dictionary with a long number of characters, the next search dictionary type information indicating the word dictionary with the shortest number of characters in the word dictionary with a long number of characters is set in the word dictionary with a short number of characters. The gist is to have a dictionary type information setting means.

(Operation) In the Japanese dictionary data management method of the present invention, each word is classified and registered in a plurality of word dictionaries according to the number of characters of the word, and a word including the same word as the first character that does not exist in the word dictionary with a short number of characters as the first character Is present in the long word dictionary,
When the word is additionally registered in a word dictionary with a short number of characters, and a word that includes the same word as the first character in the word dictionary with a short number of characters as the first character is present in the word dictionary with a long character number, even in the word dictionary with a long character number, Next search dictionary type information indicating the word dictionary having the shortest number of characters is set in the word dictionary having the shortest number of characters.

(Examples) Examples of the present invention will be described below with reference to the drawings.

FIG. 1 is an explanatory diagram of a Japanese dictionary data management system according to an embodiment of the present invention, and FIG. 2 is a flow chart showing the operation of the Japanese dictionary data management system of FIG. In FIG. 1, 1 and 2 are target character strings for word candidate extraction, respectively, as described above, and 2 is a word candidate string exhaustively extracted from this target character string. In the present embodiment, the word dictionary is configured by being divided according to the length of the word characters. For example, as shown in the figure, a two-character word dictionary 10 consisting of two-character words and three words consisting of three or more character strings. It is structured like a word dictionary 11 of characters or more.

In the 2-character word dictionary 10, reference numeral 5 is a heading key portion for the 2-character word dictionary 10 and its length is 2 characters. 6 is the next search dictionary type information, and when the two characters of the heading key portion 5 of the two-character word dictionary 10 corresponding to this information are three or more characters in the first two characters of the word dictionary 11, the next search dictionary is indicated. Type information 6 is set to "1" and a word dictionary of 3 or more characters
When it is not in the first two characters of 11, the next search dictionary type information 6 is set to "0". Furthermore, 7 is a two-letter word dictionary.
It is the data part of 10.

Further, in the word dictionary 11 having three or more characters, 8 is a heading key part and 9 is a data part.

Next, the operation will be described with reference to the flowchart shown in FIG.

In FIG. 2, the target character string 1 is set to 1, and the first character position i of the character string to be searched in the target character string 1 of this character string 1 is set to 1 (i = 1). The search key length is set to 2 (steps 110 and 120). Further, the i-th character to the j-th character of the target character string 1 are set as the first search key (step 130). Also, j is initially set to 2, and in the next step 140 it is checked whether j is greater than 2 (j> 2), and the two-letter word dictionary 10 or the three or more-letter word dictionary 11 is searched according to this result. To be done.

Specifically, in the case shown in FIG. 1, the first (i = 1) to the second in the target character string 1 for word candidate extraction.
The first two characters “patent” corresponding to the th (j = 2) are extracted as the word candidate string 2. And j in this case is 2
Since it is larger, the process proceeds to step 150, and the two-character word dictionary 10 using the character string "patent" of this word candidate string 2 as a search key.
To search. In this case, the key length of the 2-character word dictionary 10 is 2
Since it is a character, it can be searched by "patent", and it is not necessary to add a blank as in the conventional case.

When the 2-character word dictionary 10 is searched using "patent" as a search key, "patent" is registered as a word, and therefore a perfect match is found (step 160). Therefore, all of the completely matching words are searched for in the next search. (Step 170). Then, it is checked whether or not the next search dictionary type information 6 of this searched word is "1", that is, whether or not a word having "patent" as the first two characters is present in the word dictionary 11 with three characters or more. Yes (step 180). The next search dictionary type information 6 of "patent" is "0", and is a word dictionary of three or more characters.
It can be seen that there is no word in 11 that has "patent" as the first two letters. Also, it can be seen that "patented" in the word candidate sequence 2 is not in the word dictionary 11 more than 3 characters,
It is not necessary to perform a dictionary search for this "patented".
Further, as for the next "permit", it is clear that it is not a word because it is found that it is not registered when the two-character word dictionary 10 is searched.

Further, regarding the “application”, it is recognized by the same search processing as described above that the “application” is registered in the two-character word dictionary 10, and the next search dictionary type information 6 of the “application” is It can be seen that there is a word having “application” as the first two characters in the word dictionary 11 that is recognized as “1” and has three or more characters. Therefore, the search key length in this case is set to 3, and the word dictionary 11 having 3 or more characters is searched using "applicant", which is one character added to "application", as a search key (step 190 and thereafter). As a result, since "applicant" is registered in the word dictionary 11 of 3 characters or more, "applicant" is registered as a word.

As described above, 3 of “patent”, “application”, and “applicant”
One is recognized as a word. At this time, for two words of "patent" and "application", it is sufficient to search with two characters as a key, so the search time is very short compared to the conventional method.

In order to confirm the effectiveness of the Japanese dictionary data management method of this embodiment, a kanji term dictionary (words of two or more kanji characters),
The Japanese dictionary data management method of the present embodiment is applied to a system for performing morphological analysis of Japanese sentences using five dictionaries including a one-character kanji dictionary, a katakana dictionary, a hiragana dictionary, and a mixed-writing dictionary, and a conventional kanji term dictionary is applied. Is divided into two, a two-letter word dictionary and a three-letter or more word dictionary, and the results of comparing the dictionary size and the morphological analysis time before and after this division are as follows.

(1) Comparison of dictionary size Kanji term dictionary before division ・ ・ ・ 16896000 bytes Total of two dictionaries after division ・ ・ ・ 15633408 bytes (2) Comparison of morphological analysis time 39 per sentence arbitrarily extracted from the manual The results of comparing 20 sentences with the average character length are as follows.

Morphological analysis time before division ・ ・ ・ 177.7 seconds Morphological analysis time after division ・ ・ ・ 148.7 seconds Since it is more than 148.7 seconds, by using this Japanese dictionary data management method for Japanese morphological analysis, the dictionary size is reduced. 8% reduction and about 16 processing time required for morphological analysis
% Reduction.

FIG. 3 is an explanatory diagram of a Japanese dictionary data management system according to another embodiment of the present invention. In the embodiment shown in the figure, the nine words shown in the figure are converted into a two-letter word dictionary, a three-letter word dictionary, and a four-letter word dictionary.
A process for registering more than characters in the word dictionary will be described.

In FIG. 3, 12 is a heading key part of the two-character word dictionary, 13 is next search dictionary type information for a three-character word dictionary or a four-character or more word dictionary, and is in the heading key part 12 of the two-character word dictionary. If two characters are in the first character in the three-character word dictionary, "3" is set, and if four or more characters are in the first two characters in the word dictionary, "4" is set,
If neither exists, "0" is set. 14 is the data part of the two-letter word dictionary.

In the 3-letter word dictionary, 15 is a heading key part,
16 is the next search dictionary type information for the word dictionary of 4 characters or more, and 2 characters in the index key part 12 of the 3-character word dictionary are 4 characters.
If there are more than one character in the first two characters in the word dictionary, "4" is set, and if not, "0" is set.
Reference numeral 17 is a 3-character word dictionary data section.

In the word dictionary of four characters or more, 18 is a heading key part and 19 is a data part.

Nine words to register, namely AA, AAB, AB, ABA, ABAC, ABACD
E, AE, AFE, and AFED are divided and registered in either a two-character word dictionary, a three-character word dictionary, or a four-character or more word dictionary according to the character length. At this time, the next search dictionary type information of the 2-character word dictionary and the 3-character word dictionary is not set.

Next, a character string that is a word registered in the 3-character word dictionary and a word string of 4 characters or more and whose first two character strings are not registered in the 2-character word dictionary is detected.
In FIG. 3, the character string "AF" corresponds to this character string. Therefore, this character string "AF" is registered as a dummy record in the heading key part of the two-character word dictionary.

Finally, the next search dictionary type information is set. For example,
Since “AF” of the 2-character word dictionary is included in “AFE” of the 3-character word dictionary and “AFED” of the word dictionary of 4 characters or more, the next search dictionary type information 13 of the 2-character word dictionary is ,
“3” is set, which indicates the three-letter word dictionary with the shorter word length. Further, since “AA” of the two-character word dictionary is included in “AAB” of the three-character word dictionary, “3” is set in the next search dictionary type information 13. The same applies to "AB" in the two-letter word dictionary. Since “AE” of the two-letter word dictionary is not included in other dictionaries, “0” is set in the next search dictionary type information 13. Furthermore, since "AAB" of the 3-character word dictionary is not included in the word dictionary of 4 characters or more, "0" is set in the next search dictionary type information 16. "AB in the 3-letter word dictionary
“A” is a word dictionary of 4 or more letters “ABAC” and “ABACDE”
Therefore, “4” is set in the next search dictionary type information 16. Since “AFE” of the 3-character word dictionary is included in “AFED” of the word dictionary of 4 characters or more, “4” is set in the next search dictionary type information 16.

As described above, the two-letter word dictionary shown in FIG.
A character word dictionary and a word dictionary of 4 or more characters are created.

〔The invention's effect〕

As described above, according to the present invention, each word is classified and registered in a plurality of word dictionaries according to the number of characters of the word, and the number of words whose first character is the same word as a word that does not exist in the word dictionary with a short number of characters is the number of characters. Existing in the word dictionary with a long number of characters, the word is additionally registered in the word dictionary with a short number of characters, and a word including the same word as the word existing in the word dictionary with a short number of characters as the first character exists in the word dictionary with a long number of characters. Since the next search dictionary type information indicating the word dictionary with the shortest number of characters in the long word dictionary is set in the short word dictionary with the shortest number of characters, the length of the word is searched for in the word dictionary when searching for the word. Since it is not necessary to match the key length and there is no empty area in the key portion of the word dictionary, it is possible to improve the usage efficiency and shorten the search time.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of a Japanese dictionary data management system according to an embodiment of the present invention, FIG. 2 is a flow chart showing the operation of the Japanese dictionary data management system of FIG. 1, and FIG. FIG. 4 is an explanatory view of a Japanese dictionary data management method according to the embodiment of the present invention, FIG. 4 is an explanatory view of word extraction by a conventional method, and FIG.
The figure is a flow chart showing the operation of FIG. 1 …… Target character string, 2 …… Word candidate string, 5,8 …… Heading key part, 6 …… Next search dictionary type information, 7,9 …… Data part, 10 …… 2-character word dictionary, 11… … A word dictionary of 3 or more characters.

Continuation of front page (51) Int.Cl. ⁶ Identification number Office reference number FI technical display location 9288-5L G06F 15/20 520B

Claims (1)

[Claims] 1. A classification / registration means for classifying and registering each word in a plurality of word dictionaries according to the number of characters forming the word, and a word not existing in a word dictionary having a short word length among the plurality of word dictionaries. When a word including the same word as the first character exists in a word dictionary having a long number of characters, the word that does not exist in the word dictionary having a short number of characters is additionally registered in the word dictionary having a short number of characters; Of the word dictionary with the shortest number of characters in the word dictionary, if the word that contains the same word as the first character in the word dictionary with the longest number of characters exists in the word dictionary with the longest number of characters, the shortest number of characters in the word dictionary with the longest number of characters And a next search dictionary type information setting means for setting the next search dictionary type information indicating the word dictionary in the word dictionary having the short number of characters.