JP3847812B2 - Character processing method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a character processing method and apparatus, and in particular, a character processing method having a function of narrowing down the phrase structure of a target sentence for Japanese sentences and checking the phrase structure by phrase dependency and the words constituting the phrase It relates to the device.
[0002]
[Prior art]
Conventionally, as a character processing device with a function to analyze the phrase structure of an input sentence for Japanese sentences, Kana-kanji that inputs kana reading, analyzes the phrase structure, and converts it into an appropriate kanji character string There is a conversion device.
In kana-kanji conversion, input reading sequences are analyzed to create possible phrase candidates, conversion candidates are determined from the combinations, and presented in the most likely order. A candidate desired by the operator is selected from the presented conversion candidates. In order to increase the likelihood of conversion candidates presented as the first candidate, conventionally, a method of narrowing down conversion candidates by using different parts of speech or a method of arranging them in the order of word frequencies have been implemented.
[0003]
In addition, an example in which a pair of examples such as “read a book” is registered in the example dictionary in advance, and the first candidate is determined according to the example in the example dictionary by referring to the example dictionary at the time of conversion. The technique of conversion has been implemented. In addition to examples that describe the relationship between words such as “read a book”, examples that describe relationships with semantic classifications that group words together are proposed. For example, if the example of “write on [concrete object]” is registered, this example is applied to “paper” which is [concrete object], and the first conversion candidate “write on paper” is obtained. Can do. By selecting a semantically possible word pair as an example to be registered in the example dictionary and using this example conversion mechanism, the possibility of obtaining a correct conversion result has increased.
[0004]
Furthermore, a method (Japanese Patent Publication No. 5-63831) has been proposed in which example words that have not been registered in advance and are stored together with the relationship are learned according to the input of the input person. According to this, for example, a combination of the words “submarine resources” can be stored as an example, and erroneous analysis such as “revised resources” can be avoided.
[0005]
[Problems to be solved by the invention]
However, in the conventional example learning, only the combination of individual word pairs is learned, so the memory capacity required for storage becomes enormous and it is difficult to hold all of the learning results. It was.
Moreover, the effect of analyzing the relationship between clauses by example learning is limited to only the learned individual word relationship examples, and the effect was not obtained for other words having the same relationship.
[0006]
In addition, as the number of stored examples increases, the search time increases and processing time for analyzing the relationship between clauses increases, which makes it difficult to achieve high-precision analysis efficiency at low cost. .
The present invention has been made in view of the above conventional example, and an object of the present invention is to provide a character processing method and apparatus capable of high-speed text conversion with high conversion accuracy.
[0007]
[Means for Solving the Problems]
  In order to achieve the above object, a character processing method and apparatus according to the present invention comprise the following arrangement. That is, with wordsTheCombination information of words that have affinity with wordsMemorize as individual exampleDoIndividualExample dictionary, word,TheWord and affinityThe meaning concept of a wordCombination withAs a semantic example, the combinationConfidence inMeaning to remember togetherExample dictionary and the abovemeaningExample dictionaryFor example meaning ofSaidIndividualExample dictionaryIn individual examples of,Corresponds to the combination of the semantic example word and semantic conceptExamples andCorresponds to a combination of homophones and semantic concepts of the word in the semantic exampleCounter-use exampleWhenIn the ratio of the number of the extracted example and the counter example, the example extracting means for extractingAntiIf the proportion of examples is greater than a predetermined threshold,meaningThe example dictionarySemantic examplesCertainty corresponding toLowerConfidence adjustment means,As a result of the adjustment by the certainty factor adjusting unit, the example invalidating unit for invalidating the semantic example in which the certainty factor is equal to or less than a predetermined value;
Individual applicable to the phrase sequence enteredexampleOr semantic examplesBased on the phrase sequence,When an individual example is applicable, it is converted into a conversion candidate using a combination of words corresponding to the individual example. When a semantic example is applicable, a word corresponding to a combination of a word of the semantic example and a semantic concept is converted. Convert to conversion candidate using combinationConversion means.
[0008]
  Further, another invention is an individual example dictionary storage means for storing, as an individual example, combination information of a word and a word having affinity with the word,
  A semantic example dictionary storing means for storing a combination of a word and a semantic concept of a word having affinity with the word as a semantic example, and a processing means for executing various processes based on a program;An example extracting means, a certainty factor adjusting means, an example invalidating means, and a converting means, which are realized by cooperation of the processing means and the program storing means.A character processing method for processing characters in a character processing device comprising:
  SaidThe example extraction meansWith respect to the semantic examples stored in the semantic example dictionary storage means, among the individual examples stored in the individual example dictionary storage means, examples corresponding to combinations of words and semantic concepts of the semantic examples and the semantic examples An example extraction step for extracting a reusal example corresponding to a combination of a word homophone and a semantic concept;
  SaidThe confidence adjustment meansIn the ratio of the number of extracted examples and counterexamples, if the ratio of the counterexamples is larger than a predetermined threshold, the certainty factor that lowers the certainty factor stored in the semantic example dictionary storage unit corresponding to the semantic example Adjustment process;
  SaidExample invalidation meansAs a result of the adjustment in the certainty factor adjustment step, an example invalidation step of invalidating a semantic example whose certainty factor is equal to or less than a predetermined value in the semantic example dictionary storage unit;
  SaidConversion meansIndividual examples or semantic examples applicable to the input phrase string are retrieved from the individual example dictionary storage means and the semantic example dictionary storage means, and the phrase string corresponds to the individual example when the individual examples are applicable Converting to a conversion candidate using a combination of words, and when a semantic example is applicable, converting to a conversion candidate using a word combination corresponding to the combination of the word of the semantic example and the semantic concept, Prepare.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
The point of this embodiment is that each example learned as relation data between clauses is abstracted for each example having a common semantic classification, registered as a semantic classification example, and an abstract semantic classification example When the result analyzed by application is reversed, the semantic category non-applicable phrase pair is registered as a counterexample, the reliability index value of the abstract example is obtained by the number of counterexamples, and the counterexamples are individually used. It is an object of the present invention to provide a character processing device that performs control so that abstraction to a semantic classification example can be dynamically changed by using it as an index value for changing an abstraction activation condition.
[0010]
In the following, a character processing method according to an embodiment of the present invention and one of the points of the apparatus will be summarized first and then will be described in detail.
In the character processing method of the present embodiment, the learned individual examples are abstracted and registered as semantic classification examples, and whether or not the registered semantic classification examples are satisfied is determined by analyzing the inter-phrase dependency.
[0011]
For this reason, even if it is not registered as an individual example, a semantically valid phrase structure can be selected.
In addition, it is possible to improve the analysis accuracy of inter-phrase dependency.
Furthermore, by abstracting and registering as semantic classification examples, the memory capacity can be reduced, and this can be realized at low cost.
[0012]
In addition, phrase pairs that do not match the abstracted semantic classification examples can be registered as counterexamples, and the timing of abstraction processing can be dynamically changed with respect to the input depending on the number of counterexamples.
Hereinafter, in the character processing method and apparatus of the present embodiment, in particular, a case where Kana-Kanji conversion is performed using an input target as a Japanese kana reading sequence will be described in detail with reference to the drawings.
[0013]
FIG. 1 is an example of the overall configuration of the present invention.
In the configuration shown in the figure, the CPU is a microprocessor that performs operations for character processing, logical determination, and the like, and is connected to these buses via an address bus AB, a control bus CB, and a data bus DB. Control elements.
The address bus AB transfers an address signal indicating a component to be controlled by the microprocessor CPU. The control bus CB sends a control signal to each component to be controlled by the microprocessor CPU. The data bus DB transfers data between the component devices.
[0014]
Next, the ROM is a read-only fixed memory. PA is a program area in which a control procedure of a microprocessor CPU, which will be described later with reference to FIGS.
The RAM is a writable random access memory having a configuration of 16 bits per word, and is used for temporary storage of various data for each component.
[0015]
WDIC is a word dictionary for performing kana-kanji conversion. FIG. 6 shows the configuration.
YRTBL is an example dictionary that stores combinations of clauses. FIG. 7 shows the configuration.
YBUF is an input reading buffer memory for storing readings input from the keyboard KB.
[0016]
TBUF is a document buffer and is a memory for storing document information input from the keyboard KB.
BCT is a phrase candidate table that stores phrase candidates in the middle of kana-kanji conversion. FIG. 3 shows the configuration.
DBF is a homophone buffer memory that stores unconfirmed phrase candidates for Kana-Kanji conversion. FIG. 4 shows the configuration.
[0017]
DBP is a homonym pool memory that stores other candidates for kana-kanji conversion corresponding to DBF. FIG. 5 shows the configuration.
T and K are counter memories.
KB is a keyboard, and includes various function keys such as character symbol input keys such as alphabet keys, hiragana keys and katakana keys, and conversion keys for instructing conversion. In FIG. 1, YOMI is a key for inputting a reading, CON is a conversion instruction key for converting the input reading, NXT is a next candidate conversion instruction key for changing a conversion candidate and converting it to a next candidate, SEL Is a selection key for instructing to confirm the current homophone display candidate and learn as an example at the same time.
[0018]
The DISK is an external memory for storing a standard document, and the created document is stored. The stored document is called when necessary by an instruction from the keyboard.
CR is a cursor register. The contents of the cursor register can be read and written by the CPU. A CRT controller (CRTC), which will be described later, displays a cursor at a position on the display device (CRT) corresponding to the address stored here.
[0019]
The DBUF is a display buffer memory and stores display patterns corresponding to document information and the like stored in the TBUF.
The CRTC plays a role of displaying the contents stored in the cursor register (CR) and the buffer (DBUF) on the display CRT.
The CRT is a display device using a cathode ray tube or the like, and the dot configuration pattern and cursor display on the display device CRT are controlled by a CRT controller.
[0020]
Further, the CG is a character generator and stores a pattern of characters and symbols to be displayed on a display device (CRT).
The character processing apparatus according to the present embodiment composed of such components operates in response to various inputs from the keyboard KB. When an input from the keyboard KB is supplied, an interrupt signal is first generated. Various control signals sent to the microprocessor (CPU) and stored in the ROM are read out, and various controls are performed in accordance with these control signals.
[0021]
FIG. 2 is a diagram showing the configuration of the input reading buffer YBUF in the character processing device of this embodiment.
YBUF is a buffer memory for storing readings input from KB. In the present embodiment, up to 512 characters can be stored with a JIS code of 2 bytes per character. 0 is stored as a blank code indicating no input in an area where no reading is input.
[0022]
FIG. 3 is an example of the phrase candidate table BCT in the character processing device of this embodiment. The phrase candidate table is a table expressing a string of phrases that can be analyzed from the beginning to the end of the input reading string in a tree structure with one phrase candidate as a node, and is composed of five fields.
In FIG. 3, BCID is an ID (identification) number that makes each phrase candidate unique.
[0023]
BCJW represents a self-supporting part word of a phrase candidate and stores a pointer to the word dictionary WDIC in which the self-supporting part word exists.
BCFW stores an attached word string that constitutes an attached part of a phrase candidate. The attached word string stores the attached word start position and attached word end position as an index on the input reading buffer YBUF.
[0024]
In FIG. 3, “φ” means that there is no attached word string.
SLNK links another phrase candidate having the same phrase start reading position as that phrase candidate, and stores a phrase candidate ID of a link destination. The phrase candidate ID “−1” having no SLNK value means a termination code, and indicates that there is no more same reading start position phrase.
[0025]
DLNK links phrase candidates subsequent to the phrase candidate, and stores the phrase candidate ID of the link destination similar to SLNK. A DLNK value of -1 means that there are no more phrase candidates that follow, as in SLNK.
For example, the phrase candidate ID “0” “milk” has “father” or “slowly” as the phrase candidate at the same reading start position, and “greeting” as the subsequent phrase candidate. .
[0026]
FIG. 4 is an example of the homophone buffer DBF in the character processing device of this embodiment. The homophone buffer DBF is a buffer memory for storing the conversion result obtained from the phrase candidate table BCT.
In FIG. 4, HID is a homophone ID and is an ID number that makes each of the homophones created for each phrase unique.
[0027]
The JW stores the number HN from the top of the homophone pool DBF shown in FIG. 5 as the first conversion candidate of the homophone. An independent word to which an application example serial number APYR described later is applied is shown.
The CW stores the number HN from the head of the homophone word pool DBF shown in FIG. 5 for the candidate currently changed by the next candidate change of the homophone.
[0028]
When CW is “0”, it indicates that this homophone remains the first conversion candidate and has not undergone the next candidate change.
The PHID stores a homophone ID of a pair homophone with a dependency relationship (which becomes a pair of an example when an example is searched) having a dependency relationship with the homophone.
When PHID is “−1”, it means that there is no related pair homophone.
[0029]
When determining the first candidate, the ARYR stores the serial number YSN of the example dictionary YRTBL applied to the pair homophones having a dependency relationship. In FIG. 4, the homophone “Father to Father” indicates that the example of “greeting (* human)” of the example serial number “2” is applied.
FIG. 5 is an example of the homophone pool DBP in the character processing device according to the present embodiment. The homologous word pool stores all possible word candidates as candidates for the homologous word for the homologous word buffer that stores only the information of the first candidate.
[0030]
In FIG. 5, HN is a sequence number of homophones. A serial number is assigned from 1. The HID stores which homophone is a candidate with the homophone ID of the corresponding homophone buffer DBF.
The JWP stores a pointer to the word dictionary WDIC for possible independent words that can be used as homophones of the corresponding phrase.
[0031]
JWS stores the semantic classification code of the word stored in JWP. The semantic classification code is a code that the word has in the WIMI of the dictionary WDIC.
The FWP stores an attached word string for the independent part word JWP. For example, a phrase having a homophone ID “10” has three homophone candidates “father”, “milk”, and “slowly”.
[0032]
FIG. 6 is a diagram showing a configuration of the word dictionary DIC in the character processing device according to the present embodiment.
The DIC is composed of five items: reading YYOM, notation WHYOKI, part of speech WHIN, T difficult word meaning code WIMI, and word frequency WFRQ per word.
The semantic code WIMI is a classification code indicating a semantic attribute that a word does not originally have. If there is no semantic code, “¥” is stored as an indefinite symbol.
[0033]
FIG. 7A, FIG. 7B, FIG. 7C, and FIG. 7D are diagrams showing the configuration of an example table YRTBL that stores a set of clauses having a dependency relationship in the character processing device according to the present embodiment.
As will be described later, the example table is automatically abstracted from pre-stored standard individual examples and standard semantic examples, and learning individual examples, learning counterexamples, and learning individual examples that are stored by the operator's learning. Each of the learning semantic examples constructed in this way can be stored.
[0034]
FIG. 7A is a diagram showing an overall configuration of the example table YRTBL. The YRTBL is composed of an example table head TBLHD and an example table body TBLBD.
FIG. 7B is a diagram illustrating a configuration of the example table header TBLHD. TBLHD is an information storage area indicating the storage status of the example table.
[0035]
LYRC indicates the number of learning examples stored, and the initial value is zero.
LYRS stores the total area size used in the learning example, and the initial value is zero.
LYRP stores the remaining size of the learning example that can be stored by the learning of the operator. In the initial state, a value obtained by subtracting the total size of the standard individual example, standard semantic example, and learning counterexample example size from the example table body size is stored. The LHRC stores the number of stored learning counterexamples that are counterexamples that do not match the learning semantic examples. The initial value is 0.
[0036]
JHRS stores the remaining size of the learning counterexample. The learning counterexample is determined so that at least one counterexample can be stored in the initial state.
The remainder area of the example table is filled with “0” as a spare unstored area.
FIG. 7C is a diagram illustrating a configuration of the example table body.
YSN is a serial number of the example, and the example stored in the head of the example table is 1, and a serial number incremented by 1 is added.
[0037]
The USN is a serial number of the accepting clause that becomes a headline of an example to be described later.
SUN is a serial number in the example of the same heading, and the example of the same heading word information is set to 1 at the beginning, and a serial number that is sequentially increased by one is assigned.
YRTYP represents the type of the stored example, and can take the following five values.
Standard individual
Standard meaning
Individual learning
Learning meaning
Learning counterexample
KBCNT stores the number of dependency clauses KBINF related to one receiving header clause UBINF. When there is only one clause related to the receiving clause, “1” is entered.
“Cut meat with a knife”
As shown in the above, when there are two dependency clauses of “cutting knife” and “meat” in the receiving clause “cut”, “2” is entered. In this configuration, this dependency clause can be stored. That is, one example data is composed of a maximum of (N + 1) clauses.
[0038]
YCF stores a certainty value indicating the reliability of the example. The confidence value is
0 ≦ YCF ≦ 1.0
The larger the value, the more reliable the example.
[0039]
UBINF stores the receiver clause condition that is the headline of the example. UBINF has the configuration shown in FIG. 7D.
KBINF (i) stores the dependency clause condition of the example relating to the receiving clause condition UBINF. In the present embodiment, a fixed-length configuration capable of storing a maximum of N dependency-side clause conditions is used, and “¥” is entered as an unused area in the remaining dependency-side clause conditions. The structure of KBINF (i) is the same as that of UBINF shown in FIG. 7D.
[0040]
Note that the receiver clause conditions that enter UBINF are sorted in the table according to the ascending readings stored in the word reading YMYOM shown in FIG. 7D.
FIG. 7D is a diagram illustrating the configuration of the example configuration clause condition, which is a detailed configuration of the receiving-side clause condition and the dependency-side clause condition in FIG. 7C.
YMYOM is an area for storing a word constituting the phrase condition, YMHYO is a word notation, YMHIN is a word part of speech, and YMIMI is an area for storing a word semantic code. WYOMI of the word dictionary DIC shown in FIG. This corresponds to the values stored in WHYOKI, WHIN, and WIMI.
[0041]
YMCON stores the phrase end form of the words constituting the phrase condition. In the case of a dependency-side phrase condition, YMCON stores information for determining how to connect to the receiver-side phrase condition. And the following values can be taken.
Introductory form of words
Conjugated word usage
Termination form of words
Conjunctive form of words
Assumptions of usage words
Imperative form of words
The particle "ga"
Particle "O"
The particle "ni"
The particle "no"
The particle "to"
The particle "de"
The particle "kara"
Particle "Yor"
The particle "until"
In the example table description example shown in FIGS. 7C and 7D,
"Love nature"
“Greeting to (* human)”
"Meet (* human)"
“Greeting to station staff”
"Greeting to the teacher"
"Greeting to my uncle"
“Greeting to the neighborhood”
The example is described. However, the symbol “(*)” indicates not a word but a semantic classification code.
[0042]
Next, the processing procedure of the above-described embodiment will be described according to the flow.
FIG. 8 is a flowchart showing the operation of the character processing apparatus of this embodiment.
In describing the processing of the following flowchart, various variable names appear, but these are assumed to be assigned in advance to a predetermined area of the RAM (FIG. 1).
[0043]
In step S8-1, the process waits for an interrupt to be generated by pressing a key from the keyboard. When a key is input, the key is determined in step S8-2, and any one of step S8-3, step S8-4, step S8-5, step S8-6, and step S8-7 is selected according to the type of the key. Branch to step.
Step S8-3 is processing when the reading input key YOMI is pressed, and the code of the pressed reading is stored in the input reading buffer memory YBUF.
[0044]
Step S8-4 is processing when the conversion key CON is pressed, and converts the character string to be kana-kanji conversion target input in step S8-3 and stored in YBUF into kanji, Output to the output buffer.
Step S8-5 is processing when the next candidate conversion key NXT is pressed, and displays another candidate for the homophone in the homophone pool corresponding to the homophone buffer output in step S8-4.
[0045]
Step S8-6 is processing when the selection key SEL is pressed, confirms the homophone in the output buffer displayed on the screen, and outputs the confirmed character string in the document. Further, when the selected word constitutes an example, it is determined whether it is possible to abstract the individual example into a semantic classification example expressed by a common semantic classification, and if necessary, it is stored as a semantic classification example. Perform the action.
[0046]
Step S8-7 is processing when a key other than YOMI, CON, NXT, SEL (for example, a key used for document editing such as a cursor difference key) is pressed, and is generally performed in the same type of character processing apparatus. Since it is a known process, it is not particularly described.
FIG. 9 is a detailed flowchart of the “conversion process” in step S8-4.
[0047]
In step S9-1, the phrase candidate creation process detailed in FIG. 10 is performed to create a phrase candidate table BCT.
In step S9-2, the first candidate determination process detailed in FIG. 11 is performed. The candidates suitable as the first conversion candidates are narrowed down from the phrase candidates stored in the phrase candidate table BCT.
[0048]
In step S9-3, based on the first candidate determined in step S9-2, a conversion result is created and output to the homophone buffer DBF. At the same time, other candidates for the same reading start position and phrase length that have not become the first candidate are stored in the homonym pool DBF.
FIG. 10 is a detailed flowchart of the “sentence candidate creation process” in step S9-1.
[0049]
In step S10-1, the phrase candidate creation start index i on the input reading buffer YBUF and the index j of the phrase candidate table BCT are initialized to “0”.
In step S10-2, the word dictionary WDIC is searched based on the reading in the input buffer YBUF indicated by i to obtain word candidates.
[0050]
In step S10-3, a phrase connection verification process for analyzing an attached word string connected to the found word candidate is performed. As a result, phrase candidates are obtained.
In step S10-4, the obtained phrase candidates are stored in the phrase candidate table BCT. Information on independent words and adjunct word strings obtained as a result of word search and phrase connection verification is stored in the BCJW and BCFW fields, respectively, and other phrase candidates that can have this phrase candidate in SLNK and DLNK are also set. After all the information is stored in the phrase candidate table BCT, the phrase candidate table index j is counted up in step S10-5.
[0051]
In step S10-6, it is determined whether or not all the words having the same phrase starting position, i.e., the words having the same reading starting position, have been searched. Loop to S10-2 and continue the search. If all the words at the same reading start position have been searched, in step S10-7, the phrase end position indicated by the attached word reading sequence described in the phrase candidate table BCT is set to i as the subsequent start position. At this time, if all the phrase candidates are terminated, that is, if the DLNK value of the phrase candidate table BCT is “−1”, the end of the input reading has been reached, so the phrase candidate creation process Exit. Otherwise, the set i is looped to step S10-2, and the subsequent phrase candidate creation process is continued.
[0052]
FIG. 11 is a detailed flowchart of the “first candidate determination process” in step S9-2.
In step S11-1, a counter memory max that is an index value for determining the first candidate is initially set to “0”. max indicates the maximum likelihood among all the phrase candidates, and the likelihood “0” is the minimum value.
[0053]
In step S11-2, one phrase candidate string is extracted from the phrase candidate table BCT.
In step S11-3, the phrase likelihood of the extracted phrase candidate string is calculated. The phrase pair likelihood is calculated as follows.
(Phrase pair likelihood) = {(word likelihood) operation} + {sum of (inter-phrase likelihood)}
The word likelihood uses the value of the FQ field of the word dictionary WDIC.
[0054]
The phrase likelihood is a value indicating the likelihood of connection between phrases, and is fixed to “0” in the present embodiment.
In step S11-4, as will be described in detail with reference to FIG. 12, an example applicable to the extracted phrase candidate string is searched from the example table YRTBL, and the example application likelihood is added to the phrase pair likelihood.
[0055]
In step S11-4, as will be described in detail with reference to FIG. 12, an example that can apply the extracted phrase candidate string is searched from the example table YRTBL, and the example application likelihood is added to the phrase pair likelihood.
In step S11-5, it is determined whether the calculated phrase likelihood is larger than the maximum likelihood max. If it is larger, the maximum likelihood is updated to the calculated phrase pair likelihood in step S11-6. In step S11-7, the phrase candidate string corresponding to the maximum likelihood is stored in a work memory area did not shown in the RAM (FIG. 1). The work memory area did has the same configuration as the phrase candidate table BCT, but the values of SLNK and DLNK are set to be unique so as to link the phrase candidates corresponding to the phrase candidate string having the maximum likelihood. .
[0056]
In step S11-8, it is determined whether another phrase candidate string can be extracted from the phrase candidate table BCT, and if it can be extracted, the process loops to step S11-2. When all the phrase candidate strings have been processed and no more phrase candidate strings can be extracted, the process returns.
FIG. 12 is a detailed flowchart of the “example dictionary search” in step S11-4.
[0057]
In step S12-1, the example application likelihood yyd is initialized with “0”, and a state where no example is found is set as an initial state.
In step S12-2, an example search attention phrase is extracted from the phrase candidate string extracted in step S11-2, and the example dictionary YRTBL is searched for a pair phrase candidate constituting the example.
[0058]
In step S12-3, it is determined whether there is a pair clause candidate constituting the example. If not found, the process proceeds to step S12-9. If found, the process of step S12-4 and subsequent steps is performed.
The type of example found in step S12-4 is checked. If the found example is the “learning individual example”, the value yyd and the value of 1024, whichever is larger, are reset to the example application likelihood yyd in step S12-5. Similarly, in the case of “learning meaning example”, the yyd value and 512, whichever is larger, are reset in step S12-6. In the case of “standard individual example”, the larger value of yyd and 256 is reset in step S12-7, and in the case of “standard meaning example”, the larger value of yyd and 128 is reset in step S12-8. After setting the example application likelihood yyd according to the found example type, the process returns to step S12-2 to search for another example.
[0059]
When all the examples have been searched, in step S12-9, the phrase likelihood + yd is set as the phrase pair likelihood, and the process returns. In other words, when there are multiple examples applicable to the same phrase candidate for each type, the priority is higher in the order of “learning semantic examples”, “learning individual examples”, “standard individual examples”, and “standard semantic examples”. Become.
FIG. 13 is a detailed flowchart of the “next candidate display” in step S8-5.
[0060]
In step S13-1, the homophone ID of the homophone for which the next candidate is to be displayed is acquired.
In step S13-2, the next candidate having the same sound ID in the sound word pool DBP is extracted from the independent word JWP and the attached word string FWP of the sound word pool, and stored in the corresponding JW and FW fields of the sound word buffer DBF. Display this.
[0061]
FIG. 14 is a detailed flowchart of the “selection process” in step S8-6.
In step S14-1, the homophone buffer DBF having the homophone ID of the homophone for which the selection process is performed is taken out, and it is determined whether or not it becomes a target for individual example learning. If it is determined that individual example learning is possible, an individual example learning process is performed in step S14-2, and the process proceeds to step S14-3. Conversely, if it is determined that individual example learning is not possible, the process returns.
[0062]
In step S14-3, it is determined whether or not an example abstraction for replacing an example pattern by semantic classification from a set of individual examples is possible. If example abstraction is possible, individual example abstraction is performed in step S14-4, and the process returns. If example abstraction is not possible, return as is.
FIG. 15 is a flowchart detailing the determination of necessity of individual example learning in step S14-1.
[0063]
With reference to the value of the PHID field of the homophone buffer DBF extracted in step S15-1, whether or not there is a dependency relationship is determined by whether or not this value is -1. If the value of PHID is −1, it is determined that there is no clause at the destination, and value turn is performed so that no example learning is necessary. Otherwise, in step S15-2, the application example of the APYR field of the homophone buffer DBF is referred to, and it is determined whether the example is applied depending on whether this value is “0”. If the example is applied, it is determined that “example learning is not necessary”, and if the example is not applied, it is determined that “example learning is necessary” and the process returns.
[0064]
FIG. 16 is a flowchart detailing the individual example learning in step S14-2.
In step S16-1, it is determined by the procedure shown in FIG. 17 whether the phrase pair that is the subject of individual example learning is a counterexample of the learning meaning example. If it is a counterexample, it is registered in the example table YRTBL as a counterexample in step S16-3 described in detail in FIG. If it is not a counterexample, in step S16-2, the target phrase pair is registered in the example table YRTBL as an individual learning example.
[0065]
FIG. 17 is a flowchart detailing the counterexample determination of the learning meaning example in step S16-1.
In step S17-1, the example serial number APYR applied to the target clause is GETd.
In step S17-2, whether or not the type of the example indicated by APYR is a learning semantic example is checked against YRTYP in the example table YRTBL. If there is no learning meaning example, it returns that it is not a counterexample in step S17-5.
[0066]
If the correspondence of the example indicated by APYR is the learning meaning example, in step S17-3, the semantic classification JWS indicated by the post-change candidate CW of the target phrase is GET.
In step S17-4, it is checked whether the semantic classification JWS possessed by the CW matches the semantic classification of YRTBL indicated by APYR. If they match, it is determined in step S17-5 that it is a counterexample of the learning meaning example, and the process returns. If they do not match, it is determined that there is no counterexample and the process returns.
[0067]
FIG. 18 is a flowchart detailing the counterexample registration in step S16-3.
In step S18-1, the target phrase pair is registered in the example table YRTBL as a counterexample.
In step S18-2, by registering the counterexample, the validity of the learning meaning example is re-evaluated by the procedure detailed in FIG. If it is evaluated as valid, the process returns. If it is evaluated as invalid, the confidence of the learning meaning example is updated in step S18-3, and further, the learning meaning is determined from the learning individual example in step S18-4. Update the abstraction processing start condition for the example and return.
[0068]
FIG. 19 is a flowchart detailing the learning meaning example validity re-evaluation determination in step S18-2.
In step S19-1, counterexamples that are counterexamples to the target learning meaning examples are searched from the example table YRTBL, and the number is counted.
In step S19-2, an individual learning example satisfying the target learning meaning example is searched from the example table YRTBL, and the number is counted.
[0069]
In step S19-3, the number of counter examples obtained in step S19-1 is compared with the number of individual examples obtained in step S19-2. If the number of counter examples is larger, it is determined that the validity is low in step S19-4. If not, the process returns as valid in step S19-5.
FIG. 20 is a flowchart detailing the learning meaning example certainty factor update in step S18-3.
[0070]
In step S20-1, the certainty factor of the learning meaning example is set to CF from YCF of the example table YRTBL.
In step S20-2, 0.1 is subtracted from CF.
In step S20-3, it is checked whether or not CF is larger than 0. If “0” or less, the reliability as the learning meaning example is rejected, and the learning meaning example is deleted from the example table YRTBL in step S20-5. . Otherwise, in step S20-4, the value of CF is set in the certainty factor YCF of the example table YRTBL.
[0071]
FIG. 21 is a flowchart detailing the individual example abstraction activation condition update in step S18-4.
In step S21-1, the abstract activation individual example constant T is acquired.
In step S21-2, 10 is added to T, the abstraction activation condition is updated, and the process returns.
[0072]
FIG. 22 is a flowchart detailing the individual example abstraction determination in step S14-3.
In step S22-1, in the example table YRTBL, individual examples having the same semantic classification as the receiving-side clause KBINF are counted in the example learning table YRTBL and set in the counter memory K. To do.
[0073]
In step S22-2, the number K of individual examples is compared with the abstract activation individual example constant T. If K ≧ T, it is determined in step S22-3 that the individual example abstraction can be activated, and so on. In step S22-4, it is determined that abstraction is impossible, and the process returns.
FIG. 23 is a flowchart detailing the individual example abstraction in step S14-4.
[0074]
In step S23-1, the semantic classification JWS is GETted from the homophone pool of the target phrase.
In step S23-2, it is stored in the example table YRTBL as a learning semantic example stored in the classification JWS meaning the relation side phrase data of the learning individual example registered in step S14-2.
(Other embodiments)
In the above description, as a character processing device that performs inter-phrase dependency analysis, which is an embodiment of the present invention, an input target sentence is a kana character and a kana-kanji conversion device is taken as an example, but as an embodiment of the present invention, It is not limited to the Kana-Kanji conversion device. As long as it does not depart from the spirit of the present invention that analyzes the inter-phrase dependency and performs processing that prioritizes the phrase candidates that match the learned semantic examples based on the semantic examples obtained by learning and registering the relations between phrases. For example, the present invention is a character processing apparatus for analyzing the phrase structure of a Japanese sentence, as a document proofreading apparatus for analyzing a phrase structure, checking the semantic consistency of a sentence, and indicating an error, using a kana-kana mixed sentence as an input target Can be implemented.
[0075]
In the above description, the example dictionary has two word (or semantic classification) pairs as a combination of words appearing in pairs. However, there are three word pairs or generally n number of word pairs. Even a pair (n is an integer of 2 or more) can be processed similarly.
Furthermore, in the above description, as the activation condition for abstracting the individual example into the semantic classification example, the absolute number of the individual example is stored and abstracted. However, the abstracted activation condition update flowchart shown in FIG. 24 can be processed as in the flowchart shown in FIG. 24, and the individual example abstraction determination flowchart shown in FIG. 22 can be processed as in the flowchart shown in FIG. In that case, it operates as follows.
[0076]
FIG. 24 is a flowchart detailing the individual example abstraction activation condition update in step S18-4.
In step S24-1, an abstract activation individual example ratio constant R is acquired. Assume that the abstract activation individual example ratio constant R is secured in the RAM in FIG.
In step S24-2, 0.1 is added to R, and the process returns to update the abstract activation condition.
[0077]
FIG. 25 is a flowchart detailing the individual example abstraction determination in step S14-3.
In step S 25-1, the number of individual examples having the same semantic classification in the dependency-side clause KBINF is counted in the counter table K in the example table YRTBL. set.
[0078]
In step S25-2, the number of individual examples having the same receiving clause UBINF of the registered learning individual examples in the example table YRTBL is counted and set in the counter memory S. It is assumed that the counter memory S is secured in the RAM in FIG.
In step S25-3, K / S, that is, the ratio of the number of individual examples having the same semantic classification in the dependency-side clause KBINF in the total number of individual examples having the same receiving-side clause UBINF is obtained, and the abstraction activation individual example Compare the ratio constant R,
If K / S ≧ R, it is determined in step S25-4 that the individual example abstraction can be activated, and if not, it is determined in step S25-5 that abstraction is impossible and the process returns.
[0079]
As described above, the abstraction start condition from the individual example to the semantic classification example can be determined not by the absolute number of the individual example but by the ratio of the number of examples that is established when the semantic classification example is realized.
Further, in FIG. 19, as a means for determining the validity of the learning semantic example, the counter example number for the learning semantic example is determined based on whether the absolute number is larger than the number of individual examples that satisfy the learning semantic example. In step S19-3 in FIG. 19, it is possible to determine the ratio of the counterexample number to the individual example number instead of the comparison based on the absolute number and determine whether the ratio is larger than a predetermined validity determination coefficient.
[0080]
The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Needless to say, the present invention can also be applied to a case where the present invention is achieved by supplying a program to a system or apparatus.
As described above, according to the present embodiment, after the learned individual examples are abstracted and automatically registered as semantic classification examples, the reliability of the semantic classification examples is based on the number of counterexamples that do not follow the semantic classification examples. The reliability as the sex evaluation value can be updated, and the reliability of the data corresponding to the change of the input can be obtained at any time.
[0081]
Therefore, it is possible to provide a Japanese analytical character processing apparatus with good analysis accuracy and good operability.
Also, when the number of counterexamples is too large, the activation conditions for abstracting from individual examples to semantic classification examples can be automatically updated, thus suppressing the proliferation of unreliable semantic classification examples. Memory usage efficiency can be increased.
[0082]
【The invention's effect】
As described above, according to the present invention, conversion accuracy is high and sentence conversion can be performed at high speed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall configuration of a character processing apparatus according to an embodiment.
FIG. 2 is a diagram showing an example of a configuration of an input reading buffer in the character processing device according to the present embodiment.
FIG. 3 is a diagram illustrating an example of a configuration of a phrase candidate table according to the present embodiment.
FIG. 4 is a diagram showing an example of the configuration of a homophone buffer in the character processing device according to the present embodiment.
FIG. 5 is a diagram showing an example of the configuration of a homophone pool in the character processing device according to the present embodiment.
FIG. 6 is a diagram showing an example of the configuration of a word dictionary in the character processing device according to the present embodiment.
FIG. 7A is a diagram showing an example of the configuration of an example dictionary table in the character processing device according to the present embodiment;
FIG. 7B is a diagram showing an example of the configuration of an example dictionary table in the character processing device according to the present embodiment;
FIG. 7C is a diagram showing an example of the configuration of an example dictionary table in the character processing device according to the present embodiment;
FIG. 7D is a diagram showing an example of the configuration of an example dictionary table in the character processing device according to the present embodiment;
FIG. 8 is a flowchart illustrating an example of a processing procedure of the entire operation according to the present exemplary embodiment.
FIG. 9 is a flowchart illustrating an example of a processing procedure of conversion processing according to the present embodiment.
FIG. 10 is a flowchart illustrating an example of a procedure of phrase candidate creation processing according to the present embodiment;
FIG. 11 is a flowchart illustrating an example of a processing procedure of first candidate determination processing according to the present embodiment;
FIG. 12 is a flowchart illustrating an example of a processing procedure of an example dictionary search process according to the present embodiment.
FIG. 13 is a flowchart illustrating an example of a processing procedure of next candidate display processing according to the present embodiment;
FIG. 14 is a flowchart illustrating an example of a processing procedure of selection processing according to the present embodiment;
FIG. 15 is a flowchart illustrating an example of a processing procedure for determining necessity of individual example learning according to the present embodiment;
FIG. 16 is a flowchart illustrating an example of a processing procedure of individual example learning according to the present embodiment;
FIG. 17 is a flowchart illustrating an example of a counter-example determination processing procedure according to the present embodiment;
FIG. 18 is a flowchart illustrating an example of a processing procedure for counterexample registration according to the present embodiment;
FIG. 19 is a flowchart illustrating an example of a processing procedure of learning semantic example validity determination according to the present embodiment;
FIG. 20 is a flowchart illustrating an example of a processing procedure of learning meaning example certainty factor update according to the present embodiment;
FIG. 21 is a flowchart illustrating an example of an abstract activation condition update processing procedure according to the present embodiment;
FIG. 22 is a flowchart illustrating an example of an individual example abstraction determination procedure according to the present embodiment;
FIG. 23 is a flowchart illustrating an example of an individual example abstraction learning processing procedure according to the present embodiment;
FIG. 24 is a flowchart illustrating an example of an abstract activation condition update processing procedure according to another embodiment.
FIG. 25 is a flowchart illustrating an example of an individual example abstraction determination procedure according to another embodiment;

Claims (4)

An individual example dictionary for storing combination information of words and words having affinity with the words as individual examples;
A semantic example dictionary that stores a combination of a word and a semantic concept of a word having affinity with the word as a semantic example, together with a certainty factor for the combination;
Regarding the semantic examples of the semantic example dictionary, among the individual examples of the individual example dictionary, the examples corresponding to the combination of the word of the semantic example and the semantic concept, and the combination of the homophone and the semantic concept of the word of the semantic example An example extracting means for extracting a counter example corresponding to
In the ratio of the number of extracted examples and counterexamples, if the ratio of the counterexamples is greater than a predetermined threshold, a certainty factor adjustment unit that reduces the certainty factor corresponding to the semantic examples in the semantic example dictionary;
As a result of the adjustment by the certainty factor adjusting unit, the example invalidating unit for invalidating the semantic example in which the certainty factor is equal to or less than a predetermined value;
Based on the individual example or semantic example applicable to the inputted phrase string, the phrase string is converted into a conversion candidate using a combination of words corresponding to the individual example when the individual example is applicable, and the meaning is converted. When the example is applicable, a character processing device comprising: conversion means for converting into a conversion candidate using a combination of words corresponding to a combination of a word of the semantic example and a semantic concept. A selection means for selecting one conversion candidate from the conversion candidates converted by the conversion means;
The method according to claim 1, wherein if there is a candidate to which a semantic example is applied in addition to the selected conversion candidate, the selected conversion candidate is additionally registered in the individual example dictionary as a reuse example. Character processing unit. An individual example dictionary storage means for storing, as an individual example, combination information of a word and a word having affinity with the word;
A semantic example dictionary storage means for storing a combination of a word and a semantic concept of a word having affinity with the word, together with a certainty factor for the combination, a processing means for executing various processes based on a program , An example extracting means, a certainty factor adjusting means, an example invalidating means, and a converting means realized by cooperation of the processing means and the program storing means. A character processing method for processing characters in a character processing device provided,
The example extraction unit corresponds to a combination of a word and a semantic concept of the semantic example from the individual examples stored in the individual example dictionary storage unit with respect to the semantic example stored in the semantic example dictionary storage unit. An example extracting step of extracting an example and a counterexample corresponding to a combination of a homophone and a semantic concept of a word of the semantic example;
The certainty factor adjusting means is stored in the semantic example dictionary storage means corresponding to the semantic example if the ratio of the counterexamples is larger than a predetermined threshold in the ratio of the number of extracted examples and counterexamples. A confidence adjustment process to lower the confidence,
An example invalidation step in which the example invalidation unit invalidates a semantic example whose certainty factor is equal to or less than a predetermined value in the semantic example dictionary storage unit as a result of adjustment in the certainty factor adjustment step;
When the conversion means searches the individual example dictionary storage means and the semantic example dictionary storage means applicable to the inputted phrase string from the individual example dictionary storage means and the semantic example dictionary storage means, and the phrase string is applicable to the individual example Convert to a conversion candidate using a combination of words corresponding to the individual example, and if a semantic example is applicable, convert to a conversion candidate using a word combination corresponding to the combination of the word of the semantic example and a semantic concept A character processing method comprising: a conversion step. The character processing device further includes a selection unit and an additional registration unit realized by cooperation of the processing unit and the program storage unit, and the character processing method includes:
A selection step in which the selection means selects one conversion candidate from the conversion candidates converted in the conversion step ;
The additional registration means, if any candidate that is meant example is applied separately from the selected conversion candidate, the selected conversion candidate as an anti example, additional registration process for additionally registering the individual example dictionary storage means When
The character processing method according to claim 3, further comprising :

Images