JP3740541B2 - Language processing system and program - Google Patents

Description

  The present invention relates to a language processing system and a program using a system for associating a plurality of data so that a matching portion is maximized while maintaining order information such as a diff command for performing difference detection.

  Conventionally, for example, associating a Japanese sentence with an English sentence, associating a lecture with a preliminary draft, associating a question sentence with knowledge data, etc., have complicated programs as judgment processing.

  The conventional bilingual corpus mapping, the conventional lecture / preliminary mapping, and the question text / knowledge data mapping requires a complicated program and is difficult to map easily. Met.

  SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-mentioned problems and to easily perform correspondence between a plurality of language data such as correspondence between bilingual corpora and correspondence between lectures and preliminary reports using a diff command. .

  FIG. 1 shows a language processing system according to the present invention. In FIG. 1, 1 is an input means, 2 is a processing unit, 3 is a morphological analysis means, 4 is a difference detection means, and 5 is a similarity calculation means.

  The present invention has the following means in order to solve the conventional problems.

  (1): Input means 1 for inputting a plurality of language data of a first language data including a symbol that serves as a mark for association and a second language data not including the symbol, and a matching portion while maintaining order information Using a system for associating a plurality of data so as to maximize the difference between the plurality of language data and a difference detection unit 4 for detecting a difference part, wherein the difference detection unit 4 includes the common part. The language data is taken out as it is, the language data is taken out from the difference portion corresponding to the second language data from the difference portion, and the difference portion on the side corresponding to the first language data is taken out from the difference portion. By taking out the symbols and arranging them in the same order, the plurality of language data are associated with each other by having the second language data in which the symbols are inserted. For this reason, for example, by deleting the preliminary draft data (first language data) while leaving only the chapter information, which is a symbol for the correspondence, the chapter information is added to the lecture data (second language data). It can be inserted, and the correspondence between the draft and the lecture can be easily performed.

  (2): In the language processing system of (1), it is a system for associating the draft data with the lecture data, and the plurality of language data are the draft data and the lecture data. For this reason, it is possible to insert chapter information into lecture data by leaving only the chapter information, which is a symbol of association, and deleting the draft data, making it easy to associate the draft with the lecture. It can be carried out.

  (3): In the language processing system of (1) or (2), a diff command is used as a system for associating a plurality of data so as to maximize the matching portion while maintaining the order information. For this reason, it is possible to easily associate a plurality of language data.

  (4): The process of inputting a plurality of language data of the first language data including a symbol that becomes a mark of association and the second language data not including the symbol, and maximizing the matching portion while maintaining the order information A process for detecting a common part and a difference part of the plurality of language data using a system for associating a plurality of data so as to become, the language data is directly extracted from the common part, and the difference part is By extracting the language data from the difference part on the side corresponding to the second language data, and taking out the symbols from the difference part on the side corresponding to the first language data and arranging them in the same order , A program for causing a computer to realize processing for associating the plurality of language data by having second language data into which the symbols are inserted To. Therefore, by installing this program in a computer, it is possible to easily provide a language processing system that can easily realize correspondence between a plurality of language data such as a draft and a lecture.

  The present invention has the following effects.

  (1): The difference detection means extracts the language data as it is from the common part, extracts the language data from the difference part corresponding to the second language data from the difference part, and corresponds to the first language data By taking out the symbols that become the mark of correspondence from the difference part on the side to be arranged and arranging them in the order as they are, by having the second language data in which the symbols are inserted, it is possible to support multiple language data For example, the chapter information is added to the lecture data (second language data) by erasing the preliminary data (first language data), leaving only the chapter information, which is a symbol for the correspondence. It can be inserted, and the correspondence between the draft and the lecture can be easily performed.

  (2): Since the plurality of language data are the draft data and the lecture data, the lecture data is deleted by leaving only the chapter information, which is a symbol for the correspondence. Chapter information can be inserted into the data, making it easy to associate a draft with a lecture.

  (3): Since the diff command is used as a system for associating a plurality of data so as to maximize the matching portion while retaining the order information, it is possible to easily associate a plurality of language data.

(1): Description of Language Processing System FIG. 1 is an explanatory diagram of a language processing system of the present invention. In FIG. 1, an input unit 1 inputs language data for performing difference detection. The processing unit 2 processes input data. The morpheme analyzing means 3 divides input language data into optimum word strings using a dictionary and grammar. The difference detection means 4 detects a difference between a plurality of language data having a correspondence relationship using a diff command. The similarity calculation means 5 obtains the similarity between the question sentence and the knowledge data by using the difference detection means 4 and outputs a part corresponding to the question sentence of the question sentence of the knowledge data having a high similarity.

(2): Explanation of diff, mdiff, mdiffc
(1) Explanation of diff
The diff is a file comparison tool diff of UNIX (registered trademark). This diff command outputs the difference between two given files in units of lines while maintaining the order information.

For example,
Suppose you have a file (1) that says going to school today and a file (2) that says going to university today. When these diffs are taken, the difference is output in the form of <to school> to university.

(2) Explanation of mdiff By the way, the diff command has a convenient option called -D option. Using this option with the diff command will output not only the difference but also the common part. That is, file merging is realized. The difference portion is expressed by an ifdef statement used in a C (programming language) preprocessor or the like. Here, since the ifdef statement is difficult to see, the difference portion is displayed as follows.

▽▽▽▽▽▽▽
(Part only in the first file)
; ●●●
(Part only in the second file)
△△△△△△
Here, “; ▽▽▽▽▽▽” means the beginning of the difference part, “; △△△△△△” means the end of the difference part, and “; ●●●” is the two that make up the difference. It means data boundary.

  In the present embodiment, if the -D option is added and the ifdef part is further displayed (formatted) as described above and the files are merged, the diff is called mdiff (mdiff). is there).

  In fact, when mdiff is applied to the previous data (file (1) and file (2)), the following results are obtained.

Today ； ▽▽▽▽▽▽
To school; ●●●
To university △△△△△△
Iku This is the same for “Today”, “To school” and “To university” are the differences, and “Iku” is also a common part. In this way, the output of mdiff is easy to understand because the matching part is also output unlike diff.

In addition, the data of the original two files can be completely restored from the result of mdiff. If you take out only the common part and the black circle (; ●●●) of the difference part,
The information in the first file is retrieved as if going to school today. Also, if you take out only the lower part of the common part and the black circle (; ●●●) of the difference part,
The information in the second file is retrieved as if going to university today. In this way, the original information can be completely restored.

  In addition, in mdiff, only the matching part is displayed in one data and only the inconsistent part is displayed in both data, so the data amount can be reduced compared to the original two data, but the above Thus, since the original information can be completely restored, mdiff can be said to realize data compression in the sense of reducing the amount of data while it can be restored.

  FIG. 2 is an explanatory diagram of the mdiff language processing system. In FIG. 2, the mdiff language processing system is provided with a UNIXdiff processing unit 11 and a shaping unit 12. The UNIXdiff processing unit 11 outputs a difference part and a common part by diff of two input files. The shaping unit 12 shapes the output of the UNIXdiff processing unit 11 into an easily viewable expression.

  FIG. 3 is a flowchart based on mdiff. Hereinafter, a description will be given according to the processes S1 to S4 of FIG.

  S1: Two files are given to the UNIXdiff processing unit 11 as input.

  S2: The UNIXdiff processing unit 11 detects a match / mismatch portion between the two files (diff is added with option -D to detect a match portion).

  S3: The shaping unit 12 shapes the output of the UNIXdiff processing unit 11. Specifically, processing is performed to change the ifdef sentence of the output of “diff-D symbol” into an easy-to-read expression such as “; ▽▽▽▽▽▽”.

  S4: The shaping unit 12 outputs the shaped result.

  3): Explanation of mdiffc

Next, consider mdiff in units of characters. In the case of language processing, it is often desirable to take the character unit as a difference. In such a case, you can do mdiff with the file that has been output with the contents of the file once after each line break. For example, the information of the previous file (1)
now

After taking the form of going to a Japanese school, take mdiff. Applying mdiff on a character basis is called mdiffc (mdiffc c is character c).

  The display of diff is difficult to see, and mdiff completely contains the information displayed by diff, so the following explanation is done using mdiff.

(3): Explanation of difference detection and rewrite rule acquisition
(1) Explanation of differential detection of multiple system outputs Previously, when multiple versions of juman's system were confused, the output of morphological analysis was improved by merging the multiple juman outputs with mdiff. (Reference, Maki Murata, Estimation of noun indication target in Japanese sentences, Doctoral dissertation, Faculty of Engineering, Kyoto University, (1995), Mamoru Ishima, Satoshi Fujii, Tetsuya Ishikawa, Japanese morphological and syntactic analysis system JEMONI For the development and evaluation of IPSJ, see IPSJ Natural Language Processing Study Group 98-NL-127, (1998)). Here, an example of “something” will be described.

Analyzing "That", the output of juman's A version is a noun that is a verb and noun, and the output of B is a noun that is a noun and a verb that is a verb. Suppose that It is assumed that the word “had” has the ambiguity of “go” and “say”, and the version B erroneously outputs it as the word of “go”. Here is the result of mdiff:

And and particle ； ▽▽▽▽▽▽
Say the verb ； ●●●
Going verb △△△△△△△
Thing thing noun
By taking mdiff, it is possible to easily detect differences in the output of multiple systems. In this case, it can be seen that there is a difference in the output of the “like” part. Here, the operator who corrects the output determines which one is correct at a position where such a difference is detected, and if the top is correct, nothing is done, and if the bottom is correct, “x” is added to the head of “; ●●●”. Decide that you want to add. By doing so, if there is no “x”, it is separated from the information below the difference if there is no “x”, and the symbol is deleted, so that the better result is automatically obtained from each difference from the data of the work result. You can choose to produce results that are more accurate than those of each version. Also, both differences are often incorrect. In this case, it is better to actually rewrite the data above “; ●●●”.

  Using this method, the only thing that can't be corrected is the same error in both versions, and many morphological errors can be corrected. It should be noted here that multiple systems with different properties must be prepared. If the error is in the same system, many errors will be missed.

  If you have three systems, use the diff3 command. diff3 can detect differences between three files.

  In the above, morphological analysis was taken as an example, but in other analysis, the difference can be obtained with mdiff by making the analysis result line by line. If you need character units, use mdiffc.

  Here, we talked about taking the difference between the outputs of multiple systems, but one is a tagged corpus, and by comparing it with the result of analysis on some system, errors in the tagged corpus are detected. It can also be corrected.

(2) Consideration of differences and explanation of acquisition of rewrite rules This section describes research on spoken and written diffs. Using the data of spoken and written languages that have been dealt with, the difference between the spoken and written languages is considered from the difference between them, and the paraphrase rule from spoken to written language and vice versa. As the data, the oral presentation of the academic society was used as spoken language data, and the preliminary draft of the academic society that describes the contents of the oral presentation was used as written language.

  FIG. 4 is an explanatory diagram of examples of written word data and spoken word data. For example, assume that written and spoken language data is given in the form shown in FIG. Here, the morpheme analysis system (morpheme analysis unit 3) or the like is converted into a form in which one word is inserted in one line so that a difference can be easily obtained. Given such written and spoken language data, mdiff results in the results shown in FIG. FIG. 5 is an explanatory diagram of the result of diff between written language data and spoken language data. If only the difference portion is extracted from the result of FIG. 5, the result as shown in FIG. 6 is obtained. FIG. 6 is an explanatory diagram of extraction of the difference portion.

  From the result of FIG. 6, it can be seen that “e” or the like is inserted into the spoken language, and that the spoken language is smoothed by adding the expression “to say”. It can also be seen that “state” is rephrased as “state”. As you can see, mdiff can be used to detect the difference between spoken and written language, and by examining it, it can be seen that the difference between spoken and written language can be investigated. These differences can also be seen as paraphrasing rules for spoken and written words.

  For example, the “e” part can be seen as a rule that “e” is entered when converted into spoken language where there is nothing in the written language. In addition, the parts of “state” and “state” can be viewed as a rule that translates “state” into “state” when converted to spoken language. In this sense, it is understood that paraphrasing rules or conversion rules can be detected by using mdiff.

  Here, spoken and written data are used as examples, but this is possible in many places. For example, if you take mdiff between the text before English text review and the text after English text review, you can find out what kind of differences should be corrected and how to get the rules for English text review. Also, if you take mdiff between the pre-summary text and the post-summary text, you can see how they are summarized, and you can get a summary rule. In addition, various considerations and paraphrasing rules can be obtained if mdiff is applied to data with different characteristics.

(4): Explanation of data merging
(1) Explanation of bilingual corpus mapping Here, bilingual corpus mapping is considered. As a condition, it is assumed that each corpus has the same symbol in the corresponding location. Assume that the unit of association is a portion delimited by this symbol.

  FIG. 7 is an explanatory diagram of the structure of the corpus. Here, it is assumed that the Japanese corpus and the English corpus are still scattered and not associated with each other. It is assumed that section information having the same shape such as Section 1 is given to both as in the example of FIG. At this time, it is assumed that the contents of the same section are the same in Japanese and English.

  In this case, the result as shown in FIG. 8 can be obtained by taking mdiff of these data. FIG. 8 is an explanatory diagram of a bilingual corpus associated with mdiff. In the result of FIG. 8, Section 1 and the like are common parts, and other parts are non-matching parts. In this inconsistent portion, Japanese and English are stored separately on the top and bottom. By doing so, bilingual data is created using mdiff.

  The one shown here is not a detailed correspondence such as sentence-by-sentence, but it seems that it is not useful at first glance due to rough things such as sections, but sentence association is a difficult problem. There is also an idea that fine correspondence is made after making correspondence at the chapter or paragraph level where it is clear that correspondence is taken. In this sense, such rough association is also useful.

  In addition, what is shown here can be made to correspond to bilingual data in the same way by writing a program that recognizes information such as Section 1 and classifies it. However, using mdiff makes it possible to easily perform mapping without writing such a complicated program.

  FIG. 9 is an explanatory diagram of a language processing system for a bilingual corpus. In FIG. 9, the mdiff processing unit 21 is provided in the language processing system of the bilingual corpus. The mdiff processing unit 21 outputs mdiff of two files of input original text data and translation data.

  FIG. 10 is a flowchart of the bilingual corpus by mdiff. Hereinafter, a description will be given according to processing S11 to S13 of FIG.

  S11: Two files are given to the mdiff processing unit 21 as input. Here, these two files store English sentences and Japanese sentences, respectively.

  S12: The mdiff processing unit 21 takes mdiff of these two files.

  S13: The mdiff processing unit 21 outputs the result of mdiff.

(2) Explanation of correspondence between lectures and drafts Consider the correspondence between lectures and drafts. This lecture and preliminary paper correspond to the written data and spoken language data mentioned in the previous acquisition of rewriting rules. In other words, a lecture is an oral presentation of an academic conference, and a preliminary paper is a paper corresponding to the oral presentation. When such a lecture and draft are given, each part of the lecture and each part of the draft can be handled. When listening to the lecture, you can refer to the corresponding draft part and When reading, it is convenient to see the corresponding part of the lecture. Here, we will explain how to use mdiff to associate this lecture with a draft.

  Here, in particular, we will use mdiff to determine which part of the lecture corresponds to each part of the lecture. Here, it is assumed that the manuscript and the lecture are made in the same order. Further, a symbol such as “<Chapter 1>” is inserted into the draft data so that the chapter of the draft is easily recognized. FIG. 11 is an explanatory diagram of the structure of the draft data. In this form, morphological analysis is performed on the data of the manuscript and the lecture and mdiff is used with a word on each line, or mdiffc is used, or the result shown in FIG. 12 is obtained. . FIG. 12 is an explanatory diagram of the results of mdiff of the draft and the lecture. Here, if all of the upper half corresponding to the draft in the difference portion is erased except for a symbol such as “<Chapter 1>”, a result as shown in FIG. 13 is obtained. FIG. 13 is an explanatory diagram of the result of inserting chapter information into lecture data. In FIG. 13, only a symbol such as “<Chapter 1>” is inserted into the original lecture data. In other words, you can see which part of the lecture corresponds to which chapter of the draft.

  To put it simply, it uses the matching capability of mdiff to collate the manuscript with the lecture, erase the manuscript information while leaving only the chapter information, and insert the chapter information into the lecture data. Means that Such mapping between a draft and a lecture can be easily done using mdiff.

  FIG. 14 is an explanatory diagram of a language processing system for associating a lecture with a draft. In FIG. 14, the mdiff language processing system includes an mdiff processing unit 21 and a preliminary document deletion unit 22. The mdiff processing unit 21 takes mdiff of two input files and outputs them. The draft deletion unit 22 deletes all the draft data while leaving only chapter information such as <Chapter 1> in the difference part on the draft side (deleting mdiff symbols such as “; △△△△△△” etc.) ).

  FIG. 15 is a processing flowchart for associating a lecture with a draft. Hereinafter, a description will be given according to the processes S21 to S24 in FIG.

  S21: The mdiff processing unit 21 is given two files as inputs. These two files contain the text of the draft and the lecture, respectively. Also, the manuscript data shall be marked with a chapter range such as <Chapter 1>.

  S22: The mdiff processing unit 21 takes mdiff of these two files.

  S23: The draft deletion unit 22 deletes all the draft data while leaving only chapter information such as <Chapter 1> in the difference part on the draft side. Also delete the mdiff symbol “; △△△△△△” etc.

  S24: The draft deletion unit 22 outputs the result of deleting all the draft data while leaving only chapter information in the difference portion on the draft side.

(5): Explanation of question answering system using optimum matching ability This section describes a question answering system (question answering language processing system) that uses mdiff's optimum matching ability. The question answering system is, for example, a system that repeats the answer itself as “Tokyo” when asked “Where is the capital of Japan?”.

  Assuming that the knowledge is written in a natural language, the question sentence and the knowledge sentence are basically collated, and the result of the collation corresponding to the question word is output as an answer. For example, in the previous problem, a sentence “Tokyo is the capital of Japan” is searched, and “Tokyo” corresponding to the question word is output as a solution in this sentence. Here we consider doing this with mdiff.

  First, the question word part of the question sentence is replaced with X, and the end of the sentence is converted to a plain text to obtain “the capital of Japan is X”. In addition, “Tokyo is the capital of Japan” from the knowledge base. If we take these two mdiffcs, we get the following result.

The capital of Japan is
; ▽▽▽▽▽▽
X
; ●●●
Tokyo
; △△△△△△
Here, if the solution is the combination of X and the difference part, “Tokyo” can be extracted correctly.

  By the way, when using mdiffc, even if there is a slight discrepancy, you can get the answer correctly. For example, suppose that the knowledge base sentence is “Tokyo is the capital of Japan”. In this case, the result of mdiffc is as follows.

Japan
; ▽▽▽▽▽▽
; ●●●
Country
; △△△△△△
The capital city is
; ▽▽▽▽▽▽
X
; ●●●
Tokyo
; △△△△△△
The difference part increases a little, but the part corresponding to X remains “Tokyo”, and the solution can be extracted correctly.

  By the way, in the question answering system proposed by us, the transformation using the similarity as a scale is repeated, and the above matching is performed in a state where the question sentence and the sentence of the knowledge data are more consistent. For this purpose, it is necessary to define similarity.

  When mdiff is used, a matching part and a non-matching part can be recognized, and the similarity can be defined as (number of characters of matching part) / (total number of characters). Here, mdiff is used to calculate the similarity. Thus mdiff is also useful for finding similarity / similarity of sentences.

  Here, if there is a rule to paraphrase “Japan” and “Japan”, “Japan ’s capital is X” will be rephrased as “Japan's capital is X” and collation will be reduced, and the inconsistency will be reduced, so To get a solution.

  In this way, it is valuable from the viewpoint of simplicity to be able to easily construct a language processing system that responds to simple questions using just mdiff.

  FIG. 16 is an explanatory diagram of the question answering system. 16, the question answering system includes a question sentence conversion unit 31, a question sentence storage unit 32, a keyword extraction unit 33, a database sentence search unit 34, a database sentence storage unit 35, a similarity calculation unit 36, an mdiff processing unit 37, A question sentence transformation unit 38, a database sentence transformation unit 39, and a corresponding part output unit 40 are provided. The question sentence conversion unit 31 converts the question sentence into a plain sentence and the question word into X. The question sentence storage unit 32 stores the question sentence. The keyword extraction unit 33 extracts keywords from the question sentence. The database sentence search unit 34 searches for a database sentence including many keywords. The database sentence storage unit 35 stores the database sentence searched by the database sentence search unit 34. The similarity calculation unit 36 calculates the similarity by comparing the question sentence and the database sentence. The mdiff processing unit 37 takes mdiff of the question sentence and the database sentence in character units. The question sentence deforming unit 38 deforms the question sentence. The database sentence transformation unit 39 performs transformation of the database sentence. The corresponding unit output unit 40 extracts and outputs the database-side expression corresponding to “X” from the output result of the mdiff processing unit 37.

  FIG. 17 is a flowchart of the question response process. Hereinafter, a description will be given according to processing S31 to S39 of FIG.

  S31: Appropriately assign the value of the similarity S0 of the question answering system (usually set to “0” at first) and proceed to the processing S32.

  S32: A question sentence is given as an input of the question answering system, and the process proceeds to step S33.

  S33: The question sentence conversion unit 31 converts the question sentence of the given question sentence into a plain sentence and also converts the question word into X, and passes this converted result sentence to the question sentence storage unit 32.

  S34: The question sentence given as input is also passed to the keyword extracting unit 33, and the keyword is extracted from the question sentence. The extracted keyword is transferred to the database sentence search unit 34, and the process proceeds to step S35.

  S35: The database sentence search unit 34 searches the knowledge base (database not shown) for several database sentences including many keywords, and the result is stored in the database sentence storage unit 35, and the process proceeds to step S36.

  S36: The similarity calculation unit 36 compares the question sentence and the database sentence to obtain the similarity. The mdiff processing unit 37 is used for calculating the similarity. That is, the question sentence and the database sentence for which the similarity is to be obtained are input to the mdiff processing unit 37, and the number of characters in the matching part and the number of characters in the non-matching part are obtained from this mdiff result. The degree of similarity is determined in advance as (number of characters in matching part) / (total number of characters). Then, the similarity is obtained for all sentence pairs in the question sentence storage unit 32 and the database sentence storage unit 35, and the similarity degree when the similarity degree is the highest is S. Next, the similarity calculation unit 36 determines whether the similarity S is greater than S0. In this determination, if the similarity S is greater than S0, the process proceeds to S37, and if the similarity S is the same as S0, the process proceeds to S38.

  S37: The question sentence transformation unit 38 and the database sentence transformation unit 39 transform the question sentence and the database sentence, respectively, and store the transformation results in the respective storage units. The similarity calculation unit 36 sets the value of the similarity S to S0 and returns to the process S36. Here, the transformation means that “the capital of Japan is X” can be changed to “the capital of Japan is X”.

  S38: The similarity calculation unit 36 passes the output result of the mdiff processing unit 37 used for obtaining the value of the similarity S at this time to the corresponding unit output unit 40, and proceeds to processing S39.

  S39: The corresponding unit output unit 40 extracts and outputs the database-side expression corresponding to “X” from the output result of the mdiff processing unit 37.

  In the above embodiment, the diff command is used to detect the difference. However, if the system performs the association that maximizes the matching portion while maintaining the order information in a predetermined unit, other differences may be used. Detection means can be used.

(6): Description of program installation Input unit 1, processing unit 2, morpheme analysis unit 3, difference detection unit 4, similarity calculation unit 5, UNIXdiff processing unit 11, shaping unit 12, mdiff processing unit 21, draft removal unit 22 , Question sentence conversion unit 31, question sentence storage unit 32, keyword extraction unit 33, database sentence search unit 34, database sentence storage unit 35, similarity calculation unit 36, mdiff processing unit 37, question sentence transformation unit 38, database sentence transformation The unit 39, the corresponding unit output unit 40, and the like can be configured by programs, and are executed by the main control unit (CPU) and stored in the main memory. This program is generally processed by a computer. This computer is composed of hardware such as an input device which is an input means such as a main control unit, a main memory, a file device, a display device, and a keyboard.

  The program of the present invention is installed on this computer. In this installation, these programs are stored in a portable recording (storage) medium such as a floppy disk or a magneto-optical disk, and a drive device for accessing the recording medium provided in the computer is used. Alternatively, it is installed in a file device provided in the computer via a network such as a LAN. Then, the program steps necessary for processing are read from the file device into the main memory and executed by the main control unit.

It is explanatory drawing of the language processing system of this invention. It is explanatory drawing of the language processing system of mdiff in embodiment. It is a flowchart by mdiff in an embodiment. It is explanatory drawing of the example of the written word data and spoken word data in embodiment. It is explanatory drawing of the result of diff of written language data and spoken language data in embodiment. It is explanatory drawing of extraction of the difference part in embodiment. It is explanatory drawing of the structure of corpus in embodiment. It is explanatory drawing of the bilingual corpus matched by mdiff in embodiment. It is explanatory drawing of the language processing system of the bilingual corpus in embodiment. It is a flowchart by mdiff of the bilingual corpus in embodiment. It is explanatory drawing of a structure of the draft data in embodiment. It is explanatory drawing of the result of mdiff of a preliminary draft and a lecture in an embodiment. It is explanatory drawing of the insertion result of the information of the chapter to the lecture data in embodiment. It is explanatory drawing of the language processing system of matching of the lecture and preliminary draft in embodiment. It is a processing flowchart of matching of a lecture and a preliminary report in an embodiment. It is explanatory drawing of the question answering system in embodiment. It is a process flowchart of the question response in embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input means 2 Processing part 3 Morphological analysis means 4 Difference detection means 5 Similarity calculation means

Claims (4)

Input means for inputting a plurality of language data of a first language data including a symbol to be a mark of correspondence and a second language data not including the symbol;
A difference detection means for detecting a common part and a difference part of the plurality of language data using a system that associates a plurality of data so as to maximize the matching part while retaining the order information,
The difference detection means extracts the language data as it is from the common part, extracts the language data from the difference part corresponding to the second language data from the difference part, and the first language data is By extracting the symbols from the difference part on the corresponding side and arranging them in the order as they are, by having the second language data with the symbols inserted, it is possible to associate the plurality of language data A featured language processing system. A system for associating preliminary data with its lecture data,
The language processing system according to claim 1, wherein the plurality of language data are preliminary data and lecture data. 3. The language processing system according to claim 1, wherein a diff command is used as a system for associating a plurality of data so as to maximize a matching portion while maintaining the order information. A process of inputting a plurality of language data of a first language data including a symbol to be a mark of correspondence and a second language data not including the symbol;
A process for detecting a common part and a difference part of the plurality of language data using a system for associating a plurality of data so as to maximize the matching part while retaining the order information;
The language data is taken out from the common part as it is, the language data is taken out from the difference part corresponding to the second language data from the difference part, and the difference part on the side corresponding to the first language data The process of associating the plurality of language data by having the second language data in which the symbols are inserted by taking out the symbols and arranging them in the order as they are,
A program to be realized on a computer.

Images