JP5776539B2 - Extraction apparatus, extraction program, and extraction method - Google Patents

Description

  The present invention relates to an extraction device, an extraction program, and an extraction method.

  One of the elemental technologies of natural language processing is a technique for extracting a specific expression such as a person name, a place name, an organization name, a time expression, and a numerical expression from an electronic text (for example, Non-patent document 1 below).

  The technique of extracting a specific expression is used for information retrieval, information extraction, syntax analysis, text mining, or the like. In general, proper nouns such as personal names, place names, and organization names, types of words such as time expressions, and numerical expressions are collectively referred to as specific expressions.

  In addition, as one of techniques for improving the accuracy of specific expression extraction, there is a technique for extracting a plurality of words having the same notation in the extraction target text as words of the same type of specific expression (for example, Non-patent document 2 below).

  Further, as a related technique, there is a technique for correcting an extraction result based on the appearance frequency of the type of the unique expression extracted from the text (see, for example, Patent Document 1 below). In addition, there is a technique that uses two types of named entity extractors in combination (for example, see Patent Document 2 below).

  In addition, as one of machine learning techniques for automatically generating rules used for specific expression extraction, there is a Boosting learning technique (see, for example, Patent Documents 3 and 4 and Non-Patent Document 3 below). In addition, there is a learning technique using SVM (Support Vector Machine) (for example, see Non-Patent Documents 4 and 5 below).

JP 2007-148785 A JP 2006-330935 A JP 2010-33213 A JP 2010-33214 A

Kiyotaka Uchimoto, Qing Ma, Masaki Murata, Hiromi Ozaku, and Hitoshi Isahara, “Named Entity Extraction Based Amplitude Excited. 326-335 Jenny Rose Finkel, Trond Grenager, and Christopher Manning, “Incorporating Non-Local Information into Information Extraction Systems by Gibbs 05” Robert E. Schapier and Yoram Singer, “BoosTexter: A Boosting-based System for Text Categorization” Machine Learning Volume 39, Numbers 2-3, pp135-168 Vladimir N. Vapnik, “Statistical Learning Theory” Wiley-Interscience, 1998. T. T. et al. Joachims, “Training Linear SVMs in Linear Time” Proceeding KDD '06 Proceedings of the 12th ACM SIGKDD international conferencing on Knowledge discovery.

  However, in the above-described prior art, when there is a group of words with the same notation in the text to be extracted, it is assumed that the types of specific expressions of words with the same notation are likely to be the same within the same document. Based on this, it is a method of giving a high score when words of the same notation are of the same specific expression type. Therefore, if this assumption does not hold, there is a problem that the word group is mistakenly extracted as the same type of words.

  An object of the present invention is to provide an extraction apparatus, an extraction program, and an extraction method capable of improving the extraction accuracy of a specific expression in order to eliminate the above-described problems caused by the related art.

  In order to solve the above-described problems and achieve the object, according to one aspect of the present invention, a combination of co-occurrence words and words of the same notation appearing with each of the co-occurrence words are the same type of words. Information indicating whether or not, a first storage unit that stores a discrimination rule that associates the information, a combination of a co-occurrence word and a distance to the co-occurrence word, and information indicating a type of word defined according to the distance And a second storage unit that stores the extraction rule associated with the first word and the first word and the second word having the same notation as the first word are detected from the series of words It is determined whether or not the first storage unit has a determination rule in which the first word that has been generated appears with one of the co-occurrence words and the second word that has been detected appears with the other of the co-occurrence words If there is a discriminant rule, the first word and the second simple word from the discriminant rule. Are determined to be of the same type, and if it is determined to be of the same type, a word existing within a predetermined distance from each of the first word and the second word from the series of words and the word A combination with the distance to the first, and the information indicating the type of the word associated with the identified combination is extracted from the extraction rule stored in the second storage unit, the first word and the second An extraction device, an extraction program, and an extraction method for providing a word and outputting the given series of words are proposed.

  According to one aspect of the present invention, it is possible to improve the extraction accuracy of a specific expression.

FIG. 1 is an explanatory diagram showing the contents of specific expression extraction by the extraction device. FIG. 2 is a block diagram illustrating a hardware configuration example of the extraction device 100. FIG. 3 is an explanatory diagram showing an example of the stored contents of the same notation / type word discrimination rule 300. FIG. 4 is an explanatory diagram showing an example of the contents stored in the specific expression extraction rule 400. FIG. 5 is a block diagram illustrating a functional configuration of the extraction device 100. FIG. 6 is an explanatory diagram illustrating Example 1 of learning data used for the rule learning process by the extraction device 100. FIG. 7 is an explanatory diagram (part 1) showing the contents of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 600 shown in FIG. FIG. 8 is an explanatory diagram (part 2) showing the contents of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 600 shown in FIG. FIG. 9 is an explanatory diagram (part 3) showing the contents of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 600 shown in FIG. FIG. 10 is a flowchart (part 1) illustrating a detailed processing procedure of the rule learning process according to the first embodiment. FIG. 11 is a flowchart (part 2) of the detailed process procedure of the rule learning process according to the first embodiment. FIG. 12 is an explanatory diagram (part 1) of a specific example 1 of the specific expression extraction process performed by the extraction apparatus 100 according to the first embodiment. FIG. 13 is an explanatory diagram (part 2) of a specific example 1 of the named entity extraction process performed by the extraction apparatus 100 according to the first embodiment. FIG. 14 is an explanatory diagram of an output example 1 of the specific expression extraction result by the extraction device 100 according to the first embodiment. FIG. 15 is a flowchart of a detailed process procedure of the named entity extraction process according to the first embodiment. FIG. 16 is an explanatory diagram illustrating Example 2 of learning data used for the rule learning process performed by the extraction device 100. FIG. 17 is an explanatory diagram (part 1) showing the contents of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 1600 shown in FIG. FIG. 18 is an explanatory diagram (part 2) showing the contents of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 1600 shown in FIG. FIG. 19 is an explanatory diagram (part 3) showing the contents of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 1600 shown in FIG. FIG. 20 is an explanatory diagram (part 1) of a specific example 2 of the specific expression extraction process performed by the extraction apparatus 100 according to the first embodiment. FIG. 21 is an explanatory diagram (part 2) of a specific example 2 of the specific expression extraction process performed by the extraction apparatus 100 according to the first embodiment. FIG. 22 is an explanatory diagram of an output example 2 of the specific expression extraction result by the extraction device 100 according to the first embodiment. FIG. 23 is an explanatory diagram of an example of a chunk lattice used in the rule learning process by the extraction device 100 according to the second embodiment. FIG. 24 is an explanatory diagram (part 1) showing the contents of generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400 using the chunk lattice generated in FIG. FIG. 25 is an explanatory diagram (part 2) showing the contents of generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400 using the chunk lattice generated in FIG. FIG. 26 is an explanatory diagram (part 3) showing the contents of generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400 using the chunk lattice generated in FIG. FIG. 27 is a flowchart (part 1) illustrating a detailed processing procedure of the rule learning process according to the second embodiment. FIG. 28 is a flowchart (part 2) of the detailed process procedure of the rule learning process according to the second embodiment. FIG. 29 is an explanatory diagram illustrating an example of a chunk lattice that is a target of extraction of a specific expression by the extraction device 100. FIG. 30 is an explanatory diagram (part 1) of a specific example of the specific expression extraction process performed by the extraction apparatus 100 according to the second embodiment. FIG. 31 is an explanatory diagram (part 2) of a specific example of the specific expression extraction process performed by the extraction apparatus 100 according to the second embodiment. FIG. 32 is an explanatory diagram of an output example of the specific expression extraction result by the extraction device 100 according to the second embodiment. FIG. 33 is a flowchart of a detailed process procedure of the named entity extraction process according to the second embodiment.

  Exemplary embodiments of an extraction apparatus, an extraction program, and an extraction method according to the present invention will be described below in detail with reference to the accompanying drawings.

(Contents of specific expression extraction by the extractor)
First, the contents of specific expression extraction by the extraction device will be described with reference to FIG.

  FIG. 1 is an explanatory diagram showing the contents of specific expression extraction by the extraction device. In FIG. 1, an extraction apparatus 100 is an apparatus that extracts a specific expression from text data 110 that is a target of extraction of a specific expression. The extraction apparatus 100 has the same notation / type word discrimination rule 300 and a specific expression extraction rule 400 that are referred to for extraction of a specific expression.

  The same notation / type word discrimination rule 300 is a table for storing discrimination rules. The discrimination rule is a rule that is referred to in order to discriminate whether or not a set of words having the same notation is a word of the same specific expression type. The discrimination rule is, for example, a rule that associates a combination of co-occurrence words with information indicating whether or not a pair of words having the same notation appearing with each combination of the co-occurrence words is of the same type. .

  Here, the co-occurrence word is a word that appears with each of two words having the same notation in the sentence. For example, in the case of “Mr. Miyazaki from Miyazaki”, two “Miyazaki” are two words with the same notation, and “Birth” and “San” appearing with each “Miyazaki” are co-occurrence words. In addition, although co-occurrence words usually have different notations, in the present embodiment, even words having the same notation are treated as combinations of co-occurrence words. Note that “◯” in FIG. 1 is information indicating the same type, and “X” is information indicating different types.

  In the example of FIG. 1, “Miyazaki” in front of the first “from” is “place”, and “Miyazaki” in front of the next “san” is a person's name. It becomes "x" which shows that it is not the same kind. Also, for example, if the text data is “Go to Miyazaki with a friend from Miyazaki”, both “Miyazaki” are “places”, and therefore the discrimination results for these two words are of the same type. It becomes “○”.

  The specific expression extraction rule 400 is a table that stores extraction rules. The extraction rule is a rule that is referred to in order to extract what kind of word a certain word is. The extraction rule is, for example, a rule indicating what kind of word the type is to be extracted from when a certain word exists at a predetermined distance from the word whose type is to be extracted. 1 is information indicating that the co-occurrence word “origin” exists one word after the word whose type is to be extracted. “LOC (LOCATION)” is information indicating that the type of word is a place, and “PER (PERSON)” is information indicating that the type of word is a person.

  In the example of FIG. 1, the content of the specific expression extraction by the extraction apparatus 100 will be described by taking as an example the case where the text data 110 from which the specific expression is extracted is “Miyazaki from Miyazaki and going to Miyazaki.” .

  (1) First, the extraction apparatus 100 detects a word having the same notation from the text data 110 and determines whether or not the detected word is a specific expression of the same type. As a result, the extraction apparatus 100 can detect words of the same notation and the same type, and can handle the same notation and the same type of words together in the subsequent processing.

  In the example of FIG. 1, the extraction apparatus 100 detects a set of “Miyazaki” 111 and “Miyazaki” 112 as the same notation words from the text data 110. Next, the extraction apparatus 100 identifies combinations of words that appear together with each of “Miyazaki” 111 and “Miyazaki” 112 from the text data 110 as clues for determining whether or not they are of the same type. Here, the clue is a key that serves as an index when discriminating certain data. Specifically, the extraction apparatus 100, for example, combines word combinations “from & san” and “no & san” within the distance of two words before and after each of “Miyazaki” 111 and “Miyazaki” 112, It is specified as a clue to discriminate whether or not they are of the same type.

  Here, the same notation / type word discrimination rule 300 stores co-occurrence words “origin” and “san” corresponding to the identified clue “origin & san”. In the same notation / type word discrimination rule 300, a discrimination rule including a discrimination result “x” indicating that a pair of words having the same notation appearing with each of the co-occurrence words is of a different type is stored. Yes. Therefore, the extraction apparatus 100 determines that “Miyazaki” 111 and “Miyazaki” 112 are different types based on the same notation / type word determination rule 300 and the clue “from origin & san”.

  Similarly, the extraction apparatus 100 detects “Miyazaki” 111 and “Miyazaki” 113 as the same notation words from the text data 110. Next, the extraction apparatus 100 specifies from the text data 110 a combination of words “Birth & Go” and “Birth & Go” that appear together with “Miyazaki” 111 and “Miyazaki” 113 as clues.

  Here, the same notation / type word discrimination rule 300 stores the co-occurrence words “origin” and “go” corresponding to the identified clue “origin & go”. The same notation / type word discrimination rule 300 stores a discrimination rule including a discrimination result “◯” indicating that a pair of words having the same notation appearing with each of the co-occurrence words is of the same type. Yes. Therefore, the extraction apparatus 100 determines that “Miyazaki” 111 and “Miyazaki” 113 are of the same type based on the same notation / type word determination rule 300 and the clue “from & going”.

  Similarly, the extraction apparatus 100 detects “Miyazaki” 112 and “Miyazaki” 113 as the same notation words from the text data 110. Next, the extraction apparatus 100 identifies from the text data 110 the word combinations “San & He” and “San & Go” that appear with each of “Miyazaki” 112 and “Miyazaki” 113 as clues.

  Here, the same notation / type word discrimination rule 300 stores co-occurrence words “san” and “go” corresponding to the identified clue “san & go”. In the same notation / type word discrimination rule 300, a discrimination rule including a discrimination result “x” indicating that a pair of words having the same notation appearing with each of the co-occurrence words is of a different type is stored. Yes. Therefore, the extraction apparatus 100 determines that “Miyazaki” 112 and “Miyazaki” 113 are different types based on the same notation / type word determination rule 300 and the clue “San & Go”.

  Thereby, the extraction apparatus 100 determines that “Miyazaki” 111 and “Miyazaki” 113 are the same notation and the same type of word, and collects “Miyazaki” 111 and “Miyazaki” 113 together in the subsequent processing. Can be handled.

  (2) The extraction device 100 collects the words that are identified as the same notation and the same type in the text data 110 in (1). Then, the extraction device 100 identifies a clue for extracting a unique expression for each word in the text data 110 or for each grouped word, and extracts a word type from the identified clue. Thereby, the extraction apparatus 100 can extract the same word type for the same notation and the same type of words. Further, the extraction apparatus 100 can extract the type of a word separately from other words having the same notation, even if they have the same notation.

  In the example of FIG. 1, the extraction apparatus 100 collects “Miyazaki” 111 and “Miyazaki” 113 that are identified as the same notation and the same type, and within a predetermined distance from each of “Miyazaki” 111 and “Miyazaki” 113. A combination of an existing word and a distance to the word is specified as a clue for extraction. Here, the distance is a distance between words, and is determined by the number of words and the number of characters. Hereinafter, the number of words expressed as “+” in the backward direction of the sentence and the number of words expressed as “−” in the forward direction of the sentence are adopted as the distance. For example, the distance of a word existing “one word behind” a certain word from the certain word is “+1”. Further, for example, the distance of a word existing “one word before” a certain word from the certain word is “−1”. Note that the distance of a certain word itself from a certain word is “0”.

  In FIG. 1, a distance of two words before and after is adopted as the predetermined distance. Specifically, for example, since “from” is “one word behind” from “Miyazaki” 111, the extraction apparatus 100 specifies “from: +1” as a clue. Similarly, the extraction apparatus 100 identifies “no: +2”, “san: -2”, “to: −1”, “to: +1”, and “go: +2” as clues.

  Here, the specific expression extraction rule 400 stores an extraction rule indicating that the word type indicated by the clue “origin: +1” is “LOC”. In addition, the specific expression extraction rule 400 stores an extraction rule indicating that the type of the word indicated by the clue “go: +2” is “LOC”. Therefore, the extraction apparatus 100 extracts that the types of “Miyazaki” 111 and “Miyazaki” 113 are “LOC” from the specific expression extraction rules 400 and the clues “from: +1” and “go: +2”. .

  On the other hand, for “Miyazaki” 112 having no other words of the same type, the extraction apparatus 100 extracts a combination of a word existing within the distance of two words before and after “Miyazaki” 112 and the distance to the word. Identify as a clue. Specifically, for example, the extraction apparatus 100 specifies “origin: −2”, “no: −1”, “san: +1”, and “to: +2” as clues. Here, the specific expression extraction rule 400 stores an extraction rule indicating that the word type indicated by “san: +1” is “PER”. For this reason, the extraction apparatus 100 extracts that the type of “Miyazaki” 112 is “PER” from the specific expression extraction rule 400 and “san: +1”.

  Accordingly, the extraction apparatus 100 can extract the same type “LOC” as a group for “Miyazaki” 111 and “Miyazaki” 113 having the same notation and the same type. Further, the extraction apparatus 100 extracts the word type “PER” separately from “Miyazaki” 111 and “Miyazaki” 113 of the same notation for different types of “Miyazaki” 112 even if they have the same notation. Will be able to.

  As a result, the extraction device 100 prevents the extraction of the same type of word as a different type of word by extracting the same notation and the same type of word together and extracting the same type of word, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 treats different types of words separately even if they are the same notation, and extracts the type of the word, thereby preventing erroneous extraction as the same type of word and improving the extraction accuracy. Improvements can be made.

  (3) Then, the extraction device 100 outputs the extraction result. Specifically, the extraction device 100 outputs, for example, text data 110 to which the extracted word type is assigned as a tag to the display. Note that the extraction device 100 may transmit the text data 110 after the grant to another computer via a network, or may output it to a recording medium. Thereby, the extraction apparatus 100 can notify the user of the extraction apparatus 100 of the type of word in the text data 110. Further, the extraction apparatus 100 can provide text data 110 with a tag attached to other software (for example, translation software or information search software).

  Note that the extraction apparatus 100 may determine that a word with a notation swing (for example, “Miyazaki”, “Miyazaki”, “Miyazaki”, etc.) is the same notation including a case where there is a notation swing. Further, the extraction device 100 may treat the co-occurrence words as the same co-occurrence words including the cases where the co-occurrence words can be utilized (for example, “go”, “go” and “go”).

(Hardware configuration example of extraction device 100)
Next, a hardware configuration example of the extraction apparatus 100 will be described with reference to FIG.

  FIG. 2 is a block diagram illustrating a hardware configuration example of the extraction device 100. In FIG. 2, an extraction apparatus 100 includes a CPU (Central Processing Unit) 201, a ROM (Read-Only Memory) 202, a RAM (Random Access Memory) 203, a magnetic disk drive 204, a magnetic disk 205, and an optical disk drive. 206, an optical disk 207, a display 208, an I / F (Interface) 209, a keyboard 210, a mouse 211, a scanner 212, and a printer 213. Each component is connected by a bus 220.

  Here, the CPU 201 governs overall control of the extraction apparatus 100. The ROM 202 stores a program such as a boot program. The RAM 203 is used as a work area for the CPU 201. The magnetic disk drive 204 controls reading / writing of data with respect to the magnetic disk 205 according to the control of the CPU 201. The magnetic disk 205 stores data written under the control of the magnetic disk drive 204.

  The optical disk drive 206 controls reading / writing of data with respect to the optical disk 207 according to the control of the CPU 201. The optical disk 207 stores data written under the control of the optical disk drive 206, or causes the computer to read data stored on the optical disk 207.

  The display 208 displays data such as a document, an image, and function information as well as a cursor, an icon, or a tool box. As the display 208, for example, a CRT, a TFT liquid crystal display, a plasma display, or the like can be adopted.

  An interface (hereinafter abbreviated as “I / F”) 209 is connected to a network 214 such as a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet through a communication line. Connected to other devices. The I / F 209 controls an internal interface with the network 214 and controls data input / output from an external device. For example, a modem or a LAN adapter may be employed as the I / F 209.

  The keyboard 210 includes keys for inputting characters, numbers, various instructions, and the like, and inputs data. Moreover, a touch panel type input pad or a numeric keypad may be used. The mouse 211 performs cursor movement, range selection, window movement, size change, and the like. A trackball or a joystick may be used as long as they have the same function as a pointing device.

  The scanner 212 optically reads an image and captures image data into the extraction apparatus 100. The scanner 212 may have an OCR (Optical Character Reader) function. The printer 213 prints image data and document data. As the printer 213, for example, a laser printer or an ink jet printer can be adopted.

(The stored contents of the same notation / type word discrimination rule 300)
Next, the stored contents of the same notation / type word discrimination rule 300 shown in FIG. 1 will be described with reference to FIG. As described above, the same notation / type word discrimination rule 300 is a table included in the extraction apparatus 100 and stores discrimination rules. The discrimination rule is a rule that is referred to in order to discriminate whether or not a pair of words having the same notation is the same type of word. The discrimination rule is, for example, a rule that associates a combination of co-occurrence words with information indicating whether or not a pair of words having the same notation appearing with each combination of the co-occurrence words is of the same type. . The same notation / type word discrimination rule 300 is realized by the RAM 203, the magnetic disk 205, the optical disk 207, and the like.

  FIG. 3 is an explanatory diagram showing an example of the stored contents of the same notation / type word discrimination rule 300. As shown in FIG. 3, the same notation / type word discrimination rule 300 has a discrimination result item associated with each rule item, and constitutes a record for each rule.

  In the rule item, a combination of co-occurrence words appearing with each of a set of words having the same notation is stored. Specifically, for example, a combination of the co-occurrence words “from” and “san” and a combination of the co-occurrence words “from” and “go” are stored in the rule item. In the discrimination result item, information indicating whether or not a pair of words having the same notation appearing together with each of the co-occurrence words of the combination of rule items is the same type is stored. In FIG. 3, “◯” is written for the same type, and “X” is written for different types.

(Contents stored in specific expression extraction rule 400)
Next, the stored contents of the specific expression extraction rule 400 shown in FIG. 1 will be described with reference to FIG. As described above, the specific expression extraction rule 400 is a table included in the extraction apparatus 100 and stores the extraction rule. The extraction rule is a rule that is referred to in order to extract what kind of word a certain word is. The extraction rule is, for example, a rule indicating what kind of word the type is to be extracted from when a certain word exists at a predetermined distance from the word whose type is to be extracted. The specific expression extraction rules 400 are realized by the RAM 203, the magnetic disk 205, the optical disk 207, and the like.

  FIG. 4 is an explanatory diagram showing an example of the contents stored in the specific expression extraction rule 400. As shown in FIG. 4, the extraction rule has a type item in association with each rule item, and constitutes a record for each rule.

  The rule item stores a combination of the co-occurrence word of the extraction target word and the distance from the extraction target word to the co-occurrence word. As described above, the distance is a distance between words and is determined by the number of words and the number of characters. As the distance, for example, the number of words expressed as “+” in the backward direction of the sentence is adopted. For example, the distance of a word existing “one word behind” a certain word from the certain word is “+1”. Specifically, for example, “origin: +1” indicating that “origin” exists behind “one word behind” a certain word is stored in the rule item.

  The type item stores the type of word to be extracted when the combination corresponds to a combination of rule items. The types of words to be extracted include, for example, “organization name (ORGANIZATION)”, “person name (PERSON)”, “place name (LOCATION)”, “date expression (DATE)”, “time expression (TIME)”, “Money amount expression (MONEY)”, “Percentage expression (PERCENT)”, “Inherent name (ARTIFACT)”, and the like.

(Functional configuration example of the extraction device 100)
Next, a functional configuration example of the extraction device 100 will be described with reference to FIG.

  FIG. 5 is a block diagram illustrating a functional configuration of the extraction device 100. The extraction apparatus 100 includes a first storage unit 501, a second storage unit 502, an input unit 503, a detection unit 504, a determination unit 505, a specification unit 506, an extraction unit 507, and an output unit 508. The first acquisition unit 509, the second acquisition unit 510, the determination unit 511, the generation unit 512, the storage unit 513, and the conversion unit 514 are included. Specifically, the input unit 503 to the generation unit 512 and the conversion unit 514 cause the CPU 201 to execute a program stored in a storage device such as the ROM 202, the RAM 203, the magnetic disk 205, and the optical disk 207 illustrated in FIG. Or the I / F 209 realizes the function.

  For example, the extraction device 100 can extract a word having a unique expression from a series of words divided in units of words. In addition, the extraction apparatus 100 may extract a chunk of a unique expression from a lattice of chunks in which chunks are nodes and links between them. Here, a chunk is a lump of one or more words. Below, the function of the extraction apparatus 100 in the case of extracting the word of a specific expression from the series of words divided | segmented by the word unit first is demonstrated. A specific example of extracting a word of a unique expression from a series of words divided in units of words is an example described with reference to FIG. 1, an example described later with reference to FIGS. 12 to 14, or FIGS. This is an example described later with reference to FIG.

  The input unit 503 receives an input of a series of words that are the target of the specific expression extraction. Specifically, the input unit 503 receives an input of the text data 110 from the user of the extraction device 100 via the keyboard 210, for example. Specifically, the input unit 503 may accept the text data 110 received via the I / F 209, for example. The received data is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. As a result, the input unit 503 can receive an input of a series of words to be subjected to specific expression extraction and generate a trigger for starting the specific expression extraction process.

  The first storage unit 501 stores a discrimination rule that associates a combination of co-occurrence words with information indicating whether or not the same notation words appearing together with the co-occurrence words are the same type of word. . Here, a co-occurrence word is a word that appears simultaneously with a certain word in a sentence. Specifically, the first storage unit 501 is, for example, the same notation / type word discrimination rule 300 described above.

  The second storage unit 502 stores an extraction rule in which a combination of a co-occurrence word and a distance to the co-occurrence word is associated with information indicating the type of word defined according to the distance. Here, the distance is a distance between words, and is determined by the number of words and the number of characters. As the distance, for example, information indicating how many words are from the starting word can be employed.

  More specifically, for example, the distance from the starting word on the sentence to the word existing one word behind is “+1”. Further, the distance from the starting word on the sentence to the word existing one word before is “−1”. Specifically, the second storage unit 502 is, for example, the above-described specific expression extraction rule 400.

  The detection unit 504 detects a first word and a second word having the same notation as the first word from the series of words. Specifically, for example, the detection unit 504 morphologically analyzes the text data 110 input by the input unit 503, divides the text data 110 into words, and generates a series of words. Then, the detection unit 504 detects words having the same notation from the generated series of words.

  In the example of FIG. 1, the detection unit 504 performs morphological analysis on the text data 110 “Miyazaki (202) from Miyazaki (201) and goes to Miyazaki (203)”, and divides the word data into words “from Miyazaki. Go to Miyazaki with Mr. Miyazaki. " In addition, the number in () is the code | symbol 201-203 shown in FIG. Then, the detection unit 504 detects a set of “Miyazaki” 111 and “Miyazaki” 112, a set of “Miyazaki” 111 and “Miyazaki” 113, and a set of “Miyazaki” 112 and “Miyazaki” 113 with the same notation. In the following, a description will be given by taking a pair of “Miyazaki” 111 and “Miyazaki” 113 as an example.

  The detected word is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207. Thereby, the detection unit 504 can detect a set of words to be determined by the determination unit 505, and can determine whether the set of words is the same type.

  The discriminating unit 505 is a discriminating rule in which the first word detected by the detecting unit 504 appears together with one of the co-occurrence words, and the second word detected by the detecting unit 504 appears together with the other of the co-occurrence words. Is stored in the first storage unit 501. Then, when there is a determination rule, the determination unit 505 determines whether the first word and the second word are of the same type based on the determination rule.

  Specifically, the determination unit 505 specifies, for example, a combination of words existing around each of the same notation words detected by the detection unit 504 as a clue for determination. Here, the clue is a key that serves as an index for distinguishing words. In addition, the word which exists in the periphery is a word which exists within a predetermined distance, for example, is a word which is within the distance of two words before and after each of the words of the same notation.

  Next, the determination unit 505 determines whether there is a combination corresponding to any one of the specified clues among the co-occurrence word combinations of the rule items of the same notation / type word determination rule 300. Then, when there is a corresponding combination, the determination unit 505 refers to a determination result item corresponding to the combination and determines whether or not the same notation words are the same type.

  More specifically, the determination unit 505, for example, a combination of words “Birth & Go” and “Birth & Go” that exist around each of “Miyazaki” 111 and “Miyazaki” 113 detected by the detection unit 504. Are identified as clues for discrimination. Next, the determination unit 505 determines that the same notation / type word determination rule 300 includes the co-occurrence words “origin” and “go” corresponding to the identified clue “origin & go”. Then, the determination unit 505 determines that “Miyazaki” 111 and “Miyazaki” 113 are the same type because there is information indicating that they are the same type in the determination result item corresponding to the combination of the co-occurrence words. To do.

  Note that the determination unit 505 may determine words of the same notation as different types of words when there is no corresponding combination. The determination result is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. As a result, the determination unit 505 can detect a set of words to be handled together in extraction of word types.

  When the determining unit 505 determines that they are the same type, the specifying unit 506 selects a word existing within a predetermined distance from each of the first word and the second word and the word from the series of words. Identify combinations with distance. Specifically, for example, when the same notation and the same type of word exist, the specifying unit 506 uses, as a clue for extraction, a word that is a distance of two words before and after each word and the distance to the word. Identify.

  More specifically, the specifying unit 506 collectively handles, for example, “Miyazaki” 111 and “Miyazaki” 113 which are determined to be the same type by the determination unit 505. The identifying unit 506 identifies a combination of a word existing within the distance of two words before and after each of “Miyazaki” 111 and “Miyazaki” 113 and the distance to the word as a clue to extraction. Here, the identifying unit 506 uses, as clues for extraction, “origin: +1”, “no: +2”, “san: −2”, “to: −1”, “to: +1”, and “go: +2”. Is specified.

  The identified clue is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207. Thereby, the specifying unit 506 can specify a clue used by the extracting unit 507 to extract the type of word.

  The extraction unit 507 extracts information indicating the type of the word associated with the combination specified by the specifying unit 506 from the extraction rules stored in the second storage unit 502, and extracts the first word and the second Append to words.

  Specifically, the extraction unit 507, for example, a combination corresponding to any one of the clues specified by the specification unit 506 from combinations of the word of the rule item of the specific expression extraction rule 400 and the distance to the word. It is determined whether or not there is. Next, when there is a corresponding combination, the extraction unit 507 extracts the word type from the type item corresponding to the combination. Then, the extraction unit 507 assigns a tag indicating the type of the extracted word to each of the same notation and the same type of word on the sentence.

  More specifically, for example, the extraction unit 507 determines that there is a rule item corresponding to the clue “origin: +1” specified by the specification unit 506 in the specific expression extraction rule 400, and corresponds to the rule item. Information indicating that the type of the word is “LOC” is extracted from the type item. In addition, the extraction unit 507 determines that there is a rule item corresponding to the clue “go: +2” specified by the specification unit 506 in the specific expression extraction rule 400, and extracts a word from the type item corresponding to the rule item. Information indicating that the type is “LOC” is extracted. Then, the extraction unit 507 extracts “LOC” as the type of “Miyazaki” 111 and “Miyazaki” 113.

  Then, the extraction unit 507 generates the text data 110 “<LOC> Miyazaki </ LOC> from <PER> Miyazaki </ PER> and <LOC> Miyazaki </ LOC>” with the tag attached. . Information indicating the extracted word type is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Accordingly, the extraction unit 507 can extract the same word type for the same notation and the same type of word. As a result, the extraction unit 507 can separately extract word types for different types of words even if they have the same notation.

  The output unit 508 outputs a series of words given by the extraction unit 507. Specifically, the output unit 508 outputs, for example, the text data 110 provided with the extracted word type as a tag to the display 208. In addition, the output unit 508 may transmit the text data 110 after the grant to another computer via the I / F 209 or may output it to a recording medium. As a result, the output unit 508 can notify the user of the extraction device 100 of the type of word in the text data 110. Further, the output unit 508 can provide the text data 110 with the tag added to other software (for example, translation software or information search software).

  Further, the extraction apparatus 100 can automatically generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 used for the above-described extraction of the specific expression from the learning data group. Below, the function of the extraction device 100 when generating the same notation / type word discrimination rule 300 will be described first. This function is realized by the first acquisition unit 509, the second acquisition unit 510, the determination unit 511, the generation unit 512, and the storage unit 513. A specific example of generating the same notation / type word discrimination rule 300 will be described later with reference to FIGS.

  The first acquisition unit 509 acquires a combination of words having the same notation from a word string including a word to which information indicating a word type is given. Here, the word string is learning data used for generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400. The information indicating the type of word is a tag indicating the type of word assigned to the learning data. The tag is given by the creator of learning data. Here, the learning data used to generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 will be described with reference to FIG.

  FIG. 6 is an explanatory diagram illustrating Example 1 of learning data. As shown in FIG. 6, in order to generate the same notation / type word discrimination rule 300 and the unique expression extraction rule 400, a learning data group 600 including a plurality of learning data with tags attached thereto is prepared.

  For example, the learning data group 600 includes learning data 610 “<PER> Miyazaki </ LOC> from <PER> Miyazaki </ PER> and <LOC> Miyazaki </ LOC>”. The tag “LOC” is assigned to “Miyazaki” 611 in the learning data 610, the tag “PER” is assigned to “Miyazaki” 612, and the tag “LOC” is assigned to “Miyazaki” 613. ing.

  The learning data group 600 includes learning data 620 “<PER> Fukuoka </ PER> and <PER> Miyazaki </ PER> go to <LOC> Fukuoka </ LOC>”. The tag “PER” is assigned to Fukuoka 621 in the learning data 620, the tag “PER” is assigned to Miyazaki 622, and the tag “LOC” is assigned to Fukuoka 623.

  The learning data group 600 also includes learning data 630 “<PER> Miyazaki </ PER> goes to <LOC> Miyazaki </ LOC> on the Shinkansen”. The tag “PER” is assigned to Miyazaki 631 in the learning data 630, and the tag “LOC” is assigned to Miyazaki 632.

  The extraction apparatus 100 uses such a learning data group 600, so that when there is a pair of words that can be two types of “LOC” and “PER” with the same notation, the pair of words is A discrimination rule for discriminating whether or not they are the same type can be generated. Further, the extraction apparatus 100 can generate an extraction rule for extracting whether the word type is “LOC” or “PER”.

  Returning to FIG. 5, specifically, the first acquisition unit 509, for example, morphologically analyzes learning data, divides the learning data into words, and generates a word string. Then, the detecting unit 504 acquires a combination of words having the same notation from the generated word string.

  For example, the first acquisition unit 509 performs morphological analysis on the learning data 610 “Miyazaki (612) from Miyazaki (611) and goes to Miyazaki (613)”, and divides each word into “Miyazaki from Miyazaki” To go to Miyazaki. "As a word string. In addition, the number in () is the code | symbol 611-613 shown in FIG. The first acquisition unit 509 detects a pair of “Miyazaki” 611 and “Miyazaki” 612, a pair of “Miyazaki” 611 and “Miyazaki” 613, and a pair of “Miyazaki” 612 and “Miyazaki” 613 with the same notation. To do. In the following description, a set of “Miyazaki” 611 and “Miyazaki” 613 is taken as an example.

  The acquired words are stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Thereby, the first acquisition unit 509 can detect a set of words to be determined by the determination unit 511 and cause the determination unit 511 to determine whether or not the word sets are of the same type.

  The second acquisition unit 510 includes a combination of co-occurrence words that appears with each word of the combination acquired by the first acquisition unit 509 in a word string including a word to which information indicating the type of the word is given. Get from. Specifically, for example, the second acquisition unit 510 specifies a combination of words existing around each word of the combination acquired by the first acquisition unit 509 as a clue for determination. In addition, the word which exists in the periphery is a word which exists within a predetermined distance, for example, is a word which is within the distance of two words before and after each of the words of the same notation.

  More specifically, the second acquisition unit 510 is, for example, a combination of words “origin & e” existing around each of “Miyazaki” 111 and “Miyazaki” 113 acquired by the first acquisition unit 509. , "Born & Going" is obtained as a clue for discrimination. The obtained clues are stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Thereby, the second acquisition unit 510 can acquire a clue for generating a discrimination rule included in the same notation / type word discrimination rule 300.

  The determination unit 511 determines whether or not the words with the same notation are the same type of words based on the information indicating the type of word given to each word in the combination of words with the same notation. Specifically, the determination unit 511 determines, for example, whether or not tags assigned to each word of the combination match. Next, the determination part 511 determines that each word of a combination is the same kind of word, when it corresponds.

  More specifically, the determination unit 511, for example, in the text data 110 shown in FIG. 2, the tag given to “Miyazaki” 111 is “LOC”, and the tag given to “Miyazaki” 113 is Since it is “LOC” and the tags match, it is determined that they are of the same type. Note that the determination unit 511 may determine that each word in the combination is a different type of word if they do not match.

  The determination result is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207. Thereby, the determination unit 511 can determine whether the determination rules generated by the generation unit 512 are rules indicating the same type.

  The generation unit 512 generates a determination rule in which the combination of the co-occurrence words acquired by the second acquisition unit 510 and the determination result determined by the determination unit 511 are associated with each other. Specifically, the generation unit 512 uses a learning engine, for example, to identify a likely learning case from among a group of learning cases in which the acquired clues are associated with the determination result, and determines the identified learning case as a discriminating rule. Generate as Here, specifically as a learning engine, there exist a Boosting learning device and SVM which are the prior art, for example.

  As a plausible learning case, for example, a learning case whose appearance frequency in the learning case group is equal to or higher than a threshold value may be adopted. Specifically, for example, a learning case in which a combination of the co-occurrence words “from” and “go” is associated with the same type is used as a plausible learning case. Moreover, as a plausible learning example, for example, a learning example including a case particle “NO” that appears with many types of words may not be adopted. The generated discrimination rule is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Thereby, the generation unit 512 can generate a discrimination rule used for extraction of the specific expression.

  The storage unit 513 stores the determination rule generated by the generation unit 512 in the first storage unit 501. Specifically, for example, the storage unit 513 stores the generated discrimination rule in the same notation / type word discrimination rule 300. Thereby, the storage unit 513 can store the same notation / type word discrimination rule 300 used for the specific expression extraction processing.

  Next, the function of the extraction device 100 when generating the specific expression extraction rules 400 will be described. This function is realized by the detection unit 504, the determination unit 505, the specifying unit 506, the generation unit 512, and the storage unit 513. A specific example of generating the specific expression extraction rule 400 will be described later with reference to FIGS.

  The detection unit 504 detects a third word and a fourth word having the same notation as the third word from a word string including a word to which information indicating the type of word is given. Specifically, for example, the detection unit 504 morphologically analyzes learning data, divides the learning data into words, and generates a word string. Then, the detecting unit 504 detects the same notation word from the generated word string.

  For example, the detection unit 504 performs a morphological analysis on the learning data 610 “Miyazaki (612) from Miyazaki (611) and goes to Miyazaki (613)” and divides it into words “Miyazaki and Miyazaki from Miyazaki and Miyazaki. To go. " In addition, the number in () is the code | symbol 611-613 shown in FIG. Then, the detection unit 504 detects a set of “Miyazaki” 611 and “Miyazaki” 612, a set of “Miyazaki” 611 and “Miyazaki” 613, and a set of “Miyazaki” 612 and “Miyazaki” 613 with the same notation. In the following description, a set of “Miyazaki” 611 and “Miyazaki” 613 is taken as an example.

  The detected word is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207. Thereby, the detection unit 504 can detect a set of words to be determined by the determination unit 505, and can determine whether the set of words is the same type.

  The discriminating unit 505 determines a discriminating rule in which the third word detected by the detecting unit 504 appears together with one of the co-occurrence words, and the fourth word detected by the detecting unit 504 appears together with the other of the co-occurrence words. Is stored in the first storage unit 501. Then, when there is a determination rule, the determination unit 505 determines whether the first word and the second word are of the same type based on the determination rule.

  Specifically, the determination unit 505 specifies, for example, a combination of words existing around each of the same notation words detected by the detection unit 504 as a clue for determination. Next, the determination unit 505 determines whether there is a combination corresponding to any one of the specified clues among the co-occurrence word combinations of the rule items of the same notation / type word determination rule 300.

  Then, when there is a corresponding combination, the determination unit 505 refers to a determination result item corresponding to the combination and determines whether or not the same notation words are the same type. Note that the determination unit 505 may determine words of the same notation as different types of words when there is no corresponding combination.

  More specifically, the determination unit 505, for example, a combination of words “Birth & Go” and “Birth & Go” that exist around each of “Miyazaki” 611 and “Miyazaki” 613 detected by the detection unit 504. Are identified as clues for discrimination. Next, the determination unit 505 determines that the same notation / type word determination rule 300 includes the co-occurrence words “origin” and “go” corresponding to the identified clue “origin & go”. The determination unit 505 determines that “Miyazaki” 611 and “Miyazaki” 613 are of the same type because there is information indicating that they are of the same type in the determination result item corresponding to the combination of the co-occurrence words. To do.

  The determination result is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Accordingly, the determination unit 505 can detect a set of words that should be handled together in generating a word type extraction rule.

  When the determining unit 505 determines that the types are the same type, the specifying unit 506 determines whether each of the third word and the fourth word is included in the word string including the word to which the information indicating the word type is added. A combination of a word existing within a predetermined distance and a distance to the word is specified.

  Specifically, for example, when the same notation and the same type of word exist, the specifying unit 506 uses, as a clue for extraction, a word that is a distance of two words before and after each word and the distance to the word. Identify.

  More specifically, the specifying unit 506 handles, for example, “Miyazaki” 611 and “Miyazaki” 613 collectively determined as the same type by the determination unit 505. The identifying unit 506 identifies a combination of a word existing within the distance of two words before and after each of “Miyazaki” 111 and “Miyazaki” 113 and the distance to the word as a clue to extraction. Here, the identifying unit 506 uses, as clues for extraction, “origin: +1”, “no: +2”, “san: −2”, “to: −1”, “to: +1”, and “go: +2”. Is specified.

  The identified clue is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207. Thereby, the specifying unit 506 can specify a clue used by the generating unit 512 to generate a word type extraction rule.

  The generation unit 512 includes a combination of the word specified by the specifying unit 506 and the distance to the word, and information indicating the type of word assigned to one of the third word and the fourth word , And an extraction rule associated with each other. Specifically, the generation unit 512 uses, for example, a learning engine to generate a likelihood from among a group of learning cases in which the types of words indicated by tags attached to the same notation and the same type of words as the acquired clues are associated. A specific learning case is specified, and the specified learning case is generated as an extraction rule. Specifically, for example, a learning example in which “from: +1” and the word type “LOC” are associated with each other is adopted as a plausible learning example. The generated extraction rule is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Thereby, the generation unit 512 can generate an extraction rule used for extraction of the specific expression.

  The storage unit 513 stores the extraction rule generated by the generation unit 512 in the second storage unit 502. Specifically, for example, the storage unit 513 stores the generated extraction rule in the specific expression extraction rule 400. Thereby, the storage unit 513 can store the specific expression extraction rule used for the specific expression extraction processing.

  Next, the function of the extraction apparatus 100 in the case of extracting a chunk that is a unique expression from a chunk lattice having a chunk as a node will be described. A specific example of extracting a chunk of unique expressions from a lattice of chunks divided in units of chunks will be described later with reference to FIGS.

  The conversion unit 514 converts a series of words into a plurality of word strings including chunks. Specifically, the conversion unit 514 divides a series of sentences into words by, for example, morphological analysis. Then, the conversion unit 514 generates a lattice of a plurality of chunks including chunks obtained by connecting the words.

  Here, as an example of a series of sentences, “an employee of B Trading goes to B Trading and returns from B Trading” is taken as an example. Specifically, for example, the conversion unit 514 divides a series of sentences into words by morphological analysis, and generates “B trading employee goes to B trading and returns from B trading”. A space indicates a word break.

  Next, the conversion unit 514 generates chunks by concatenating words of a predetermined number of words. Here, it is assumed that “3” is designated as the number of words. For example, as the chunk generated from the word “B” that appears first, chunks up to the specified number of words such as “B”, “B Trading”, and “B Trading” are generated. As chunks generated from the next word “trade”, chunks of “trade”, “trade” and “trade employee” are created.

  In this way, chunks are generated by concatenating words of the designated number of words starting from each word, and the last word of each chunk and the chunk starting from the next appearing word are connected, so that FIG. , A chunk lattice as shown in FIG. 29 is generated.

  The converted chunk lattice is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207.

  The detection unit 504 detects the first chunk and the second chunk having the same notation as the first chunk from the chunk lattice. Specifically, the detection unit 504 detects, for example, chunks having the same notation from the chunk lattice. More specifically, the detection unit 504, for example, from a lattice of chunks generated from “an employee of B trading goes to B trading and returns from B trading”, three sets of chunks “B” with the same notation, Three sets of “trade” and three sets of “B trade” are detected. In the following description, the second “B Trading” and the third “B Trading” from the beginning of the sentence will be described as an example.

  The detected chunk is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Thereby, the detection unit 504 can detect a set of chunks to be determined by the determination unit 505, and can determine whether the set of chunks is the same type.

  The discriminating unit 505 determines a discriminating rule in which the first chunk detected by the detecting unit 504 appears together with one of the co-occurrence words, and the second chunk detected by the detecting unit 504 appears together with the other of the co-occurrence words. Is stored in the first storage unit 501. Then, when there is a determination rule, the determination unit 505 determines whether or not the first chunk and the second chunk are the same type from the determination rule.

  Specifically, the determination unit 505 specifies, for example, a combination of chunks present around each of the same notation chunks detected by the detection unit 504 as a clue for determination. Next, the determination unit 505 determines whether there is a combination corresponding to one of the specified clues among the combinations of co-occurrence chunks of the rule items of the same notation / type chunk determination rule. When there is a corresponding combination, the determination unit 505 refers to a determination result item corresponding to the combination and determines whether the same notation chunks are the same type. Note that when there is no corresponding combination, the determination unit 505 may determine the same notation chunks as different types of chunks.

  More specifically, the discriminating unit 505 discriminates, for example, the word combinations “To & From” and “Go & Return” that exist around each of the “B Trading” groups detected by the detecting unit 504. Identify as a clue. Next, when the same notation / type word discrimination rule 300 includes the co-occurrence words “going” and “returning” corresponding to the identified clue “going & returning”, the discriminating unit 505 determines the combination of the co-occurrence words. Information indicating whether or not they are the same type in the corresponding discrimination result item is extracted. Then, the determination unit 505 determines whether the set of “B trading” detected by the detection unit 504 is the same type from the extracted information.

  The determination result is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Thereby, the discrimination unit 505 can detect a set of chunks to be handled collectively in the extraction of the chunk type.

  When the determining unit 505 determines that there is a determination rule, the specifying unit 506 includes a word existing within a predetermined distance from each of the first chunk and the second chunk and the word from the converted word string Specify the combination with the distance. Specifically, for example, when the same notation and the same type of chunks exist, the specifying unit 506 uses, as a clue for extraction, a word that is a distance of two words before and after each chunk and the distance to the word. Identify.

  More specifically, for example, the specifying unit 506 collectively handles a set of “B trading” determined as the same type by the determining unit 505. Then, the specifying unit 506 specifies a combination of a word existing within a distance of two words before and after each of the “B trading” pairs and a distance to the word as a clue for extraction. Here, the identifying unit 506 uses “employee: −2”, “ha: −1”, “to: +1”, “going: +2”, “going: −2”, “going: − as clues for extraction. 1 ”,“ From: +1 ”, and“ Return: +2 ”.

  The identified clue is stored in a storage area such as the RAM 203, the magnetic disk 205, or the optical disk 207. Thereby, the specifying unit 506 can specify a clue used by the extracting unit 507 to extract the type of chunk.

  The extraction unit 507 extracts information indicating the type of word associated with the combination identified by the identification unit 506 from the extraction rules stored in the second storage unit 502, and extracts the first chunk and the second chunk Grant to a chunk.

  Specifically, the extraction unit 507, for example, a combination corresponding to any one of the clues specified by the specification unit 506 from combinations of the word of the rule item of the specific expression extraction rule 400 and the distance to the word. It is determined whether or not there is. Next, when there is a corresponding combination, the extraction unit 507 extracts the word type from the type item corresponding to the combination. Then, the extraction unit 507 assigns a tag indicating the type of the extracted word to each of the same notation and the same type of chunk on the sentence.

  More specifically, the extraction unit 507, for example, if the specific expression extraction rule 400 includes a rule item corresponding to the clue “origin: +1” specified by the specification unit 506, the type corresponding to the rule item Information indicating the type of word is extracted from the item. Then, the extraction unit 507 gives, for example, 1 score for “LOC” as the type of “B Trading” from the extracted information. In this way, a score that can be a unique expression is assigned to each chunk.

  After applying the rule to all chunks, the extraction unit 507 determines the types of unique expressions that can be taken on the path of the chunk path that can be taken from the beginning to the end of the sentence based on the score of each unique expression assigned to each chunk. Among the combinations, select the column of chunks with the largest sum of scores and the respective unique expression type. As a result, for example, “an employee of <ORG> B trading </ ORG> goes to <LOC> B trading </ LOC> and returns from <LOC> B trading </ LOC>” is generated. The method of determining the score of whether or not each chunk is a unique expression can be determined by the total number of times identified as each unique expression by the rule or the score of the applied rule if the rule has a score. Use the sum of

  Information indicating the type of the extracted chunk is stored in a storage area such as the RAM 203, the magnetic disk 205, and the optical disk 207. Accordingly, the extraction unit 507 can extract the same chunk type for the same notation and the same type of chunk. As a result, the extraction unit 507 can extract the types of chunks separately for different types of chunks even with the same notation.

  The output unit 508 outputs the word string given by the extraction unit 507. Specifically, the output unit 508 outputs, for example, a lattice of chunks to which the extracted word type is assigned as a tag to the display 208. Further, the output unit 508 may transmit the lattice of the assigned chunk to another computer via the I / F 209 or may output it to a recording medium. As a result, the output unit 508 can notify the user of the extraction device 100 of the type of chunk. Further, the output unit 508 can provide the lattice of the chunk to which the tag is added to other software (for example, translation software or information search software).

Example 1
Next, Example 1 will be described. Example 1 is an example in the case of extracting the word of a specific expression from the series of words divided | segmented by the word unit demonstrated using FIG.

(Specific example 1 of rule learning processing by the extraction apparatus 100 according to the first embodiment)
Next, specific example 1 of the rule learning process performed by the extraction apparatus 100 according to the first embodiment will be described with reference to FIGS. The rule learning process is a process for generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 600 shown in FIG. 6, and the process described with reference to FIG. is there.

  FIG. 7 to FIG. 9 are explanatory diagrams showing the contents for generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 using the learning data group 600 shown in FIG. In FIG. 7, the extraction apparatus 100 first selects unprocessed learning data from the learning data group 600. Here, it is assumed that the extraction device 100 has selected the learning data 610.

  (1) Next, the extraction apparatus 100 selects “Miyazaki” 611 and “Miyazaki” 612, “Miyazaki” 611 and “Miyazaki” 613 as combinations of words having the same notation from the selected learning data 610. And “Miyazaki” 612 and “Miyazaki” 613 are extracted.

  The extraction apparatus 100 uses different combinations of “Miyazaki” 611 and “Miyazaki” 612, and therefore, the combination of words around each of “Miyazaki” 611 and “Miyazaki” 612 is converted into a word set. As a clue that indicates that they are of different types. Then, the extraction apparatus 100 generates a learning example for discriminating that a pair of words such as “from & san” and “no & san” is a different type of clue.

  Further, since the extraction device 100 has the same type for the set of “Miyazaki” 611 and “Miyazaki” 613, the combination of words around each of “Miyazaki” 611 and “Miyazaki” 613 is converted into a set of words. Is obtained as a clue that indicates that they are of the same type. Then, the extraction apparatus 100 generates a learning example for determining that the word sets such as “Birth & Go”, “Birth & Go”, etc. are the same type.

  Further, since the extraction device 100 has different types of “Miyazaki” 612 and “Miyazaki” 613 sets, the combination of words around each of “Miyazaki” 612 and “Miyazaki” 613 is converted into a set of words. As a clue that indicates that they are of different types. Then, the extraction apparatus 100 generates a learning example for discriminating that a pair of words such as “san & he” and “san & go” is the same type as a clue.

  If the unprocessed learning data remains in the learning data group 600 after generating the learning case from the learning data 610, the extracting device 100 also generates the learning case from the unprocessed learning data.

  (2) Then, using the learning engine, the extraction device 100 identifies a likely learning case among the generated learning case group. For example, the learning engine identifies a learning case whose appearance frequency is greater than or equal to a threshold in the learning case group as a likely learning case.

  For example, when the learning engine group generated from the learning data group 600 includes a predetermined number or more of learning cases of “origin & go” indicating that the word set is of the same type, Identify a study case of “Born & Going” as a plausible study case. In addition, the learning engine has a clue “no & san” that includes case particles “no” that appear with many types of words, even if the word pairs are of the same type or different types. Do not adopt as a plausible learning case because there is a possibility.

  Note that examples of the learning engine include a Boosting learning device and an SVM. Since the Boosting learner and the SVM are conventional techniques, description thereof is omitted. The extraction device 100 generates a discrimination rule from the learning case specified by the learning engine. For example, the extraction apparatus 100 generates a discrimination rule that associates information indicating that the word pairs are of the same type with a combination of the co-occurrence words “from” and “go”. Then, the extraction apparatus 100 generates the same notation / type word determination rule 300 including the generated determination rule.

  In FIG. 8, the extraction apparatus 100 selects unprocessed learning data from the learning data. Next, the extraction apparatus 100 generates target data from which the tags are excluded from the selected learning data. Here, it is assumed that the extraction apparatus 100 selects the learning data 610 and generates the target data 800 from the learning data 610.

  (1) Next, the extraction apparatus 100 sets a pair of “Miyazaki” 801 and “Miyazaki” 802 and a pair of “Miyazaki” 801 and “Miyazaki” 803 from the target data 800 as pairs of words having the same notation. And a set of “Miyazaki” 802 and “Miyazaki” 803 are extracted.

  Then, the extraction apparatus 100, for the set of “Miyazaki” 801 and “Miyazaki” 802, combines word combinations “from & san”, “no & san”, etc. around each of “Miyazaki” 801 and “Miyazaki” 802. As a clue. In addition, the extraction apparatus 100 uses a combination of words “Birth & Go”, “Birth & Go”, etc. in the vicinity of “Miyazaki” 801 and “Miyazaki” 803 for the set of “Miyazaki” 801 and “Miyazaki” 803, etc. As a clue. In addition, the extraction apparatus 100 uses a combination of words “Miyazaki” 801 and “Miyazaki” 802 in the vicinity of “Miyazaki” 801 and “Miyazaki” 802, such as “san & e” and “san & go”. As a clue.

  (2) Then, using the discrimination engine, the extraction apparatus 100 identifies a clue that corresponds to the discrimination rule of the same notation / type word discrimination rule 300 among the generated clues. When there are a plurality of applicable discrimination rules, the discrimination engine specifies a plausible discrimination rule. The extraction apparatus 100 determines a set of “Miyazaki” 801 and “Miyazaki” 802, a set of “Miyazaki” 801 and “Miyazaki” 803, a set of “Miyazaki” 802 and “Miyazaki” 803 according to the discrimination rule specified by the discrimination engine. And whether each is the same type. Here, the extraction apparatus 100 determines that the combination of “Miyazaki” 801 and “Miyazaki” 803 is of the same type based on the same notation / type word discrimination rule 300 and the combination of the co-occurrence words “from” and “go”. Determine.

  In FIG. 9, the extraction apparatus 100 selects “Miyazaki” 611 and “Miyazaki” 613 corresponding to the set of “Miyazaki” 801 and “Miyazaki” 803 determined as the same type in the selected learning data 610 in FIG. 8. Is identified. The extraction apparatus 100 handles the identified “Miyazaki” 611 and “Miyazaki” 613 together in the subsequent processing.

  (1) The extraction apparatus 100 identifies a word type by referring to a tag for each word in the learning data 610 or for each grouped word, and a clue for extracting the identified word type Is identified. Specifically, the extraction apparatus 100, for example, “origin: +1”, which is a clue for extracting “LOC” from the words around each of “Miyazaki” 611 and “Miyazaki” 613 grouped together. , “No: +2”, “san: -2”, “to: −1”, “to: +1”, and “go: +2”. Then, the extraction apparatus 100 generates a learning case for determining that the identified word is a clue and the word type is “LOC”.

  In addition, the extraction apparatus 100 can obtain a clue for extracting “PER” from words in the vicinity of “Miyazaki” 612 that does not have other words of the same type. , “San: +1” and “to: +2” are specified. Then, the extraction apparatus 100 generates a learning case for determining that the identified word is a clue and the word type is “PER”.

  If the unprocessed learning data remains in the learning data group 600 after generating the learning case from the learning data 610, the extracting device 100 also generates the learning case from the unprocessed learning data.

  (2) Then, using the learning engine, the extraction apparatus 100 identifies a likely learning case in the determination of whether or not it is the same kind of specific expression from among the generated learning case group. For example, the learning engine identifies a learning case whose appearance frequency is greater than or equal to a threshold in the learning case group as a likely learning case. The extraction device 100 generates an extraction rule from the learning case specified by the learning engine. For example, the extraction apparatus 100 generates an extraction rule in which information indicating the word type “LOC” is associated with “origin: +1”.

  Then, the extraction apparatus 100 generates a specific expression extraction rule 400 including the generated extraction rule. Further, the extraction apparatus 100 may identify a clue for the word “from” of the type “O (Other)” other than the specific expression, and generate a learning example. Then, the extraction apparatus 100 may generate an extraction rule indicating the word type “O” from the generated learning case.

  Thereby, the extraction apparatus 100 can generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 by machine learning using the learning data group 600. Therefore, the user of the extraction apparatus 100 can reduce the trouble of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400.

(Detailed processing procedure of the rule learning process according to the first embodiment)
Next, a detailed processing procedure of the rule learning process according to the first embodiment will be described with reference to FIGS. 10 and 11. The rule learning process according to the first embodiment is a process executed by the extraction device 100 illustrated in FIGS. 7 to 9.

  10 and 11 are flowcharts illustrating a detailed processing procedure of the rule learning process according to the first embodiment. As shown in FIG. 10, first, the extraction apparatus 100 selects unprocessed learning data from the learning data group 600 (step S1001).

  Next, the extraction apparatus 100 determines whether there is a word with the same notation in the selected learning data (step S1002). Here, when there is no word of the same notation (step S1002: No), the extraction apparatus 100 moves to step S1008.

  On the other hand, when there is a word with the same notation (step S1002: Yes), the extraction apparatus 100 selects a set of unprocessed words from the set of words with the same notation in the selected learning data (step S1003). ).

  Next, the extraction apparatus 100 determines whether or not the selected set of words is the same type of word (step S1004). Here, if the words are of the same type (step S1004: Yes), the extraction apparatus 100 indicates that the combination of co-occurrence words appearing with each of the selected word sets and the word set are of the same type. A learning case including information is generated (step S1005). Then, the extraction device 100 proceeds to step S1007.

  On the other hand, when it is a different kind of word (step S1004: No), the extraction apparatus 100 is information which shows that the combination of the co-occurrence word which appears with each of the selected word set and the word set are different types. And a learning example including (Step S1006). Then, the extraction device 100 proceeds to step S1007.

  Next, the extraction apparatus 100 determines whether there is a set of unprocessed words (step S1007). If there is a set of unprocessed words (step S1007: Yes), the extraction apparatus 100 returns to step S1003.

  On the other hand, when there is no unprocessed word set (step S1007: No), the extraction apparatus 100 determines whether there is unprocessed learning data in the learning data group 600 (step S1008). Here, when there is unprocessed learning data (step S1008: Yes), the extraction device 100 returns to step S1001.

  On the other hand, when there is no unprocessed learning data (step S1008: No), the extraction apparatus 100 generates the same notation / type word discrimination rule 300 from the generated learning case group (step S1009). Then, the extraction device 100 proceeds to step S1101 in FIG.

  In FIG. 11, the extraction device 100 selects unprocessed learning data from the learning data group 600, and generates target data from which the tags of the selected learning data are removed (step S1101). Next, the extraction apparatus 100 refers to the same notation / type word discrimination rule 300 and identifies a set of words having the same notation and the same type in the generated target data (step S1102).

  Next, the extraction apparatus 100 selects an unprocessed word from the selected learning data (step S1103). Then, the extraction apparatus 100 determines whether there is a word of the same notation and the same type as the selected word from the identification result of step S1102 (step S1104).

  Here, when there are words of the same notation and the same type (step S1104: Yes), the extraction apparatus 100 sets the clue specified from each of the same notation and the same type of word and the set of the word specified from the tag. A learning example including the type of (step S1105). Then, the extraction device 100 proceeds to step S1107.

  On the other hand, when there is no word of the same notation and the same type (step S1104: No), the extraction device 100 includes the clue specified from the selected word and the type of the word specified from the tag. A learning example is generated (step S1106). Then, the extraction device 100 proceeds to step S1107.

  Next, the extraction apparatus 100 determines whether or not there is an unprocessed word in the selected learning data (step S1107). Here, when there is an unprocessed word (step S1107: Yes), the extraction apparatus 100 returns to step S1103.

  On the other hand, when there is no unprocessed word (step S1107: No), the extraction apparatus 100 determines whether there is unprocessed learning data (step S1108). Here, when there is unprocessed learning data (step S1108: Yes), the extraction device 100 returns to step S1101.

  On the other hand, when there is no unprocessed learning data (step S1108: No), the extraction apparatus 100 generates a specific expression extraction rule 400 from the generated learning case group (step S1109). Then, the extraction device 100 ends the rule learning process. Thereby, the extraction apparatus 100 can generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 by machine learning using the learning data group 600. Therefore, the user of the extraction apparatus 100 can reduce the trouble of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400.

(Specific example 1 of specific expression extraction processing by the extraction apparatus 100 according to the first embodiment)
Next, specific example 1 of the specific expression extraction process by the extraction device 100 according to the first embodiment will be described with reference to FIGS. 12 and 13. The specific expression extraction process according to the first embodiment is a process of extracting a word of a specific expression from data to be extracted of a specific expression, and is the process described with reference to FIG.

  12 and 13 are explanatory diagrams of a specific example 1 of the specific expression extraction process performed by the extraction apparatus 100 according to the first embodiment. In FIG. 12, the extraction apparatus 100 receives input data 1200 that is a target of extraction of a specific expression.

  (1) First, the extraction apparatus 100 includes a set of “Fukuoka” 1201 and “Fukuoka” 1202, a set of “Fukuoka” 1201 and “Fukuoka” 1203 as a set of words having the same notation from the input data 1200. , “Fukuoka” 1202 and “Fukuoka” 1203 are extracted.

  Then, the extraction apparatus 100, for the set of “Fukuoka” 1201 and “Fukuoka” 1202, combines word combinations “Origin & San”, “No & San”, etc. around each of “Fukuoka” 1201 and “Fukuoka” 1202. As a clue. In addition, the extraction apparatus 100 uses a combination of words “Fukuoka” 1201 and “Fukuoka” 1203 in the vicinity of “Fukuoka” 1201 and “Fukuoka” 1203, such as “from &&” and “from &&”. As a clue. In addition, the extraction apparatus 100 uses a combination of words “San & He”, “San & Go”, etc., around each of “Fukuoka” 1202 and “Fukuoka” 1203 for the group of “Fukuoka” 1202 and “Fukuoka” 1203. As a clue.

  (2) Then, using the discrimination engine, the extraction apparatus 100 identifies a clue that corresponds to the discrimination rule of the same notation / type word discrimination rule 300 among the generated clues. When there are a plurality of applicable discrimination rules, the discrimination engine specifies a plausible discrimination rule. The extraction apparatus 100 determines a set of “Fukuoka” 1201 and “Fukuoka” 1202, a set of “Fukuoka” 1201 and “Fukuoka” 1203, a set of “Fukuoka” 1202 and “Fukuoka” 1203 according to the discrimination rule specified by the discrimination engine. And whether each is the same type.

  Here, the extraction apparatus 100 determines that the combination of “Fukuoka” 1201 and “Fukuoka” 1203 is of the same type based on the same notation / type word discrimination rule 300 and the combination of the co-occurrence words “from” and “go”. Determine.

  In FIG. 13, the extraction apparatus 100 treats “Fukuoka” 1201 and “Fukuoka” 1203 that are determined to be the same type in FIG.

  (1) The extraction apparatus 100 specifies a clue for extracting the type of word for each word in the input data 1200 or for each grouped word. Specifically, for example, the extraction device 100 uses, as a clue, words from “Fukuoka” 1201 and “Fukuoka” 1203 that are grouped together as “clue: +1”, “no: +2”, “san: −”. 2 ”,“ To: −1 ”,“ To: +1 ”, and“ Go: +2 ”. In addition, the extraction apparatus 100 uses the words in the vicinity of “Fukuoka” 1202 that do not have other words of the same type as clues “from: -2”, “no: −1”, “san: +1”, and “ And: +2 ”.

  (2) Next, the extraction apparatus 100 specifies the extraction rule of the specific expression extraction rule 400 corresponding to the generated clue using the discrimination engine. When there are a plurality of corresponding extraction rules, the discrimination engine specifies a plausible extraction rule. The extraction device 100 extracts word types for each word in the input data 1200 or for each word grouped according to the extraction rules specified by the discrimination engine. Here, the extraction apparatus 100 extracts “LOC” as the type of “Fukuoka” 1201 and “Fukuoka” 1203 and extracts “PER” as the type of “Fukuoka” 1202.

  The extraction apparatus 100 may also identify a clue for the word “from” and extract the type of the word “from”. Here, the extraction apparatus 100 extracts a word type “O” that is not a unique expression for “born”.

  Thereby, the extraction apparatus 100 can extract the same word type for the same notation and the same type of words. Further, the extraction apparatus 100 can extract the type of a word separately from other words having the same notation, even if they have the same notation. As a result, the extraction device 100 prevents the extraction of the same type of word as a different type of word by extracting the same notation and the same type of word together and extracting the same type of word, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 treats different types of words separately even if they are the same notation, and extracts the type of the word, thereby preventing erroneous extraction as the same type of word and improving the extraction accuracy. Improvements can be made.

(Output example 1 of the result of extraction of specific expressions by the extraction apparatus 100 according to the first embodiment)
Next, an output example 1 of the named entity extraction result by the extraction apparatus 100 according to the first embodiment will be described with reference to FIG. The extraction apparatus 100 outputs the specific expression extraction results in FIGS. 12 and 13.

  FIG. 14 is an explanatory diagram of an output example 1 of the specific expression extraction result by the extraction device 100 according to the first embodiment. As shown in FIG. 14A, the extraction apparatus 100 uses, for example, input data 1200 “<LOC> Fukuoka </ LOC> from <PER> Fukuoka </ PER> with the extracted word type as a tag. Go to <LOC> Fukuoka </ LOC>. Further, as shown in FIG. 14B, for example, when the extraction apparatus 100 displays the input data 1200 “Going to Fukuoka with Mr. Fukuoka from Fukuoka”, the color indicating the type of the extracted word is displayed. Give and display.

(Detailed processing procedure of specific expression extraction processing according to embodiment 1)
Next, a detailed processing procedure of the specific expression extraction processing according to the first embodiment will be described with reference to FIG. The specific expression extraction processing according to the first embodiment is processing performed by the extraction device 100 illustrated in FIGS. 12 to 14.

  FIG. 15 is a flowchart of a detailed process procedure of the named entity extraction process according to the first embodiment. As shown in FIG. 15, first, the extraction device 100 accepts input data (step S1501). Next, the extraction apparatus 100 refers to the same notation / type word discrimination rule 300 and identifies a set of words having the same notation and the same type in the input data (step S1502).

  Next, the extraction apparatus 100 selects an unprocessed word from the input data (step S1503). Then, the extraction apparatus 100 determines whether or not there is a word having the same notation and the same type as the selected word from the identification result of step S1502 (step S1504).

  Here, when there is a word of the same notation and the same type (step S1504: Yes), the extraction device 100 includes a clue specified from each of the set of the same notation and the same type of word, a proper expression extraction rule 400, The type of a set of words having the same notation and the same type is extracted from (step S1505). Then, the extraction apparatus 100 proceeds to step S1507.

  On the other hand, when there is no word of the same notation and type (step S1504: No), the extraction device 100 selects the word selected from the clue specified from the selected word and the specific expression extraction rule 400. Are extracted (step S1506). Then, the extraction apparatus 100 proceeds to step S1507.

  Next, the extraction apparatus 100 determines whether there is an unprocessed word in the input data (step S1507). If there is an unprocessed word (step S1507: YES), the extraction apparatus 100 returns to step S1503.

  On the other hand, when there is no unprocessed word (step S1507: No), the extraction apparatus 100 outputs an extraction result (step S1508). Then, the extraction apparatus 100 ends the specific expression extraction process.

  Thereby, the extraction apparatus 100 can extract the same word type for the same notation and the same type of words. Further, the extraction apparatus 100 can extract the type of a word separately from other words having the same notation, even if they have the same notation. As a result, the extraction device 100 prevents the extraction of the same type of word as a different type of word by extracting the same notation and the same type of word together and extracting the same type of word, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 treats different types of words separately even if they are the same notation, and extracts the type of the word, thereby preventing erroneous extraction as the same type of word and improving the extraction accuracy. Improvements can be made.

(Specific example 2 of rule learning processing by the extraction apparatus 100 according to the first embodiment)
Next, specific example 2 of the rule learning process performed by the extraction apparatus 100 according to the first embodiment will be described with reference to FIGS. Specific example 2 is an example in which the types of words are subdivided compared to specific example 1. For example, in the specific example 2, as the word type, the organization name “ORG” is subdivided, the organization name top “B-ORG”, the organization name “I-ORG”, and the organization name tail “E-E”. -ORG ".

  FIG. 16 is an explanatory diagram illustrating Example 2 of learning data used for the rule learning process performed by the extraction device 100. As shown in FIG. 16, a learning data group 1600 including a plurality of learning data to which tags indicating types subdivided from the learning data group 600 of FIG. 6 are provided.

  For example, the learning data group 1600 includes learning data 1610 “<B-ORG> A </ B-ORG> <E-ORG> Corporation </ E-ORG> </ B-ORG> Corporation </ B- ORG> <I-ORG> A </ I-ORG> <E-ORG> Laboratory </ E-ORG>, <B-ORG> Corporation </ B-ORG> <I-ORG> A </ I-ORG> <E-ORG> Laboratory </ E-ORG> is in <LOC> Kawasaki City </ LOC>.

  A tag “B-ORG” is assigned to “A” 1611 in the learning data 1610, and a tag “E-ORG” is assigned to “corporation” 1612. Also, “B-ORG” tag is assigned to “corporation” 1613, “I-ORG” tag is assigned to “A” 1614, and “E-ORG” is assigned to “Laboratory” 1615. A tag is attached. Also, “B-ORG” tag is assigned to “corporation” 1616, “I-ORG” tag is assigned to “A” 1617, and “E-ORG” is assigned to “laboratory” 1618. A tag is attached. The tag “LOC” is assigned to “Kawasaki city” 1619.

  The learning data group 1600 includes <PER> of <B-ORG> Miyazaki </ B-ORG> <E-ORG> Trading </ E-ORG> in the learning data 1620 “<LOC> Miyazaki </ LOC>”. > President Miyazaki </ PER> is from <LOC> Miyazaki </ LOC>. The tag “LOC” is assigned to “Miyazaki” 1621 in the learning data 1620, and the tag “B-ORG” is assigned to “Miyazaki” 1622. In addition, the tag “E-ORG” is assigned to “Shosho” 1623, the tag “PER” is assigned to “Miyazaki” 1624, and the tag “LOC” is assigned to “Miyazaki” 1625. .

  The extraction apparatus 100 uses the learning data group 1600 as described above, so that the same notation / type word discrimination rule and specific expression corresponding to the types subdivided compared to the case where the learning data group 600 shown in FIG. 6 is used. Extraction rules can be generated.

  FIGS. 17 to 19 are explanatory diagrams showing the contents for generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400 using the learning data group 1600 shown in FIG. In FIG. 17, the extraction apparatus 100 first selects unprocessed learning data from the learning data group 1600. Here, it is assumed that the extraction apparatus 100 has selected the learning data 1610.

  (1) Next, the extraction apparatus 100 selects “A” 1611 and “A” 1614, “A” 1611, and “A” 1617 as a pair of words having the same notation from the selected learning data 1610. And a set of “A” 1614 and “A” 1617 are extracted.

  The extraction apparatus 100 has different combinations of “A” 1611 and “A” 1614, and therefore a combination of words “A Co. & Research” around each of “A” 1611 and “A” 1614. A learning example is generated to determine whether a set of words such as “place” is a clue and is not the same type.

  Further, the extraction apparatus 100 uses different combinations of “A” 1611 and “A” 1617, and therefore, the combination of the words “A” 1611 and “A” 1617 around “A” 1611 and “A” 1617. A learning example is generated to determine whether a set of words such as “place” is a clue and is not the same type.

  In addition, the extraction apparatus 100 has the same type of “A” 1614 and “A” 1617, and therefore, a combination of words around each of “A” 1614 and “A” 1617 “laboratory & research” A learning example is generated for discriminating that a set of words such as “Tokoro” and “Corporation & Co., Ltd.” are clues and are of the same type.

  After generating the learning case for the word “A”, the extraction apparatus 100 also generates the learning case for the other words “corporation” and “laboratory” having the same notation. In addition, after generating a learning case from the learning data 1610, the extraction device 100 generates a learning case from the unprocessed learning data if unprocessed learning data remains in the learning data group 1600.

  (2) Then, using the learning engine, the extraction apparatus 100 identifies a likely learning case in the determination of whether or not it is the same kind of specific expression from the generated learning case group. For example, the learning engine identifies a learning case whose appearance frequency is greater than or equal to a threshold in the learning case group. In addition, since there is no similar learning case, the learning engine may specify a learning case that can determine whether or not the word sets are of the same type with an accuracy equal to or higher than a threshold value. The extraction device generates, for example, a discrimination rule that associates information indicating that a pair of words is of the same type with a combination of the co-occurrence words “stock” and “stock”. Then, the extraction apparatus 100 generates the same notation / type word determination rule 300 including the generated determination rule.

  In FIG. 18, the extraction device 100 selects unprocessed learning data from the learning data group 1600. Next, the extraction apparatus 100 generates target data from which the tags are excluded from the selected learning data. Here, it is assumed that the extraction apparatus 100 selects the learning data 1610 and generates the target data 1800 from the learning data 1610.

  (1) Next, the extraction apparatus 100 sets “A” 1801 and “A” 1802 and “A” 1801 and “A” 1803 as pairs of the same notation from the target data 1800. Then, a set of “A” 1802 and “A” 1803 is extracted.

  Then, the extraction apparatus 100 identifies a combination of words “A” 1801 and “A” 1802 around each of “A” 1801 and “A” 1802 using “Corporation & Research Institute” as a clue. To do. Further, the extraction apparatus 100 identifies the combination of “A” 1801 and “A” 1803 with a combination of words “Corporation & Research Institute” around each of “A” 1801 and “A” 1803 as a clue. To do. In addition, the extraction apparatus 100 uses a combination of words “Laboratory & Laboratory”, “Co., Ltd.”, and “A” 1802 and “A” 1803 in the vicinity of “A” 1802 and “A” 1803. Identify “Corporation” as a clue.

  (2) Then, using the discrimination engine, the extraction device 100 identifies the discrimination rule of the same notation / type word discrimination rule 300 corresponding to the generated clue. When there are a plurality of applicable discrimination rules, the discrimination engine specifies a plausible discrimination rule based on, for example, the discrimination accuracy on the learning example.

  The extraction apparatus 100 determines a set of “A” 1801 and “A” 1802, a set of “A” 1801 and “A” 1803, a set of “A” 1802 and “A” 1803 according to the discrimination rule specified by the discrimination engine. And whether each is the same type. In this case, the extraction apparatus 100 uses the same notation / type word discrimination rule 300, the combination of the co-occurrence words “laboratory” and “laboratory”, and the combination of “corporation” and “corporation”. "1802" and "A" 1803 are determined to be of the same type.

  In FIG. 19, the extraction apparatus 100 selects “A” 1614 and “A” 1617 corresponding to the combination of “A” 1802 and “A” 1803 determined as the same type in FIG. 18 from the selected learning data 1610. Is identified. The extraction apparatus 100 handles the specified “A” 1614 and “A” 1617 together in the subsequent processing.

  (1) The extraction apparatus 100 refers to a tag for each word in the learning data 1610 or for each grouped word, identifies a word type, and a clue for extracting the identified word type Is identified. Specifically, for example, the extraction apparatus 100 can be used as a clue to extract “I-ORG” from a group of words in the vicinity of “A” 1614 and “A” 1617. 1 ”and“ Lab: +1 ”are identified. Then, the extraction apparatus 100 generates a learning case for determining that the identified word is a clue and the word type is “I-ORG”.

  Further, the extraction apparatus 100 identifies “corporation: +1” that is a clue for extracting “B-ORG” from words around “A” 1611 that does not have other words of the same type. Then, the extraction apparatus 100 generates a learning case for determining that the identified word is a clue and the type of the word is “B-ORG”.

  After generating the learning case for the word “A”, the extraction apparatus 100 also generates the learning case for the other words “corporation” and “laboratory” having the same notation. In addition, after generating a learning case from the learning data 1610, the extraction device 100 generates a learning case from the unprocessed learning data if unprocessed learning data remains in the learning data group 1600.

  (2) Then, using the learning engine, the extraction device 100 identifies a likely learning case among the generated learning case group. For example, the learning engine identifies a learning case whose appearance frequency is greater than or equal to a threshold in the learning case group as a likely learning case. The extraction device 100 generates an extraction rule from the learning case specified by the learning engine. The extraction apparatus 100 generates, for example, an extraction rule that associates the word type “I-ORG” with “corporation: −1”. Then, the extraction apparatus 100 generates a specific expression extraction rule 400 including the generated extraction rule.

  Accordingly, the extraction apparatus 100 can generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 by machine learning using the learning data group 1600. Therefore, the user of the extraction apparatus 100 can reduce the trouble of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400. Here, a rule learning method based on a threshold is used, but as other learning methods, Non-Patent Documents 3, 4, 5, and the like can also be used. When Non-Patent Document 3 is used as a learning technique, the same notation is used based on the sum of scores of each rule, and Non-Patent Documents 4 and 5 are based on the sum of scores calculated using a model (corresponding to a rule). Determine the words that have the same specific expression and the type of specific expression for each word.

(Specific example 2 of specific expression extraction processing by the extraction apparatus 100 according to the first embodiment)
Next, a specific example 2 of the specific expression extraction process performed by the extraction apparatus 100 according to the first embodiment will be described with reference to FIGS. 20 and 21.

  20 and 21 are explanatory diagrams of a specific example 2 of the specific expression extraction process performed by the extraction apparatus 100 according to the first embodiment. In FIG. 20, the extraction apparatus 100 receives input data 2000 that is a target for extraction of a specific expression.

  (1) First, the extraction apparatus 100 includes a group of “B” 2001 and “B” 2002, a group of “B” 2001 and “B” 2003 as a group of words having the same notation from the input data 2000. , “B” 2002 and “B” 2003 are extracted.

  Then, the extraction apparatus 100 identifies the combination of “B” 2001 and “B” 2002 with a combination of words “Corporation & Research Institute” around each of “B” 2001 and “B” 2002. To do. In addition, the extraction apparatus 100 identifies a combination of “B” 2001 and “B” 2003 with a combination of words “Corporation & Research Institute” around each of “B” 2001 and “B” 2003 as a clue. To do. In addition, the extraction apparatus 100 uses a combination of words “Laboratory & Research Institute”, “Co., Ltd.” and “B” 2002 and “B” 2003 in combination with words around each of “B” 2002 and “B” 2003. Identify “Corporation” as a clue.

  (2) Then, the extraction apparatus 100 specifies the discrimination rule of the same notation / type word discrimination rule 300 corresponding to the identified clue using the discrimination engine. When there are a plurality of applicable discrimination rules, the discrimination engine specifies a plausible discrimination rule. The extraction apparatus 100 uses a discrimination rule specified by the discrimination engine to set “B” 2001 and “B” 2002, “B” 2001 and “B” 2003, and “B” 2002 and “B” 2003. And whether each is the same type.

  In this case, the extraction apparatus 100 uses the same notation / type word discrimination rule 300, the combination of the co-occurrence words “laboratory” and “laboratory”, and the combination of “corporation” and “corporation”. "2002" and "B" 2003 are determined to be of the same type.

  In FIG. 21, the extraction apparatus 100 collectively handles “B” 2002 and “B” 2003 determined to be the same type in FIG. 20 in the subsequent processing.

  (1) The extraction apparatus 100 specifies a clue for extracting the type of word for each word in the input data 2000 or for each grouped word. Specifically, the extraction apparatus 100 specifies, for example, “corporation: −1” and “laboratory: +1” as clues from the words around “B” 2002 and “B” 2003 that are grouped together. To do. In addition, the extraction apparatus 100 identifies “corporation: +1” as a clue from words in the vicinity of “B” 2001 that has no other words of the same type.

  (2) Next, the extraction apparatus 100 specifies the extraction rule of the specific expression extraction rule 400 corresponding to the specified clue using the discrimination engine. When there are a plurality of corresponding extraction rules, the discrimination engine specifies a plausible extraction rule. The extraction device 100 extracts word types for each word in the input data 2000 or for each word grouped according to the extraction rules specified by the discrimination engine.

  Here, the extraction apparatus 100 extracts “I-ORG” as the type of “B” 2002 and “B” 2003 and extracts “B-ORG” as the type of “B” 2001. Note that the extraction apparatus 100 similarly extracts word types for the other words “corporation” and “laboratory” having the same notation.

  Thereby, the extraction apparatus 100 can extract the same word type for the same notation and the same type of words. Further, the extraction apparatus 100 can extract the type of a word separately from other words having the same notation, even if they have the same notation. As a result, the extraction device 100 prevents the extraction of the same type of word as a different type of word by extracting the same notation and the same type of word together and extracting the same type of word, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 treats different types of words separately even if they are the same notation, and extracts the type of the word, thereby preventing erroneous extraction as the same type of word and improving the extraction accuracy. Improvements can be made.

(Output example 2 of the named entity extraction result by the extraction apparatus 100 according to the first embodiment)
Next, an output example 2 of the named entity extraction result by the extraction apparatus 100 according to the first embodiment will be described with reference to FIG. The extraction apparatus 100 outputs the specific expression extraction results in FIGS. 20 and 21.

  FIG. 22 is an explanatory diagram of an output example 2 of the specific expression extraction result by the extraction device 100 according to the first embodiment. As illustrated in FIG. 22, the extraction apparatus 100, for example, uses input data 2000 “<B-ORG> B </ B-ORG> <E-ORG>, Inc. </ E>, to which the extracted word type is assigned as a tag. -ORG> is a subsidiary of <B-ORG> Corporation </ B-ORG> <I-ORG> B </ I-ORG> <E-ORG> Laboratory </ E-ORG>, <B- ORG> Corporation </ B-ORG> <I-ORG> B </ I-ORG> <E-ORG> Laboratory </ E-ORG> is in Kawasaki City ”.

  In addition, when a tag expressing a case where a single unique expression is formed by a plurality of words, such as B-ORG, I-ORG, and E-ORG, <ORG> B Corporation <ORG> Is a subsidiary of <ORG> B Laboratory Inc. </ ORG>, and <ORG> B Laboratory Ltd. </ ORG> is located in Kawasaki City. It is also possible to output as a single tag.

(Example 2)
Example 1 is an example in which a word with a unique expression is extracted from a series of words divided in units of words. On the other hand, Embodiment 2 is an embodiment in the case of extracting a chunk of a unique expression from the lattice of chunks divided in units of chunks described with reference to FIG.

(Contents of Rule Learning Processing by Extracting Device 100 According to Embodiment 2)
Next, the contents of the rule learning process performed by the extraction apparatus 100 according to the second embodiment will be described with reference to FIGS. The rule learning process is a process of generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400 using chunk lattices obtained by converting each of the learning data groups.

  FIG. 23 is an explanatory diagram of an example of a chunk lattice used in the rule learning process by the extraction device 100 according to the second embodiment. As shown in FIG. 23, in the extraction apparatus 100, employees of learning data “<ORG> C company </ ORG> go to <LOC> C company </ LOC> and return from <LOC> C company </ LOC>. ”To generate a chunk lattice 2300.

  Specifically, for example, the employee of the learning data “<ORG> C Company </ ORG> goes to <LOC> C Company </ LOC> and returns from <LOC> C Company </ LOC>. "Is morphologically analyzed and separated into words. The learning data divided for each word is, for example, “an employee of company C goes to company C and returns from company C”.

  The extraction apparatus 100 generates a chunk in which two words are combined from learning data divided for each word. For example, the extraction apparatus 100 generates chunks such as “Company C”, “Company”, “Employees”, and “Employees”. Then, the extraction apparatus 100 generates a chunk lattice from the generated chunk. Then, the extraction apparatus 100 performs rule learning so as to correctly determine the path in the chunk lattice including the chunk “Company C” to which the tag is attached from among the plurality of chunk lattices.

  24 to 26 are explanatory diagrams showing the contents of generating the same notation / type word discrimination rule 300 and the unique expression extraction rule 400 using the chunk lattice generated in FIG. In FIG. 24, the extraction apparatus 100 first selects unprocessed learning data from the learning data group. Then, as illustrated in FIG. 23, the extraction apparatus 100 generates a chunk lattice from the selected learning data. Here, it is assumed that the extraction apparatus 100 has generated a chunk lattice 2300.

  (1) The extraction apparatus 100 selects “C Company” 2301 and “C Company” 2302 as a set of chunks having the same notation from the selected chunk lattice 2300, and “C Company” 2301 and “C Company”. ”2303 and“ C company ”2302 and“ C company ”2303 are extracted. Further, the extraction apparatus 100 may extract a set of “C” or a set of “company” as a set of chunks having the same notation, for example.

  The extraction apparatus 100 uses different combinations of “Company C” 2301 and “Company C” 2302, and therefore a combination of words “C Company” 2301 and “Company C” 2302 around each “ "No & Go", "Employee & Go", etc. are generated as clues indicating that the word pairs are of different types, and learning cases that are not identical are generated.

  Further, since the extraction apparatus 100 has different types of “Company C” 2301 and “Company C” 2303, the word combination “N” around each of “Company C” 2301 and “Company C” 2303 "& Kara", "employee & return", etc. are generated as clues indicating that the word pairs are of different types, and a learning case that is not the same is generated.

  Further, the extraction apparatus 100 has the same type of “Company C” 2302 and “Company C” 2303, and therefore, the combination of words “C Company” 2302 and “C Company” 2303 around each “ “To & From”, “Go & Return”, etc. are generated as clues indicating that the word pairs are of the same type, and a learning case that is the same is generated.

  After generating a learning example from the chunk lattice 2300, the extraction apparatus 100 converts the unprocessed learning data into a chunk lattice if there is unprocessed learning data in the learning data group. A learning example is also generated from the lattice of the chunk.

  (2) Then, using the learning engine, the extraction device 100 identifies a likely learning case among the generated learning case group. For example, the learning engine identifies a learning case whose appearance frequency is equal to or higher than a threshold in the learning case group as a likely clue. The extraction device 100 generates a discrimination rule from the learning case specified by the learning engine. For example, the extraction device 100 generates a discrimination rule that associates, for example, information indicating that the combination of chunks is the same type with a combination of the co-occurrence words “go” and “return”. Then, the extraction apparatus 100 generates the same notation / type word determination rule 300 including the generated determination rule. Further, the extraction apparatus 100 may not generate a discrimination rule for “C”, “Company”, “Company”, or the like that is not a chunk of the unique expression.

  In FIG. 25, the extraction apparatus 100 selects unprocessed learning data from the learning data group. Then, as illustrated in FIG. 23, the extraction apparatus 100 generates a chunk lattice from the selected learning data. Here, it is assumed that the extraction apparatus 100 has generated a chunk lattice 2300.

  Further, the extraction apparatus 100 generates a target lattice from which tags are excluded from the chunk lattice 2300. Here, it is assumed that the extraction apparatus 100 generates the target lattice 2500 from the chunk lattice 2300.

  (1) Next, the extraction apparatus 100 sets “C company” 2501 and “C company” 2502 sets, “C company” 2501 and “C company” as a set of chunks having the same notation from the target lattice 2500. "2503" and "C company" 2502 and "C company" 2503 are extracted. Further, the extraction apparatus 100 may extract a set of “C” or a set of “company” as a set of chunks having the same notation, for example.

  Then, the extraction apparatus 100, for the set of “C company” 2501 and “C company” 2502, combines the word combinations “No & He” and “Employees” around “C company” 2501 and “C company” 2502. “& Go” etc. as a clue. In addition, the extraction apparatus 100 uses a combination of words “No & Kara” and “employee” in the vicinity of “C company” 2501 and “C company” 2503 for the set of “C company” 2501 and “C company” 2503. & "Return" etc. as a clue. In addition, the extraction apparatus 100, for a set of “Company C” 2502 and “Company C” 2503, combines word combinations “To & From” and “Go” around each of “Company C” 2502 and “Company C” 2503. & "Return" etc. as a clue.

  (2) Then, the extraction apparatus 100 specifies the discrimination rule of the same notation / type word discrimination rule 300 corresponding to the identified clue using the discrimination engine. When there are a plurality of applicable discrimination rules, the discrimination engine specifies a plausible discrimination rule. The extraction apparatus 100 determines the combination of “Company C” 2501 and “Company C” 2502, the combination of “Company C” 2501 and “Company C” 2503, “Company C” 2502, and “ It is determined whether or not each of the group of “Company C” 2503 is the same type. Here, the extraction apparatus 100 determines that the combination of “Company C” 2502 and “Company C” 2503 is of the same type based on the same notation / type word discrimination rule 300 and the combination of the co-occurrence words “go” and “return”. Determine that there is.

  In FIG. 26, the extraction apparatus 100 includes “C company” 2302 and “C” corresponding to a set of “C company” 2502 and “C company” 2503 identified as the same type in FIG. “Company” 2303 is specified. The extraction apparatus 100 handles the specified “Company C” 2302 and “Company C” 2303 together in the subsequent processing.

  (1) The extraction apparatus 100 refers to the tag for each chunk in the chunk lattice 2300 or for each chunk that is grouped, identifies the chunk type, and extracts the identified chunk type Identify clues. Specifically, for example, the extraction apparatus 100 can be used as a clue to extract “LOC” from chunks around “C company” 2302 and “C company” 2303 that are grouped together. ”,“ Ha: -1 ”,“ To: +1 ”,“ To: +2 ”,“ To: -2 ”,“ To: −1 ”,“ From: +1 ”, and“ Return: +2 ” . Then, the extraction apparatus 100 identifies the “employee: −2”, “ha: −1”, “to: +1”, “bound: +2”, “bound: -2”, “bound: −1”, “ From: +1 ”and“ Return: +2 ”are clues, and a learning example for determining that the type of chunk is“ ORG ”is generated.

  In addition, the extraction apparatus 100 uses “no: +1” and “employee: +2” as clues for extracting “ORG” from chunks around “C company” 2301 that does not have other chunks of the same type. Is identified. Then, the extraction apparatus 100 generates a learning case for determining that the identified “no: +1” and “employee: +2” are clues and the type of chunk is “ORG”.

  After generating a learning example from the chunk lattice 2300, the extraction apparatus 100 converts the unprocessed learning data into a chunk lattice if there is unprocessed learning data in the learning data group. A learning example is also generated from the lattice of the chunk.

  (2) Then, using the learning engine, the extraction device 100 identifies a likely learning case among the generated learning case group. For example, the learning engine identifies a learning case whose appearance frequency is greater than or equal to a threshold in the learning case group as a likely learning case. The extraction device 100 generates an extraction rule from the learning case specified by the learning engine. For example, the extraction device 100 generates an extraction rule that associates information indicating the word type “LOC” with “bound: +2”. Then, the extraction apparatus 100 generates a specific expression extraction rule 400 including the generated extraction rule.

  Further, the extraction apparatus 100 may generate an extraction rule indicating that “C”, “Company”, “Company”, or the like that is not a chunk of the unique expression is not a unique expression. Further, the extraction apparatus 100 may not generate an extraction rule for “C”, “Company”, “Company”, or the like that is not a chunk of the unique expression.

  Thereby, the extraction device 100 can generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 by machine learning using the learning data group. Therefore, the user of the extraction apparatus 100 can reduce the trouble of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400.

(Detailed processing procedure of the rule learning process according to the second embodiment)
Next, a detailed processing procedure of the rule learning process according to the second embodiment will be described with reference to FIGS. 27 and 28. The rule learning process according to the second embodiment is a process executed by the extraction device 100 illustrated in FIGS.

  FIGS. 27 and 28 are flowcharts illustrating the detailed processing procedure of the rule learning process according to the second embodiment. As shown in FIG. 27, first, the extraction apparatus 100 selects unprocessed learning data from the learning data group (step S2701).

  Next, the extraction apparatus 100 converts the selected learning data into a lattice of chunks divided in units of chunks (step S2702). The extraction apparatus 100 determines whether there is a chunk with the same notation in the converted chunk lattice (step S2703). Here, when there is no chunk with the same notation (step S2703: No), the extraction apparatus 100 proceeds to step S2709.

  On the other hand, when there are chunks with the same notation (step S2703: Yes), the extraction apparatus 100 selects a set of unprocessed chunks from the pair of chunks with the same notation in the chunk lattice (step S2704). .

  Next, the extraction apparatus 100 determines whether or not the selected set of chunks is the same type of chunk (step S2705). If the chunks are the same type (step S2705: Yes), the extraction apparatus 100 indicates that the combination of co-occurrence chunks that appear with each of the selected chunk pairs and the chunk set are of the same type. A discrimination rule including information is generated (step S2706). Then, the extraction device 100 proceeds to step S2708.

  On the other hand, in the case of different types of chunks (step S2705: No), the extraction apparatus 100 indicates that the combination of co-occurrence chunks that appears with each of the selected chunk pairs and the type of chunks are different. Are generated (step S2707). Then, the extraction device 100 proceeds to step S2708.

  Next, the extraction apparatus 100 determines whether there is an unprocessed chunk pair (step S2708). If there is an unprocessed chunk pair (step S2708: YES), the extraction apparatus 100 returns to step S2704.

  On the other hand, if there is no unprocessed chunk pair (step S2708: No), the extraction apparatus 100 determines whether there is an unprocessed chunk lattice in the chunk lattice group (step S2709). If there is a lattice of unprocessed chunks (step S2709: YES), the extraction apparatus 100 returns to step S2701.

  On the other hand, when there is no lattice of unprocessed chunks (step S2709: No), the extraction apparatus 100 generates the same notation / type word determination rule 300 from the generated determination rule group (step S2710). Then, the extraction apparatus 100 proceeds to step S2801 in FIG.

  In FIG. 28, the extraction apparatus 100 selects unprocessed learning data from the learning data group, and generates target data from which the tags of the selected learning data are removed (step S2801). Next, the extraction apparatus 100 converts the generated target data into a chunk lattice divided in units of chunks (step S2702).

  Then, the extraction apparatus 100 refers to the same notation / type word discrimination rule 300 and identifies a set of chunks of the same notation and the same type in the chunk lattice (step S2803). Next, the extraction apparatus 100 selects an unprocessed chunk from the chunk lattice (step S2804). Then, the extraction apparatus 100 determines whether there is a chunk having the same notation and the same type as the selected chunk from the identification result of step S2802 (step S2805).

  Here, when there is a chunk of the same notation and the same type (step S2805: Yes), the extraction apparatus 100 sets the clue specified from each of the same notation and the same type of chunk and the set of the chunk specified from the tag. And an extraction rule including the type of (step S2806). Then, the extraction device 100 proceeds to step S2808.

  On the other hand, when there is no same notation and same type of chunk (step S2805: No), the extraction device 100 includes the clue specified from the selected chunk and the type of the chunk specified from the tag. An extraction rule is generated (step S2807). Then, the extraction device 100 proceeds to step S2808.

  Next, the extraction apparatus 100 determines whether there is an unprocessed chunk in the lattice of the selected chunk (step S2808). If there is an unprocessed chunk (step S2808: YES), the extraction apparatus 100 returns to step S2804.

  On the other hand, when there is no unprocessed chunk (step S2808: No), the extraction apparatus 100 determines whether there is a lattice of unprocessed chunks (step S2809). If there is an unprocessed chunk lattice (step S2809: YES), the extraction apparatus 100 returns to step S2801.

  On the other hand, when there is no lattice of unprocessed chunks (step S2809: No), the extraction apparatus 100 generates a specific expression extraction rule 400 from the generated extraction rule group (step S2810). Then, the extraction device 100 ends the rule learning process.

  Thereby, the extraction device 100 can generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 by machine learning using the learning data group. Therefore, the user of the extraction apparatus 100 can reduce the trouble of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400.

(Specific Example of Specific Expression Extraction Processing by Extraction Device 100 According to Embodiment 2)
Next, a specific example of the specific expression extraction process performed by the extraction apparatus 100 according to the second embodiment will be described with reference to FIGS. 29 to 32. The specific expression extraction process according to the second embodiment is a process for extracting a chunk of a specific expression from a lattice of chunks obtained by converting data to be extracted of a specific expression, and is the process described with reference to FIG. is there.

  FIG. 29 is an explanatory diagram illustrating an example of a chunk lattice that is a target of extraction of a specific expression by the extraction device 100. As illustrated in FIG. 29, the extraction apparatus 100 receives input data “an employee of B trading goes to B trading and returns from B trading”, which is a target of extraction of a specific expression. Then, the extraction apparatus 100 generates a chunk lattice 2900 that is a target of extraction of the specific expression from the input data.

  Specifically, for example, the extraction apparatus 100 performs morphological analysis on input data “an employee of B Trading goes to B Trading and returns from B Trading”, and divides it into words. The learning data divided into words is, for example, “Employees of B Trading go to B Trading and return from B Trading”.

  The extraction apparatus 100 generates a chunk in which two words are combined from learning data divided for each word. For example, the extraction apparatus 100 generates chunks such as “B Trading”, “Commerce Trading”, “Employees”, and “Employees”. Then, the extraction apparatus 100 generates a chunk lattice from the generated chunk. Then, the extraction apparatus 100 identifies the generated chunk lattice as the chunk lattice that is the target of the specific expression extraction.

  FIG. 30 and FIG. 31 are explanatory diagrams illustrating a specific example of the specific expression extraction process performed by the extraction apparatus 100 according to the second embodiment. In FIG. 30, (1) the extraction apparatus 100 includes a set of “B trading” 2901 and “B trading” 2902, “B trading” 2901, and “ A set of “B trading” 2903 and a set of “B trading” 2902 and “B trading” 2903 are extracted. Further, for example, the extraction apparatus 100 may extract a “B” set or a “trade” set as a set of chunks having the same notation.

  Then, the extraction apparatus 100, for the set of “B Shoji” 2901 and “B Shoji” 2902, combines the words “No & H”, “Employees” around “B Shoji” 2901 and “B Shoji” 2902. “& Go” etc. as a clue. Further, the extraction apparatus 100 uses a combination of words “no & to” and “employee” in the vicinity of “B trading” 2901 and “B trading” 2903 for the set of “B trading” 2901 and “B trading” 2903. & "Return" etc. as a clue. In addition, the extraction apparatus 100, for the set of “B Shoji” 2902 and “B Shoji” 2903, combines word combinations “To & From” and “Go To” around “B Shoji” 2902 and “B Shoji” 2903. & "Return" etc. as a clue.

  (2) Then, using the discrimination engine, the extraction device 100 identifies the discrimination rule of the same notation / type word discrimination rule 300 corresponding to the generated clue. When there are a plurality of applicable discrimination rules, the discrimination engine specifies a plausible discrimination rule. The extraction apparatus 100 uses a discrimination rule specified by the discrimination engine to set “B trading” 2901 and “B trading” 2902, “B trading” 2901 and “B trading” 2903, and “B trading” 2902 and “ It is determined whether or not each of the group “B Trading” 2903 is of the same type.

  Here, the extraction apparatus 100 uses the same notation / type word discrimination rule 300 and the combination of the co-occurrence words “go” and “return”, and the combination of “B trading” 2902 and “B trading” 2903 is of the same type. Determine that there is.

  In FIG. 31, the extraction apparatus 100 handles “B Trading” 2902 and “B Trading” 2903 collectively determined as the same type in FIG.

  (1) The extraction apparatus 100 specifies a clue for extracting the type of chunk for each chunk in the chunk lattice 2900 or for each chunk. Specifically, for example, the extraction apparatus 100 uses, as a clue, “employee: −2”, “ha: −1”, and the like from a group of words around “B trading” 2902 and “B trading” 2903. “To: +1”, “To: +2”, “To: -2”, “To: −1”, “From: +1”, and “Return: +2” are specified. Further, the extraction apparatus 100 identifies “no: +1” and “employee: +2” as clues from words around “B Shoji” 2901 that does not have other chunks of the same type.

  (2) Next, the extraction apparatus 100 specifies the extraction rule of the specific expression extraction rule 400 corresponding to the generated clue using the discrimination engine. When there are a plurality of corresponding extraction rules, the discrimination engine specifies a plausible extraction rule. The extraction apparatus 100 extracts the type of chunk for each chunk in the chunk lattice 2900 or for each chunk that is grouped according to the extraction rule specified by the learning engine. Here, the extraction apparatus 100 extracts “LOC” as the type of “B trading” 2902 and “B trading” 2903 and extracts “ORG” as the type of “B trading” 2901.

  Thereby, the extraction apparatus 100 can extract the same chunk type together for the same notation and the same type of chunk. Further, the extraction apparatus 100 can extract the type of chunk separately from other chunks of the same notation for different types of chunks even if they have the same notation. As a result, the extraction apparatus 100 prevents the extraction of the same type of chunks as different types of chunks by extracting the same type of chunks with the same notation and the same type of chunks, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 prevents different types of chunks from being mistakenly extracted as the same type of chunks by handling different types of chunks separately and extracting the types of chunks, even if they have the same notation. Improvements can be made.

(Output example of specific expression extraction result by extraction apparatus 100 according to embodiment 2)
Next, an output example of the named entity extraction result by the extraction apparatus 100 according to the second embodiment will be described with reference to FIG. The extraction apparatus 100 outputs the specific expression extraction results in FIGS. 29 to 31.

  FIG. 32 is an explanatory diagram of an output example of the specific expression extraction result by the extraction device 100 according to the second embodiment. As shown in FIG. 32, the extraction apparatus 100, for example, uses a chunk lattice 2900 “<ORG> B Shoji </ ORG> as an employee for <LOC> B Shoji </ LOC>” to which the extracted word type is assigned as a tag. Go to <LOC> B Trading </ LOC> ”.

(Detailed processing procedure of specific expression extraction processing according to embodiment 2)
Next, the detailed processing procedure of the specific expression extraction processing according to the second embodiment will be described with reference to FIG. The specific expression extraction process according to the second embodiment is a process performed by the extraction apparatus 100 illustrated in FIGS. 29 to 32.

  FIG. 33 is a flowchart of a detailed process procedure of the named entity extraction process according to the second embodiment. As shown in FIG. 33, first, the extraction device 100 accepts input data (step S3301). Next, the extraction apparatus 100 converts the input data into a chunk lattice divided in units of chunks (step S3302).

  Then, the extraction apparatus 100 refers to the same notation / type word discrimination rule 300 and identifies a set of chunks of the same notation and the same type in the input data (step S3303). Next, the extraction apparatus 100 selects an unprocessed chunk from the input data (step S3304). Then, the extraction device 100 determines whether or not there is a chunk of the same notation and type as the selected chunk from the identification result of step S3303 (step S3305).

  Here, when there is a chunk of the same notation and the same type (step S3305: Yes), the extraction apparatus 100 includes a clue specified from each of the set of chunks of the same notation and the same type, a rule 400 for extracting the unique expression, A score for specifying the type of the unique expression of the same notation and the same type of chunk set is assigned using a rule (step S3306). Then, the extraction device 100 proceeds to step S3308.

  On the other hand, if there is no chunk of the same notation and the same type (step S3305: No), the extraction device 100 selects the chunk selected from the clue specified from the selected chunk and the specific expression extraction rule 400. A score for specifying the type of proper expression is assigned using a rule (step S3307). Then, the extraction device 100 proceeds to step S3308.

  Next, the extraction apparatus 100 determines whether or not there is an unprocessed chunk in the input data (step S3308). If there is an unprocessed chunk (step S3308: Yes), the extraction apparatus 100 returns to step S3304.

  On the other hand, if there is no unprocessed chunk (step S3308: No), the final result is selected. Selection of the final result is based on the score of each unique expression assigned to each chunk, and the sum of the scores among the combinations of chunk path paths that can be taken from the beginning to the end of the sentence and the types of unique expressions that can be taken on that path. Select the column of chunks that has the largest and each named entity type. Thereafter, the extraction apparatus 100 outputs the extraction result (step S3309). Then, the extraction apparatus 100 ends the specific expression extraction process.

  Thereby, the extraction apparatus 100 can extract the same chunk type together for the same notation and the same type of chunk. Further, the extraction apparatus 100 can extract the type of chunk separately from other chunks of the same notation for different types of chunks even if they have the same notation. As a result, the extraction apparatus 100 prevents the extraction of the same type of chunks as different types of chunks by extracting the same type of chunks with the same notation and the same type of chunks, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 prevents different types of chunks from being mistakenly extracted as the same type of chunks by handling different types of chunks separately and extracting the types of chunks, even if they have the same notation. Improvements can be made.

  As described above, when there is a word with the same notation in the text data 110 to be extracted from the specific expression, the extraction apparatus 100 determines whether the word is the same type of word. Next, the extracting means extracts the same word type by grouping the same notation and the same type of word, and extracts different word types for different types of words even if they have the same notation.

  As a result, the extraction device 100 prevents the extraction of the same type of word as a different type of word by extracting the same notation and the same type of word together and extracting the same type of word, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 treats different types of words separately even if they are the same notation, and extracts the type of the word, thereby preventing erroneous extraction as the same type of word and improving the extraction accuracy. Improvements can be made.

  Further, the extraction device 100 can generate the same notation / type word discrimination rule 300 and the specific expression extraction rule 400 from the learning data by machine learning. Therefore, the user of the extraction apparatus 100 can reduce the trouble of generating the same notation / type word discrimination rule 300 and the specific expression extraction rule 400.

  Further, the extraction device 100 generates a chunk lattice by dividing the text data 110 from which the specific expression is to be extracted into chunks. If there are chunks with the same notation in the chunk lattice, the chunks are the same. Determine if it is a type of chunk. Next, the extracting unit extracts chunk types of chunks having the same notation and the same type together, and extracts different types of chunks even if they have the same notation.

  As a result, the extraction apparatus 100 prevents the extraction of the same type of chunks as different types of chunks by extracting the same type of chunks with the same notation and the same type of chunks, thereby improving the extraction accuracy. Improvements can be made. Further, the extraction apparatus 100 prevents different types of chunks from being mistakenly extracted as the same type of chunks by handling different types of chunks separately and extracting the types of chunks, even if they have the same notation. Improvements can be made.

  The extraction method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. The extraction program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The extraction program may be distributed via a network such as the Internet.

  The following additional notes are disclosed with respect to the embodiment described above.

(Additional remark 1) The 1st which memorize | stores the discrimination rule which linked | related the combination of a co-occurrence word, and the information which shows whether the word of the same description which appears with each of the said co-occurrence word is the same kind of word A storage unit;
A second storage unit that stores an extraction rule that associates the combination of the co-occurrence word and the distance to the co-occurrence word and information indicating the type of the word defined according to the distance;
A detection unit for detecting a first word and a second word having the same notation as the first word from a series of words;
A discrimination rule in which the first word detected by the detection unit appears together with one of the co-occurrence words and the second word detected by the detection unit appears together with the other of the co-occurrence words is A determination unit that determines whether the first word and the second word are of the same type based on the determination rule;
When it is determined by the determination unit that they are of the same type, a word existing within a predetermined distance from each of the first word and the second word from the series of words and a distance to the word A specific part that identifies the combination of
Information indicating the type of word associated with the combination specified by the specifying unit is extracted from the extraction rule stored in the second storage unit, and is given to the first word and the second word An extractor to perform,
An output unit for outputting the series of words given by the extraction unit;
An extraction device comprising:

(Supplementary Note 2) A conversion unit that converts the series of words into a plurality of word strings including chunks,
The detector is
Detecting a first chunk and a second chunk having the same notation as the first chunk from the plurality of word strings converted by the conversion unit;
The discrimination unit
A discrimination rule in which the first chunk detected by the detection unit appears with one of the co-occurrence words and the second chunk detected by the detection unit appears with the other of the co-occurrence words is the first rule. Whether or not the first word and the second word are of the same type from the determination rule;
The specific part is:
When it is determined by the determination unit that they are of the same type, a word existing within a predetermined distance from each of the first chunk and the second chunk and the word from the converted word string Identify the combination with distance,
The extraction unit includes:
Information indicating the type of word associated with the combination specified by the specifying unit is extracted from the extraction rule stored in the second storage unit, and is given to the first chunk and the second chunk The extraction apparatus according to Supplementary Note 1, wherein:

(Additional remark 3) The 1st acquisition part which acquires the combination of the word of the same description from the word sequence containing the word to which the information which shows the kind of word was provided,
Second acquisition of acquiring a combination of co-occurrence words appearing with each word of the combination acquired by the first acquisition unit from a word string including a word to which information indicating the type of the word is given. And
A determination unit configured to determine whether or not the word of the same notation is the same type of word based on the information indicating the type of the word given to each word of the combination of the same notation words;
A generation unit that generates a determination rule that associates the combination of co-occurrence words acquired by the second acquisition unit with the determination result determined by the determination unit;
A storage unit that stores the discrimination rule generated by the generation unit in the first storage unit;
The extraction apparatus according to appendix 1 or 2, characterized by comprising:

(Additional remark 4) The said detection part is
A third word and a fourth word having the same notation as the third word are detected from a word string including a word to which information indicating a word type is given;
The discrimination unit
A discrimination rule in which the third word detected by the detection unit appears together with one of the co-occurrence words and the fourth word detected by the detection unit appears together with the other of the co-occurrence words is Whether or not the first word and the second word are of the same type from the determination rule;
The specific part is:
When determined to be the same type by the determination unit, a predetermined distance from each of the third word and the fourth word from a word string including a word to which information indicating the type of the word is given Identify the combination of the word that is within and the distance to that word,
The generator is
The combination of the word specified by the specifying unit and the distance to the word is associated with information indicating the type of the word given to any one of the third word and the fourth word. Generated extraction rules,
The storage unit
4. The extraction device according to appendix 3, wherein the extraction rule generated by the generation unit is stored in the second storage unit.

(Appendix 5)
Detecting a first word and a second word having the same notation as the first word from the series of words;
In a first storage unit for storing a discrimination rule that associates a combination of co-occurrence words and information indicating whether or not the same notation word appearing with each of the co-occurrence words is the same type of word, Determining whether there is a discrimination rule in which the detected first word appears with one of the co-occurrence words and the detected second word appears with the other of the co-occurrence words; If there is, determine whether the first word and the second word are the same type from the determination rule,
When it is determined that they are of the same type, a combination of a word existing within a predetermined distance from each of the first word and the second word and a distance to the word is identified from the series of words And
A combination of the co-occurrence word and the distance to the co-occurrence word and information indicating the type of word defined according to the distance are stored in a second storage unit that stores an extraction rule that associates the information. Information indicating the type of the word associated with the specified combination is extracted from the extraction rule, and is added to the first word and the second word,
Outputting the given sequence of words,
An extraction program characterized by causing processing to be executed.

(Appendix 6)
Detecting a first word and a second word having the same notation as the first word from the series of words;
In a first storage unit for storing a discrimination rule that associates a combination of co-occurrence words and information indicating whether or not the same notation word appearing with each of the co-occurrence words is the same type of word, Determining whether there is a discrimination rule in which the detected first word appears with one of the co-occurrence words and the detected second word appears with the other of the co-occurrence words; If there is, determine whether the first word and the second word are the same type from the determination rule,
When it is determined that they are of the same type, a combination of a word existing within a predetermined distance from each of the first word and the second word and a distance to the word is identified from the series of words And
A combination of the co-occurrence word and the distance to the co-occurrence word and information indicating the type of word defined according to the distance are stored in a second storage unit that stores an extraction rule that associates the information. Information indicating the type of the word associated with the specified combination is extracted from the extraction rule, and is added to the first word and the second word,
Outputting the given sequence of words,
An extraction method characterized by executing processing.

100 Extraction Device 300 Same Notation / Type Word Discrimination Rule 400 Specific Expression Extraction Rule 501 First Storage Unit 502 Second Storage Unit 503 Input Unit 504 Detection Unit 505 Discrimination Unit 506 Identification Unit 507 Extraction Unit 508 Output Unit 509 First Acquisition unit 510 second acquisition unit 511 determination unit 512 generation unit 513 storage unit 514 conversion unit

Claims (5)

A first storage unit that stores a discrimination rule that associates a combination of co-occurrence words and information indicating whether or not the same notation words appearing with each of the co-occurrence words are of the same type;
A second storage unit that stores an extraction rule that associates the combination of the co-occurrence word and the distance to the co-occurrence word and information indicating the type of the word defined according to the distance;
A detection unit for detecting a first word and a second word having the same notation as the first word from a series of words;
A discrimination rule in which the first word detected by the detection unit appears together with one of the co-occurrence words and the second word detected by the detection unit appears together with the other of the co-occurrence words is A determination unit that determines whether the first word and the second word are of the same type based on the determination rule;
When it is determined by the determination unit that they are of the same type, a word existing within a predetermined distance from each of the first word and the second word from the series of words and a distance to the word A specific part that identifies the combination of
Information indicating the type of word associated with the combination specified by the specifying unit is extracted from the extraction rule stored in the second storage unit, and is given to the first word and the second word An extractor to perform,
An output unit for outputting the series of words given by the extraction unit;
An extraction device comprising: A first acquisition unit that acquires a combination of words having the same notation from a word string including a word to which information indicating a word type is given;
Second acquisition of acquiring a combination of co-occurrence words appearing with each word of the combination acquired by the first acquisition unit from a word string including a word to which information indicating the type of the word is given. And
A determination unit configured to determine whether or not the word of the same notation is the same type of word based on the information indicating the type of the word given to each word of the combination of the same notation words;
A generation unit that generates a determination rule that associates the combination of co-occurrence words acquired by the second acquisition unit with the determination result determined by the determination unit;
A storage unit that stores the discrimination rule generated by the generation unit in the first storage unit;
The extraction apparatus according to claim 1, comprising: The detector is
A third word and a fourth word having the same notation as the third word are detected from a word string including a word to which information indicating a word type is given;
The discrimination unit
A discrimination rule in which the third word detected by the detection unit appears together with one of the co-occurrence words and the fourth word detected by the detection unit appears together with the other of the co-occurrence words is Whether or not the first word and the second word are of the same type from the determination rule;
The specific part is:
When determined to be the same type by the determination unit, a predetermined distance from each of the third word and the fourth word from a word string including a word to which information indicating the type of the word is given Identify the combination of the word that is within and the distance to that word,
The generator is
The combination of the word specified by the specifying unit and the distance to the word is associated with information indicating the type of the word given to any one of the third word and the fourth word. Generated extraction rules,
The storage unit
3. The extraction apparatus according to claim 2, wherein the extraction rule generated by the generation unit is stored in the second storage unit. On the computer,
Detecting a first word and a second word having the same notation as the first word from the series of words;
In a first storage unit for storing a discrimination rule that associates a combination of co-occurrence words and information indicating whether or not the same notation word appearing with each of the co-occurrence words is the same type of word, Determining whether there is a discrimination rule in which the detected first word appears with one of the co-occurrence words and the detected second word appears with the other of the co-occurrence words; If there is, determine whether the first word and the second word are the same type from the determination rule,
When it is determined that they are of the same type, a combination of a word existing within a predetermined distance from each of the first word and the second word and a distance to the word is identified from the series of words And
A combination of the co-occurrence word and the distance to the co-occurrence word and information indicating the type of word defined according to the distance are stored in a second storage unit that stores an extraction rule that associates the information. Information indicating the type of the word associated with the specified combination is extracted from the extraction rule, and is added to the first word and the second word,
Outputting the given sequence of words,
An extraction program characterized by causing processing to be executed. Computer
Detecting a first word and a second word having the same notation as the first word from the series of words;
In a first storage unit for storing a discrimination rule that associates a combination of co-occurrence words and information indicating whether or not the same notation word appearing with each of the co-occurrence words is the same type of word, Determining whether there is a discrimination rule in which the detected first word appears with one of the co-occurrence words and the detected second word appears with the other of the co-occurrence words; If there is, determine whether the first word and the second word are the same type from the determination rule,
When it is determined that they are of the same type, a combination of a word existing within a predetermined distance from each of the first word and the second word and a distance to the word is identified from the series of words And
A combination of the co-occurrence word and the distance to the co-occurrence word and information indicating the type of word defined according to the distance are stored in a second storage unit that stores an extraction rule that associates the information. Information indicating the type of the word associated with the specified combination is extracted from the extraction rule, and is added to the first word and the second word,
Outputting the given sequence of words,
An extraction method characterized by executing processing.

Images