JP5297972B2 - Document determination condition generation apparatus and document determination condition generation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a document determination condition for making more highly precise document determination. <P>SOLUTION: A document determination condition generation device is configured to store a plurality of document data with which any category among a plurality of categories is preliminarily associated, and to store a plurality of keywords corresponding to an object category as the object of the generation of a document determination condition and an adaptation indicating the degree of adaptation of the keywords to the object category in association, and to compare the adaptation associated with the keywords included in a keyword list with a predetermined threshold, and to extract the keywords corresponding to the adaptation which is less than the threshold, and to extract document data in which the extracted keywords are included, and to extract correct document data with which the object category has been associated from among the extracted document data, and to extract additional keywords as words whose adaptation to the object category is relatively high from the words included in the extracted correct document data, and to generate document determination conditions indicating that the extracted keywords and the additional keywords are included. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a technique for generating a document determination condition for determining a category indicating a classification according to the content according to a word included in document data.

  In recent years, document data in which various types of information are digitized has been processed by a computer, and a technique for classifying such document data into categories according to the contents thereof has been studied (for example, Patent Document 1). Patent Document 2). For example, for each of a plurality of predetermined categories, a keyword that appears characteristically in the document data of the category is determined, and a word included in the document data to be determined as a category is compared with the determined keyword. Then, it is determined whether or not the document data to be determined belongs to the category by determining whether or not the keyword is included. By using such technology, for example, document data that is news articles of various genres are classified according to the contents, or the document data that is e-mail received from various customers by a company that sells products. It can be classified for each product according to its contents.

JP 2007-79948 A JP 2008-21028 A

  However, when performing document determination as to which category the document data belongs to using a single keyword as described above, document data is determined to be a category that does not correspond to the content, or document data is not determined to be a corresponding category. There was a case. For example, when the keyword “recovery” is associated as the keyword of the medical category, the classification noise is generated such that the document data of the economic category including the word “recovery” about the economy is classified into the medical category. May occur. Here, it is desirable to perform document determination using a more appropriate keyword corresponding to the category as a document determination condition to reduce classification noise.

  The present invention has been made in view of such a situation, and provides a document determination condition generation apparatus and a document determination condition generation method for generating a document determination condition for performing document determination with less classification noise.

  In order to solve the above-described problem, the present invention provides a document determination that generates a document determination condition for determining a category indicating a classification according to the content of document data in accordance with a word included in the defined document data. A condition generation device, a document data storage unit storing a plurality of document data in which any one of a plurality of categories is associated in advance, and a target for generating a document determination condition among the plurality of categories A keyword list storage unit in which a plurality of keywords corresponding to a certain target category and a matching rate indicating a degree of matching of the keyword with the target category are stored in association with each other, and a keyword included in the keyword list The keyword to be improved is a keyword associated with a matching rate that is less than the threshold by comparing the matching rate with a predetermined threshold. The target document extraction unit for extracting the keyword from the keyword list storage unit, and correct document data that is the document data including the improvement target keyword extracted by the improvement target keyword extraction unit and associated with the target category is stored in the document data An additional keyword that extracts a first additional keyword that is a word having a relatively high matching rate with respect to the target category from words included in the correct document data extracted by the document data extracting unit and the document data extracting unit An extraction unit; and a condition generation unit that generates a document determination condition indicating that the improvement target keyword extracted by the improvement target keyword extraction unit is included and that the first additional keyword extracted by the additional keyword extraction unit is included It is characterized by providing.

  Further, the present invention provides a morphological analysis unit that performs morphological analysis on a sentence included in correct document data extracted by a document data extraction unit, and outputs a morpheme included in correct document data and a part of speech corresponding to the morpheme, and a morphological analysis A processing target word extraction unit that extracts a processing target word that is a word including a morpheme whose part of speech is a noun based on the result of the morphological analysis by the part, and the additional keyword extraction unit is A first additional keyword is extracted from the extracted processing target word.

  In the present invention, the document data extraction unit includes the improvement target keyword extracted by the improvement target keyword extraction unit and the first additional keyword extracted by the additional keyword extraction unit, and corresponds to a category other than the target category. Incorrect document data, which is attached document data, is extracted, and the additional keyword extraction unit has relative relevance ratios with respect to categories other than the target category from words included in the incorrect answer document data extracted by the document data extraction unit. The second additional keyword that is a high word is extracted, and the condition generation unit includes the improvement target keyword extracted by the improvement target keyword extraction unit, and the second additional keyword extracted by the additional keyword extraction unit is A document determination condition indicating that it is not included is generated.

  The present invention also reproduces the ratio of the number of correct document data extracted by the document data extraction unit and the number of document data extracted from the document data stored in the document data storage unit using the document determination condition. The recall rate calculation unit that calculates the rate, the recall rate calculated by the recall rate calculation unit, and a predetermined recall rate threshold value are compared to determine whether the recall rate is less than the threshold value. A comparison unit, and the additional keyword extraction unit extracts a plurality of first additional keywords having a relatively high relevance ratio with respect to the target category from words included in the correct document data extracted by the document data extraction unit. The condition generation unit includes the improvement target keyword extracted by the improvement target keyword extraction unit and the additional keyword extraction when the reproduction rate comparison unit determines that the reproduction rate is less than the threshold value. And generating a document determination conditions shown to include one of the keywords of the extracted plurality of first additional keyword by.

  Further, according to the present invention, the additional keyword extraction unit includes a plurality of second additional keywords having a relatively high relevance rate for a category other than the target category from words included in the incorrect answer document data extracted by the document data extraction unit. The condition generation unit includes the improvement target keyword extracted by the improvement target keyword extraction unit and is extracted by the additional keyword extraction unit when the reproduction rate comparison unit determines that the reproduction rate is equal to or greater than the threshold. A document determination condition indicating that none of the plurality of second additional keywords is included is generated.

  Further, according to the present invention, the relevance ratio of a keyword to a category is the number of document data associated with a category among the document data stored in the document data storage unit, and the document data of all categories including the keyword. It is represented by the ratio with the number.

  In addition, the present invention provides a document data storage unit that stores a plurality of document data in which any one of a plurality of categories indicating classification according to the contents of the defined document data is associated in advance, and a plurality of document data Among the categories, a plurality of keywords corresponding to the target category for generating a document determination condition for determining the category of the document data according to a word included in the document data, and the keyword matches the target category. A document determination condition generation method of a document determination condition generation device including a keyword list storage unit that stores a matching ratio indicating a degree in association with the matching ratio that is associated with a keyword included in the keyword list And a predetermined threshold value are compared, and a keyword corresponding to the matching rate that is less than the threshold value is extracted from the keyword list storage unit. And extracting the correct document data, which is the document data that includes the extracted keyword and the associated category, from the document data storage unit, and from the words included in the extracted correct document data, Extracting a first additional keyword that is a word having a relatively high relevance rate; generating a document determination condition indicating that the extracted keyword is included and the extracted first additional keyword is included; It is characterized by providing.

  As described above, according to the present invention, a plurality of document data in which any one of a plurality of categories is associated in advance is stored, and a plurality of items corresponding to a target category for which a document determination condition is generated is stored. Are stored in association with the matching rate indicating the degree of matching of the keyword with the target category, and the matching rate associated with the keyword included in the keyword list is compared with a predetermined threshold value. , A keyword corresponding to the relevance ratio that is less than the threshold is extracted, document data including the extracted keyword is extracted, and correct document data associated with the target category is extracted from the extracted document data and extracted. Extract additional keywords that are words with relatively high relevance to the target category from the words included in the correct document data, and extract the key A document judgment condition indicating that a keyword and an additional keyword are included is generated, and a keyword having a matching rate less than a threshold among keywords corresponding to the target category is included in the document data including the keyword. In order to perform document determination with less classification noise, it is possible to extract an additional keyword having a relatively high matching rate with respect to the target category and generate a document determination condition that is a compound condition with the keyword. It is possible to provide a document determination condition generation device that generates the document determination condition.

It is a block diagram which shows the structural example of the document determination condition production | generation apparatus by one Embodiment of this invention. It is a figure which shows the relationship between the category and keyword by one Embodiment of this invention. It is a flowchart which shows the operation example of the document determination condition production | generation apparatus by one Embodiment of this invention. It is a flowchart which shows the operation example of the additional keyword extraction process by one Embodiment of this invention. It is a flowchart which shows the operation example of the document determination condition production | generation process by one Embodiment of this invention. It is a figure which shows the relationship between the recall and the number of extraction documents according to the document determination condition production | generation process by one Embodiment of this invention. It is a figure which shows the example of the document determination conditions by one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration of a document determination condition generation device 100 according to the present embodiment. The document determination condition generation device 100 is a computer device that generates a document determination condition for determining a category indicating a classification according to the content of document data in accordance with a word included in the defined document data. FIG. 2 is a diagram conceptually illustrating the relationship between a set of document data associated with a specific category and a set of document data including a specific keyword. For example, among a set of document data in which categories corresponding to contents are associated in advance, a document data set in which medical categories are associated in advance (reference a) and a document data set in which economic categories are associated in advance (references) b). Each category is associated in advance with a keyword that appears characteristically in the document data of that category.

  For example, keywords such as “bone fracture” (symbol a1), “knee” (symbol a2), “lymphocytic leukemia” (symbol a3), and “recovery” (symbol a4) are associated with the medical category. Document data belonging to the medical category can be extracted by extracting a document data set including the keyword using such a keyword as a document determination condition. Here, the category of the document data is determined depending on whether or not the keyword is included in the document data whose category is unknown.

  In this example, the document data set including the keyword “lymphocytic leukemia” indicated by the symbol a3 is a subset included in the document data set associated with the medical category. That is, if the keyword “lymphatic leukemia” is included in the document data whose category is unknown, the document can be determined with high accuracy if it is determined that the category of the document data is medical. On the other hand, for example, the document data set including the keyword “recovery” indicated by the symbol a4 is included in the medical category and the economic category. That is, if the document data whose category is unknown includes the keyword “recovery”, if it is determined that the category of the document data is medical, the document data of the economic category may be determined as the medical category. Such a determination is called classification noise.

  Therefore, in the present embodiment, a document determination condition that enables document determination with higher accuracy is generated, and classification noise is reduced. For example, in a document data set including the keyword “recovery”, keywords (AND additional keywords) characteristic to document data belonging to the medical category are combined with the keyword “recovery” by a logical product operator (AND). According to the document determination condition, it is considered possible to improve the accuracy of document determination. In this case, it is possible to extract the document data of the area of the code a4-1 belonging to the medical category from the document data including the keyword “recovery”. On the other hand, the accuracy of document determination can be improved also by a document determination condition in which a keyword (NOT additional keyword) characteristic to document data belonging to the economic category is combined with the keyword “recovery” by a negative operator (NOT). It is considered possible. In this case, from the document data including the keyword “recovery”, it is possible to extract and exclude the document data in the area of the code a4-2 that does not belong to the medical category.

  For example, in the medical category document data, according to the document determination condition in which the keyword “recovery” and the additional keyword “condition” corresponding to the keyword “recovery” and characteristic to the medical category are combined by an AND operator. Therefore, it is considered possible to perform document determination with higher accuracy. As described above, the document determination condition generation apparatus 100 according to the present embodiment determines an optimum additional keyword for performing document determination with higher accuracy for document data with unknown categories, and generates a document determination condition. is there. Here, as a method for determining document data according to the content, there is a classification method using a model constructed by machine learning based on the appearance rate, etc. in addition to classification by keyword, but the basis of classification is clear As shown in this embodiment, the keyword classification method is considered to be superior.

  Returning to FIG. 1, the document determination condition generation device 100 includes a keyword list storage unit 111, an improvement target keyword extraction unit 112, a document data storage unit 110, a document data extraction unit 121, a morpheme analysis unit 122, and a processing target. A word extraction unit 123, an additional keyword extraction unit 124, a first reproduction rate calculation unit 131, a first reproduction rate comparison unit 132, an extracted document number comparison unit 133, and a NOT additional keyword candidate re-extraction unit 134 , A second reproduction rate calculation unit 135, a second reproduction rate comparison unit 136, and a condition generation unit 137.

  The document data storage unit 110 stores a plurality of document data in which any one of a plurality of predetermined categories is associated in advance. The document data includes a plurality of sentences or words. For example, categories such as politics, economy, society, sports, and entertainment are stored in association with each other according to the content of document data that is a news article. For example, the document data storage unit 110 stores a document ID (Identifier) that identifies document data, document data, and a category corresponding to the content of the document data in association with each other. The category associated with the document data may be associated with a category input according to the content of the document data by the administrator, or may be associated with a category predetermined by the creator of the document data. Alternatively, the categories determined using the document determination conditions generated by the document determination condition generation apparatus 100 of the present embodiment may be associated with each other. Here, as document data stored in the document data storage unit 110, in addition to news articles, for example, medical receipt data, various academic papers, inquiry contents data to call centers of companies, e-mails, insurance claims, etc. Payment classification data can be applied.

  In the keyword list storage unit 111, for each of a plurality of predetermined categories, a keyword list in which a plurality of keywords corresponding to the category are associated with a matching rate indicating a degree of matching of the keyword with the category. Is memorized. For example, in association with political categories, multiple keywords such as “election”, “political party”, “policy”, political party names, politician names, etc. are associated with the relevance ratio of each keyword to the political category. A keyword list is stored.

  The relevance ratio of the keyword indicating the degree of conformity with the target category is, for example, the document data stored in the document data storage unit 110 that is associated with the target category and includes the keyword. It is represented by the ratio between the number and the number of document data of all categories including the keyword. Here, “relevance ratio = (number of document data including the corresponding keyword among document data belonging to the corresponding category) / (number of document data of all categories including the corresponding keyword)”). The For example, the relevance rate of the keyword “election” with respect to the political category is “relevance rate = (the number of document data including the keyword“ election ”among the document data belonging to the political category stored in the document data storage unit 110) / (Of all the document data stored in the document data storage unit 110, the number of document data including the keyword “election”) ”.

  The improvement target keyword extraction unit 112 compares the matching rate associated with the keyword included in the keyword list stored in the keyword list storage unit 111 with a predetermined threshold value, and handles the matching rate that is less than the threshold value. Keywords to be improved are extracted. That is, the improvement target keyword extraction unit 112 reads a keyword list corresponding to a category for which a keyword is to be reviewed from the keyword list storage unit 111, and associates the keyword list with a keyword among a plurality of keywords included in the read keyword list. The matching precision is compared with a predetermined threshold, and a keyword associated with a precision that is less than the threshold is extracted as an improvement target keyword to be reviewed as a document determination condition. Here, the improvement target keyword extraction unit 112 stores in advance a threshold value of an adaptation rate such as “80%” in its own storage area. In this way, the improvement target keyword extraction unit 112, among the keywords included in the keyword list corresponding to the target category, has a large amount of classification noise when used as a document determination condition, and is an improvement target keyword that seems to have low classification accuracy. To extract. Here, when a plurality of improvement target keyword groups whose relevance ratio is less than the threshold value are extracted by the improvement target keyword extraction unit 112, processing by the following functional units is performed for each of the extracted improvement target keyword groups. .

  The document data extraction unit 121 extracts, from the document data storage unit 110, correct document data that is document data that includes the improvement target keyword extracted by the improvement target keyword extraction unit 112 and is associated with the target category. Further, the document data extraction unit 121 extracts incorrect document data that is document data that includes the improvement target keyword extracted by the improvement target keyword extraction unit 112 and is associated with a category other than the target category. For example, the document data extraction unit 121 extracts the document data including the improvement target keyword from the document determination condition generation device 100, and determines that the document is correct if the category associated with the extracted document data is the target category. If the category associated with the document data is a category other than the target category, the document is determined to be an incorrect answer document.

  The morpheme analyzing unit 122 performs morphological analysis on the sentence included in the correct document data or the incorrect document data extracted by the document data extracting unit 121, and outputs the morpheme included in the correct document data and the part of speech corresponding to the morpheme. . Morphological analysis is a process of analyzing a sentence and dividing it into the smallest meaningful units (morphemes) based on a predetermined grammar and dictionary. For example, the morpheme analysis unit 122 stores a dictionary in which a plurality of words and information indicating the part of speech of the words are associated with each other in its storage area in advance. In addition, the morpheme analysis unit 122 stores information indicating the grammatical rule indicating the connection relation of parts of speech in its storage area in advance. The morpheme analysis unit 122 divides the sentence to be analyzed into words included in a dictionary stored in advance. The morphological analysis unit 122 determines whether or not the connection relation of the part of speech corresponding to each divided word is a correct arrangement order according to a predetermined grammar rule, and the division is performed when the correct arrangement order is obtained. Make the word correct. For example, when the sentence to be analyzed is “the lymphatic leukemia has developed”, the morphological analysis unit 122 sets the sentence to be analyzed as “lymph” (noun), “sex” (noun), “leukemia” (noun), “ Is divided into seven words: “on” (particle), “onset” (noun), “shi” (verb), and “ta” (auxiliary verb).

  The processing target word extraction unit 123 extracts a morpheme whose part of speech is a noun based on the result of the morphological analysis by the morpheme analysis unit 122. Here, the processing target word extraction unit 123 may not extract a noun such as a non-independent noun or an adverb noun among words whose part of speech is a noun. Further, the processing target word extraction unit 123 generates a compound word that is a combination of consecutive nouns based on the extracted morphemes. For example, when three morphemes of “lymph”, “sex”, and “leukemia” are consecutive, compound words of “lymphoid”, “sexual leukemia”, and “lymphocytic leukemia” are generated. Further, the processing target word extraction unit 123 deletes an inappropriate morpheme or compound word from the morpheme extracted by the morpheme analysis unit 122 and the generated compound word based on a predetermined condition. For example, the processing target word extraction unit 123 deletes a compound word starting with a suffix, a compound word ending with a prefix, a morpheme consisting of only one alphabet or hiragana, a morpheme or compound word composed only of numbers, and the like as inappropriate. In this example, the compound word “sexual leukemia” is deleted because it begins with a suffix, and the five morphemes and compound of “lymph”, “sex”, “leukemia”, “lymphoid”, and “lymphocytic leukemia” are combined. A word is extracted as a processing target word.

  The additional keyword extraction unit 124 extracts, as AND additional keywords, a plurality of words having a relatively high matching rate with respect to the target category from the words included in the correct document data extracted by the document data extraction unit 121. The AND addition keyword is a keyword that is combined with the improvement target keyword by a logical product operator. The additional keyword extraction unit 124 selects a predetermined number (for example, 50) from the processing target words extracted by the processing target word extraction unit 123 among the words included in the correct document data extracted by the document data extraction unit 121. Keyword) AND addition keyword is extracted. Here, the AND additional keyword extracted by the additional keyword extraction unit 124 is combined with the improvement target keyword by a logical product operator. In addition, when a plurality of AND addition keywords are extracted, the plurality of AND addition keywords are combined by a logical sum operator.

  Here, the additional keyword extraction unit 124 calculates the relevance ratio with respect to the target category for each of the processing target words extracted by the processing target word extraction unit 123. The additional keyword extracting unit 124 is included in the relevance ratio of the processing target word extracted by the processing target word extracting unit 123 to the target category, the number of document data to which the processing target word belongs among the document data belonging to the target category, and the processing target word. A predetermined number of AND added keywords are extracted based on the number of morphemes and the like. The additional keyword extraction unit 124 stores in advance a threshold for the number of additional keywords to be extracted in its own storage area. Similarly, the additional keyword extraction unit 124 includes a plurality of NOT additional keywords that are words having relatively high relevance ratios for categories other than the target category from words included in the incorrect document data extracted by the document data extraction unit 121. To extract. As described above, the additional keyword extraction unit 124 extracts an additional keyword including an AND additional keyword having a relatively high matching rate for the target category and a NOT additional keyword having a relatively high matching rate for a category other than the target category.

  The first recall ratio calculation unit 131 is stored in the document data storage unit 110 using the number of correct document data extracted by the document data extraction unit 121 and a document determination condition generated by a condition generation unit 137 described later. A reproduction ratio that is a ratio with the number of document data extracted from the document data is calculated. Here, the first recall ratio calculation unit 131 generates and generates a document determination condition in which the improvement target keyword and a plurality of AND additional keywords extracted by the additional keyword extraction unit 124 are combined by a logical product operator. Document data that matches the document determination condition is extracted from the document data storage unit 110, and the recall is calculated. The recall rate indicates the ratio of the number of correct documents extracted using the document determination condition with the AND additional keyword or the NOT additional keyword added to the number of correct documents extracted only with the improvement target keyword. That is, the recall rate is “recall rate = (an improvement operator keyword and an AND additional keyword extracted as the first candidate among a plurality of AND additional keywords extracted by the additional keyword extracting unit 124 using a logical product operator. The number of document data extracted based on the combined document determination condition) / (the number of correct document data extracted by the document data extraction unit 121) ”. In addition, when the first reproduction rate calculation unit 131 determines that the calculated reproduction rate is less than the allowable reproduction rate by the first reproduction rate comparison unit 132, the next candidate extracted by the additional keyword extraction unit 124. The document addition condition of the AND addition keyword is combined with the improvement target keyword by the logical product operator, the document data matching the generated document determination condition is extracted from the document data storage unit 110, and the reproduction rate is calculated. To do. As described above, the calculation process of the reproduction rate by the first reproduction rate calculation unit 131 is repeatedly looped every time it is determined that the reproduction rate calculated by the first reproduction rate comparison unit 132 is less than the reproduction rate. Done. Here, the AND addition keyword added to the improvement target keyword by the first recall ratio calculation unit 131 is combined with the improvement target keyword by a logical product operator. Further, when a plurality of AND addition keywords are added to the improvement target keyword, the plurality of AND addition keywords are combined by a logical sum operator.

  The first reproduction rate comparison unit 132 compares the reproduction rate calculated by the first reproduction rate calculation unit 131 with a predetermined reproduction rate threshold (allowable reproduction rate), and the reproduction rate is less than the threshold. It is determined whether or not there is. The allowable recall is a value such as 80 (%), for example, and is stored in advance in its own storage area. The allowable reproduction rate is a threshold value that determines how much the correct document extraction can be reproduced by using the reproduction rate as an index to allow the document determination condition. For example, if the allowable recall is 80%, when the correct answer document extracted only with the improvement target keyword is 100, the correct answer document extracted using the category determination condition with the additional keyword added is 80% or more. If so, the document determination condition is allowed to be valid. Here, if the first recall ratio comparison unit 132 determines that the recall rate is less than the allowable recall rate, the first recall ratio calculation unit 131 sets the document determination condition to which the next candidate AND addition keyword is added. The recall ratio used is calculated, and the determination process by the first recall ratio comparison unit 132 is repeatedly looped.

  The extracted document number comparison unit 133 uses the document data number (extracted document number) when the document data is extracted from the document data storage unit 110 using only the improvement target keyword as the document determination condition, and the first reproduction rate calculation unit 131 or The number of document data (the number of extracted documents) when the document data is extracted from the document data storage unit 110 using the document determination condition generated by the second reproduction rate calculation unit 135 is compared.

  The NOT additional keyword candidate re-extracting unit 134 uses the extracted document number comparing unit 133 to extract the number of extracted documents based only on the improvement target keyword, and the number of extracted documents based on the document determination condition generated by the first recall ratio calculating unit 131. Are determined to be inconsistent with each other, the AND additional keywords including the improvement target keyword and added by the first recall calculation unit 131 (if there are a plurality of AND additional keywords, Document data including any one of them is extracted from the document data storage unit 110. Then, the NOT keyword candidate re-extraction unit 134 re-extracts the NOT additional keyword using the incorrect document data of the extracted document data as a new incorrect document.

  The second recall ratio calculating unit 135 adds a negative operator to the NOT additional keyword extracted as the first candidate among the NOT additional keywords extracted by the additional keyword extracting unit 124 or the NOT keyword candidate re-extracting unit 134. , Generating a document determination condition in which a NOT addition keyword to which a negation operator is added and an improvement target keyword are combined by a logical product operator, and extracting document data that matches the generated document determination condition from the document data storage unit 110, Calculate recall. In addition, when the second recall ratio calculation unit 135 determines that the calculated recall ratio is equal to or higher than the allowable recall rate by the second recall ratio comparison unit 136, the second candidate extracted by the additional keyword extraction unit 124. The NOT addition keyword is added to the NOT addition keyword, a document determination condition is generated by combining the improvement target keyword with the logical product operator, and the document data matching the generated document determination condition is extracted from the document data storage unit 110 And the recall is calculated.

  The second reproduction rate comparison unit 136 compares the reproduction rate calculated by the second reproduction rate calculation unit 135 with a predetermined reproduction rate threshold (allowable reproduction rate), and the reproduction rate is equal to or greater than the threshold. It is determined whether or not there is. The allowable recall is a value such as 80 (%), for example, and the value stored in the first recall comparison unit 132 is referred to.

  The condition generation unit 137 generates a document determination condition that includes the improvement target keyword extracted by the improvement target keyword extraction unit 112 and includes the AND additional keyword extracted by the additional keyword extraction unit 124. In addition, the condition generation unit 137 generates a document determination condition indicating that the keyword extracted by the improvement target keyword extraction unit 112 is included and the NOT additional keyword extracted by the additional keyword extraction unit 124 is not included. Here, the condition generation unit 137 stores variables such as the minimum number of extracted documents, the optimal number of AND additional keywords, and the optimal number of AND additional keywords in its storage area. It is updated by the recall rate calculation unit 131, the second recall rate calculation unit 135, and the like.

  Here, the condition generation unit 137 combines the number of AND additional keywords indicated by the optimal number of AND additional keywords among the plurality of AND additional keywords extracted by the additional keyword extraction unit 124 by a logical sum operator (OR). . In addition, the condition generation unit 137 combines a plurality of AND addition keywords combined by a logical sum operator and the keywords extracted by the improvement target keyword extraction unit 112 by a logical product operator.

  In addition, the condition generation unit 137 adds a negative operator to the number of NOT additional keywords indicated by the optimal number of NOT additional keywords among the plurality of NOT additional keywords extracted by the additional keyword extraction unit 124, and adds a negative operator. A plurality of added NOT keywords are combined by a logical product operator. In addition, the condition generation unit 137 combines a plurality of NOT addition keywords combined by a logical product operator and the keywords extracted by the improvement target keyword extraction unit 112 by a logical product operator.

  Next, an operation example of the document determination condition generation device 100 according to the present embodiment will be described. FIG. 3 is a flowchart illustrating an example of an operation in which the document determination condition generation device 100 generates a determination condition for a target category that is a target for review of the determination condition. Here, the document determination condition generation device 100 includes, for example, a display that is a display device, a keyboard and a mouse that are input devices that receive input from a user, and displays a plurality of predetermined categories on the display device. The input of the target category for reviewing the judgment condition is accepted.

  The improvement target keyword extraction unit 112 of the document determination condition generation apparatus 100 reads a keyword list corresponding to the target category from the keyword list storage unit 111. The improvement target keyword extraction unit 112 compares the relevance rate associated with the keyword and the predetermined threshold value among the plurality of keywords included in the read keyword list, and the relevance rate that is less than the threshold value is associated. The determined keywords are extracted as improvement target keywords (in a case where there are a plurality of improvement target keywords), the determination conditions are reviewed (step S100).

  Next, the document determination condition generation device 100 performs a process of extracting an additional keyword for the improvement target keyword extracted in step S100 (step S200). FIG. 4 is a flowchart showing additional keyword extraction processing by the document determination condition generation device 100. The document data extraction unit 121 reads out correct document data and incorrect document data including the improvement target keyword extracted by the improvement target keyword extraction unit 112 from the document data storage unit 110 (step S201). The morpheme analysis unit 122 performs a morpheme analysis process of the sentence included in the document data extracted by the improvement target keyword extraction unit 112, and outputs information on the morpheme and the part of speech corresponding to the morpheme (step S202). The processing target word extraction unit 123 extracts a morpheme whose part of speech is a noun from the morpheme output by the morpheme analysis unit 122. In addition, the processing target word extraction unit 123 generates a plurality of compound words that are combinations of morphemes whose part of speech is a noun (step S203). The processing target word extraction unit 123 deletes the determined inappropriate morpheme or compound word from the extracted morpheme and the generated multiple compound words, and outputs the processing target word (step S204).

  The additional keyword extraction unit 124 calculates the relevance ratio of the processing target word output from the processing target word extraction section 123 to the target category, and extracts the processing target word having the highest relevance ratio of the target category to the correct document (step S210). . Here, the additional keyword extraction unit 124 excludes processing target words that have already been extracted as AND additional keyword candidates from the extraction target. Then, the additional keyword extraction unit 124 determines whether there are a plurality of processing target words extracted in step S210 (step S211). Here, if the extracted processing target word is one (step S211: NO), the process proceeds to step S217. If there are a plurality of processing target words extracted (step S211: YES), the additional keyword extraction unit 124 calculates the number of correct document data including the processing target word among the processing target words extracted in step S210. Then, processing target words having a large number of document data are extracted (step S212).

  The additional keyword extraction unit 124 determines whether there are a plurality of processing target words extracted in step S212 (step S213). If the extracted processing target word is one (step S213: NO), the process proceeds to step S217. If there are a plurality of processing target words extracted (step S213: YES), the additional keyword extraction unit 124 has a large number of morphemes included in the processing target word among the processing target words extracted in step S212. A word is extracted (step S214).

  The additional keyword extraction unit 124 determines whether or not there are a plurality of processing target words extracted in step S214 (step S215). If the extracted processing target word is one (step S215: NO), the process proceeds to step S217. If there are a plurality of processing target words extracted (step S215: YES), the additional keyword extraction unit 124 extracts an arbitrary processing target word from the processing target words extracted in step S214 (step S216). Here, when extracting an arbitrary processing target word, the additional keyword extraction unit 124 extracts, for example, the processing target word extracted first or last from the document data.

The additional keyword extraction unit 124 determines the processing target word extracted in any of step S210, step S212, step S214, or step S216 as an AND additional keyword candidate (step S217). The additional keyword extraction unit 124 determines whether or not a predetermined number of AND additional keyword candidates have been extracted (step S218). If the additional keyword extraction unit 124 has not extracted a predetermined number of AND additional keyword candidates (step S218: NO), the process returns to step S210 to extract other AND additional keyword candidates. On the other hand, if the additional keyword extraction unit 124 has extracted a predetermined number of AND additional keyword candidates (step S218: YES), the process proceeds to step S220. Similar to steps S210 to S218, the additional keyword extraction unit 124, in advance, from the processing target words extracted by the processing target word extraction unit 123 based on the incorrect document data extracted by the document data extraction unit 121 in step S201. A predetermined number of NOT additional keywords are extracted (steps S220 to S228).
In this example, in step S210, the additional keyword extraction unit 124 excludes a processing target word that has already been extracted as an AND additional keyword candidate from the extraction target, thereby extracting a plurality of the same AND additional keyword candidates. For example, in step S217, the document data including the extracted AND added keyword may be removed from the correct document, and the process may return to step S210. Similarly, when extracting NOT additional keyword candidates, a plurality of the same NOT additional keyword candidates are extracted by excluding from the extraction target processing target words already extracted as NOT additional keyword candidates in step S220. In step S227, the document data including the extracted NOT additional keyword may be excluded from the incorrect document, and the process may return to step S220.

  Returning to FIG. 3, the document determination condition generation device 100 generates an optimal document determination condition based on the plurality of additional keywords extracted in step S200 (step S300). FIG. 5 is a flowchart showing document determination condition generation processing by the document determination condition generation apparatus 100. The condition generation unit 137 stores variables such as the minimum number of extracted documents, the optimal number of AND keywords, and the optimal number of NOT keywords in its own storage area, and initializes the respective values (step S301). The minimum number of extracted documents is a value indicating the smallest number of documents among the number of document data (number of documents) extracted from the document data storage unit 110 using the plurality of generated document determination conditions. The optimum number of AND keywords is a value indicating the number of AND keywords added under the document determination condition corresponding to the minimum number of extracted documents. The optimum number of NOT keywords is a value indicating the number of NOT keywords added under the document determination condition corresponding to the minimum number of extracted documents.

  The first recall ratio calculating unit 131 combines the improvement target keyword and the AND additional keyword extracted as the first candidate among the plurality of AND additional keywords extracted by the additional keyword extracting unit 124 using a logical product operator. The document determination condition is generated, document data that matches the generated document determination condition is extracted from the document data storage unit 110, and the reproduction rate is calculated. The first reproduction rate comparison unit 132 compares the reproduction rate calculated by the first reproduction rate calculation unit 131 with the allowable reproduction rate stored in its own storage area, and whether the reproduction rate exceeds the allowable reproduction rate. It is determined whether or not (step S302). If the first recall ratio comparison unit 132 determines that the recall ratio exceeds the allowable recall ratio (step S302: YES), the process proceeds to step S304.

  On the other hand, when the first recall ratio comparing unit 132 determines that the recall rate does not exceed the allowable recall rate (step S302: NO), the first recall rate calculating unit 131 is extracted by the additional keyword extracting unit 124. A document determination condition is generated by adding the next candidate AND addition keyword among the AND addition keywords (step S303). Here, the first recall calculation unit 131 combines a plurality of AND addition keywords with a logical OR operator, and combines the plurality of AND addition keywords and an improvement target keyword with a logical product operator. The first reproduction rate calculation unit 131 returns to step S302, extracts document data that matches the generated document determination condition from the document data storage unit 110, and calculates the reproduction rate. Then, the document determination condition generation apparatus 100 repeats the processes of step S302 and step S302 until the first reproduction ratio comparison unit 132 determines that the reproduction ratio exceeds the allowable reproduction ratio.

  In step S302, if the first reproduction rate comparison unit 132 determines that the reproduction rate exceeds the allowable reproduction rate, the extracted document number comparison unit 133 uses only the improvement target keyword as a document determination condition, and the document data storage unit 110. The document data is extracted from the document data storage unit 110 by using the document data number (extracted document number) when the document data is extracted from the document data and the document determination condition generated by the first recall calculation unit 131 in step S303. The number of document data in this case (the number of extracted documents) is compared (step S304). When the extracted document number comparison unit 133 determines that the number of extracted documents based only on the improvement target keyword matches the number of extracted documents based on the document determination condition generated by the first recall ratio calculation unit 131 (step S304: YES). ), The second recall ratio calculation unit 135 substitutes 0 (zero) for the variable of the optimal number of AND keywords stored in the storage area of the condition generation unit 137 (step S305).

  The second recall ratio calculation unit 135 adds a NOT operator to the NOT addition keyword extracted as the first candidate among the NOT addition keywords extracted by the addition keyword extraction unit 124, and adds a NOT operator to the NOT addition keyword. A document determination condition in which the additional keyword and the improvement target keyword are combined by a logical product operator is generated, document data that matches the generated document determination condition is extracted from the document data storage unit 110, and a reproduction rate is calculated. In addition, the condition generation unit 137 stores the number of extracted documents using the document determination condition at this point in its storage area as the minimum number of extracted documents. The second reproduction rate comparison unit 136 compares the reproduction rate calculated by the second reproduction rate calculation unit 135 with the allowable reproduction rate stored in its own storage area, and the reproduction rate exceeds the allowable reproduction rate. It is determined whether or not there is (step S306).

  In step S306, if the second recall ratio comparison unit 136 determines that the recall rate exceeds the allowable recall rate (step S306: YES), the second recall ratio calculation unit 135 is extracted by the additional keyword extraction unit 124. A document determination condition is generated by adding the next candidate NOT addition keyword among the NOT addition keywords that have been added (step S307). Here, the second recall ratio calculation unit 135 combines a plurality of NOT addition keywords added with a negation operator with a logical sum operator, and performs a logical product operation on the combined plurality of NOT addition keywords and the improvement target keyword. Join with children. The second reproduction rate calculation unit 135 returns to step S306, extracts document data that matches the generated document determination condition from the document data storage unit 110, and calculates the reproduction rate. Then, the document determination condition generation device 100 repeats the processes of step S306 and step S307 until the second reproduction rate comparison unit 136 determines that the reproduction rate does not exceed the allowable reproduction rate. In step S306, when the second reproduction rate comparison unit 136 determines that the reproduction rate does not exceed the allowable reproduction rate (step S306: NO), the second reproduction rate comparison unit 136 sets the reproduction rate to the allowable reproduction rate. The larger number of NOT additional keywords is substituted into the variable of the optimum NOT keyword number stored in the storage area of the condition generation unit 137 (step S308).

  On the other hand, in step S304, if the extracted document number comparison unit 133 does not match the number of extracted documents based only on the improvement target keyword and the number of extracted documents based on the document determination condition generated by the first recall calculation unit 131. When the determination is made (step S304: NO), the NOT additional keyword candidate re-extracting unit 134 includes the improvement target keyword and is added by the first recall ratio calculating unit 131 with a plurality of AND additional keywords (a plurality of AND additional keywords). In this case, document data including any one of a plurality of AND additional keywords) is extracted from the document data storage unit 110.

  Similar to step S306, the second recall ratio calculation unit 135 adds a negative operator to the NOT additional keyword extracted as the first candidate among the NOT additional keywords extracted by the additional keyword extraction unit 124, and negates it. A document determination condition in which a NOT addition keyword to which an operator is added and an improvement target keyword are combined with a logical product operator is generated, and document data that matches the generated document determination condition is extracted from the document data storage unit 110, and the reproduction rate Is calculated. In addition, the condition generation unit 137 stores the number of extracted documents using the document determination condition at this point in its storage area as the minimum number of extracted documents. The second reproduction rate comparison unit 136 compares the reproduction rate calculated by the second reproduction rate calculation unit 135 with the allowable reproduction rate stored in its own storage area, and the reproduction rate exceeds the allowable reproduction rate. It is determined whether or not there is (step S311).

  In step S311, when the second recall ratio comparison unit 136 determines that the recall rate exceeds the allowable recall rate (step S311: YES), the second recall ratio calculation unit 135 is extracted by the additional keyword extraction unit 124. A document determination condition is generated by adding the next candidate NOT addition keyword among the NOT addition keywords that have been added (step S312). The second reproduction rate calculation unit 135 returns to step S311, extracts document data that matches the generated document determination condition from the document data storage unit 110, and calculates the reproduction rate. Then, the document determination condition generation device 100 repeats the processes of step S311 and step S312 until the second reproduction ratio comparison unit 136 determines that the reproduction ratio does not exceed the allowable reproduction ratio. If the second reproduction rate comparison unit 136 determines in step S311 that the reproduction rate does not exceed the allowable reproduction rate (step S311: NO), the extracted document number comparison unit 133 stores the lowest value stored in the condition generation unit 137. The number of extracted documents is compared with the number of documents extracted from the document data storage unit 110 using the document determination condition generated by the second reproduction rate calculation unit 135 in step S311 (step S313).

  If the extracted document number comparison unit 133 determines that the minimum number of extracted documents stored in the condition generation unit 137 is equal to or greater than the number of extracted documents based on the document determination condition generated by the second recall ratio calculation unit 135 in step S311 ( In step S313: YES), the condition generation unit 137 substitutes the number of extracted documents based on the document determination condition generated by the second reproduction rate calculation unit 135 in step S311 into the variable of the optimum number of extracted documents. Further, the number of AND additional keywords added by the first recall ratio calculating unit 131 is substituted for the optimal number of AND keywords, and the NOT additional keyword added by the second recall ratio calculating unit 135 is substituted for the optimal number of NOT keywords. And the optimal solution for the document determination condition is overwritten (step S314).

  When the optimum solution overwriting process of the document determination condition in step S314 is performed, or when the extracted document number comparison unit 133 determines in step S313 that the minimum extracted document number is not equal to or greater than the extracted document number based on the document determination condition (step S313: NO), the condition generation unit 137 determines whether the recall based on the document determination condition is 100% or whether all of the AND additional keyword candidates extracted by the additional keyword extraction unit 124 are set as the document determination condition. Determination is made (step S315).

  When the condition generation unit 137 determines that the recall based on the document determination condition is not 100% and all of the AND additional keyword candidates extracted by the additional keyword extraction unit 124 are not set as the document determination condition (step S315). : NO), the next AND addition keyword candidate is added to the document determination condition, and the process returns to step S310. On the other hand, in step S315, the condition generation unit 137 has a recall rate of 100% based on the document determination condition or sets all AND additional keyword candidates extracted by the additional keyword extraction unit 124 as the document determination condition. If it determines (step S315: YES), it will determine with the optimal AND keyword number memorize | stored in its memory | storage part, and the optimal NOT keyword number being an optimal solution (step S309). The condition generation unit 137 generates a document determination condition based on the optimum number of AND keywords and the optimum number of NOT keywords (step S309).

  FIG. 6 is a diagram illustrating the relationship between the number of extracted documents and the recall rate in the document determination condition generation process described with reference to FIG. In this figure, the X axis indicates the number of extracted documents, and the Y axis indicates the recall rate. Here, let T0 be the number of extracted documents when only the improvement target keyword is used as the document determination condition. The number of extracted documents indicated by T0 is considered to be a number including classification noise. Reference symbol a1 corresponds to the recall rate and the number of extracted documents when one AND added keyword is added to the improvement target keyword. Thus, when AND addition keywords are combined and added by a logical sum operator, the recall and the number of extracted documents increase. The first recall ratio calculation unit 131 adds AND addition keywords until the recall ratio exceeds the allowable recall ratio (symbol a2 to a8). If eight AND added keywords are added and the recall rate exceeds the allowable recall rate, the NOT added keyword is added. Reference numeral d1 is a point corresponding to the reproduction rate and the number of extracted documents when one additional NOT keyword is added to the document determination condition in which eight AND additional keywords are added. The second recall ratio calculation unit 135 adds the NOT additional keyword in this way, and reduces the number of extracted documents extracted according to the document determination condition.

  The second reproduction rate calculation unit 135 continues to add the NOT addition keyword to the document determination condition, and determines the maximum number of NOT addition keywords that the reproduction rate does not fall below the allowable reproduction rate. In this example, from the point of reference a8, when six NOT added keywords are added (d6), the recall rate is less than the allowable recall rate, so the number of extracted documents (T1) when five NOT added keywords are added. ) Is substituted for the minimum number of extracted documents, 5 is substituted for the optimal number of NOT keywords, and 8 is substituted for the optimal number of AND keywords. Here, the recall rate is not 100%, and all of the AND keyword candidates are not added to the document determination condition (the process proceeds to NO in step S315 in the above flow), so the ninth AND keyword is added. Similarly, the NOT keyword is added to the ninth AND addition keyword. In this way, the document determination condition generation apparatus 100 calculates the reproduction rate and the number of extracted documents based on a plurality of document determination conditions, satisfies the allowable reproduction rate, and satisfies the document determination condition when the number of extracted documents is the smallest. Are output as optimum document determination conditions. That is, the document data group satisfying the allowable reproduction rate may include an incorrect answer document (noise), and therefore, the document having the smallest number of extracted documents among the document data group satisfying the allowable reproduction rate. The data group is considered to be the document data group with the least noise. For this reason, the category determination condition generation device 100 outputs the document determination condition when the number of extracted documents is the smallest as the optimum document determination condition.

FIG. 7 is a diagram illustrating an example of a conventional document determination condition and a document determination condition generated by the document determination condition generation apparatus 100 according to the present embodiment. For example, conventionally, keywords such as “finger cut”, “knee”, “fracture”, and “speech disorder” are set independently as document determination conditions for extracting document data belonging to the “disorder” category. It is assumed that the matching rate of the keywords “knee” and “fracture” is low, and there is a lot of classification noise when these are used. Here, the optimum AND addition keyword and NOT addition keyword are determined for these and added to the document determination condition. For example, for the keyword “knee”, an AND addition keyword “cut” and an AND addition keyword “necrosis” combined by a logical sum operator (Or), and a NOT addition keyword “both legs” to which a negation operator is added. Are combined by the conjunction operator (&). In addition, an AND addition keyword “femur” is added to the keyword “fracture”, a NOT operator is added, and NOT addition keywords “not recognized” and “mild” are combined by a logical product operator (&). Are combined by AND operator (&).
As described above, according to the present embodiment, it is possible to minimize document classification noise and generate an optimal document determination condition while maintaining a reproduction rate exceeding an allowable reproduction rate.

  Note that a program for realizing the function of the processing unit in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute the document determination condition. Generation may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 Document determination condition production | generation apparatus 110 Document data memory | storage part 111 Keyword list memory | storage part 112 Improvement object keyword extraction part 121 Document data extraction part 122 Morphological analysis part 123 Processing object word extraction part 124 Additional keyword extraction part 131 1st reproduction rate calculation part 132 First recall ratio comparison section 133 Extracted document number comparison section 134 NOT keyword candidate re-extraction section 135 Second recall ratio calculation section 136 Second recall ratio comparison section 137 Condition generation section

Claims (7)

A document determination condition generation device that generates a document determination condition for determining a category indicating a classification according to the content of document data according to a word included in the defined document data,
A document data storage unit that stores a plurality of document data in which any one of the plurality of categories is associated in advance;
Among the plurality of categories, a plurality of keywords corresponding to a target category that is a target for generating the document determination condition and a matching rate indicating a degree of matching of the keyword with the target category are stored in association with each other. A keyword list storage unit,
The relevance ratio associated with the keyword included in the keyword list is compared with a predetermined threshold, and the improvement target keyword that is the keyword associated with the relevance ratio that is less than the threshold is An improvement target keyword extraction unit to be extracted from the keyword list storage unit;
A document data extraction unit for extracting correct document data, which is the document data including the improvement target keyword extracted by the improvement target keyword extraction unit and associated with the target category, from the document data storage unit;
An additional keyword extraction unit that extracts, from words included in the correct document data extracted by the document data extraction unit, a first additional keyword that is a word having a relatively high precision with respect to the target category;
A condition generation unit that generates the document determination condition indicating that the improvement target keyword extracted by the improvement target keyword extraction unit is included and that the first additional keyword extracted by the additional keyword extraction unit is included When,
A document determination condition generation apparatus comprising: A morpheme analysis unit that morphologically analyzes a sentence included in the correct document data extracted by the document data extraction unit, and outputs a morpheme included in the correct document data and a part of speech corresponding to the morpheme;
A processing target word extraction unit that extracts a processing target word that is a word including the morpheme whose part of speech is a noun based on the result of the morphological analysis by the morpheme analysis unit;
The document determination condition generation apparatus according to claim 1, wherein the additional keyword extraction unit extracts the first additional keyword from the processing target word extracted by the processing target word extraction unit. The document data extraction unit includes the improvement target keyword extracted by the improvement target keyword extraction unit and the first additional keyword extracted by the additional keyword extraction unit, and the category other than the target category includes Extracting incorrect document data, which is the associated document data,
The additional keyword extraction unit is a second addition that is a word having a relatively high relevance rate with respect to the category other than the target category from words included in the incorrect answer document data extracted by the document data extraction unit. Extract keywords,
The condition determination unit includes the document determination that indicates that the improvement target keyword extracted by the improvement target keyword extraction unit is included and that the second additional keyword extracted by the additional keyword extraction unit is not included The document determination condition generation apparatus according to claim 1, wherein the condition generation is performed. Reproduction that is a ratio between the number of correct document data extracted by the document data extraction unit and the number of document data extracted from the document data stored in the document data storage unit using the document determination condition A recall rate calculation unit for calculating a rate;
A reproduction rate comparison unit that compares the reproduction rate calculated by the reproduction rate calculation unit with a predetermined reproduction rate threshold value and determines whether the reproduction rate is less than the threshold value; ,
The additional keyword extraction unit extracts a plurality of the first additional keywords having a relatively high relevance ratio for the target category from words included in the correct document data extracted by the document data extraction unit,
The condition generation unit includes the improvement target keyword extracted by the improvement target keyword extraction unit when the reproduction rate determination unit determines that the reproduction rate is less than the threshold, and the additional keyword extraction The document determination condition indicating that any one of the plurality of first additional keywords extracted by the section is included is generated. 4. The method according to claim 1, wherein the document determination condition is generated. The document determination condition generation device described in 1. The additional keyword extraction unit includes a plurality of the second additional keywords having a relatively high relevance ratio for a category other than the target category from words included in the incorrect answer document data extracted by the document data extraction unit. Extract
The condition generation unit includes the improvement target keyword extracted by the improvement target keyword extraction unit and the additional keyword extraction when the reproduction rate comparison unit determines that the reproduction rate is equal to or greater than the threshold value. 5. The document determination condition generation device according to claim 4, wherein the document determination condition is generated to indicate that none of the plurality of second additional keywords extracted by the section is included. The matching rate of the keyword with respect to the category is the number of the document data associated with the category and the document data of all categories including the keyword among the document data stored in the document data storage unit. The document determination condition generation device according to claim 1, wherein the document determination condition generation device is expressed by a ratio to the number of A document data storage unit storing a plurality of document data in which any one of the plurality of categories indicating a classification according to the content of the defined document data is associated in advance, and among the plurality of the categories, A plurality of keywords corresponding to a target category that is a target for generating a document determination condition for determining a category of the document data according to a word included in the document data, and a degree to which the keyword matches the target category. A document determination condition generation method of a document determination condition generation device including a keyword list storage unit that stores therein a matching rate to be indicated,
The matching rate associated with the keyword included in the keyword list is compared with a predetermined threshold, and the keyword corresponding to the matching rate that is less than the threshold is extracted from the keyword list storage unit. Steps,
Extracting correct document data, which is the document data including the extracted keyword and associated with the target category, from the document data storage unit;
Extracting a first additional keyword that is a word having a relatively high accuracy rate for the target category from words included in the extracted correct document data;
Generating the document determination condition indicating that the extracted keyword is included and the extracted first additional keyword is included;
A document determination condition generation method comprising:

Images