JP5402377B2 - Keyword selection apparatus, method, program, and system - Google Patents

Description

  The present invention relates to a keyword selection device, method, program, and system, and in particular, a keyword selection device for selecting a keyword that is a word string representing a document from a plurality of word strings included in each of a plurality of documents. The present invention relates to a method, a program, and a system.

  A technology for extracting a keyword, which is a word string representing the content of a document, from each document of a given document set is an element technology such as an information search system or a hyperlink automatic generation system that automatically generates hyperlinks between documents. As important.

  Non-Patent Document 1 discloses a tf / idf method that is often used for information retrieval indexing. According to this method, the top M words with high tf / idf values are selected as keywords from each document (M is a constant). Now, assume that the tf / idf value of the word t included in the document d of the document set D is obtained. Here, tf is an abbreviation of term frequency and represents the number of appearances of the word t in the document d or the total number of appearances of the word t in the document set D. Idf is an abbreviation for inverse document frequency and is the reciprocal of the number of appearing documents of the word t in the document set D. The product of tf and idf is the td / idf value.

  Related techniques are disclosed in Patent Documents 1 to 5. Patent Document 1 discloses a technique for calculating the importance of a word extracted from a document based on the appearance frequency in the document and the appearance frequency (including the number of documents) in the document set. Yes. Patent Document 2 discloses a technique related to a document search apparatus that appropriately determines unnecessary words in a search word. In particular, the document search apparatus according to Patent Document 2 determines the number of unnecessary words by comparing the number of occurrence documents of words with a threshold value. Patent Document 3 discloses a technique for extracting a keyword from access history information to a document and lowering the priority at the time of keyword extraction for a keyword that is frequently used. Patent Document 4 discloses a technique for determining whether a candidate word is appropriate as a keyword based on the importance of the candidate word. Patent Document 5 discloses a technique for determining a word candidate as a keyword candidate when the number of appearing documents of the word is equal to or greater than a minimum value.

JP 2000-076254 A Japanese Patent Laid-Open No. 2005-050239 JP 2006-120126 A Japanese Unexamined Patent Publication No. 07-319882 JP-A-11-224255

Takenobu Tokunaga, Language and Computation-5 Information Retrieval and Language Processing, The University of Tokyo Press, pp.27-32.

  However, Non-Patent Document 1 and Patent Documents 1 to 5 have a problem that the accuracy of a selected keyword cannot be obtained sufficiently. In particular, the problem of keyword extraction using tf / idf according to Non-Patent Document 1 is that there are only two types of indices for determining the importance of a word string, tf and idf, so that sufficient performance cannot be obtained. That is. There is a certain correlation between these indices and the importance of word strings. However, in reality, there are important word strings that should be used as keywords other than word strings in which the number of occurrences of word strings in the document is large and the number of word string occurrences is small. That is, even in a document in which a word string appears, the importance of the word string in the document is not always high. Thus, there is a possibility that the importance of such a word string becomes relatively low only by tf / idf. Also, in Patent Documents 1 to 5, the same problem may occur because only the index corresponding to idf is used. Therefore, it is necessary to introduce new constraints for more accurate keyword extraction.

  The present invention has been made to solve such a problem, and is a keyword selection capable of selecting a keyword of each document with high accuracy from a plurality of word strings included in each of a plurality of documents. An object is to provide an apparatus, a method, a program, and a system.

  The keyword selection device according to the first aspect of the present invention uses, for each of a plurality of documents, a word string having a high priority in the document among at least one word string extracted from each document as a temporary keyword. Temporary keyword selection means to select, selected document number calculation means to calculate the number of selected documents, which is the total number of documents in which the temporary keyword is selected for each word string among the plurality of documents, and the number of selected documents is a document When a predetermined threshold according to the number is exceeded, the priority of at least some of the word strings in each document in which the temporary keyword is selected is set so that the number of selected documents is equal to or less than the predetermined threshold. And a keyword selection unit that selects a keyword for each of the plurality of documents based on the corrected priority.

  In the keyword selection method according to the second aspect of the present invention, for each of a plurality of documents, a word string having a high priority in the document among the word strings extracted from each document is used as a temporary keyword. A temporary keyword selection step for selecting, a selected document number calculation step for calculating a selected document number that is the total number of documents in which the temporary keyword is selected for each word string among the plurality of documents, and the selected document number is a document When a predetermined threshold according to the number is exceeded, the priority of at least some of the word strings in each document in which the temporary keyword is selected is set so that the number of selected documents is equal to or less than the predetermined threshold. And a priority selection step for selecting a keyword based on the corrected priority for each of the plurality of documents.

  The keyword selection program according to the third aspect of the present invention uses, for each of a plurality of documents, a word string having a high priority in the document among at least one word string extracted from each document as a temporary keyword. Temporary keyword selection processing to be selected, selected document number calculation processing for calculating the number of selected documents, which is the total number of documents in which the temporary keyword is selected for each word string among the plurality of documents, and the number of selected documents is a document When a predetermined threshold according to the number is exceeded, the priority of at least some of the word strings in each document in which the temporary keyword is selected is set so that the number of selected documents is equal to or less than the predetermined threshold. And a priority selection process for correcting each of the plurality of documents and a keyword selection process for selecting a keyword based on the corrected priority for each of the plurality of documents. To be executed by the Yuta.

  A keyword selection system according to a fourth aspect of the present invention includes a priority storage unit that stores a priority level of a word string extracted from a document for each of a plurality of documents, and the priority storage unit. For each of the plurality of documents, a temporary keyword selection unit that selects a word string having a high priority in the document as a temporary keyword, and a document that includes the temporary keyword selected by the temporary keyword selection unit. A keyword storage unit that stores the selected document number, a selected document number calculation unit that calculates the total number of documents in which the temporary keyword for each word string is selected among the plurality of documents, and the calculated selection. A selected document number storage unit that stores the number of documents, and a temporary keyword stored in the keyword storage unit with reference to the selected document number storage unit A keyword determination unit that determines whether or not the number of selected documents exceeds a predetermined threshold according to the number of documents, and each of the provisional keywords selected when the keyword determination unit determines that the number exceeds a predetermined threshold Priority correction means for correcting the priority stored in the priority storage means so as to lower the priority of the word string selected as the temporary keyword in the document, the temporary keyword selection means, The temporary keyword is selected again based on the priority updated by the priority correction unit, and the keyword determination unit determines that the number of selected documents of the temporary keyword stored in the keyword storage unit is equal to the predetermined threshold value. When it is determined that it is the following, the temporary keyword stored in the keyword storage means is determined as a keyword.

  The keyword selection method according to the fifth aspect of the present invention includes, for each of a plurality of documents, a priority storage step of storing, in a priority storage unit, a priority in the document of a word string extracted from the document; Referring to the priority storage means, for each of a plurality of documents, a temporary keyword selection step for selecting a word string having a high priority in the document as a temporary keyword, and the temporary keyword selected by the temporary keyword selection step A step of storing in a keyword storage means in association with a document including a temporary keyword, and calculating a number of selected documents, which is a total number of documents in which the temporary keyword is selected for each word string among the plurality of documents. A step of storing the calculated number of selected documents in the selected document number storage unit; and referring to the selected document number storage unit; A keyword determination step for determining whether or not the number of selected documents of the temporary keyword stored in the keyword storage means exceeds a predetermined threshold according to the number of documents, and it is determined by the keyword determination step that the predetermined threshold is exceeded. A priority correction step for correcting the priority stored in the priority storage means so as to lower the priority of the word string selected as the temporary keyword in each document in which the temporary keyword is selected; The temporary keyword selection step selects the temporary keyword again based on the priority updated by the priority correction step, and the keyword determination step stores the temporary keyword stored in the keyword storage means. When it is determined that the number of selected documents of a keyword is equal to or less than the predetermined threshold, the keyword storage unit Determining the stored provisional keyword as a keyword.

  The keyword selection program according to the sixth aspect of the present invention includes, for each of a plurality of documents, a priority storage process for storing, in a priority storage unit, the priority in the document of a word string extracted from the document, Referring to the priority storage unit, for each of a plurality of documents, the temporary keyword selection process for selecting a word string having a high priority in the document as a temporary keyword, and the temporary keyword selected by the temporary keyword selection process, A process of storing in a keyword storage unit in association with a document including a temporary keyword, and calculating a number of selected documents that calculates the number of selected documents that is the total number of documents in which the temporary keyword is selected for each word string among the plurality of documents. Processing, storing the calculated number of selected documents in the selected document number storage unit, referring to the selected document number storage unit, and storing the selected document number in the keyword storage unit When it is determined that the number of selected documents of the temporary keyword exceeds a predetermined threshold corresponding to the number of documents, and the keyword determination process determines that the temporary keyword exceeds the predetermined threshold, A priority correction process for correcting the priority stored in the priority storage unit so as to lower the priority of the word string selected as the temporary keyword in each selected document, and the temporary keyword The selection process selects the temporary keyword again based on the priority updated by the priority correction process, and the keyword determination process determines the number of selected documents of the temporary keyword stored in the keyword storage unit. The temporary keyword stored in the keyword storage unit is determined as a keyword when it is determined that the threshold is equal to or less than the predetermined threshold. To execute processing for the computer.

  According to the present invention, it is possible to provide a keyword selection device, method, program, and system that can select a keyword of each document from a plurality of word strings included in each of the plurality of documents with high accuracy.

It is a block diagram which shows the structure of the keyword selection apparatus concerning Embodiment 1 of this invention. It is a flowchart figure which shows the flow of the keyword selection process concerning Embodiment 1 of this invention. It is a figure which shows the example of the word string and temporary keyword before the priority correction process concerning Embodiment 1 of this invention. It is a figure which shows the example of the word string and keyword after the priority correction process concerning Embodiment 1 of this invention. It is a figure which shows the example of the word string and the number of selection documents concerning Embodiment 1 of this invention. It is a block diagram which shows the structure of the keyword selection system concerning Embodiment 2 of this invention. It is a flowchart figure which shows the flow of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 1 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 2 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 3 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 4 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 5 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 6 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 7 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and temporary keyword in the cycle 8 of the keyword selection process concerning Embodiment 2 of this invention. It is a figure which shows the example of the word string and the number of selection documents concerning Embodiment 2 of this invention. It is a block diagram which shows the structure of the keyword selection system concerning Embodiment 3 of this invention. It is a flowchart figure which shows the flow of the keyword selection process concerning Embodiment 3 of this invention.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description will be omitted as necessary for the sake of clarity.

<Embodiment 1 of the Invention>
FIG. 1 is a block diagram showing a configuration of a keyword selection device 1 according to the first exemplary embodiment of the present invention. The keyword selection device 1 includes a temporary keyword selection unit 11, a selected document number calculation unit 12, a priority correction unit 13, and a keyword selection unit 14.

  The temporary keyword selection unit 11 selects, as a temporary keyword, a word string having a high priority in the document among at least one word string extracted from each document for each of a plurality of documents. That is, a temporary keyword is selected for each of a plurality of documents. The document only needs to include at least one word string. In addition, it is assumed that the word string is extracted from each document in advance. Further, it is assumed that priorities are set in advance for the word strings. Here, the priority may be, for example, a priority order in the document determined by a magnitude relationship such as tf / idf. The priority is not limited to tf / idf and priority. For example, the priority may be expressed by a numerical value, and the order can be determined by the magnitude of the priority.

  The temporary keyword selection unit 11 may select a plurality of temporary keywords per document. For example, the temporary keyword selection means 11 may select the top M words from the word strings with high priority among the plurality of word strings extracted from the document as the temporary keywords. Alternatively, the temporary keyword selection unit 11 may select the upper M% of the total number of word strings in the document among the plurality of word strings as the temporary keyword. That is, the temporary keyword selection unit 11 may change the number of temporary keywords to be selected for each document.

  The selected document number calculation means 12 calculates the number of selected documents, which is the total number of documents in which a temporary keyword for each word string is selected from among a plurality of documents. In other words, even if the document includes the same word string as the temporary keyword, if the word string is not selected as the temporary keyword in the document, the document is not included in the number of selected documents of the temporary keyword. .

  When the number of selected documents exceeds a predetermined threshold (N) corresponding to the number of documents, the priority correction unit 13 prioritizes at least some of the word strings in each word string in which the temporary keyword is selected. Is corrected so that the number of selected documents is equal to or less than a predetermined threshold (N). Here, the predetermined threshold value (N) according to the number of documents is, for example, an absolute numerical value with respect to the total number of documents, or a relative value, that is, a constant ratio. Note that M and N described above are set in advance by the user and stored in a storage device (not shown) in the keyword selection device 1.

  Further, correcting the priority so that the number of selected documents is N or less indicates the following. First, by correcting the priority of at least a part of a plurality of word strings in a document, the word string that can be selected as a temporary keyword in the document when the temporary keyword selection unit 11 is performed. Is changed. When the word string that can be selected as the temporary keyword is changed, the number of selected documents when the selected document number calculation unit 12 is performed is changed. In other words, correcting the priority so that the number of selected documents is N or less can be said to correct the priority so as to change the word string selected as the temporary keyword.

  For example, it is desirable that the priority correction unit 13 corrects the priority of the word string selected as the temporary keyword to be low. That is, it is desirable that the priority correction unit 13 corrects the priority of the word string selected as the temporary keyword so that the number of selected documents is N or less. As a result, the word string selected as the temporary keyword has a relatively low priority in the document, so that it can be prevented from being selected as the temporary keyword. In this case, since the number of selected documents is reduced, the number of selected documents is N or less.

  Alternatively, the priority correction unit 13 may correct the priority of the word string having a lower priority than the word string selected as the temporary keyword among the word strings other than the temporary keyword with the number of selected documents exceeding N. desirable. As a result, the word string having the low priority has a relatively high priority in the document, and therefore, the word string selected as the temporary keyword can be prevented from being selected as the temporary keyword. In this case, since the number of selected documents is reduced, the number of selected documents is N or less.

  Further, the priority correction unit 13 selects the priority of the word string having a priority one lower than the word string selected as the temporary keyword among the word strings other than the temporary keyword with the number of selected documents exceeding N, and the temporary document It is desirable to correct by exchanging the priority of the word string selected as the keyword. That is, the priority of the word string selected as the temporary keyword is lowered by one, and the priority of the word string having a low priority is raised by one. As a result, since the word string selected as the temporary keyword has a relatively low priority in the document, the word string selected as the temporary keyword can be prevented from being selected as the temporary keyword. In this case, since the number of selected documents is reduced, the number of selected documents is N or less.

  Further, the priority correction unit 13 corrects the priority of the word string selected as the temporary keyword to be low, and if the word string is not selected as the temporary keyword in the document, the priority correction unit 13 is extracted from the document. It is desirable to delete a word string selected as the temporary keyword from a plurality of word strings. As a result, once the word string deviating from the temporary keyword is prevented from being selected again as a temporary keyword by lowering the priority of the other word strings, the accuracy of keyword selection can be improved. it can.

  The keyword selection means 14 selects a keyword based on the corrected priority for each of a plurality of documents. For example, after the priority is corrected by the priority correction unit 13, the keyword selection unit 14 performs priority after correction in the document among a plurality of word strings extracted from each document for each of the plurality of documents. A word string having a high degree is selected as a keyword. That is, due to the priority correction by the priority correction means 13, the number of selected temporary keyword documents in a plurality of documents falls below N, and all the temporary keywords have a high priority only in some documents. Therefore, it can be said that a word string having a high priority in each document is a keyword that is a word string representing the document.

  FIG. 2 is a flowchart showing a keyword selection process according to the first embodiment of the present invention. First, the temporary keyword selection unit 11 selects, for each of a plurality of documents, a word string having a high priority in the document among a plurality of word strings extracted from each document as a temporary keyword (S11). Next, the selected document number calculation means 12 calculates the number of selected documents, which is the total number of documents in which the temporary keyword for each word string is selected from among a plurality of documents (S12). Then, when the number of selected documents exceeds a predetermined threshold according to the number of documents, the priority correction unit 13 sets the priority of at least a part of the word strings in the word strings in each document for which the temporary keyword is selected. The number of selected documents is corrected so as to be equal to or less than a predetermined threshold (S13). Thereafter, the keyword selecting unit 14 selects a keyword based on the corrected priority for each of the plurality of documents (S14).

  Here, an example of keyword selection processing according to the first exemplary embodiment of the present invention will be described. FIG. 3 is a diagram illustrating an example of a word string and a temporary keyword before the priority correction process according to the first embodiment of the present invention. Here, an example where M = 5 and N = 3 will be described. First, it is assumed that the plurality of documents d1 to d4 include word strings A, B,. Further, it is assumed that priorities are set in advance for each document in the word strings A, B,. In FIG. 3, the documents d1 to d4 are arranged such that the word strings with high priority are on the left, and the documents with low priority are on the right.

  At this time, since the temporary keyword selection means 11 is M = 5, the word strings A, B, C, D and E having the highest priority among the word strings of the document d1 are selected as temporary keywords. . Similarly, the temporary keyword selection unit 11 selects temporary keywords for the documents d2 to d4. As a result, the temporary keyword group VK is selected in step S11.

  Next, the selected document number calculation means 12 calculates the number of selected documents for each word string. FIG. 5 is a diagram showing an example of the word string and the number of selected documents according to the first embodiment of the present invention. Here, for comparison, a case is shown in which the number of selected documents for all word strings A to J is calculated. For example, since there are two documents d1 and d4 in which the word string A is selected as a temporary keyword, the number of selected documents in the word string A is “2”. Here, the number of selected documents of the word string E is “4”, which is indicated by shading in FIG. That is, the word string E is selected as a temporary keyword for all of the documents d1 to d4. For this reason, the word string E cannot be said to be a word string representing the differences between the documents d1 to d4, and thus can be said to be a low-accuracy keyword.

  Here, since the number of selected documents “4” of the word string E exceeds 3 among the number of selected documents shown in FIG. 5, the priority correction unit 13 determines the word strings E of the documents d1 to d4 selected as temporary keywords. The priority is corrected by exchanging the priority and the word string of the priority one lower than the word string E. For example, the priority correction unit 13 swaps the priorities of the word string E and the word string F of the document d1. Similarly, the priority correction unit 13 corrects the priority for the documents d2 to d4. At this time, the priority correction means 13 may reduce only the priority of the word string E. Alternatively, the priority correction unit 13 may increase the priority of a word string having a lower priority than the word string E. At this time, the word string E may be deleted from the set of word strings in the document d1. Alternatively, the word string E may be set to the lowest priority in the document d1.

  FIG. 4 is a diagram illustrating an example of a word string and keywords after priority correction processing by the priority correction unit 13 according to the first embodiment of the present invention. In FIG. 4, when the number of selected documents is calculated for a word string to be selected as a temporary keyword, the number of selected documents after correction in FIG. 5 is obtained. That is, the number of selected documents after the priority correction of the word string E is “3”. In other words, the priority correction unit 13 has corrected the number of selected documents in the word string E to be 3 or less.

  Here, the keyword selection means 14 selects the keyword group RK based on the corrected priority shown in FIG. In any of the keyword groups RK, the number of selected documents is 3 or less. In particular, the word string E is not selected from the document d1 compared to before the priority correction. For this reason, it can be said that the word string E is also highly accurate as a keyword in each document, like the other selected word strings.

  As described above, according to the first embodiment of the present invention, it is possible to select a keyword of each document from a plurality of word strings included in each of a plurality of documents with high accuracy.

<Embodiment 2 of the Invention>
FIG. 6 is a block diagram showing a configuration of the keyword selection system 2 according to the second exemplary embodiment of the present invention. The keyword selection system 2 may be one in which the keyword selection device 1 according to the first embodiment of the present invention described above is applied to a computer system. The keyword selection system 2 includes a data processing device 20 that operates under program control, and a storage device 30 that stores information. The data processing device 20 includes a temporary keyword selection unit 21, a selected document number calculation unit 22, a keyword determination unit 23, and a priority correction unit 24.

  The temporary keyword selection means 21 selects a word string having a high priority (M or higher) in the document as a temporary keyword for each of a plurality of documents. The selected document number calculation means 22 calculates the number of selected documents, which is the total number of documents in which a temporary keyword for each word string is selected from among a plurality of documents. The keyword determination means 23 determines whether or not the number of selected documents for the temporary keyword exceeds a predetermined threshold (N) corresponding to the number of documents. The keyword determination unit 23 determines the temporary keyword as a keyword when determining that the number of selected documents for the temporary keyword is N or less.

  The priority correction unit 24 gives priority to lower the priority of the word string selected as the temporary keyword in each document in which the temporary keyword is selected when the keyword determination unit 23 determines that N is exceeded. Correct the degree. Thereafter, the temporary keyword selection unit 21 selects the temporary keyword again based on the priority updated by the priority correction unit 24.

  The storage device 30 includes a priority storage unit 31, a selected document storage unit 32, and a keyword storage unit 33. The storage device 30 is realized by a main storage device such as a memory or an auxiliary storage device such as a hard disk.

  The priority storage unit 31 stores, for each of a plurality of documents, the priority within the document of the word string included in the document. It is assumed that the word string has already been extracted from the document. The selected document storage unit 32 stores the number of selected documents calculated by the selected document number calculating unit 22. The keyword storage unit 33 stores the temporary keyword selected by the temporary keyword selection unit 21 in association with the document including the temporary keyword.

  In the second embodiment of the present invention, a case where a systematic document set is targeted as an example of a plurality of documents will be described. Here, the systematic document set means a document set created based on the overall management of the editor or creator. For example, one electronic manual or one Web site corresponds to the systematic document set. In this case, one section of the electronic manual or one page of the website corresponds to one document.

  The plurality of documents is not limited to the systematic document set, and may be an arbitrary document set. However, the structured document set has a feature that there is a high possibility that the same topic is described so as not to be distributed. Therefore, in the second embodiment of the present invention, when a plurality of documents is a systematic document set, a keyword that is a word string representing the contents of the document is selected by paying attention to the above-described features. Thereby, the topic keyword can be selected from the systematized document with high accuracy by adding the number of documents selected as the topic keyword as a restriction.

  The value of N is N = 1 when an ideal structured document set is targeted. This is because in an ideal structured document set, one topic should be described in one document (section) without duplication. However, in reality, there are few completely systematized document sets, and it is considered that there may be a certain degree of topic overlap, so it is preferable to set a value slightly larger than 1 (for example, N = 5). Further, considering that there is a correlation between the number of documents registered in the document set and the overlap of topics, N may be a ratio (for example, 1%) to the total number of word strings included in the document instead of the number.

  The keyword selection policy in the keyword selection system 2 is such that the number of documents in which the same word string is selected in the entire structured document set is N while maintaining the priority stored in the priority storage unit 31 as much as possible. Minimal changes to be made:

  FIG. 7 is a flowchart showing a keyword selection process according to the second embodiment of the present invention. First, the temporary keyword selection means 21 selects a temporary keyword (S21). Specifically, the temporary keyword selection unit 21 refers to the priority storage unit 31 and selects a word string having a high priority in the document as a temporary keyword for each of a plurality of documents. Then, the temporary keyword selection unit 21 stores the selected temporary keyword in the keyword storage unit 33 in association with the document including the temporary keyword.

  Next, the selected document number calculation means 22 calculates the number of selected documents (S22). Specifically, the selected document number calculation unit 22 calculates the number of selected documents, which is the total number of documents in which a temporary keyword for each word string is selected from among a plurality of documents. Then, the selected document number calculating unit 22 stores the calculated selected document number in the selected document storage unit 32.

  Then, the keyword determination unit 23 determines whether or not the number of selected documents exceeds the threshold value N (S23). Specifically, the keyword determining unit 23 refers to the selected document storage unit 32 and determines whether or not the number of selected documents for each temporary keyword stored in the keyword storage unit 33 exceeds N. If it is determined in step S23 that the number of selected documents exceeds the threshold value N, the priority correction unit 24 corrects the priority (S24). Specifically, the priority correction unit 24 stores the priority stored in the priority storage unit 31 so as to lower the priority of the word string selected as the temporary keyword in each document in which the temporary keyword is selected. Correct. Then, it returns to step S21. That is, the temporary keyword selection unit 21 selects a temporary keyword again based on the priority updated in step S24. Thereafter, steps S21 to S24 are repeated until the number of all selected documents becomes N or less in step S23.

  If it is determined in step S23 that the number of selected documents is equal to or less than the threshold value N, the keyword determination unit 23 determines the selected temporary keyword as a keyword (S25). Specifically, when the keyword determination unit 23 determines that the number of selected documents of the temporary keyword stored in the selected document storage unit 32 is N or less, the temporary keyword stored in the keyword storage unit 33 is used as a keyword. decide.

  In the example of FIGS. 3, 4 and 5 according to the first embodiment of the present invention described above, since N = 3, the number of selected documents for all temporary keywords is 3 or less in one priority correction process. However, when N = 2, the effect becomes more significant by applying the keyword selection system 2. Therefore, an application example in the case of N = 2 will be described below.

  FIG. 8 is a diagram showing an example of word strings and temporary keywords in the cycle 1 of the keyword selection process according to the second exemplary embodiment of the present invention. Here, it is assumed that M = 5 and N = 2 are set. In FIG. 8, as in FIG. 3, the plurality of documents d1 to d4 include word strings A, B,. Further, it is assumed that priorities are set in advance for each document in the word strings A, B,. In FIG. 8, the documents d1 to d4 are arranged such that the word strings with the higher priority are on the left and the documents with the lower priority are on the right. FIG. 8 shows the initial state, that is, cycle 1.

  At this time, since the temporary keyword selection means 21 is M = 5, the word strings A, B, C, D and E having the highest priority among the word strings of the document d1 are selected as temporary keywords. . Similarly, the temporary keyword selection unit 21 selects temporary keywords for the documents d2 to d4. As a result, the temporary keyword group V1 is selected in step S21.

  Next, the selected document number calculation means 22 calculates the number of selected documents for each word string. FIG. 16 is a diagram showing an example of word strings and the number of selected documents according to the second embodiment of the present invention. Here, for comparison, a case is shown in which the number of selected documents for all word strings A to J is calculated. For example, since there are two documents d1 and d4 in which the word string A is selected as a temporary keyword, the number of selected documents in the word string A is “2”.

  Here, the keyword determination unit 23 determines whether there are more than two of the selected documents shown in FIG. Here, four word strings B, C, D, and E correspond to each other and are indicated by shading in FIG. Therefore, the process proceeds to step S24, and the priority correction unit 24 swaps the priorities of the word string in which the number of selected documents exceeds N and the word string F one level lower. In addition, when word strings having N or more selected documents continue, the word strings with lower priority are replaced. For example, in the document d1, a word string having two or more selected documents continues to B, C, D, and E. In this case, first, the word string E and the word string F are exchanged, then the word string F and the word string D are exchanged, and then the word string F and the word string C are exchanged. The word string B is replaced. The documents d2 to d4 are similarly replaced. As a result, the priority shown in FIG. 9 is obtained.

  The priority correction unit 24 is not limited to the replacement as described above. For example, when word strings having N or more selected documents are consecutive, the priority correction unit 24 takes the consecutive word strings as a group and gives priority to a word string having a priority lower than the continuous word string. You may change the degree. That is, in the case of the document d1, the word strings B, C, D, and E and the word string F may be interchanged. Alternatively, it is possible to reduce only the priority of a word string having N or more selected documents without replacing. That is, the priority of the word strings B, C, D, and E may be corrected to be lower than the word string F. Alternatively, it is possible to correct only the priority of a word string having a priority one level lower than the continuous word string without correcting the priority of a word string having the number of selected documents of N or more. That is, you may correct | amend only the priority of the word string F so that it may become the high priority of the word strings B, C, D, and E.

  Furthermore, the priority correction unit 24 corrects the priority of the word string selected as the temporary keyword so that the word string is not selected as the temporary keyword in the document and is extracted from the document. It is desirable to delete a word string selected as the temporary keyword from a plurality of word strings. For example, the word string E in the document d1 has a priority lower than M by lowering the priority, and is not selected as a temporary keyword. Therefore, the priority correction unit 24 deletes the word string E from the word string set of the document d1. As a result, the word string E will no longer be higher than the priority M in the document d1, so that it is not selected as a keyword, and the accuracy can be improved.

  FIG. 9 is a diagram showing examples of word strings and temporary keywords in the cycle 2 of the keyword selection process according to the second embodiment of the present invention. Subsequently, in cycle 2, the temporary keyword selection means 21 selects the temporary keyword group V2 for the word string in FIG. Further, the selected document number calculating means 22 calculates the number of selected documents in cycle 2 in FIG.

  At this time, for example, in the cycle 1, since the document selected as the temporary keyword is two documents d2 and d3, the keyword determination unit 23 does not determine that the number of selected documents exceeds two. , It was not a processing target of the priority correction means 24. However, since the priority correction means 24 corrects the priority of at least some of the word strings of the documents d1 to d4, the word string F is newly selected as a temporary keyword in the document d1. Therefore, in cycle 2, the word string F becomes the processing target of the priority correction means 24 because the number of selected documents is “3”.

  Similarly, in the word string D, since three documents d1, d2, and d3 are selected as the temporary keywords in the cycle 1, the keyword determination unit 23 determines that the number of selected documents exceeds 2, It was a processing target of the priority correction means 24. However, the priority of the word string D and the word string C in the document d2 is switched by the priority correction unit 24 correcting the priority of at least some of the word strings in the documents d1 to d4. Has been deleted from document d2. Therefore, in cycle 2, the word string D has the number of selected documents of “2” and is no longer a processing target of the priority correction unit 24.

  Thereafter, steps S21 to S24 are similarly repeated from cycle 3 to cycle 7. 10, FIG. 11, FIG. 12, FIG. 13 and FIG. 14 are diagrams showing examples of word strings and temporary keywords in the keyword selection processing cycles 3, 4, 5, 6 and 7 according to the second embodiment of the present invention. It is.

  FIG. 15 is a diagram showing examples of word strings and temporary keywords in the keyword selection process cycle 8 according to the second embodiment of the present invention. In cycle 8, the temporary keyword selection means 21 selects the temporary keyword group V8. Then, the selected document number calculation means 22 calculates the number of selected documents shown in FIG. Here, the keyword determination means 23 determines that the number of all selected documents is 2 or less. Then, the keyword determination unit 23 determines the temporary keyword group V8 as the keyword group K8. That is, the keyword determination means 23 selects each word string of the temporary keyword group V8 as a keyword.

  It should be noted that the keyword determination unit 23 performs the same steps even when there are more than N selected documents, even when there is no word string to be replaced with the priority of the temporary keyword corresponding to the selected documents. You may advance to S25. For example, in FIG. 15, even if the number of selected documents in the word string I exceeds N, there is no word string having a lower priority than the word string I in the document d2, so the process proceeds to step S25.

  In the above example, the keyword determination unit 23 uses the word string priority in the case of NO in step S23 as the final word string priority. However, the following method may be used. It stores the priority of the word string at the start of each cycle up to the cycle when the processing is completed, and the total number of word strings for which the number of selected documents exceeds N, and the priority of the word string having the smallest value. This is a method in which the degree is set as the final word string priority. In this case, the priority correction unit 23 corrects the priority of the word strings so that the total number of word strings selected as temporary keywords with the number of selected documents exceeding N is minimized.

  For example, as shown in FIG. 16, in cycle 1, the total number of word strings in which the number of selected documents exceeds 2 is 13. Similarly, 13 for cycle 2, 13 for cycle 3, 8 for cycle 4, 7 for cycle 5, 3 for cycle 6, 3 for cycle 7, and 0 for cycle 8. Here, the state of FIG. 15 which is the priority of the word string in cycle 8 is the final priority of the word string. Note that in the keyword selection process according to the second exemplary embodiment of the present invention, the total number of word strings in which the number of selected documents exceeds N may be greater than the previous cycle by changing the priority. In that case, keywords are determined based on priorities other than the final cycle.

  Alternatively, a threshold value for the total number of word strings in which the number of selected documents exceeds N may be determined in advance, and the keyword determination unit 23 may determine whether the total number is equal to or less than the threshold value. If it is determined that the total number is equal to or less than the threshold, the keyword determination unit 23 may perform the process of step S25. Thereby, the processing load of the keyword selection apparatus 1 can be suppressed.

  Further, the keyword determination unit 23 may end the process when the number of cycles set in advance ends. Thereby, the processing load of the keyword selection apparatus 1 can be suppressed. Further, the priority correction means 24 replaces the priority of the word string having the selected document number exceeding N with the word string having the priority lower by one, but deletes the word string having the selected document number exceeding N. You may do that. Thereby, a cycle can be decreased and the processing load of the keyword selection apparatus 1 can be suppressed. In addition to selecting the top M words with high priority from each document, the temporary keyword selection means 21 selects the top M% word strings with high priority among the word strings included in the document. Also good.

  Further, the priority correction means 24 replaces the priority of the word string in which the number of selected documents exceeds N with a word string having a priority one lower than that of the selected document, and the provisional correction unit 24 temporarily calculates the total number of documents included in the document set. The priority of a word string in which the ratio of the number of documents selected as a keyword exceeds N% may be replaced with a word string having a priority lower by one.

  As described above, according to the second embodiment of the present invention, it is possible to select a keyword of each document from a plurality of word strings included in each of the plurality of documents with high accuracy. In particular, when a structured document set is targeted, keywords can be selected from the structured document set with high accuracy.

<Third Embodiment of the Invention>
FIG. 17 is a block diagram showing a configuration of the keyword selection system 2a according to the third embodiment of the present invention. The keyword selection system 2a includes a data processing device 20a that operates under program control, and a storage device 30a that stores information. The data processing device 20a is obtained by adding priority calculation means 25 to the data processing device 20 according to the second embodiment of the present invention. The storage device 30a is obtained by adding a systematic document set storage unit 34 to the storage device 30 according to the second embodiment of the present invention. Below, it demonstrates centering around the difference with Embodiment 2 of this invention, and detailed description and illustration are abbreviate | omitted about the same structure and operation | movement. In the third embodiment of the present invention, the keyword selection system 2a targets a systematic document set.

  The structured document set storage unit 34 stores the set of structured documents described above. The priority calculation means 25 extracts a plurality of word strings included in the document for each of the plurality of documents stored in the systematic document set storage unit 34, and extracts each of the extracted plurality of word strings. Calculate the priority within the document. The priority storage unit 31 stores the plurality of extracted word strings and the priority calculated by the priority calculation unit 25 in association with each other for each of the plurality of documents.

  FIG. 18 is a flowchart showing the flow of keyword selection processing according to the third embodiment of the present invention. The priority calculation means 25 calculates the priority (S31). Specifically, the priority calculation unit 25 reads a plurality of documents from the systematic document set storage unit 34, extracts a plurality of word strings included in the read document, and for each of the extracted plurality of word strings, The priority in the extracted document is calculated. Then, the priority calculation means 25 stores the plurality of extracted word strings in association with the calculated priority in the priority storage unit 31 for each of the plurality of documents.

  Here, the extraction of the word string is performed according to the following procedure. First, the morphological analysis is used to divide the document into words, and the utilization words are converted into original forms, and parts of speech are given to the words. Next, narrow down words using clues. For example, only “noun” is selected. However, the type of part of speech to be selected is not limited to this. Finally, among the remaining words, consecutive words are connected to form a word string.

  For example, it is assumed that “cartridge / replacement / procedure /. Following / of / procedure / replacement / to do /./” (“/” represents a word break) is given. At this time, if only “noun” is selected and consecutive words are connected, “cartridge replacement procedure”, “following”, “procedure” and “exchange” can be extracted as a word string.

  At this time, the partial word string of the extracted word string may be a word string. In this case, in addition to the word string “cartridge exchange procedure”, “cartridge exchange”, “exchange procedure”, “cartridge”, “exchange”, and “procedure” are also word strings.

  Next, a temporary keyword selection unit 21 and a selected document number calculation unit 22. The keyword determination unit 23 and the priority correction unit 24 perform keyword selection processing (S32). Here, it is assumed that the keyword selection processing is either Embodiment 1 or 2 of the present invention. Therefore, detailed description is omitted.

  As described above, according to the third embodiment of the present invention, in addition to the effects of the first or second embodiment of the present invention, the initial priority of the word string can be automatically determined. The reason is that it has priority calculation means 25 for extracting a word string included in a document and determining the priority of the extracted word string in the document.

<Other embodiments of the invention>
The keyword selection device 1 according to the first exemplary embodiment of the present invention may target the above-described structured document set as a plurality of documents.

  The keyword selection apparatus and system according to the first to third embodiments of the present invention can be used as an element technology of an information search system and a hyperlink automatic generation system that automatically generates a hyperlink between documents.

  Furthermore, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention described above. For example, in the above-described embodiment, the present invention has been described as a hardware configuration, but the present invention is not limited to this. The present invention can also realize arbitrary processing by causing a CPU (Central Processing Unit) to execute a computer program. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another transmission medium. Examples of the storage medium include a flexible disk, hard disk, magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), and BD (Blu-ray (registered trademark) Disc). ROM (Read Only Memory) cartridge, battery-backed RAM (Random Access Memory) memory cartridge, flash memory cartridge, nonvolatile RAM cartridge, and the like. The communication medium includes a wired communication medium such as a telephone line, a wireless communication medium such as a microwave line, and the like.

DESCRIPTION OF SYMBOLS 1 Keyword selection apparatus 11 Temporary keyword selection means 12 Selected document number calculation means 13 Priority correction means 14 Keyword selection means 2 Keyword selection system 2a Keyword selection system 20 Data processing apparatus 20a Data processing apparatus 21 Temporary keyword selection means 22 Calculation of number of selected documents Means 23 Keyword determination means 24 Priority correction means 25 Priority calculation means 30 Storage device 30a Storage device 31 Priority storage portion 32 Selected document storage portion 33 Keyword storage portion 34 Systematic document set storage portion d1 document d2 document d3 document d4 document A word string B word string C word string D word string E word string F word string G word string H word string I word string J word string VK temporary keyword group RK keyword group V1 temporary keyword group V2 temporary keyword group V3 temporary keyword group V4 Temporary key Word group V5 Temporary keyword group V6 Temporary keyword group V7 Temporary keyword group V8 Temporary keyword group K8 Keyword group

Claims (24)

Temporary keyword selection means for selecting, as a temporary keyword, a word string having a high priority in the document among at least one word string extracted from each document for each of a plurality of documents;
A selected document number calculating means for calculating a selected document number that is a total number of documents in which the temporary keyword for each word string is selected among the plurality of documents;
When the number of selected documents exceeds a predetermined threshold according to the number of documents, the priority of at least some of the word strings in each document in which the temporary keyword is selected is set to the priority of the selected document number. Priority correction means for correcting to be equal to or lower than the threshold value,
A keyword selection device comprising: a keyword selection unit that selects a keyword based on the corrected priority for each of the plurality of documents.   The keyword selection apparatus according to claim 1, wherein the priority correction unit corrects the priority of the word string selected as the temporary keyword to be low.   The priority correction unit corrects the priority of a word string having a lower priority than a word string selected as the temporary keyword among word strings other than the temporary keyword exceeding the predetermined threshold. The keyword selection device according to claim 1 or 2.   The priority correction means selects a priority of a word string having a priority one lower than that of the word string selected as the temporary keyword, and a temporary keyword selected from the word strings other than the temporary keyword exceeding the predetermined threshold. The keyword selection device according to claim 1, wherein the priority is corrected by exchanging the priority of the word string.   The priority correction unit corrects the priority of the word string selected as the temporary keyword to be low, so that when the word string is not selected as the temporary keyword in the document, a plurality of extracted words are extracted from the document. The keyword selection device according to claim 1, wherein the word string selected as the temporary keyword is deleted from the word string.   2. The priority correction unit corrects the priority of word strings so that the total number of word strings selected as temporary keywords whose number of selected documents exceeds the predetermined threshold is minimized. The keyword selection device according to any one of 1 to 5. A keyword selection method using a computer having a CPU (Central Processing Unit),
The CPU is
For each of a plurality of documents, a temporary keyword selection step of selecting, as a temporary keyword, a word string having a high priority in the document among at least one word string extracted from each document;
A selected document number calculating step of calculating a selected document number that is a total number of documents in which the temporary keyword for each word string is selected from among the plurality of documents;
When the number of selected documents exceeds a predetermined threshold according to the number of documents, the priority of at least some of the word strings in each document in which the temporary keyword is selected is set to the priority of the selected document number. A priority correction step for correcting to be equal to or less than a threshold value of
For each of the plurality of documents, keyword selection method of performing a keyword selection step of selecting the keywords based on the corrected priority. The keyword selection method according to claim 7, wherein in the priority correction step, the CPU corrects the priority of the word string selected as the temporary keyword to be low. In the priority correction step, the CPU corrects a priority of a word string having a lower priority than a word string selected as the temporary keyword among word strings other than the temporary keyword exceeding the predetermined threshold. The keyword selection method according to claim 7 or 8, wherein: In the priority correction step, among the word strings other than the temporary keyword exceeding the predetermined threshold , the CPU has a priority of a word string having a priority lower than that of the word string selected as the temporary keyword, The keyword selection method according to any one of claims 7 to 9, wherein correction is performed by exchanging the priority of the word string selected as the temporary keyword. In the priority correction step, when the CPU corrects the priority of the word string selected as the temporary keyword to be low, and the word string is not selected as the temporary keyword in the document, it is extracted from the document. The keyword selection method according to any one of claims 7 to 10, wherein a word string selected as the temporary keyword is deleted from the plurality of word strings. In the priority correction step, the CPU corrects the priority of word strings so that the total number of word strings selected as temporary keywords whose number of selected documents exceeds the predetermined threshold is minimized. The keyword selection method according to any one of claims 7 to 11. For each of a plurality of documents, a temporary keyword selection process for selecting, as a temporary keyword, a word string having a high priority in the document among at least one word string extracted from each document;
A selected document number calculation process for calculating a selected document number that is the total number of documents in which the temporary keyword for each word string is selected from among the plurality of documents;
When the number of selected documents exceeds a predetermined threshold according to the number of documents, the priority of at least some of the word strings in each document in which the temporary keyword is selected is set to the priority of the selected document number. Priority correction processing for correcting to be equal to or less than the threshold value,
A keyword selection program for causing a computer to execute a process including a keyword selection process for selecting a keyword based on the corrected priority for each of the plurality of documents.   The keyword selection program according to claim 13, wherein the priority correction process corrects the priority of the word string selected as the temporary keyword to be low.   The priority correction processing corrects a priority of a word string having a lower priority than a word string selected as the temporary keyword among word strings other than the temporary keyword exceeding the predetermined threshold. The keyword selection program according to claim 13 or 14.   The priority correction processing is performed by selecting a priority of a word string having a priority one lower than that of the word string selected as the temporary keyword, and selecting the temporary keyword as the temporary keyword. The keyword selection program according to any one of claims 13 to 15, wherein a correction is made by exchanging the priority of the word string.   The priority correction process corrects the priority of the word string selected as the temporary keyword to be low, and when the word string is no longer selected as the temporary keyword in the document, a plurality of extracted words are extracted from the document. The keyword selection program according to any one of claims 13 to 16, wherein the word string selected as the temporary keyword is deleted from the word string.   14. The priority correction processing corrects the priority of word strings so that the total number of word strings selected as temporary keywords whose number of selected documents exceeds the predetermined threshold is minimized. 18. The keyword selection program according to any one of items 17 to 17. Priority storage means for storing the priority of the word string extracted from the document in the document for each of the plurality of documents;
Referring to the priority storage means, for each of a plurality of documents, a temporary keyword selection means for selecting a word string having a high priority in the document as a temporary keyword;
Keyword storage means for storing the temporary keyword selected by the temporary keyword selection means in association with a document containing the temporary keyword;
A selected document number calculating means for calculating a selected document number that is a total number of documents in which the temporary keyword for each word string is selected among the plurality of documents;
Selected document number storage means for storing the calculated selected document number;
A keyword determination unit that refers to the selected document number storage unit and determines whether the selected document number of the temporary keyword stored in the keyword storage unit exceeds a predetermined threshold according to the document number;
When the keyword determination unit determines that a predetermined threshold is exceeded, the priority storage unit reduces the priority of the word string selected as the temporary keyword in each document in which the temporary keyword is selected. Priority correction means for correcting the priority stored in the
The temporary keyword selection means selects the temporary keyword again based on the priority updated by the priority correction means,
The keyword determination unit determines the temporary keyword stored in the keyword storage unit as a keyword when determining that the number of selected documents of the temporary keyword stored in the keyword storage unit is equal to or less than the predetermined threshold. A keyword selection system characterized by that. For each of the plurality of documents, a plurality of word strings included in the document are extracted, and each of the extracted plurality of word strings further includes a priority calculation unit that calculates a priority in the extracted document,
The keyword selection system according to claim 19, wherein the priority storage means stores the plurality of extracted word strings and the calculated priority in association with each other for the plurality of documents. A keyword selection method using a computer having a CPU (Central Processing Unit),
The CPU is
A priority storage step of storing, in each of the plurality of documents, the priority in the document of the word string extracted from the document in the priority storage unit;
A temporary keyword selection step of referring to the priority storage means and selecting, as a temporary keyword, a word string having a high priority in the document for each of a plurality of documents;
Storing the temporary keyword selected in the temporary keyword selection step in a keyword storage unit in association with a document including the temporary keyword;
A selected document number calculating step of calculating a selected document number that is a total number of documents in which the temporary keyword for each word string is selected from among the plurality of documents;
Storing the calculated selected document number in a selected document number storage unit;
A keyword determination step of referring to the selected document number storage unit and determining whether or not the selected document number of the temporary keyword stored in the keyword storage unit exceeds a predetermined threshold according to the number of documents;
When it is determined by the keyword determination step that a predetermined threshold value is exceeded, the priority storage unit reduces the priority of the word string selected as the temporary keyword in each document in which the temporary keyword is selected. Executing a priority correction step for correcting the priority stored in
In the temporary keyword selection step, the CPU selects the temporary keyword again based on the priority updated in the priority correction step,
In the keyword determination step, when the CPU determines that the number of selected documents of the temporary keyword stored in the keyword storage unit is equal to or less than the predetermined threshold, the temporary keyword stored in the keyword storage unit is determined. A keyword selection method characterized by determining as a keyword. The CPU is
For each of the plurality of documents, a plurality of word strings included in the document are extracted, and a priority calculation step of calculating a priority in the extracted document for each of the extracted plurality of word strings is further executed. ,
The keyword selection method according to claim 21, wherein in the priority storage step, the CPU stores the plurality of extracted word strings and the calculated priority in association with each other for each of the plurality of documents. . For each of a plurality of documents, a priority storage process for storing, in the priority storage unit, the priority in the document of the word string extracted from the document;
Referring to the priority storage unit, for each of a plurality of documents, a temporary keyword selection process of selecting a word string having a high priority in the document as a temporary keyword;
A process of storing the temporary keyword selected by the temporary keyword selection process in a keyword storage unit in association with a document including the temporary keyword;
A selected document number calculation process for calculating a selected document number that is the total number of documents in which the temporary keyword for each word string is selected from among the plurality of documents;
A process of storing the calculated number of selected documents in a selected document number storage unit;
A keyword determination process for referring to the selected document number storage unit and determining whether the selected document number of the temporary keyword stored in the keyword storage unit exceeds a predetermined threshold according to the document number;
When it is determined by the keyword determination process that a predetermined threshold is exceeded, the priority storage unit reduces the priority of the word string selected as the temporary keyword in each document in which the temporary keyword is selected. And a priority correction process for correcting the priority stored in
The temporary keyword selection process selects the temporary keyword again based on the priority updated by the priority correction process,
The keyword determination process determines the temporary keyword stored in the keyword storage unit as a keyword when it is determined that the number of selected documents of the temporary keyword stored in the keyword storage unit is equal to or less than the predetermined threshold. The keyword selection program which makes a computer perform the process characterized by this. For each of the plurality of documents, a plurality of word strings included in the document are extracted, and for each of the extracted plurality of word strings, a priority calculation process for calculating a priority in the extracted document is further included,
The keyword selection program according to claim 23, wherein the priority storing process stores the plurality of extracted word strings and the calculated priority in association with each other for each of the plurality of documents.

Images