JP4592566B2 - Topic extraction method and apparatus, program, and computer-readable recording medium - Google Patents

Description

  The present invention relates to a topic extraction method and apparatus, a program, and a computer-readable recording medium, and in particular, in a situation where documents containing new information can be obtained one after another from one or a plurality of information sources. The present invention relates to a topic extraction method and apparatus, a program, and a computer-readable recording medium for extracting a characteristic word that is treated as a phrase.

  With the development of information media such as the Internet, anyone can easily send information, and documents are being sent from various senders every day. Under such circumstances, it is considered that it is possible to extract the topic that is a topic in each document by analyzing the document information created so far.

  For large and diverse document collections such as document collections uploaded on the Internet, feature words / phrases treated as topics in the document are considered in consideration of the field and temporal frequency of words / phrases in the document. Several techniques for extraction have been proposed.

As a conventional technology, documents uploaded on a network system are acquired together with their creation time information, and automatically classified into a plurality of fields set in advance according to the contents of the documents. There is a technique for calculating a topic value indicating a topic level as a feature word representing a topic with respect to a word or phrase whose appearance frequency has increased characteristically and does not appear in other fields (for example, , See Patent Document 1).
JP 2005-13531 A

  However, in the above-described conventional technique, since the field items to be manually classified are set in advance, the tendency of documents uploaded to the network changes with time, and the field items are re-read every time a new field is generated. It was necessary to set again.

  In addition, since documents with contents that are not included in the set field items are not accurately classified, they are processed together with documents in other fields. There is a problem that a high topic level is not calculated because the document does not show a characteristic appearance frequency in the field in which the document is classified.

  The present invention has been made in view of the above points, and has become a hot topic in the field handled by a document without manually setting candidates for the field of contents handled by the document to be processed. It is an object of the present invention to provide a topic extraction method and apparatus, a program, and a computer-readable recording medium capable of calculating a high topic level for a word representing a matter.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a topic degree calculation method for analyzing a set of documents created by a large number of information transmission sources and calculating the strength of topicality for a word or phrase included in a processing target document. ,
When a set of documents having information on a transmission source that has created the document is input to the phrase extraction unit, the set of documents is analyzed, and a phrase that is subject to topicality evaluation is cut out from the document, and the transmission source A phrase extraction step (step 1) for adding information and recording it in the phrase database;
The transmission source relevance calculating means accumulates in the transmission source feature amount buffer a transmission source feature amount list having the number of appearances of a word and phrase extracted from the transmission source document in units of the transmission source as one set of transmission sources. a source source relevance calculating step (step 2) of calculating a relevance Rij indicating the degree of similarity between i and the source j from the source feature list of the source and recording it in the relevance database;
The topic level calculation means accepts the processing target document as an input, based on the reference relevance distribution based on the transmission source i of the processing target document and the phrase relevance distribution obtained for each of the phrases k included in the processing target document. , A topic degree calculating step for calculating the degree of topic of the word k,
And
In the topic level calculation step,
The degree of association Rij between the transmission source i of the document to be processed and the other transmission source j is obtained from the degree-of-association database and corresponds to the degree of the degree of relevance for each degree of relevance divided into a predetermined number N. A reference relevance distribution which is information obtained by aggregating the number of transmission sources j having relevance Rij
The transmission source l having the phrase k is obtained by referring to the phrase database, the relevance level Ril between the transmission source i and the transmission source l is acquired from the relevance degree database, and the relevance degree is incremented for each relevance degree step. The phrase relevance distribution, which is information obtained by counting the number of transmission sources 1 having the corresponding relevance Ril, is obtained.
Comparing the reference relevance distribution and the phrase relevance distribution from the reference relevance distribution and the phrase relevance distribution, the phrase relevance distribution of the word k is biased to a range having a higher relevance than the reference relevance distribution. In this case, it is regarded as a topic word that is often handled in a document highly related to the transmission source i of the processing target document, and the center of gravity of the reference relevance distribution is determined from the relevance n in the step for each determined relevance step. The value obtained by subtracting the relevance value n0 from the value obtained by multiplying the value of the word relevance distribution in the step by the value obtained by subtracting the value of the standard relevance distribution is obtained, and the sum of these values is added to the word. k is the degree of the topic.

According to the present invention (claim 2), the number of relevance steps in the word relevance distribution is set to the total number of times the word k is used at the transmission source l corresponding to the step .

Further, the present invention (Claim 3) normalizes the reference relevance distribution and the phrase relevance distribution respectively, and uses the normalized reference relevance distribution and the phrase relevance distribution to Find the degree of being .

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 4) is a topic degree calculation device that analyzes a set of documents created by a large number of information transmission sources and calculates the strength of topicality for words included in a processing target document. ,
When a set of documents having information on a transmission source that has created the document is input to the phrase extraction unit, the set of documents is analyzed, and a phrase that is subject to topicality evaluation is cut out from the document, and the transmission source A phrase extraction unit 210 that adds information and records it in the phrase database 220;
A source feature quantity list having the number of occurrences of a word extracted from the source document as a feature quantity is stored in the source feature quantity buffer in units of the source, and the similarity between one set of source i and source j A source relevance calculating means for calculating the relevance Rij indicating the degree from the transmission source feature list of the transmission source and recording it in the relevance database 240;
From the reference relevance distribution based on the source i of the processing target document as an input and the word relevance distribution obtained for each of the words k included in the processing target document, the topic of the word k Topic level calculation means 250 for calculating the degree of
The topic level calculation means 250
The degree of association Rij between the transmission source i of the document to be processed and the other transmission source j is acquired from the degree-of-relationship database 240, and the degree of relevance is calculated for each degree of relevance divided into a predetermined number N. A reference relevance distribution, which is information obtained by counting the number of transmission sources j having the corresponding relevance Rij, is obtained.
The transmission source l having the phrase k is acquired with reference to the phrase database 220, the relevance level Ril between the transmission source i and the transmission source l is acquired from the relevance level database 240 , and Finding the phrase relevance distribution, which is information obtained by tabulating the number of sources 1 having the relevance Ril corresponding to the step ,
Comparing the reference relevance distribution and the phrase relevance distribution from the reference relevance distribution and the phrase relevance distribution, the phrase relevance distribution of the word k is biased to a range having a higher relevance than the reference relevance distribution. In this case, it is regarded as a topic word that is often handled in a document highly related to the transmission source i of the processing target document, and the center of gravity of the reference relevance distribution is determined from the relevance n in the step for each determined relevance step. The value obtained by subtracting the relevance value n0 in the value and the value obtained by subtracting the value of the reference relevance distribution from the value of the word relevance distribution at the step is obtained, and the sum of these values is added to the word. It includes means for determining the degree of topic of k.

Further, according to the present invention (claim 5), in the topic level calculation means 250,
The number of relevance steps of the word relevance distribution is included as a sum of the number of times the word k is used at the transmission source l corresponding to the step .

Further, according to the present invention (claim 6 ), the topic level calculation means 250
Means for normalizing the reference relevance distribution and the phrase relevance distribution, respectively, and using the normalized reference relevance distribution and the phrase relevance distribution to determine the degree of the topic k being the topic.

The present invention (Claim 7 ) is a topic degree calculation program for causing a computer to function as each means constituting the topic degree calculation apparatus according to any one of Claims 4 to 6 .

The present invention (Claim 8 ) is a computer-readable recording medium storing a topic degree calculation program characterized by storing the program according to Claim 7 .

  As described above, according to the present invention, with respect to a document group dealing with various fields created by various transmission sources such as a diary on the Web, the degree of association between each transmission source and the transmission source of the processing target document. Compared to the distribution of relevance and the distribution of relevance when focusing only on sources that have dealt with each word or phrase contained in the document to be processed, By setting a high weight to a certain word / phrase, it is possible to obtain a high degree of topic with respect to a word / phrase characteristic in the field to which the document to be processed belongs without setting the field item in advance.

  Further, according to the present invention, a time weight is given to words that are characteristically handled at the same time as the creation time of the document to be processed, and a new topic degree that considers both the topic degree and the time weight is calculated. Thus, it is possible to set a high topic level only for words that are characteristically used in a certain field at the time when the processing target document is created.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 3 shows the configuration of the topic level calculation device according to the first embodiment of the present invention.

  The topic level calculation apparatus shown in FIG. 2 is connected to a document database 200 that stores document data that is input to the apparatus, and a phrase topic level recording apparatus 290 that records the topic level of words output by the apparatus. .

  The topic level calculation device according to the present embodiment includes a word / phrase extraction unit 210 in a document, a seed topic information database 220, a transmission source relevance level calculation unit 230, a relevance level database 240, a topic level calculation unit 250, and a topic level output unit 260. Composed.

  FIG. 4 is a flowchart of an outline operation in the first embodiment of the present invention.

  Step 101) In the in-document phrase extracting unit 210, all documents included in the document are read out by reading out a group of documents collected in advance and stored in the document database 200, and analyzing them using an existing technique such as morphological analysis. Are extracted from each of the extracted words and phrases into the seed topic information database (corresponding to the “phrase database” in the claims) 220 by combining the source and creation time of the document containing the words. Store.

  Step 102) Next, referring to the seed topic information database 220 created in Step 101 in the transmission source relevance calculation unit 230, for each transmission source, the phrase included in the document created by the transmission source in the past Group and the number of times each phrase in the phrase group is used, a phrase group vector for the source is created, and existing vector comparison techniques such as cosine similarity are used for the phrase group vectors of the two sources Then, the process of calculating the similarity between the use phrase vectors as the relevance between the transmission sources is performed for all possible combinations of the transmission sources, and the obtained relevance is output to the relevance database 240.

  Step 103) In the topic level calculation unit 250, for each word included in a document to be processed from the outside (hereinafter referred to as a processing target document), a transmission source group and a process that have transmitted the same word / phrase as the word / phrase Compare the distribution of relevance with the source of the target document by comparing the distribution of relevance between all sources or sources other than the source that created the processing target document and the source that created the processing target document, When the distribution of relevance of the source group that has transmitted the relevant phrase is biased to a higher relevance range, the relevant phrase appears as a feature phrase that frequently appears in the field related to the processing target document. Set a high topic level.

  Step 104) The topic level obtained in Step 103 is output from the topic level output unit 260 to the phrase topic level recording device 290.

  Next, the detailed operation of the topic level calculation device will be described based on the configuration diagram of FIG.

  The document database 200 stores a plurality of documents to which creation time and transmission source information are added for each document. For example, the creation time such as “2004 4/25 13:55” and the source information such as the domain name of the published website and the name of the author are added to the document published on the web. The document database 200 can be constructed by recording as inputs one after another. In particular, it is desirable that new documents are sequentially created, such as a diary site on the Internet, and a document from an information source that can be identified by the caller is input. Further, when a document on the site is updated, it may be collected assuming that a new document has been created.

  The in-document phrase extraction unit 210 obtains documents stored in the document database 200 one document at a time, performs morphological analysis, and decomposes each part of speech. For example, the sentence “delicious chocolate donut” is broken down into “delicious”, “chocolate”, and “donut”. Select and extract only nouns from the decomposed part-of-speech group. At this time, if necessary, consecutive nouns such as “chocolate” and “donut” may be connected to form a compound noun “chocolate donut”, and the compound noun may be treated as one noun.

  Also, a phrase treated as a noun such as “new program in autumn” may be extracted as a noun. In the following description, the extracted nouns, compound nouns, and phrases treated as nouns are collectively referred to as words. For each word obtained in this way, the creation time and source information of the document containing the word before the analysis is added, for example, “Choco Donut 2005/01/06 11:36 blog.temporary .ex.xx "is stored in the seed topic information database 220. In the following description, information on a set of the above phrase, creation time, and transmission source information is referred to as seed topic information.

  An example of seed topic information stored in the seed topic information database 220 is shown in FIG. If the same word / phrase is used more than once in the processed document, multiple seed / topic information with the same content will be stored. It is good.

  The transmission source relevance calculation unit 230 refers to the seed topic information stored in the seed topic information database 220 for each fixed processing time, and extracts a phrase used by each transmission source. In addition, the number of times these words are used is calculated. The extracted phrase and the calculated number of times the phrase is used are set as the source feature amount. By comparing the obtained transmission source feature quantity for each pair of different transmission sources, the degree of association between each transmission source is calculated, and the degree of association is output to the association degree database 240 in the form of a table.

  An example of the relevance level information stored in the relevance level database 240 is shown in FIG.

  Here, the operation of the transmission source relevance calculation unit 230 will be described in detail.

  FIG. 7 is a flowchart of processing of the transmission source association degree calculation unit in the first embodiment of the present invention.

  Step 510) When the process is started, the transmission source relevance calculating unit 230 acquires all transmission source information stored in the seed topic information database 220, and removes the overlapping portion, thereby obtaining a document data buffer (not shown). A list of sources that are input sources.

  Step 520) Next, one transmission source information is extracted from the transmission source list obtained in Step 510, a word / phrase in the seed topic information including the transmission source information is acquired from the seed topic information database 220, and a phrase information buffer is obtained. (Not shown). At this time, in order to reduce processing, for example, only the seed topic information having a creation time within a certain time range, such as a subject for which a phrase is acquired is set to seed topic information whose creation time is two months before the processing time. It may be narrowed down to.

Step 530) For each word in the word information buffer (not shown), the number of appearances for each word is obtained, and the processing source S _i composed of the word w _k and the number of appearances V _i (w _k ) of the word. creating a word feature quantity C _i. For example, if the word / phrase in the word / phrase information buffer (not shown) accumulated in step 520 is “baseball”, “world cup”, “baseball”, “baseball”, “final”, “world cup”, “baseball”, The phrase feature amount C _i of _i is a set of a set of a phrase w _k such as “baseball 4, World Cup 2, final 1” and the number of appearances of the phrase V _i (W _k ). The obtained phrase feature value C _i is stored in a transmission source feature value buffer (not shown) as a transmission source feature value list.

  Step 540) It is determined whether or not the processing of Step 520 and Step 530 has been performed for all the transmission sources in the list of transmission sources created in Step 510 above, and there is unprocessed transmission source information. Returns to step 520. If not, the process proceeds to step 550.

Step 550) The phrase feature amounts C _i and C _j of the set of source sources S _i and S _i are extracted from the source feature list of the source feature buffer (not shown), and the relationship between the C _i and C _j is extracted. The degree is calculated as the degree of association R _ij between the transmission source i and the transmission source j.

For example, the degree of association R _ij between the transmission source i and the transmission source j is calculated using the following equation (1).

上記の式（１）において、Ｗは語句特徴量Ｃ_ｉとＣ_ｊに含まれるすべての語句を表すとする。また、Ｖ_ｉ(w_k)，Ｖ_ｊ（w_k）は全て正の値であるため、Ｒ_ｉｊは０以上１以下となり、Ｒ_ｉｊの値が大きいほど発信源Ｓ_ｉと発信源Ｓ_ｊの関連が大きいことを表す。

In the above equation (1), W represents all the phrases included in the phrase feature amounts C _i and C _j . In addition, since V _i (w _k ) and V _j (w _k ) are all positive values, R _ij is 0 or more and 1 or less, and the larger the value of R _ij , the more the transmission source S _i and the transmission source S _j . Represents a large relationship.

The obtained relevance R _ij (j ≠ i) is stored in two locations, i row j column and j row i column, in the relevance database 240.

  Step 560) It is determined whether or not the processing of Step 550 has been performed for all combinations of transmission sources. If there is an unprocessed source combination, the process proceeds to step 550. When step 550 has been processed for all combinations of the transmission sources, the processing of the transmission source relevance calculation unit 230 ends.

  The topic level calculation unit 250 accepts a processing target document as an input, and for each word / phrase in the document, the value of the degree of association between the transmission source of the processing target document and a transmission source that has handled the word / phrase in the past Compared with the distribution of the degree of relevance between the source of the processing target document and all other sources, the phrase frequently used in the source of the processing target document On the other hand, it is assumed that the topic level is high, and the topic level having a large value is calculated.

  The operation of the topic level calculation unit 250 will be described in detail.

  FIG. 8 is a flowchart of the operation of the topic level calculation unit in the first embodiment of the present invention.

  Step 610) When the processing starts, the topic level calculation unit 250 accepts a processing target document as an input from the outside. Assume that a transmission source is assigned to the processing target document. The processing target document is preferably a document included in the document database 200.

Step 620) A collection of association degrees R _ij between the transmission source S _{i of the} document to be processed and all transmission sources S _j (j ≠ i) other than the transmission source is obtained from the association degree database 240, and the distribution of the values is obtained. Tally. Referring to FIG. 6, when the transmission source of the document to be processed is the transmission source “8”, the relationship between the transmission source “8” and all other transmission sources (transmission sources 1 to 7, 9 to N). As a degree, 0.282, 0.166, 0.217, 0.327, 0.313, 0.275... Are acquired. For the set of relevances R _8j (j ≠ 8) obtained in this way, the distribution of the values is obtained as the reference relevance distribution R _s (n). For example, when counting in increments of 0.01,
0.01 * n ≦ R _8j <0.01 * (n + 1)
The number of transmission sources that apply to the above formula is aggregated in the range of n = 0 to 100, and the reference relevance distribution R _s (n) (n = 0... 99)
Ask for.

The words contained in the step 630) the target document D _i, obtained by the same processing as the document within the phrase extraction section 210, the resulting word to the processing target phrases buffer topic calculation unit 250 (not shown) to accumulate.

Step 640) By taking one phrase wk from the processing target phrase buffer (not shown), referring to the seed topic information database 220, obtaining source information in the seed topic information having the phrase, and summing up, Phrase usage source information consisting of a combination of a source S _j that has used the word w _k and the number of times X (S _j , w _k ) that used the word is created.

  For example, the source information acquired from the seed topic information database 220 for the phrase “baseball” is “source j” “source j” “source j” “source k” “source n” “source”. In the case of “n”, the phrase use transmission source information of “transmission source j 3 times, transmission source k 1 time, transmission source n 2 times” is created. However, if the phrase usage count is not used in step 650, the phrase usage count need not be included in the phrase usage source information.

Step 650) A set of relevance R _ij between the source Si of the specified document information and each source S _j included in the phrase use source information is obtained from the relevance level database 240, and the value is set as Step 620. Similarly, the word relevance distribution Rwk (n) is obtained. For example, when counting in increments of 0.01,
0.01 * n ≦ R _ij <0.01+ (n + 1)
The number of transmission sources satisfying the above formula is tabulated from n = 0 to 100, and the phrase relevance distribution R _wk (n) (n = 0... 100) is obtained. At this time, instead of counting the number of transmission sources having a degree of relevance within a certain range, the total number of times of use of phrases of the transmission sources having a degree of relevance within a certain range may be totaled. For example, a process for _{obtaining a} phrase relevance distribution R _wk (n) for the phrase w _k at the source of the processing target document is S ₈ ,
0.01 * 5 ≦ R _8j <0.01 * (5 + 1)
When there are three sources S ₆ , S ₁₀ , and S ₂₁ that satisfy the condition, “3” that is the number of sources included in the range is not set in R _wk (5), but S ₆ , X (S ₆ , w _k ) + X (S ₁₀ , w _k ) + X (S ₂₁ , w _k ), which is the sum of the number of times the phrases S ₁₀ and S ₂₁ are used
May be set to R _wk (5).

Step 660) When word w _k is often used at the originating source associated high degree of source of S _i of the target document D _i, to set a high value as a characteristic word in the field, the phrase related Normalization of the degree distribution and the reference relevance distribution using the sum of the distribution values in the entire range, and the normalized phrase relevance distribution in the high relevance range is the normalized reference relevance When the distribution is larger than the distribution, the topic score Score _r (w _k ) is calculated to a high value.

For example, when the calculation is performed within the range of the degree of relevance in increments of 0.01 as in step 620 and step 650 described above, Score _r (w _k ) is obtained by equation (4).

式（４）中のｎ_０は、基準関連度分布Ｒ_ｓ（ｎ）の重心を示す値であり、式（５）により求められる。

N ₀ in Expression (4) is a value indicating the center of gravity of the reference relevance distribution R _s (n), and is obtained from Expression (5).

ステップ６７０） 得られた語句ｗ_ｋと話題度Score_r(w_k)の組を、話題度出力部２６０に出力する。

Step 670) The set of the obtained word / phrase w _k and topic level Score _r (w _k ) is output to the topic level output unit 260.

  Step 680) It is determined whether or not the processing from Step 640 to Step 670 has been performed for all the words / phrases in the processing target word / phrase buffer (not shown). If there are unprocessed words / phrases, the process proceeds to Step 640. If all words have been processed, the topic level calculation unit 250 ends.

  The topic level output unit 260 outputs a combination of the phrase received from the topic level calculation unit 250 and the topic level of the phrase to the phrase topic level recording device 290. At this time, in order to reduce the output amount, only words having a topic degree equal to or higher than a preset value may be output. FIG. 9 shows an example of a phrase and its topic level that are output to the phrase topic level recording device 290.

[Second Embodiment]
FIG. 10 shows the configuration of the topic level calculation apparatus according to the second embodiment of the present invention.

  The topic level calculation apparatus shown in FIG. 2 is connected to a document database 200 that stores document data that is input to the apparatus, and a phrase topic level recording apparatus 290 that records the topic level of words output by the apparatus. .

  The topic level calculation device according to the present embodiment includes an in-document word / phrase extraction unit 210, a seed topic information database 220, a transmission source relevance level calculation unit 230, a relevance level database 240, a topic level calculation unit 250, a time weight calculation unit 860, and a topic. It is composed of a degree output unit 870.

  Among them, the in-document phrase extraction unit 210, the seed topic information database 220, the transmission source relevance calculation unit 230, the relevance database 240, and the topic calculation unit 250 operate in the same manner as in the first embodiment. .

  In the present embodiment, a document including creation time and transmission source information is a processing target.

  In the method of calculating a high topic level for a word that is often handled in the field to which the processed document belongs as shown in the first embodiment, a field to be used such as a technical term in the field is specified. A high degree of topic is also calculated for words that are commonly used in the field.

  On the other hand, a time weight calculation unit 860 that weights words that are characteristically handled at the same time as the creation time of the processing target document is provided, and a topic obtained by the topic level calculation unit 250 in the topic level calculation unit 250 By calculating a new topic degree that takes into account both the degree and the time weight obtained by the time weight calculation unit 860, only the words that are characteristically used in a certain field at the time when the processing target document was created It is possible to set a high topic level.

  FIG. 11 is a flowchart of an outline operation in the second exemplary embodiment of the present invention.

  Since steps 101 to 103 shown in FIG. 4 in the first embodiment are the same operations, the description thereof is omitted.

  Step 201) In the time weight calculation unit 860, for each word / phrase included in the processing target document, the degree of use of the processing target document at the creation time of the processing target document is set as the time weight value of the word / phrase.

  Step 202) In the topic level output unit 870, for each word / phrase included in the processing target document, the topic level of the word / phrase obtained by the topic level calculation unit 250 and the word / phrase obtained by the time weight calculation unit 860 are calculated. The value obtained by multiplying the time weight is set as the topic level of the word / phrase considering the change in the attention level with time.

  Step 203) The topic level output unit 870 outputs the topic level obtained in Step 202 to the phrase topic level recording device 290.

  Next, the detailed operation of the topic level calculation device in the present embodiment will be described based on the configuration diagram of FIG.

  Hereinafter, the operations of the time weight calculation unit 860 and the topic degree calculation unit 870 different from those in the first embodiment will be described.

Similar to the topic level calculation unit 250, the time weight calculation unit 860 receives the designation of the processing target document that is the subject of the topic phrase extraction process as an input, and is obtained by performing the same processing as the in-document phrase extraction unit 210. For all the words in the processing target document, for example, at the creation time of the processing target document as disclosed in Japanese Patent Application Laid-Open No. 2005-276115, a high weight is given to the words whose frequency of occurrence of the word is increasing. The time weight Score _t (w _k ) is calculated by applying the attaching method with reference to the seed topic information database 220.

  The topic level output unit 870 generates, for each word / phrase wk included in the processing target document, a word that appears characteristically in the field including the processing target document and is frequently used at the creation time of the processing target document. A high topic level is calculated and output as a word representing the topic of the document.

Topic of the output unit 870 receives the word w _k topics calculator 250 from the w _k topic of Score _r (w _k), the time from the weight calculation section 860 of the w _k time weight Score _t a (w _k), respectively , Score _r (w _k ) and Score _t (w _k ) are multiplied to calculate the topic score Score (w _k ) of the phrase that evaluates both the domain feature and temporal feature, It is output to the phrase topic level recording device 290. At this time, in order to reduce the output amount, only words having a topic degree equal to or higher than a preset value may be output.

  In the present invention, the operation of the topic level calculation device shown in the first and second embodiments is constructed as a program, and the seed topic information database 220 and the relevance level database 240 that operate as the topic level calculation device can be accessed. It can also be installed on a simple computer or distributed via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk device or a flexible disk / CD-ROM, installed in a computer to be executed, or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a technique for calculating the topic level of words in a document.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the topic degree calculation apparatus in the 1st Embodiment of this invention. It is a flowchart of the outline | summary operation | movement in the 1st Embodiment of this invention. It is an example of the seed topic information accumulate | stored in the seed topic information database in the 1st Embodiment of this invention. It is an example of the relevance degree information accumulate | stored in the relevance degree database in the 1st Embodiment of this invention. It is a flowchart of operation | movement of the transmission source related degree calculation part in the 1st Embodiment of this invention. It is a figure explaining the process of the topic degree calculation part in the 1st Embodiment of this invention. It is an example of the output to the phrase topic degree recording device in the first embodiment of the present invention. It is a block diagram of the topic degree calculation apparatus in the 2nd Embodiment of this invention. It is a flowchart of the outline | summary operation | movement in the 2nd Embodiment of this invention.

Explanation of symbols

200 Document Database 210 Phrase Extraction Unit, In-Document Phrase Extraction Unit 220 Phrase Database, Species Topic Information Database 230 Transmission Source Relevance Calculation Unit, Transmission Source Relevance Calculation Unit 240 Relevance Database 250 Topic Degree Calculation Unit, Topic Degree Calculation Unit 260 Topic level output means, topic level output unit 290 Phrase topic level recording device 860 Time weight calculation unit 870 Topic output unit

Claims (8)

A topic degree calculation method for analyzing a set of documents created by a large number of information transmission sources and calculating the strength of topicality with respect to a word or phrase included in a processing target document,
When a set of documents having information on a transmission source that has created the document is input to the phrase extraction unit, the set of documents is analyzed, and a phrase that is subject to topicality evaluation is cut out from the document, and the transmission source A phrase extraction step of adding information and recording it in a phrase database;
The transmission source relevance calculating means accumulates in the transmission source feature quantity buffer a transmission source feature quantity list whose feature quantity is the number of appearances of the phrase extracted from the transmission source document in units of transmission sources, and sets a set of transmissions. A source source relevance calculating step of calculating a relevance Rij indicating the degree of similarity between the source i and the source j from the source feature list of the source and recording it in the relevance database;
The topic level calculation means accepts a processing target document as an input, a standard relevance distribution based on the transmission source i of the processing target document, and a phrase relevance distribution obtained for each of the phrases k included in the processing target document From the topic level calculation step of calculating the degree of the topic k is a topic,
And
In the topic level calculation step,
The degree of association Rij between the source i of the document to be processed and the other source j is obtained from the degree-of-association database, and the degree of the degree of association is obtained for each degree of association divided into a predetermined number N. Obtaining the reference relevance distribution, which is information obtained by aggregating the number of transmission sources j having the relevance Rij corresponding to
A source l having the phrase k is obtained by referring to the phrase database, and a relevance level Ril between the source i and the source l is obtained from the relevance level database , Obtaining the phrase relevance distribution which is information obtained by tabulating the number of transmission sources l having relevance Ril corresponding to the relevance step ;
By comparing the reference relevance distribution and the phrase relevance distribution from the reference relevance distribution and the phrase relevance distribution, the phrase relevance distribution of the word k is biased to a range having a higher relevance than the reference relevance distribution. In the case where the processing target document is highly related to the transmission source i of the document to be processed, it is regarded as a topic word that is often handled. The value obtained by multiplying the value obtained by subtracting the relevance value n0 at the center of gravity and the value obtained by subtracting the value of the reference relevance distribution from the value of the word relevance distribution at the step is obtained, and the sum of these values is obtained. The degree of the topic k is a topic,
The topic degree calculation method characterized by this. The topic degree calculation method according to claim 1 , wherein the number of relevance levels in the phrase relevance distribution is a sum of the number of times the phrase k is used at the transmission source l corresponding to the step . The reference relevance distribution and the phrase relevance distribution are respectively normalized, and the normalized degree of topic k is obtained using the normalized reference relevance distribution and the phrase relevance distribution. The topic degree calculation method according to 1 or 2. A topic degree calculation device that analyzes a set of documents created by a large number of information transmission sources and calculates the strength of topicality with respect to a phrase included in a processing target document,
When a set of documents having information on a transmission source that has created the document is input to the phrase extraction unit, the set of documents is analyzed, and a phrase that is subject to topicality evaluation is cut out from the document, and the transmission source A phrase extraction means for adding information and recording the phrase database;
A transmission source feature amount list having the number of appearances of the phrase extracted from the transmission source document as a feature amount in the transmission source unit is accumulated in the transmission source feature amount buffer, and a set of the transmission source i and the transmission source j A source relevance calculating means for calculating a relevance Rij indicating the degree of similarity from the source characteristic list of the source and recording it in a relevance database;
The processing target document is received as an input, and the reference relevance distribution based on the transmission source i of the processing target document and the phrase relevance distribution obtained for each of the phrases k included in the processing target document are used. A topic degree calculating means for calculating the degree of topic,
The topic level calculation means includes:
The degree of association Rij between the source i of the document to be processed and the other source j is obtained from the degree-of-association database, and the degree of the degree of association is obtained for each degree of association divided into a predetermined number N. Obtaining the reference relevance distribution, which is information obtained by aggregating the number of transmission sources j having the relevance Rij corresponding to
A source l having the phrase k is obtained by referring to the phrase database, and a relevance level Ril between the source i and the source l is obtained from the relevance level database , Obtaining the phrase relevance distribution which is information obtained by tabulating the number of transmission sources l having relevance Ril corresponding to the relevance step ;
By comparing the reference relevance distribution and the phrase relevance distribution from the reference relevance distribution and the phrase relevance distribution, the phrase relevance distribution of the word k is biased to a range having a higher relevance than the reference relevance distribution. In the case where the processing target document is highly related to the transmission source i of the document to be processed, it is regarded as a topic word that is often handled. The value obtained by subtracting the relevance value n0 at the center of gravity and the value obtained by subtracting the reference relevance distribution value from the value of the word relevance distribution at the step is obtained, and the sum of these values is obtained. Including means for determining the degree of topic of the phrase k,
A topic degree calculation device characterized by that. The topic level calculation means includes:
The topic degree calculation device according to claim 4, further comprising means for setting the number of relevance steps of the word relevance distribution for each step of the word k at the transmission source l corresponding to the step . The topic level calculation means includes:
Means for normalizing the reference relevance distribution and the phrase relevance distribution respectively, and using the normalized reference relevance distribution and the phrase relevance distribution to determine the degree of topic of the word k The topic degree calculation device according to claim 4 or 5.   A topic degree calculation program for causing a computer to function as each means constituting the topic degree calculation apparatus according to claim 4.   A computer-readable recording medium storing a topic degree calculation program, wherein the program according to claim 7 is stored.

Images