JP4309933B2 - Document classification apparatus and classification method - Google Patents

Description

  The present invention relates to a document classification device and a document classification method, and more particularly to improvement of classification accuracy for an acquired target document group.

  Conventionally, a large number of documents collected by, for example, a document search system may be classified into several categories according to the contents of the documents. When such a large number of documents are classified, it is generally classified automatically using a mechanical method such as clustering. The machine classification referred to here is a method of classifying a feature quantity based on the appearance frequency of a word in a document as a vector element based on the similarity between the vectors.

  However, automatic classification by machine can classify a large amount of documents in a short time, but because of low accuracy, it is often not classified as intended by the user. Therefore, it has been troublesome to manually re-examine the machine classification result again.

  In order to solve the above-described problem, there is a method of improving accuracy by incorporating the user's classification intention into the machine classification as a constraint. When classifying, it may be known in advance that a document and a document always belong to the same category. For example, Non-Patent Document 1 realizes a classification result that matches the user's intention by incorporating such prior knowledge into machine classification as a restriction in classification. Specifically, we give a constraint called Must-link if any two documents are known to belong to the same category, and Cannot-link if they are in different categories. Improve accuracy by classifying constraints with priority.

Wagstaff, K., Cardie, C., Rogers, S. & Schroedl, S. Constrained K-Means clustering with background knowledge.Proc. Of the 18th International Conference on Machine Learning.pp.577-584. 2001.

  However, since the above method is a restriction applied between documents, it cannot be applied when it is necessary to add between documents and categories. In addition, since the restriction has only information that “any two documents belong to the same (different) category”, when a Must-link is placed between two documents with different classification results based on similarity, which category The problem of whether both should belong to cannot be solved.

  According to the present invention, when a user moves a document from a certain category to another category with respect to a tabular classification map obtained by machine classification, the same classification constraint is placed between the document and the destination category. Set. The same classification constraint is set between the created classification restriction document and the moved document by creating a centroid calculated by a document included in the destination category as a new document (same classification restriction document). . Furthermore, a movement flag is set for the moved document among the two documents connected by the same classification constraint.

  On the other hand, in the machine classification after manual classification map correction, each document is classified according to the similarity of the vectors, but there is a document that is linked to the same classification constraint in a certain document, and the similarity between the documents depends on the similarity. If the classification results are different, the document with the movement flag belongs to the same category as the document without the movement flag.

  As described above, by using the same classification restriction document and movement flag set by manual classification at the time of machine classification, a highly accurate classification result reflecting the user's classification intention can be obtained.

  FIG. 1 is a block diagram of the entire document classification apparatus according to an embodiment of the present invention. The document classification apparatus includes an information terminal 10, a document data DB 110, a classification operation DB 111, a bibliographic information DB 112, and a classification map DB 113, and a network 114. The information terminal 10 and the four databases are connected by the network 114, but the four databases may be included in the information terminal 10.

  The information terminal 10 includes a CPU 101, a memory 102, a keyboard / mouse 103, a display 104, a machine classification control unit 105, a classification restriction document generation unit 106, a manual classification control unit 107, a document display unit 108, and a data communication unit 109.

  The CPU 101 performs various processes by calling and executing various programs stored in the machine classification control unit 105, the classification restriction document generation unit 106, the manual classification control unit 107, the document display unit 108, and the data communication unit 109. The memory 102 provides a work area for temporarily storing data used by the CPU 101 for various processes. The keyboard / mouse 103 is used as a means for a user to input information. The display 104 displays the classification result and the like. The data communication unit 109 is a network interface. For example, it is composed of a LAN card that can communicate using the TCP / IP protocol. The information terminal 10 can communicate with a database arranged on the network 114 by the data communication unit 109.

  The document data DB 110 stores frequency information of words included in the document. In the classification operation DB 111, various processing results obtained from the machine classification control unit 105, the same classification constraint document generation unit 106, and the manual classification control unit 107 are stored. The bibliographic information DB 112 stores bibliographic information of documents such as creator and creation date. The classification map DB 113 stores category information of the created classification map.

FIG. 2 is a configuration diagram of the manual classification control unit 107 in FIG.
The fixed flag generation unit 201 generates a fixed flag for a target document when a category fixing instruction is issued from the user. The movement flag generation unit 202 generates a movement flag for the target document when the user moves the document from one category to another category. The category division control unit 203 divides the category into a plurality of categories, and controls the fixed flag and the movement flag when the document is moved to the divided category. The category combination control unit 204 controls the fixed flag and the movement flag when a plurality of categories are integrated into one.

  An overview of processing performed by the machine classification control unit 105, the classification restriction document generation unit 106, the manual classification control unit 107, and the document display unit 108 will be described with reference to FIG. FIG. 3 is a process flow diagram of the document classification apparatus according to the embodiment of the present invention. Thereafter, in accordance with the processing flow of FIG. 3 and the classification result display example of FIG. 17, various classification processes (machine classification control unit 105, classification restriction document generation unit 106, manual classification control unit 107) and DB (document data DB 110, Details of the classification operation DB 111, the bibliographic information DB 112, and the classification map DB 113) will be described.

  First, a user uses a keyboard / mouse 103 to input a document set collected by a search system or the like into the document classification apparatus of the present invention. The document classification apparatus that has received the target document set (S301) passes the target document set to the machine classification control unit 105 and executes initial machine classification when the classification result information is not given to the documents in the set (S302, S303). If the target document set is classified, the document classification device passes the target document set to the document display unit 108, and the document display unit 108 displays the classification result on the display 104 (S304). Here, the case where a classification is given to the target document set means, for example, a result of previously classifying the target document set by the document classification apparatus of the present invention.

  When the machine classification control unit 105 executes the initial machine classification (S303), the user inputs the number of division categories of the target document set. Upon receiving the target document set and the number of division categories, the machine classification control unit 105 executes initial machine classification (S303). In addition, in the initial machine classification performed here, a general hierarchical clustering method and a K-Means method are combined. For example, clustering is performed with the number of divided categories using the hierarchical clustering method, the centroid of each category is obtained from the classification result, and the result obtained by the K-Means method with the centroid as the initial centroid is used as the final initial result. Use machine classification results. Further, as a method for obtaining the initial centroid used in the K-Means method, the user may directly input a document having the initial centroid instead of using the hierarchical clustering method. For the hierarchical clustering method and K-Means method, see, for example, `` The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, Springer-Verlag, 2001 '' I want to be.

  Next, when the category of the target document set is determined, the machine classification control unit 105 passes the classification result to the document display unit 108, and the document display unit 108 displays the classification result on the display 104 (S304).

  FIG. 17 is a display example 1701 of the classification result of the target document set. As shown in the classification map 1702, the classification results are displayed by two axes, vertical and horizontal. In this example, the document 1705 is arranged as “creator” information 1703 of a document obtained from the bibliographic information DB 112 with row components (vertical axis) and category information 1704 with column components (horizontal axis) classified by machine classification. In the example of FIG. 17, creator information is used as the row component, but it can be freely selected from items registered in the bibliographic information DB 112. Reference numeral 1706 is a check box for fixing the category of the corresponding document, 1707 is a button for performing machine classification, 1708 is a button for moving the document to another category, and 1709 is a category. The button 1710 is a button for dividing the categories, and 1710 is a button for combining the categories.

  After displaying the map of the classification target document set, the user repeats the machine classification (S306) and manual classification (S308) by trial and error (S305-S309), thereby completing the desired classification map 1702. In the following, the configuration of the document data DB 110, the classification operation DB 111, the bibliographic information DB 112, and the classification map DB 113 will be described first, and then the classification map 1702, check box 1706, and buttons 1707 to 1710 in FIG. Details of the machine classification and manual classification operations will be described.

  FIG. 13 shows a data format and data example of data stored in the document data DB 110. Reference numeral 1301 denotes a number for identifying a document to be stored. 1302 stores words characteristic of the corresponding document and their appearance frequencies.

  FIG. 14 shows a data format and an example of data stored in the classification operation DB 111. Reference numeral 1401 denotes a number for identifying a document to be stored. Reference numeral 1402 denotes a number for identifying a column to which the corresponding document belongs in the classification map 1702. Reference numeral 1403 denotes an identification number of the source column when the corresponding document is moved to another category by manual classification. When this value is 0, it indicates that the corresponding document has not been moved. Reference numeral 1404 denotes 0 and 1 values indicating the state of the fixed flag check box 1706 of the corresponding document in the classification map 1702, where 0 indicates an off state and 1 indicates an on state. Reference numeral 1405 denotes an identification number of a document having the same classification constraint relationship as the corresponding document. When this value is 0, it indicates that there is no same classification restriction document. In the case of -1, this indicates that this document is the same classification restriction document. In the example of FIG. 14, the document with the document id 2 indicates that the document with the document id 20 has the same classification constraint relationship.

  FIG. 15 shows a data format and data example of data stored in the bibliographic information DB 112. Reference numeral 1501 denotes a number for identifying a document to be stored. 1502 is a creation date, 1503 is a creator, 1504 is some classification assigned to the document, and 1505 is a title. This data format is merely an example, and what should be defined as a column element depends on the type of target document.

  FIG. 16 shows the data format and data example of data stored in the classification map DB 113. Reference numeral 1601 denotes a number for identifying a category in the classification map. The category name 1602 is the name of the corresponding category.

  FIG. 4 is a processing flow of the machine classification operation (S306). In the machine classification, after the machine classification control unit 105 receives the target document set (S401), the same classification restriction document related to the documents is generated (S402), and the machine classification is performed using the target document set and the same classification restriction document. Execute (S403). Finally, the movement flags of all documents in the classification operation DB 111 are updated to 0 (S404). In the following, details of the classification constraint document generation and machine classification will be described.

FIG. 5 is a processing flow for generating the classification restriction document (S402). The machine classification control unit 105 acquires the target document (S501), and obtains a movement flag of the document from the classification operation DB 111 (S502). If the movement flag is 0 (S502 → No), the classification restriction document generation process is terminated. When the movement flag is 1 or more (S502 → Yes), the machine classification control unit 105 acquires the same classification restriction document id of the document from the classification operation DB 111. When the same classification restriction document id is 1 or more (S503 → Yes), the machine classification control unit 105 deletes the document entry having the same document id as the same classification restriction document id from the document data DB 110 (S504). The target document is passed to the constraint document generation unit 106. Then, the classification restriction document generation unit 106 generates a new classification restriction document (S505). If the same classification constraint document id is 0 in S503 (S503 → No), the same classification constraint document is generated (S505), and the same classification constraint document generation processing is terminated. The following four cases follow the above flow. In the classification operation DB 111, the target document
Case 1: The movement flag is 0 and the classification restriction document id is 0.
Case 2: The movement flag is 0, and the classification restriction document id is 1 or more.
Case 3: The movement flag is 1 or more and the classification restriction document id is 0.
Case 4: When the movement flag is 1 or more and the classification restriction document id is 1 or more. First, Case 1 is a case where the target document has never been moved, and the process ends without generating the same classification constraint document (S502 → No). Case 2 is a case where the document has been moved in the past but has not been moved by the previous manual classification operation, and the processing is terminated without generating the same classification constraint document ( S502 → No). Case 3 is a case where the document has moved for the first time, and executes the same classification constraint document generation process (S503 → No). Case 4 is a case where movement has been performed in the past and movement has been performed in the previous manual classification (for example, manual document movement → machine classification execution → manually moving the document again → machine) In this case, since the immediately preceding movement process is prioritized, the currently held same classification restriction document is deleted, and a new same classification restriction document is generated according to the previous movement process (S503 → Yes).

  FIG. 6A is a processing flow of classification restriction document generation (S505). The classification constraint document generation unit 106 acquires the target document from the machine classification control unit 105 (S601). Next, the classification constraint document generation unit 106 acquires the document id of the document group having the column id equal to the column id of the target document from the classification operation DB 111 (S602), and stores the corresponding document data in the document data DB 110. The center of gravity is calculated using the word frequency information (S603). Finally, the center of gravity is stored as an entry in various DBs (S604).

An example of a series of operations from the same-classification constraint document generation process of FIG. 5 to the same-classification constraint document generation process of FIG. 6A will be described by taking a document with a document id of 2 in FIG. Equivalent to Case 4). First, since the movement flag of the document with the document id 2 is 1 or more from the classification operation DB 111 (S502 → Yes), the value of the classification restriction document id is acquired (S503). Since the value is 20 (S503 → Yes), the document with the document id 20 is searched from the document data DB 110, the classification operation DB 111, and the bibliographic information DB 112. In this example, the lowermost document of each DB is a document with a document id of 20, and these entries are deleted (S504). Then, using the column id value (= 2) in the classification operation DB 111, processing for generating the same classification constraint document is performed (to S601). All documents having a column id value of 2 are extracted from the classification operation DB 111 (S602), and the center of gravity formed by these documents is calculated (S603). In this example, the documents having the same column id as the document with the document id of 2 are the documents with the document ids of 3 and 5, as shown in FIG. The following calculation is performed according to the value of the document data DB 110.
Communication terminal Carrier terminal ... Congestion ...
(8 + 7 + 8) / 3 (4 + 3 + 3) / 3 (0 + 0 + 0) / 3 (1 + 2 + 1) / 3 ... (0 + 0 + 0) / 3 ...

  The entry is registered in the three DBs with the document id of the center of gravity (document) calculated as 21 as 21. In the document data DB 110, word frequency information obtained by the above calculation is registered as an entry. In the classification operation DB 111, an entry with a column id (the document id is the same value as the column id of 2) 2, a movement flag, a fixed flag 0, and a classification restriction document id −1 is registered. In the bibliographic information DB 112, entries that are all empty except for the document id are registered. In addition, the value of the classification restriction document id of the entry whose document id is 2 in the classification operation DB 111 is changed to 21 (S604). 6B and 6C show examples of the results of the document data DB 110 and the classification operation DB 111 obtained by the same classification constraint document generation process for the entry with the document id of 2. This completes the classification constraint document generation process.

  FIG. 18 is a processing flow for acquiring the machine classification result (S403). The machine classification control unit 105 acquires a target document group to be classified (S1801), and calculates the center of gravity of each category (S1802). The center of gravity is calculated by regarding documents with the same column id in the classification operation DB 111 as the same category. At this time, a document whose classification restriction document id is −1 in the classification operation DB 111 is not included in the centroid calculation. Next, each document is classified into a corresponding category based on the similarity to the center of gravity (S1803). The classification method based on the similarity will be described in detail later with reference to FIG. After all the documents are classified, the center of gravity is recalculated using the classification result (S1804), and the square value V of (old center of gravity−new center of gravity) is calculated (S1805). Also in the calculation of the new center of gravity, a document whose classification restriction document id is −1 in the classification operation DB 111 is not included. If the value of V is larger than a preset threshold value, the old center of gravity is updated to the new center of gravity (S1806), classification is performed again based on the similarity of each document (S1805 → No), and if it is smaller, the machine classification is terminated. (S1805 → Yes).

  FIG. 7 is a processing flow for classifying documents by similarity (S1803). If there is the same classification restriction document, the document is classified according to the restriction. The machine classification control unit 105 acquires the target document (referred to as document i) (S701), and checks the value of the fixed flag of the document i from the classification operation DB 111 (S702). If the value of the fixed flag is 1, this classification process is terminated (S702 → No), and if it is 0 (S702 → Yes), the document i is classified based on the similarity to the center of gravity (this classification result is c_i). (S703). Next, it is checked from the classification operation DB 111 whether or not the classification restriction document id of the document i is 1 or more (S704). If it is not 1 or more, the column id in the classification operation DB 111 of the document i is set as c_i, and the process is ended (S704 → No). If it is 1 or more, the document corresponding to the id is acquired from the document data 110 (referred to as document j) (S704 → Yes). Then, the document j is classified based on the similarity (S705), and the classification result c_i of the document i is compared with the classification result c_j of the document j (S706). If c_i is equal to c_j, the column id in the classification operation DB 111 for document i is set to c_i, and the process is terminated (S706 → No). If c_i and c_j are not equal (S706 → Yes), the movement flag of the document i and the document j is checked from the classification operation DB 111, and the classification result of the document whose movement flag is 0 is made the same as the non-zero classification result. The process ends (S707). For example, in FIG. 14, if the document i is a document with a document id of 2 and the document j is a document with a document id of 20, the movement flag of the document with a document id of 20 is 0. The column id is updated to 3 (which is the column id of the document whose document id is 20).

  The above is the description of the machine classification result acquisition (S306) process. After the machine classification is completed, the document display unit 108 displays the classification result according to the classification operation DB 111, the bibliographic information DB 112, and the classification map DB 113. At that time, the document with the same classification restriction document id −1 in the classification operation DB 111 is not displayed because it is a virtual document for restriction.

  FIG. 8 is an operation flow of the manual classification operation (S308). Manual classification includes document category fixing (S802), document movement (S803), category division (S804), and category combination (S805), and these operations can be repeated (S801, S806). .

  The document category fixing (S802) is an operation for always classifying the document into the corresponding category after the subsequent machine classification execution, and each document displayed on the classification map 1702 shown in FIG. When the check box 1706 at the head of the document is turned on, the “fix” function is enabled. FIG. 9 shows a processing flow of the fixed flag generation unit 201 related to category fixing. The fixed flag generation unit 201 acquires the document identification number of the corresponding document (S901), and changes the value of the fixed flag of the corresponding document in the classification operation DB 111 (S902). The fixed flag 1404 is set to 1 when the check box 1706 is turned on, and is set to 0 when the check box 1706 is turned off. If there are a plurality of target documents to be acquired in step S901, this process is repeated for the number of documents.

  The document movement (S803) is an operation of moving the document from the current category to another category. The user selects a document to be moved from the classification map 1702 and then clicks a move button 1708 to select a destination category. FIG. 10 shows a processing flow of the movement flag generation unit 202 regarding category movement. The movement flag generation unit 202 acquires the document id of the corresponding document (S1001). Then, the migration flag of the corresponding document in the classification operation DB 111 is changed to the identification number of the source column (S1002), and the column id is changed to the identification number of the destination column (S1003). Finally, if the fixed flag 1404 of the document is 1, it is set to 0 (S1004). If 0, do not change. For example, in the classification map 1702 of FIG. 17, when a document whose document id is 1 in the classification operation DB 111 of FIG. 14 is moved to the classification B, the movement flag is set to 1 (from the classification map DB 113 of FIG. Set the column id to 2 (because the category id of classification B is 2). If there are a plurality of target documents to be acquired in S1001, this process is repeated for the number of documents.

  The category division (S804) is an operation of dividing an existing category into two. The division operation is realized by two steps of creating a new empty category in the classification map 1702 and moving the document to the category. FIG. 11 shows a processing flow of the category division control unit 203 related to category division. The user clicks the division button 1709 in FIG. 17 and inputs the category name of the newly created category. In response to the above operation, the category division control unit 203 adds an empty column having the category name to the classification map 1702 (S1101), and adds an entry in which the category id without duplication and the category name input by the user are set. Created in the DB 113 (S1102). Thereafter, the user moves the document to the newly added empty category. Accordingly, the category division control unit 203 performs document movement processing (S1103). Since the document moving operation after adding an empty category is the same as the processing in S803, the description is omitted.

  The combination of categories (S805) is an operation of combining existing categories into one. The user clicks a combining button 1710 and inputs a plurality of categories to be combined (hereinafter referred to as a combination target category group) and a combined category name to combine them. FIG. 12A shows a processing flow of the category combination control unit 204 related to category combination. The category combination control unit 204 acquires the category id of the combination target category group (S1201). Thereafter, an empty category is created in the classification map 1702 (S1202), and an entry having a category id without duplication and a category name input by the user is added to the classification map DB 113 (S1203). Next, all the documents in the combination target category group are moved to the newly created category (S1204), the category id of the new category is combined with the column id of these documents, and the movement flag is combined. If it is any column id value of the target category group, the value is set to 0 (S1205). Finally, all the join target category groups that are empty are deleted (S1206), and the join target category entries in the classification map DB 113 are deleted (S1207).

  For example, consider a case where a new category X is created by combining the categories A and B of the category map 1702 in FIG. First, an empty column whose category information 1704 is classification X is created in the classification map 1702, and an entry whose category id is 5 and category name is classification X is added to the classification map DB 113. Next, among the entries in the classification operation DB 111, the column id of the document whose column id is 1 or 2 (in the example of FIG. 14, the documents with document ids 1, 2, 3, and 5) is updated to 5, Of these documents, the movement flag of the document whose movement flag is equal to the column id value of the combination target category (the document whose document id is 3 in the example of FIG. 14) is updated to 0. Then, the columns of classification A and classification B in the classification map 1702 are deleted, the corresponding entries in the classification map DB 113 are also deleted, and the combining process is terminated. The state of the classification operation DB 111 after the end of the combining process is shown in FIG. 12B.

This is the end of the description of the manual classification operation (S308).
According to the invention according to the first embodiment described above, for manual document movement, it is possible to restrict classification for a category by connecting the centroid of the destination category and the moved document with the same classification restriction document. did. In addition, by combining this classification restriction document with the movement flag and fixed flag, it is possible to classify into the category specified by the user even in repeated machine classification, and the classification intention of the user is reflected in the obtained classification map A highly accurate document classification map can be created.

  In the first embodiment, the result of the machine classification is arranged in the column component, and the classification based on the bibliographic information such as the document creator is arranged in the row component (FIG. 17). The result of machine classification is a result of classifying documents to be classified according to the difference in contents (actually vector similarity) in the present invention. In the second embodiment, machine classification is also performed for row components. Place the result of. That is, a document is classified from two viewpoints (contents), and a classification map is created. For example, when a patent document is a classification target document, the patent document has viewpoints such as “conventional technology”, “technical problem”, “solution”, and “effect of the invention”. Therefore, it is possible to create a biaxial map from the viewpoint of using the column component as a technical problem and the row component as a solution. FIG. 24 is a diagram showing an example of a biaxial classification map in which a classification target document group is a patent document relating to plasma display panel technology, a column component 2403 is a technical problem, and a row component 2404 is a solution.

  In order to realize the present embodiment, in the vectorization of a document when machine classification is performed using a vector space model, it is necessary to generate a vector using a word representing a viewpoint unique to the document. In the second embodiment, a document classification apparatus will be described in detail in which the vector generation technique according to the document viewpoint is combined with the first embodiment. Further, even if the classification map is based on the biaxial viewpoint, the basic processing is almost the same as in the first embodiment. For both machine and manual classification operations, if the instruction is a classification for a row, the operation shown in the first embodiment is performed for the row, and for a column, the operation is performed for the column. In the following description, the description of the same points as those of the document classification apparatus according to the first embodiment will be omitted, and only different points will be described.

  The difference from the first embodiment is a block diagram of a document classification device (FIG. 19), a feature vector generation unit 1906, a document data DB 1911 (FIG. 20), a classification operation DB 1912 (FIG. 21A), and a classification map DB 1914 (FIG. 22). , Viewpoint DB 1915 (FIG. 23), classification restriction document DB 1916 (FIG. 21B), and classification map display example (FIG. 24).

  FIG. 19 is a block diagram of the entire document classification apparatus according to the second embodiment of the present invention. It differs from the block diagram of the first embodiment in that a feature vector generation unit 1906 and a viewpoint DB 1915 are added. Regarding other DBs, the data structure in the DB is different from that of the first embodiment. Therefore, in the following, each difference will be described first, and then the classification map display example of FIG. The second embodiment will be described in detail.

  The feature vector generation unit 1906 extracts a part having a description related to the viewpoint in the full text data of the document data DB 1911 according to a viewpoint defined in advance by the system side, and features based on the frequency of words characteristic of the part. Generate a vector.

  FIG. 20 shows a data format and data example of data stored in the document data DB 1911. Reference numeral 2001 denotes a number for identifying a document to be stored. 2002 stores the full-text data of the corresponding document.

  FIG. 21A shows the data format and data example of the data stored in the classification operation DB 1912. Stores the classification operation results for each row and column. In the first embodiment, the ID 1402, the movement flag 1403, and the fixed flag 1404 for the column are stored. In this embodiment, the ID, the movement flag, the fixed flag, and the classification restriction document ID are stored for both the row and the column. .

  FIG. 22 shows a data format and data example of data stored in the classification map DB 1914. Stores the category name for each row and column. The axis type 2203 is a value indicating whether the category name is a row name or a column name, and 0 indicates a column and 1 indicates a row. The viewpoint 2204 stores an identification number indicating which viewpoint the category is defined in the viewpoint DB 1915.

  FIG. 23 shows a data format and an example of data stored in the viewpoint DB 1915. The viewpoint id 2301 indicates the identification number of the viewpoint defined in advance on the system side, and the viewpoint name 2302 indicates the name.

  FIG. 21B shows a data format and a data example of data stored in the classification restriction document DB 1916. The restriction document generated by the classification restriction document generation process (FIG. 25, S2503) is stored. The restriction document id 2110 corresponds to the same row / column classification restriction document id in the classification operation DB 1912.

  FIG. 24 is a display example of a map created by classifying both rows and columns by category (viewpoint). As described at the beginning of the present embodiment, this is an example in which a classification map is created with a group of documents to be classified as a patent document relating to plasma display panel technology, a column component 2403 as a technical problem, and a row component 2404 as a solution. This embodiment is different from the first embodiment in that there are check boxes (2406 and 2407) for fixing the classification for each row and column.

  The operation flow at the information terminal 20 in this embodiment is basically the same as that shown in FIG. However, in the present embodiment, in the initial machine classification result acquisition step (S303), in addition to the number of division categories, the user selects from which viewpoint the system provides each row / column component.

  FIG. 25 is a processing flow of the machine classification operation. The processing of the first embodiment shown in FIG. 4 is that there is a step (S2502) of obtaining information on which axis (row / column / both rows and columns) the machine classification processing is executed by the user. It is different from the flow. If it is a machine classification instruction for either a row or a column, the same processing as in the first embodiment is performed on the corresponding axis. The same applies to both axes, after classifying one of them and then classifying the other. Further, the word frequency information of the classification restriction document generated by the generation of the classification restriction document (S2503) is stored in the document data DB 110 in the first embodiment, but in the classification restriction document DB 1916 in the present embodiment. Store.

  FIG. 26 is a processing flow for acquiring the machine classification result. It is obtained from the classification map DB 1914 and the viewpoint DB 1915 which axis the axis (row / column / both) for which the user designates the machine classification processing is the axis related to the viewpoint, and from the full-text part corresponding to the viewpoint using the document data DB 1911. It differs from the processing flow of the first embodiment shown in FIG. 18 in that there is step S2602 in which the feature vector generation unit 1906 generates a feature vector. In the similarity classification in S2604, the processing flow is the same as in FIG. 7, but the word frequency information referred to when classifying the same classification restriction document by similarity (S705) is the document in the first embodiment. In contrast to the data DB 110, the present embodiment refers to the same classification constraint document DB 1916. The feature vector referred to here refers to a vector composed of words and keywords peculiar to the document in the part where the viewpoint is described. For details on extraction of words specific to a document and a specific part of the document and a method of constructing a vector, “Information Retrieval Algorithm, Kenji Kita, Kazuhiko Tsuda, Masami Sasabori, Kyoritsu Shuppan, 2002” is detailed.

  According to the invention according to the second embodiment described above, the target document set can be classified in more detail according to the content (viewpoint) of the document. For example, if patent documents are classified, creating a classification map with "invention issues" in the rows and "invention solving means" in the columns, there is room for development in the technology from which viewpoints Can be created with less effort than a conventional document classification apparatus.

  In the first embodiment, every time a user moves a document by manual classification, a document vector of the classification restriction document is created and an entry is registered in various DBs. In the third embodiment, the document ids of three documents are registered instead of the created document vector itself. By doing so, when the document vector of the same classification restriction document is required in the machine classification, the center of gravity formed by the three documents is calculated, and the same classification as that registered in the first embodiment Make sure that the constraint document is available. Therefore, the storage capacity required for the DB can be saved. Although three document ids are registered, it may be more than that. In the following description, the description of the same points as those of the document classification apparatus according to the first embodiment will be omitted, and only different points will be described.

  The difference from the first embodiment is a block diagram of the document classification device (FIG. 27), a representative document calculation control unit 2706, and a classification operation DB 2712 (FIG. 28).

  FIG. 27 is a block diagram of the entire document classification apparatus according to the third embodiment of the present invention. It differs from the block diagram of the first embodiment in that a representative document calculation control unit 2706 is added. The representative document calculation control unit 2706 selects a plurality of documents, for example, three documents, whose centroid vectors are substantially equal to the centroid when the centroid vectors of the destination category are calculated as the classification restriction documents.

  FIG. 28 shows a data format and an example of data stored in the classification operation DB 2712. In the first embodiment, the ID of the centroid vector document actually registered in various DBs is registered as the same classification constraint document id. However, in the third embodiment, the representative document calculation control unit 2706 selects Register the ids of the three documents.

  FIGS. 29 and 30 are flow charts of machine classification processing using the same classification restriction document in the third embodiment.

  FIG. 29 is a processing flow for obtaining the same classification constraint document. In the first embodiment, the entry of the center of gravity vector registered in the various DBs is deleted in S504, but in this embodiment, only the same classification constraint document id in the classification operation DB 2712 is deleted in S2904. In step S2905, after calculating the center of gravity of the destination category, the representative document calculation control unit 2706 selects three documents represented, and registers these three documents in the classification operation DB 2712. As a method of selecting the representative three documents, for example, a method of selecting the three documents closest to the true centroid among the documents in the category, or the true centroid and the three documents from all combinations of selecting three from the documents in the category A method of selecting the three documents having the closest centroids by the method is conceivable.

  FIG. 30 is a processing flow for acquiring a machine sentence result. After obtaining the same classification constraint document id in S3004, step S3005 for calculating a centroid vector using those ids is added. The calculated center-of-gravity vector is temporarily stored in the memory 2702, and is deleted when this processing flow ends.

  According to the above-described invention according to the third embodiment, by registering three documents such that a vector equal to the document vector can be obtained by calculating the center of gravity, the classification restriction document vector itself is recorded. It is possible to save the required DB storage capacity rather than keeping it. In the above embodiment, the difference from the first embodiment has been described, but the same applies to the second embodiment.

FIG. 2 is a block diagram illustrating a configuration example of a document classification device in the first embodiment. The figure which shows the structural example of the manual classification control part in a 1st Example. The flowchart which shows an example of the process in the information terminal in a 1st Example. The flowchart which shows an example of the machine classification result acquisition process in a 1st Example. The flowchart which shows an example of the same classification | category restrictions document acquisition process in a 1st Example. The flowchart which shows an example of the classification | category restrictions document production | generation process in a 1st Example. The figure which shows the example of document data DB changed by the same classification | category restrictions document production | generation process in a 1st Example. The figure which shows the example of classification operation DB changed by the same classification | category restrictions document production | generation process in a 1st Example. The flowchart which shows an example of the classification process by the similarity with the category gravity center of each document in a 1st Example. The flowchart which shows an example of the manual classification result acquisition process in a 1st Example. The flowchart which shows an example of the fixed flag production | generation process in a 1st Example. The flowchart which shows an example of the movement flag production | generation process in a 1st Example. The flowchart which shows an example of the division | segmentation process control process of the category in a 1st Example. The flowchart which shows an example of the combination process control process of the category in a 1st Example. The figure which shows the example of classification operation DB changed by the combination process of the category in a 1st Example. The figure which shows the structural example of document data DB in a 1st Example. The figure which shows the structural example of classification operation DB in a 1st Example. The figure which shows the structural example of bibliographic information DB in a 1st Example. The figure which shows the structural example of the classification map DB in a 1st Example. The figure which shows the example of the display screen of the classification map in a 1st Example. The flowchart which shows an example of the machine classification result acquisition process in a 1st Example. The block diagram which shows the structural example of the document classification | category apparatus in a 2nd Example. The figure which shows the structural example of document data DB in a 2nd Example. The figure which shows the structural example of classification operation DB in a 2nd Example. The figure which shows the structural example of the classification restriction | limiting document DB in a 2nd Example. The figure which shows the structural example of the classification map DB in a 2nd Example. The figure which shows the structural example of viewpoint DB in a 2nd Example. The figure which shows the example of the display screen of the classification map in a 2nd Example. The flowchart which shows an example of the machine classification result acquisition process in a 2nd Example. The flowchart which shows an example of the machine classification result acquisition process in a 2nd Example. FIG. 10 is an example of a block diagram of an entire document classification device and a block diagram of an information terminal in a third embodiment. The figure which shows the structural example of classification operation DB in a 3rd Example. The flowchart which shows an example of the classification restrictions document acquisition process in a 3rd Example. The flowchart which shows an example of the classification process by the similarity with the category gravity center of each document in a 3rd Example.

Explanation of symbols

10: Information terminal, 101: CPU, 102: Memory, 103: Keyboard / mouse, 104: Display, 105: Machine classification control unit, 106: Same classification restriction document generation unit, 107: Manual classification control unit, 108: Document display 109: Data communication unit 110: Document data DB 111: Classification operation DB 112: Bibliographic information DB 113: Classification map DB 114: Network

Claims (10)

A display unit that displays a classification map that categorizes the documents in the document database;
An operation input unit for operating the classification map displayed on the display unit;
A machine classification control unit that performs initial classification to classify documents in the document database into a given number of categories, and then reclassification that reflects the corrections made by the operation input unit;
A classification constraint document generation unit,
In the initial classification, the machine classification control unit includes a word in the document database based on similarity between vectors having a word included in a document as an element and a numerical value calculated based on the appearance frequency of the word as an element value. Classify documents into the given number of categories,
When an operation for moving a document belonging to a certain category to another category is performed by the operation input unit,
The same-classification constraint document generation unit generates a same-classification constraint document corresponding to a centroid vector of a document group belonging to a destination category, associates the moved document with the same-classification constraint document,
The machine classification control unit calculates a centroid vector of a document group belonging to each category, reclassifies each document into a corresponding category based on a similarity between a vector representing the document and the centroid vector, and moves the moved document. Are classified into the same category as the same classification constraint document with which the document is associated,
A document classification apparatus, wherein a classification map reflecting a result of reclassification by the machine classification control unit is displayed on the display unit.   2. The document classification apparatus according to claim 1, wherein the classification map is a two-dimensional map in which categories are arranged on one axis and bibliographic information of the document is arranged on the other axis.   2. The document classification apparatus according to claim 1, wherein when an operation for fixing a specific document to a current category is performed by the operation input unit, the category is not changed by the reclassification for the document. Feature document classification device.   The document classification apparatus according to claim 1, wherein a new category is created by the operation input unit, and an operation for moving a part of a group of documents belonging to one category to the new category is performed. A document classification apparatus, wherein the same processing as when an operation of moving a document belonging to a certain category to another category is performed on each document of the document group moved to (1). A display unit for displaying a two-dimensional classification map in which documents in a document database are categorized by arranging a category in a first viewpoint on one axis and a category in a second viewpoint on the other axis; ,
A first vector having a word included in a part having a description related to the first viewpoint in the document as an element and a numerical value calculated based on the appearance frequency of the word as an element value is generated, and the second aspect A feature vector generating unit that generates a second vector having a value included in a part of the description as an element and a numerical value calculated based on the appearance frequency of the word;
An operation input unit for operating the classification map displayed on the display unit;
An initial classification for classifying the documents in the document database into a given number of first viewpoint categories and second viewpoint categories, and then a reclassification reflecting the corrections made by the operation input unit; A machine classification control unit to execute;
A classification constraint document generation unit,
In the initial classification, the machine classification control unit classifies the documents in the document database into the given number of first viewpoint categories based on the similarity between the first vectors, Classifying the documents in the document database into the given number of second viewpoint categories based on the similarity between the second vectors;
When an operation for moving a document belonging to a certain category of the first viewpoint to another category of the first viewpoint is performed by the operation input unit,
The same-classification constraint document generation unit generates a same-classification constraint document corresponding to a centroid vector of a first vector of a document group belonging to a destination category, and associates the moved document with the same-classification constraint document ,
The machine classification control unit calculates a centroid vector of a first vector of a document group belonging to each category of the first viewpoint, and each document is similar to the first vector and the centroid vector of the document. Reclassify to the category of the first viewpoint corresponding to the degree, and classify the moved document into the same category of the first viewpoint as the same classification constraint document to which the document is associated,
Display a classification map reflecting the result of reclassification by the machine classification control unit on the display unit,
When an operation for moving a document belonging to a certain category of the second viewpoint to another category of the second viewpoint is performed by the operation input unit,
The same-classification constraint document generation unit generates a same-classification constraint document corresponding to a centroid vector of a second vector of a document group belonging to a destination category, and associates the moved document with the same-classification constraint document ,
The machine classification control unit calculates a centroid vector of a second vector of a document group belonging to each category of the second viewpoint, and each document is similar to the second vector and the centroid vector of the document. Reclassify to the category of the second viewpoint corresponding to the degree, and classify the moved document into the category of the second viewpoint that is the same as the classification restriction document to which the document is associated,
A document classification apparatus, wherein a classification map reflecting a result of reclassification by the machine classification control unit is displayed on the display unit. A document by a document classification apparatus having a display unit that displays a classification map in which documents in the document database are categorized, an operation input unit that operates the classification map displayed on the display unit, and a machine classification control unit A classification method,
The machine classification control unit is provided with the document in the document database based on a similarity between vectors having a word included in the document as an element and a numerical value calculated based on the appearance frequency of the word as an element value. An initial classification process for classifying into a number of categories;
Receiving an operation of moving a document belonging to a certain category to another category by the operation input unit;
Generating a same-classification constraint document corresponding to a centroid vector of a document group belonging to a destination category, and associating the moved document with the same-classification constraint document;
The machine classification control unit calculates the centroid vector of each document group belonging to each category, reclassifies each document into a corresponding category based on the similarity between the vector representing the document and the centroid vector, and the moved Classifying the document into the same category as the same classification constraint document with which the document is associated;
And a step of displaying a classification map reflecting a result of reclassification by the machine classification control unit on the display unit. 7. The document classification method according to claim 6 , wherein the classification map is a two-dimensional map in which categories are arranged on one axis and bibliographic information of the document is arranged on the other axis. 7. The document classification method according to claim 6 , wherein when an operation for fixing a specific document to a current category is performed by the operation input unit, the category is not changed by the reclassification for the document. Feature document classification method. The document classification method according to claim 6 ,
Accepting creation of a new category by the operation input unit;
Receiving an operation of moving a part of a document group belonging to one category to the new category;
A step of performing, for each document in the document group moved to the new category, processing similar to that performed when an operation for moving a document belonging to the certain category to another category is performed. Document classification method. A display unit for displaying a two-dimensional classification map in which documents in a document database are categorized by arranging a category in a first viewpoint on one axis and a category in a second viewpoint on the other axis; , A document classification method by a document classification device having an operation input unit for performing an operation on the classification map displayed on the display unit, and a machine classification control unit,
Inputting the first viewpoint and its number and the second viewpoint and its number;
For each document in the document database, generating a first vector having as an element a numerical value calculated based on the appearance frequency of the word, including the word contained in a part having a description relating to the first viewpoint, Generating a second vector having as an element a numerical value calculated based on the appearance frequency of a word including an element included in a part having a description relating to the second aspect;
The machine classification control unit classifies the document in the document database into the given number of first viewpoint categories based on the similarity between the first vectors, and the second vector. Performing an initial classification to classify the documents in the document database into the given number of second viewpoint categories based on the similarity between them;
Displaying the result of the initial classification on the display unit as a classification map;
Receiving a document movement operation between categories by the operation input unit,
When the movement operation of the document is movement between categories of the first viewpoint,
Generating a same-classification constraint document corresponding to a centroid vector of a first vector of a document group belonging to a destination category, and associating the moved document with the same-classification constraint document;
The machine classification control unit calculates a centroid vector of a first vector of a document group belonging to each category of the first viewpoint, and each document is similar to the first vector and the centroid vector of the document. Reclassifying the corresponding category of the first viewpoint according to the degree and classifying the moved document into the same category of the first viewpoint as the classification restriction document to which the document is associated;
Performing a step of displaying a classification map reflecting a result of reclassification by the machine classification control unit on the display unit,
When the movement operation of the document is movement between categories of the second viewpoint,
Generating a same-classification constraint document corresponding to a centroid vector of a second vector of a document group belonging to a destination category, and associating the moved document with the same-classification constraint document;
The machine classification control unit calculates a centroid vector of a second vector of a document group belonging to each category of the second viewpoint, and each document is similar to the second vector and the centroid vector of the document. Reclassifying the corresponding second viewpoint category according to the degree and classifying the moved document into the same second viewpoint category as the same classification constraint document with which the document is associated;
And a step of displaying a classification map reflecting a result of reclassification by the machine classification control unit on the display unit.

Images