JP7243402B2 - DOCUMENT PROCESSING METHOD, DOCUMENT PROCESSING PROGRAM AND INFORMATION PROCESSING DEVICE - Google Patents

Description

The present invention relates to a document processing method and the like.

When a failure occurs in a product operating in the field, a maintenance worker repairs the target product, prepares a failure report (MR) describing the details of the failure, and reports it to the maintenance management department.

The maintenance department analyzes a plurality of reported MRs to detect, for example, failures that occur frequently. The maintenance management department documents the content of the most frequently detected failures and countermeasures, and disseminates them to the field to prevent failures that may occur in products.

A company that sells products improves the maintainability and quality of its products by repeatedly implementing the above-described measures. In order to speed up the cycle of such efforts, there is a need to use AI (artificial intelligence) to improve the efficiency of MR analysis work, which currently relies on human labor.

Until now, descriptions of MRs have included spelling variations, synonyms, and synonyms, and it has been difficult for a computer to determine whether or not MRs have the same disorder. However, by utilizing the distributed representation of words, it is possible to vectorize words and associate MRs including spelling variations, synonyms, and synonyms. For example, there is also a prior art that calculates a document (MR) vector by weighted synthesis of each word vector and quantifies the similarity between documents.

There is also a conventional technique (clustering) that calculates similarity between documents using vectors of documents and classifies documents with high similarity into the same cluster. For example, it is possible to perform clustering based on the similarity of each MR, and regard a cluster containing a predetermined number or more of MRs as a cluster of MRs corresponding to failures that occur frequently.

For example, a cluster consisting of only MRs with a similarity of 1 can be regarded as a cluster with the same failure. Better.

JP 2009-146397 A JP 2017-194727 A JP 2018-106390 A

MRs may contain sentences other than the sentence describing the failure details, and when clustering is performed on such MRs, MRs that do not describe common failure details are classified into the same cluster, Proper clusters are not generated.

MRs are characterized by having a very small number of characters and sentences compared to general documents, and the existence of sentences unrelated to the content of failures contained in MRs tends to affect the results of clustering.

For example, the MR "Syn flood attacks are frequently detected. Please tell me how to deal with them." contains. Of these sentences A and B, sentence A is a sentence describing the content of failure, and sentence B is not a sentence describing the content of failure.

Here, when the number of MRs including sentence B is larger than the number of MRs including sentence A, the MRs including sentence B are classified into one cluster (first cluster) regardless of the fault content. . The number of MRs included in the first cluster increases, but since this first cluster is a cluster of MRs that have in common the sentence B that is not related to the content of the failure, when this first cluster is detected, false positives occur. lead to the occurrence of

Also, MRs containing sentence A but not B are classified into a cluster (second cluster) different from MRs containing sentence B. FIG. Although the second cluster is a cluster that classifies MRs corresponding to failure details, MRs that include both sentences A and B belong to the first cluster, so the number of MRs included in the second cluster becomes smaller, the second cluster is excluded from detection targets, and detection omission occurs.

For this reason, it is required to exclude other sentences from documents such as MRs, while leaving predetermined sentences describing the details of failures and the like.

In one aspect, the present invention provides a document processing method, a document processing program, and an information processing apparatus capable of leaving a sentence describing the content of a failure while excluding other sentences unrelated to the content of the failure from the document. intended to

In the first alternative, the computer performs the following processes. A computer obtains a plurality of documents consisting of one or more sentences. The computer identifies a first sentence of interest composed of a sentence that satisfies a preset condition from among a plurality of documents. The computer acquires a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest from among the plurality of documents. The computer identifies a second sentence of interest composed of a sentence other than the identified first sentence of interest from among the plurality of acquired first documents. The computer acquires a plurality of second documents composed of a plurality of sentences including the second sentence of interest from among the plurality of documents. The computer selects a second sentence of interest from among the plurality of documents based on the relationship between the number of identical documents included in each of the plurality of first documents and the plurality of second documents and the number of documents other than the identical document. exclude.

Other sentences unrelated to the content of the failure can be excluded from the document while leaving the sentence describing the content of the failure.

FIG. 1 is a diagram for explaining the processing of the information processing apparatus according to the embodiment. FIG. 2 is a functional block diagram showing the configuration of the information processing apparatus according to this embodiment. FIG. 3 is a diagram showing an example of the data structure of the document DB. FIG. 4 is a diagram showing an example of the data structure of the set S table. FIG. 5 is a diagram showing an example of the data structure of the set M table. FIG. 6 is a diagram showing an example of the data structure of the set M' table. FIG. 7 is a diagram showing an example of the data structure of a discriminant model table. FIG. 8 is a diagram (1) for explaining an example of the processing of the exclusion processing unit; FIG. 9 is a diagram (2) for explaining an example of the processing of the exclusion processing unit; FIG. 10 is a diagram (3) for explaining an example of the processing of the exclusion processing unit; FIG. 11 is a diagram (4) for explaining an example of the processing of the exclusion processing unit; FIG. 12 is a diagram (5) for explaining an example of the processing of the exclusion processing unit; FIG. 13 is a diagram (1) showing an example of a detection result. FIG. 14 is a diagram (2) showing an example of the detection result. FIG. 15 is a flowchart (1) showing the processing procedure of the information processing apparatus according to the present embodiment. FIG. 16 is a flowchart (2) showing the processing procedure of the information processing apparatus according to the embodiment. FIG. 17 is a diagram showing an example of the hardware configuration of a computer that implements the same functions as the information processing apparatus according to this embodiment.

Exemplary embodiments of the document processing method, the document processing program, and the information processing apparatus disclosed in the present application will be described below in detail with reference to the drawings. In addition, this invention is not limited by this Example.

FIG. 1 is a diagram for explaining the processing of the information processing apparatus according to the embodiment. The information processing device detects a document S1 that includes a sentence 1a describing the content of the failure and does not include other sentences from a plurality of documents (for example, MR).

The information processing device also detects documents M1, M2, and M3 (and other documents) containing sentence 1a and other sentences, and M4 and M5 (and other documents) that do not contain sentence 1a. In FIG. 1, sentences 1a and 2a are sentences describing different fault contents. Sentences 1b, 1c, and other sentences are sentences in which it is unclear whether or not they describe the details of the failure.

In the same document, other sentences co-occurring with the sentence 1a describing the content of the failure are either sentences describing the content of the failure or sentences not describing the content of the failure. In addition, there is a feature that sentences not describing the content of failure often co-occur with sentences describing the content of failure, regardless of specific content of the failure. Conversely, it can be said that sentences co-occurring with sentences describing various failures do not describe the contents of failures.

Here, each document M1 to M5 is classified into sections 10A, 10B, and 10C. Section 10A includes documents M2 and M3 (other documents not shown) in which sentence 1a and sentence 1b co-occur and which do not include sentence 1c. Section 10B includes documents M4 and M5 (other documents not shown) in which sentence 2a and sentence 1c co-occur. Section 10C includes document M1 in which sentence 1a, sentence 1b, and sentence 1c co-occur.

Here, documents M2, M3 and other documents (not shown) included in section 10A have sentences 1a and 1b co-occurring, and "sentence 1b is a sentence related to the failure content of sentence 1a." I can say. On the other hand, in section 10B, sentence 1c co-occurs with sentence 2a, which describes a failure content different from that of sentence 1a. Therefore, the information processing device excludes the sentence 1c from the document M1.

As described above, the information processing apparatus according to the present embodiment detects a document including a sentence describing the failure content of interest, and among the detected documents, a document including a plurality of sentences is related to the failure content of interest. Leave a sentence with In addition, the information processing device performs processing for deleting sentences unrelated to the focused failure content (sentences not describing the failure content). In this way, sentences related to the sentence describing the content of the failure can be left, and sentences not related can be deleted. Therefore, in failure detection by clustering processing, erroneous detection and detection omission can be suppressed.

Next, an example of the configuration of the information processing apparatus according to this embodiment will be described. FIG. 2 is a functional block diagram showing the configuration of the information processing apparatus according to this embodiment. As shown in FIG. 2 , this information processing apparatus 100 has a communication section 110 , an input section 120 , a display section 130 , a storage section 140 and a control section 150 .

The communication unit 110 is a processing unit that performs data communication with an external device via a network. Communication unit 110 is an example of a communication device. A control unit 150 , which will be described later, exchanges data via the communication unit 110 . For example, the communication unit 110 receives information of a document describing the details of the failure from the external device.

The input unit 120 is an input device for inputting various kinds of information to the information processing device 100 . For example, the input unit 120 corresponds to a keyboard, mouse, touch panel, and the like. The user may operate the input unit 120 to input the information of the document describing the content of the failure to the information processing apparatus 100 .

The display unit 130 is a display device that displays various information output from the control unit 150 . The display unit 130 corresponds to a display device such as a liquid crystal display and a touch panel.

The storage unit 140 has a document DB (Data Base) 141 , a set S table 142 , a set M table 143 and a discrimination model table 145 . The storage unit 140 corresponds to semiconductor memory devices such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, and storage devices such as HDD (Hard Disk Drive).

The document DB 141 is a database for registering information of a plurality of documents (MR) describing failure details. FIG. 3 is a diagram showing an example of the data structure of the document DB. As shown in FIG. 3, the document DB 141 associates document identification information with document information. Document identification information is information that uniquely identifies a document. The document information is document information including one or more sentences describing the content of the failure. For example, each sentence included in one piece of document information is distinguished from other sentences by punctuation marks.

In FIG. 3, the document information corresponding to the document identification information "MR1" is "A syn flood attack has been detected. Please tell me how to deal with it.". This document information includes the sentence "A syn flood attack has been detected." and the sentence "Please tell me how to deal with it."

In FIG. 3, the document information corresponding to the document identification information "MR100" is "A syn flood attack was detected." This document information includes the sentence "A syn flood attack was detected." In this way, when the document information contains only one sentence, the one sentence can be regarded as a sentence describing the details of the failure.

The set S table 142 is a table for registering document information including one sentence among the document information registered in the document DB 141 . FIG. 4 is a diagram showing an example of the data structure of the set S table. As shown in FIG. 4, this set S table 142 associates document identification information with document information (one sentence).

The set M table 143 is a table for registering document information including a plurality of sentences among the document information registered in the document DB 141 . FIG. 5 is a diagram showing an example of the data structure of the set M table. As shown in FIG. 5, this set M table 143 associates document identification information with document information (a plurality of sentences).

The set M' table 144 is a table for registering information obtained by dividing the document information registered in the set M table 143 for each sentence. FIG. 6 is a diagram showing an example of the data structure of the set M' table. As shown in FIG. 6, this set M' table 144 associates document identification information, document sub-identification information, deletion flags, and document information. The document identification information corresponds to the document identification information described with reference to FIG. The document sub-identification information is information for identifying each sentence included in document information of multiple sentences. The deletion flag is a flag indicating whether or not to delete the corresponding document information. If it is to be deleted, it is "on", and if it is not to be deleted, it is "off". The deletion flag is set in the exclusion processing unit 154, which will be described later. The initial value of the delete flag is "off". Document information is information of one sentence.

The determination model table 145 is a table for registering a model for determining whether or not a document is similar to the target document information. FIG. 7 is a diagram showing an example of the data structure of a discriminant model table. As shown in FIG. 7, the discriminant model table 145 associates target document identification information with discriminant models. The target document identification information is information indicating the document identification information of the document information that is the target of the discriminant model, or document sub-identification information (described later). The discriminant model is information of a discriminant model for discriminating a document similar to the document of the target document identification information.

Returning to the description of FIG. The control unit 150 has an acquisition unit 151 , a first identification unit 152 , a second identification unit 153 , an exclusion processing unit 154 and a detection unit 155 . The control unit 150 can be implemented by a CPU, MPU (Micro Processing Unit), or the like. The control unit 150 can also be realized by hardwired logic such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array).

The acquisition unit 151 acquires document information describing the details of the failure from an external device or the like via the network, and registers the acquired document information in the document DB 141 . The document identification information corresponding to the document information may be preset in the document information, or the acquisition unit 151 may assign unique document identification information to the document information. The acquisition unit 151 may acquire document information via the input unit 120 .

The first identification unit 152 identifies document information consisting of one sentence from among multiple pieces of document information contained in the document DB 141 , and registers the identified document information and document identification information in the set S table 142 .

Further, the first identifying unit 152 selects one sentence S1 from each document information (sentence) registered in the set S table 142, and executes the following process to determine sentences similar to the sentence S1. Generate a discriminant model.

An example of a process of generating a discriminant model by the first identifying unit 152 will be described. The first identifying unit 152 calculates the degree of similarity between the sentence S1 and each sentence (including the sentence S1) included in the set S table 142, and finds the sentences included in the set S table 142 that have the highest similarity. identify the sentence

For example, the first identifying unit 152 calculates the vector of each word included in the sentence in the same manner as word2vec, and multiplies the vectors of each word included in the sentence to calculate the vector of the sentence. The first identifying unit 152 calculates the distance between the vector of the sentence S1 and the vector of each sentence registered in the set S table 142 as the degree of similarity. The first identifying unit 152 increases the degree of similarity as the distance between the vectors is shorter. The first identifying unit 152 sorts the sentences in descending order of similarity, and identifies sentences included in the top n as sentences with the highest similarity.

The first identifying unit 152 performs PU (Positive Unlabeled) learning with the identified high-ranking similarity sentences as “positive examples” and generates a discrimination model for determining whether or not the sentence is similar to the sentence S1. The first identifying unit 152 associates the target document identification information for identifying the sentence S1 with the discriminant model information and registers them in the discriminant model table 145 .

Here, PU learning is a learning method that performs machine learning when only some positive examples are given as training data. A discriminant model learned by PU learning is an estimation model for estimating the probability of a positive case for data whose positive or negative is unknown. Also, the discriminant model learned by PU learning is a discriminant model weighted by the calculated positive case probability. Although this PU learning is used in the subsequent processing, it is also possible to regard sentences with low similarity as negative examples and construct a discriminant model using a normal machine learning method.

For example, the first identifying unit 152 defines random variables x, y, and z when performing PU learning. Here, let x ∈ R (all real numbers), y ∈ {-1, 1}, s ∈ {0, 1}. x is the input (sentence vector), y is the class label (negative example = -1, positive example = 1), s is whether the data is labeled (unlabeled s = 0, labeled = 1).

First, the first identifying unit 152 learns an estimation model of p(s=1|x). As described above, the sentence with the highest degree of similarity is given the “positive” label (s = 1), and the other sentences are not labeled (s = 0), so they are labeled. The data (vector of sentences) are positive examples. Therefore, it can be said that the estimation model for p(s=1|x) is a model for estimating the “probability of being a positive example”. The first identifying unit 152 learns the estimation model of p(s=1|x) by, for example, learning to adjust the parameters of the NN (Neural Network).

Subsequently, the first identification unit 152 regards the output of the estimation model of p(s=1|x) as likely to be a positive example, and determines the discriminant model p(y=1|x)=p(s=1|x) /p(s=1|y=1). The first identifying unit 152 learns a discriminant model of p(y=1|x) by performing learning for adjusting parameters of the NN. When a sentence vector is input to this discriminant model, the probability that the sentence is similar to sentence S1 is output.

The first identification unit 152 generates a discriminant model by executing the same processing as for the sentence S1 for the other sentences (S2 to Sn) registered in the set S table 142, and identifies the target sentence identification information, It is registered in the discriminant model table 145 in association with the discriminant model information.

The second identifying unit 153 identifies document information composed of a plurality of sentences among a plurality of pieces of document information contained in the document DB 141 , and registers the identified document information and document identification information in the set M table 143 .

The second identifying unit 153 generates a set M' table 144 based on the set M table 143. FIG. For example, the second identifying unit 153 selects a record of the set M table 143, divides multiple sentences included in the selected record into individual sentences, and assigns document sub-identification information to each sentence. The second specifying unit 153 registers the document identification information, the document sub-identification information, and the document information (one sentence) in the set M′ table 144 in association with each other.

The second identifying unit 153 acquires the discriminant model of the sentence S1 from the discriminant model of each sentence included in the discriminant model table 145 . The second identifying unit 153 applies each piece of document information (one sentence) included in the set M′ table 144 to the discriminant model of the sentence S1, thereby, among the pieces of document information included in the set M′ table 144, Document information similar to sentence S1 is specified.

For example, the second identifying unit 153 inputs the vector of the document information included in the set M′ table 144 to the discriminant model, and if the likelihood value output from the discriminant model is equal to or greater than a threshold, the document information is specified as document information similar to the sentence S1.

The second identifying unit 153 compares the document sub-identification information of the document information similar to the identified sentence S1 with the set M' table 144 to identify the document identification information corresponding to the document sub-identification information. The document identification information obtained is generated as a list L1 (S1).

The second identification unit 153 performs the same processing as for sentence S1 on sentences S2 to Sn to generate lists L1(S2) to L1(Sn).

The exclusion processing unit 154 is a processing unit that excludes sentences unrelated to the failure content from the document information (plural sentences) in the set M table 143 by executing the following processing. For example, the exclusion processing unit 154 performs a process of generating a discriminant model, a process of generating a list L2, a process of setting a deletion flag, and a process of excluding.

The “process for generating a discriminant model” executed by the exclusion processing unit 154 will be described. Here, as an example, the list L1 (S1) will be used for explanation. The exclusion processing unit 154 selects sentences that are not similar to sentence S1 from among multiple sentences included in the document information included in list L1 (S1), and generates a discrimination model for the selected sentences.

FIG. 8 is a diagram (1) for explaining an example of the processing of the exclusion processing unit; As shown in FIG. 8, for example, it is assumed that document identification information MR1, MR2, . Document sub-identification information T11 and T12 are included in the document with document identification information MR1. Assuming that the sentence with the document sub-identification information T11 is similar to the sentence S1, the exclusion processing unit 154 selects the sentence with the document sub-identification information T12 and generates a discrimination model for the selected sentence. The exclusion processing unit 154 also selects sentences that are not similar to the sentence S1 from among the documents with the document identification information MR2, .

As an example, an example of processing for generating a discriminant model of document sub-identification information T11 will be described. In the following description, the document information of the document sub-identification information T12 is referred to as "sentence T12". The exclusion processing unit 154 calculates the degree of similarity between the sentence T12 and each sentence (including the sentence T12) included in the set M′ data, and selects sentences with the highest similarity among the sentences included in the set M′ table. Identify sentences. Similar to the first identification unit 152, the exclusion processing unit 154 uses sentence vectors to calculate the similarity of each sentence.

The exclusion processing unit 154 performs PU learning using the identified high-similarity sentences as “positive examples” and generates a discrimination model for determining whether or not the sentences are similar to the sentence T12. The exclusion processing unit 154 associates the target document identification information for identifying the sentence T12 with the discriminant model information and registers them in the discriminant model table 145 . The PU learning performed by the exclusion processing unit 154 is the same as the PU learning performed by the first identifying unit 152 .

The exclusion processing unit 154 generates a discriminant model by executing the same process as for sentence T11 for other sentences not similar to sentence S1 included in the list L1 (S1), and generates a discriminant model, the target document identification information, It is registered in the discriminant model table 145 in association with the discriminant model information.

The exclusion processing unit 154 generates a discriminant model by executing the same process as for sentence T12 for other sentences not similar to sentences S2 to Sn included in the list L1 (S2 to Sn), and extracts the target document. The identification information and the discriminant model information are associated and registered in the discriminant model table 145 .

Next, the “processing for generating the list L2” executed by the exclusion processing unit 154 will be described. The exclusion processing unit 154 generates a plurality of lists L2 for each list L1 (S1 to Sn). For example, the number of lists L2 corresponding to one list L1 (S1) is the number of sentences not similar to sentence S1 among the sentences of the list L1 (S1).

FIG. 9 is a diagram (2) for explaining an example of the processing of the exclusion processing unit; For example, it is assumed that document identification information MR1, MR2, and MR10 are registered in list L1 (S1). It is also assumed that the document with document identification information MR1 includes document sub-identification information T11 and T12, and the sentence with document sub-identification information T11 is similar to sentence S1.

It is assumed that the document with document identification information MR2 includes document sub-identification information T21 and T22, and the sentence with document sub-identification information T21 is similar to sentence S1. It is assumed that the document with document identification information MR10 includes document sub-identification information T101 and T102, and the sentence with document sub-identification information T101 is similar to sentence S1. In this case, the exclusion processing unit 154 uses list L2 (T12), list L2 (T22), list L2 (T102) based on document sub-identification information T12, T22, and T102 as list L2 corresponding to list L1 (S1). to generate

Here, as an example, a case of generating a list L2 (T12) of sentences (T12) that are not similar to sentence S1 will be described.

The exclusion processing unit 154 acquires the discriminant model of the sentence T12 from the discriminant model of each sentence included in the discriminant model table 145. FIG. The exclusion processing unit 154 applies each piece of document information (single sentence) included in the set M′ table 144 to the discrimination model for the sentence T12, so that among the pieces of document information included in the set M′ table 144, sentence Identify document information similar to T11.

For example, the exclusion processing unit 154 inputs a vector of document information included in the set M′ table 144 to the discriminant model, and if the likelihood value output from the discriminant model is equal to or greater than a threshold, the document information is , as document information similar to the sentence T12.

The exclusion processing unit 154 compares the document sub-identification information of the document information similar to the identified sentence T12 with the set M' table 144, and identifies the document identification information corresponding to the document sub-identification information. Document identification information is generated as a list L2 (T12).

Exclusion processing unit 154 generates list L2 (T22) using the discriminant model of sentence T22 in the same manner as the process for generating list L2 (T12). The exclusion processing unit 154 uses the discriminant model of sentence T102 to generate list L2 (T102).

Also, the exclusion processing unit 154 generates a plurality of lists L2 for each list L1 (S2 to Sn) in the same manner as the process for generating a plurality of lists L2 for the list L1 (S1).

Next, the “deletion flag setting process” executed by the exclusion processing unit 154 will be described. As an example, the process of determining whether or not to exclude the sentence of the document sub-identification information T12 based on the list L1 (S1) and the list L2 (T12) will be described. For example, when the list L1 (S1) and the list L2 (T12) have a large number of pieces of common document information, the exclusion processing unit 154 determines that the sentences S1 and T12 are related and that the sentence T12 should be left.

On the other hand, the exclusion processing unit 154 determines that the sentence T12 is excluded because the sentence S1 and the sentence T12 are not related when the number of pieces of document information common to the list L1 (S1) and the list L2 (T12) is small. . When the exclusion processing unit 154 determines to exclude the sentence T12, it sets the deletion flag of the document sub-identification information “T12” in the set M′ table 144 to “on”. Further, the exclusion processing unit 154 detects other sentences (similar sentences) similar to the sentence T12 from the set M′ table 144 based on the discriminant model of the document sub-identification information “T12”, and Set the deletion flag corresponding to (similar sentence) to "on".

FIG. 10 is a diagram (3) for explaining an example of the processing of the exclusion processing unit; In the example shown in FIG. 10, it is assumed that document information of document identification information "MR1, MR2, . . . , MR10" is registered in the list L1 (S1). It is assumed that list L2 (T12) includes document information with document identification numbers "MR1, MR2, MR3, MR11, . . . , MR20".

Also, the exclusion processing unit 154 compares the list L1 (S1) and the list L2 (T12) to obtain Table 20A. As shown in FIG. 10, it is assumed that the number of pieces of document information included in list L1 (S1) and included in list L2 (T12) is "three". It is assumed that the number of pieces of document information included in the list L1 (S1) and not included in the list L2 (T12) is "7". It is assumed that the number of document information not included in the list L1 (S1) and included in the list L2 (T12) is "10". Assume that the number of document information not included in the list L1 (S1) and not included in the list L2 (T12) is "980".

The exclusion processing unit 154 performs a test (correctness probability test, chi-square test, etc.) on Table 20A to determine whether sentence S1 and sentence T12 are related. For example, in the test for Table 20A, at 5% risk, p-value = 1.96 x 10 ^-4 , p-value values below the threshold determine association. In this case, the exclusion processing unit 154 determines not to exclude sentence T12.

FIG. 11 is a diagram (4) for explaining an example of the processing of the exclusion processing unit; In the example shown in FIG. 11, it is assumed that document information of document identification information "MR1, MR2, . . . , MR10" is registered in the list L1 (S1). It is assumed that list L2 (T102) includes document information with document identification numbers "MR1, MR21, MR22, . . . , MR40".

Also, the exclusion processing unit 154 compares the list L1 (S1) and the list L2 (T102) to obtain Table 20B. As shown in FIG. 11, it is assumed that the number of pieces of document information included in list L1 (S1) and included in list L2 (T102) is "one". Assume that the number of pieces of document information included in the list L1 (S1) and not included in the list L2 (T102) is "9". It is assumed that the number of document information not included in the list L1 (S1) and included in the list L2 (T102) is "20". Assume that the number of document information not included in the list L1 (S1) and not included in the list L2 (T102) is "970".

The exclusion processing unit 154 performs a test (correctness probability test, chi-square test, etc.) on Table 20B to determine whether sentence S1 and sentence T102 are related. In the test for Table 20B, the risk level is 5%, p-value = 0.196, and the value of p-value is equal to or greater than the threshold, so it is determined that there is no association. The exclusion processing unit 154 sets the deletion flag corresponding to the document sub-identification information T102 in the set M' table 144 to "on".

The exclusion processing unit 154 compares the list L1 (S1 to Sn) with the corresponding list L2 to determine whether or not each sentence is related. Repeat the process to turn it on.

Next, the “exclusion processing” executed by the exclusion processing unit 154 will be described. The exclusion processing unit 154 compares the document DB 141 and the set M' table 144, and deletes from the document DB 141 sentences whose deletion flags are "ON".

FIG. 12 is a diagram (5) for explaining an example of the processing of the exclusion processing unit; As shown in FIG. 12, since the deletion flag corresponding to the document sub-identification information T12 is "ON", the exclusion processing unit 154 removes the document sub-identification information from the document information corresponding to the document identification information "MR1". Delete the sentence "Please tell me how to deal with it" corresponding to T12.

Since the deletion flag corresponding to the document sub-identification information T22 is "on", the exclusion processing unit 154 extracts the sentence "handling" corresponding to the document sub-identification information T12 from the document information corresponding to the document identification information "MR2". I don't know how." is deleted.

The exclusion processing unit 154 updates the document DB 141 by repeatedly executing the process of deleting sentences with the deletion flag set to "on" for other document information as well. The updated document DB 141 is referred to as document DB 141a. Here, the exclusion processing unit 154 refers to the document DB 141a, and registers, in the set S table 142, the document information that has become one sentence by the above exclusion processing among the document information contained in the document DB 141a.

Returning to the description of FIG. The detection unit 155 classifies each piece of document information included in the document DB 141a into a plurality of clusters by clustering the document DB 141a. The detection unit 155 detects clusters in which the number of pieces of document information belonging to the cluster is equal to or greater than a predetermined number among the plurality of clusters. A cluster to which more than a predetermined number of pieces of document information belong can be said to be document information describing failure details.

For example, the detection unit 155 calculates a vector of each piece of document information in the document DB 141a. If the document information contains one sentence, the vector of the document information is the vector of such sentence. If the document information contains a plurality of sentences, the vector of the document information is calculated by integrating the vectors of each sentence. The detection unit 155 calculates the degree of similarity between the vectors of each piece of document information, and performs clustering so that the pieces of document information whose degree of similarity is greater than or equal to a threshold belong to the same cluster.

The detection unit 155 may output the detection result to the display unit 130 for display, or may notify an external device of the result via a network. FIG. 13 is a diagram (1) showing an example of a detection result. As shown in FIG. 13, the detection unit 155 generates a plurality of clusters 30A to 30C by clustering the updated document DB 141a. For example, when the number of pieces of document information belonging to cluster 30A is equal to or greater than a predetermined number, detection unit 155 detects cluster 30A as the detection result. When referring to the cluster 30A, the administrator can specify the "syn flood attack" as a frequent failure because the number of document information items related to the "syn flood attack" is equal to or greater than a predetermined number.

By the way, if clustering is performed on the document DB 141 that has not been updated as it is and clusters are detected in the same way, the result shown in FIG. 14 is obtained. FIG. 14 is a diagram (2) showing an example of the detection result. As shown in FIG. 14, the detection unit 155 temporarily clusters the document DB 141 to generate a plurality of clusters 31A to 31C. For example, when the number of pieces of document information belonging to cluster 31A is equal to or greater than a predetermined number, detection unit 155 detects cluster 31A as the detection result. Each piece of document information included in the cluster 31A belongs to the same cluster due to the influence of the sentence "Please tell me the target method" that is not related to the failure content. Since the number of occurrences of each failure is less than the predetermined number, it is determined that there are no multiple failures.

Next, an example of the processing procedure of the information processing apparatus 100 according to this embodiment will be described. 15 and 16 are flowcharts showing the processing procedure of the information processing apparatus according to this embodiment. As shown in FIG. 15, the first specifying unit 152 of the information processing apparatus 100 registers document information composed of one sentence in the set S table 142 among the pieces of document information in the document DB 141 (step S101). The second specifying unit 153 of the information processing apparatus 100 registers document information composed of a plurality of sentences among the document information in the document DB 141 in the set M table 143 (step S102).

The second identifying unit 153 generates the set M' table 144 based on the set M table 143 (step S103). The first identifying unit 152 extracts one sentence (for example, sentence S1) from the set S table 142 (step S104). When all the sentences in the set S table 142 have been extracted (when the extraction is not successful) (step S105, No), the information processing apparatus 100 ends the process.

On the other hand, when the first identification unit 152 succeeds in extracting the sentence from the set S table 142 (step S105, Yes), it generates a sentence discrimination model (step S106). The second identification unit 153 applies the sentence judgment model to the set M' table 144, detects a plurality of pieces of document information including sentences similar to the sentence, and registers them in the list L1 (step S107).

The exclusion processing unit 154 of the information processing apparatus 100 extracts one sentence (for example, T11) other than the similar sentence from the document information of the list L1 (step S108), and proceeds to step S109 in FIG.

16 will be described. If the extraction is successful (step S109, Yes), the exclusion processing unit 154 generates a discriminant model of the sentence extracted from the list L1 (step S110). The exclusion processing unit 154 applies the sentence discrimination model extracted from the list L1 to the set M' table, detects document information including sentences similar to the sentence, and registers them in the list L2 (step S111).

Exclusion processing unit 154 determines whether there is a relationship between the extracted sentences (for example, sentence S1 and sentence T12) based on list L1 and list L2 (step S112). If the sentences are not related (step S113, No), the exclusion processing unit 154 proceeds to step S115.

On the other hand, if the sentences are related (step S113, Yes), the exclusion processing unit 154 turns on the deletion flags corresponding to the sentence extracted from the list L1 and similar sentences (step S114). ).

The exclusion processing unit 154 extracts unselected sentences from the list L1 (step S115), and proceeds to step S109.

By the way, if the extraction is not successful (step S109, No), the exclusion processing unit 154 deletes sentences whose deletion flag is turned on from each piece of document information in the document DB 141 (step S116). The exclusion processing unit 154 adds the document information that has been deleted into one sentence to the set S table 142 (step S117), and proceeds to step S104 in FIG.

Next, the effects of the information processing apparatus 100 according to this embodiment will be described. The information processing apparatus 100 detects a document containing a sentence describing the content of the failure of interest, and among the detected documents, for a document including a plurality of sentences, a sentence related to the content of the failure of interest (a sentence describing the content of the failure) is detected. sentence). In addition, the information processing device performs processing for deleting sentences unrelated to the focused failure content (sentences not describing the failure content). In this way, sentences related to the sentence describing the content of the failure can be left, and sentences not related can be deleted. Therefore, in failure detection by clustering processing, erroneous detection and detection omission can be suppressed.

For example, as described with reference to FIG. 13, by leaving the document information describing the content of the failure and deleting the document information not describing the content of the failure, the document information related to the similar content of the failure can be classified into clusters. This makes it easier to identify multiple failures. As explained with reference to FIG. 14, if document information that does not describe the content of the failure remains, document information that commonly includes document information that does not describe the content of the failure will be classified into the same cluster. It is difficult to detect multiple failures.

In addition, when discriminating other sentences similar to a sentence, the information processing apparatus 100 generates a discrimination model of the sentence based on PU learning, and based on the discrimination model, discriminates other similar sentences. discriminate. This makes it possible to discriminate similar sentences even when there is little teacher data about sentences.

Next, an example of a hardware configuration of a computer that implements the same functions as the information processing apparatus 100 shown in this embodiment will be described. FIG. 17 is a diagram showing an example of the hardware configuration of a computer that implements the same functions as the information processing apparatus according to this embodiment.

As shown in FIG. 17, a computer 500 has a CPU 501 that executes various arithmetic processes, an input device 502 that receives data input from a user, and a display 503 . The computer 500 also has a reading device 504 that reads programs and the like from a storage medium, and an interface device 505 that exchanges data with an external device or the like via a wired or wireless network. The computer 500 has a RAM 506 that temporarily stores various information and a hard disk device 507 . Each device 501 - 507 is then connected to a bus 508 .

The hard disk device 507 has an acquisition program 507a, a first identification program 507b, a second identification program 507c, an exclusion processing program 507d, and a detection program 507e. The CPU 501 reads out the acquisition program 507 a , the first identification program 507 b , the second identification program 507 c , the exclusion processing program 507 d and the detection program 507 e and develops them in the RAM 506 .

Acquisition program 507a functions as acquisition process 506a. The first identification program 507b functions as a first identification process 506b. The second identification program 507c functions as a second identification process 506c. The exclusion processing program 507d functions as an exclusion processing process 506d. Detection program 507e functions as detection process 506e.

The processing of the acquisition process 506 a corresponds to the processing of the acquisition unit 151 . The processing of the first identification process 506 b corresponds to the processing of the first identification unit 152 . The processing of the second identification process 506c corresponds to the processing of the second identification unit 550c. The processing of the exclusion processing process 506 d corresponds to the processing of the exclusion processing unit 154 . The processing of the detection process 506 e corresponds to the processing of the detection unit 155 .

Note that the programs 507a to 507e do not necessarily have to be stored in the hard disk device 507 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), CD-ROM, DVD disk, magneto-optical disk, IC card, etc. inserted into the computer 500 . Then, the computer 500 may read out and execute each of the programs 507a to 507e.

The following additional remarks are disclosed regarding the embodiments including the above examples.

(Appendix 1) A document processing method executed by a computer,
Get multiple documents consisting of one or more sentences,
identifying a first sentence of interest composed of a sentence that satisfies a preset condition from among the plurality of documents;
Obtaining, from among the plurality of documents, a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest;
identifying a second sentence of interest composed of one sentence other than the identified first sentence of interest from among the plurality of acquired first documents;
acquiring, from among the plurality of documents, a plurality of second documents composed of a plurality of sentences including the second sentence of interest;
the second sentence of interest from among the plurality of documents based on the relationship between the number of identical documents included in each of the plurality of first documents and the plurality of second documents and the number of documents other than the same document; A document processing method characterized by performing a process of excluding

(Supplementary note 2) A supplementary note characterized by further executing a process of classifying the plurality of documents excluded from the second sentence of interest into a plurality of clusters based on the degree of similarity between the documents. 1. The document processing method according to 1.

(Appendix 3) The first sentence of interest is a sentence describing the content of the failure, and based on the number of documents belonging to the plurality of clusters, a cluster related to the sentence describing the content of the failure is detected. 3. The method of claim 2, further comprising performing processing.

(Appendix 4) By performing learning using a sentence similar to the first sentence of interest among the plurality of sentences included in the plurality of documents as a positive example, whether or not the sentence is similar to the first sentence of interest is determined. The process of generating a discriminant model for discrimination is further executed, and the process of acquiring the first document is performed based on the discriminant model, from among the plurality of documents, a plurality of sentences including the specified first sentence of interest. 4. The document processing method of claim 1, 2 or 3, comprising obtaining a plurality of structured first documents.

(Appendix 5) to the computer,
Get multiple documents consisting of one or more sentences,
identifying a first sentence of interest composed of a sentence that satisfies a preset condition from among the plurality of documents;
Obtaining, from among the plurality of documents, a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest;
identifying a second sentence of interest composed of one sentence other than the identified first sentence of interest from among the plurality of acquired first documents;
obtaining a plurality of second documents composed of a plurality of sentences including the second sentence of interest from among the plurality of documents;
the second sentence of interest from among the plurality of documents based on the relationship between the number of identical documents included in each of the plurality of first documents and the plurality of second documents and the number of documents other than the same document; A word processing program characterized by causing a process to be executed.

(Supplementary note 6) A supplementary note characterized by further executing a process of classifying the plurality of documents from which the second sentence of interest is excluded by the exclusion process into a plurality of clusters based on the degree of similarity between the documents. 5. The document processing program according to 5.

(Appendix 7) The first sentence of interest is a sentence describing the content of the failure, and based on the number of documents belonging to the plurality of clusters, a cluster related to the sentence describing the content of the failure is detected. 7. The document processing program of Claim 6, further performing processing.

(Appendix 8) By performing learning using a sentence similar to the first sentence of interest among the plurality of sentences included in the plurality of documents as a positive example, whether or not the sentence is similar to the first sentence of interest is determined. The process of generating a discriminant model for discrimination is further executed, and the process of acquiring the first document is performed based on the discriminant model, from among the plurality of documents, a plurality of sentences including the specified first sentence of interest. 8. A document processing program according to Clause 5, 6 or 7, wherein a plurality of structured first documents are obtained.

(Appendix 9) Acquiring a plurality of documents consisting of one or more sentences,
a first identifying unit that identifies a first sentence of interest composed of a sentence that satisfies a preset condition from among the plurality of documents;
Obtaining a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest from among the plurality of documents, and obtaining the specified first sentence of interest from among the plurality of acquired first documents a second identifying unit that identifies a second sentence of interest composed of a sentence other than a sentence;
the second sentence of interest from among the plurality of documents based on the relationship between the number of identical documents included in each of the plurality of first documents and the plurality of second documents and the number of documents other than the same document; and an exclusion processing unit that excludes the information processing apparatus.

(Supplementary Note 10) A supplementary note characterized by further comprising a detection unit that classifies a plurality of documents from which the second sentence of interest is excluded by the exclusion processing unit into a plurality of clusters based on the similarity between the documents. 10. The information processing device according to 9.

(Supplementary Note 11) The first sentence of interest is a sentence describing the content of the failure, and the detection unit detects the sentence describing the content of the failure based on the number of documents belonging to the plurality of clusters. 11. The information processing apparatus according to appendix 10, further executing a process of detecting a cluster that

(Supplementary Note 12) The first identifying unit performs learning using, among a plurality of sentences included in a plurality of documents, sentences similar to the first sentence of interest as positive examples. Further executing a process of generating a discriminant model for discriminating whether or not the sentence includes the first sentence of interest specified from among the plurality of documents based on the discriminant model 12. The information processing apparatus according to appendix 9, 10, or 11, wherein a plurality of first documents composed of a plurality of sentences are obtained.

100 information processing device 110 communication unit 120 input unit 130 display unit 140 storage unit 141 document DB
142 set S table 143 set M table 144 set M' table 145 discrimination model table 150 control unit 151 acquisition unit 152 first identification unit 153 second identification unit 154 exclusion processing unit 155 detection unit

Claims (6)

A computer-implemented document processing method comprising:
Get multiple documents consisting of one or more sentences,
identifying a first sentence of interest composed of a sentence that satisfies a preset condition from among the plurality of documents;
Obtaining, from among the plurality of documents, a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest;
identifying a second sentence of interest composed of one sentence other than the identified first sentence of interest from among the plurality of acquired first documents;
acquiring, from among the plurality of documents, a plurality of second documents composed of a plurality of sentences including the second sentence of interest;
the second sentence of interest from among the plurality of documents based on the relationship between the number of identical documents included in each of the plurality of first documents and the plurality of second documents and the number of documents other than the same document; A document processing method characterized by performing a process of excluding 2. The method according to claim 1, further comprising classifying the plurality of excluded documents of the second sentence of interest into a plurality of clusters based on the similarity between the documents in the excluding processing. document processing method. The first sentence of interest is a sentence describing the content of the failure, and based on the number of documents belonging to the plurality of clusters, a process of detecting a cluster related to the sentence describing the content of the failure is further executed. 3. The document processing method according to claim 2, wherein: A discriminant model that determines whether or not the sentence is similar to the first sentence of interest by performing learning using sentences similar to the first sentence of interest among the plurality of sentences included in the plurality of documents as positive examples. and obtaining the first document, based on the discriminant model, comprises a plurality of sentences including the specified first sentence of interest from among the plurality of documents. 4. A document processing method according to claim 1, 2 or 3, wherein a plurality of first documents are obtained. to the computer,
Get multiple documents consisting of one or more sentences,
identifying a first sentence of interest composed of a sentence that satisfies a preset condition from among the plurality of documents;
Obtaining, from among the plurality of documents, a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest;
identifying a second sentence of interest composed of one sentence other than the identified first sentence of interest from among the plurality of acquired first documents;
acquiring, from among the plurality of documents, a plurality of second documents composed of a plurality of sentences including the second sentence of interest;
the second sentence of interest from among the plurality of documents based on the relationship between the number of identical documents included in each of the plurality of first documents and the plurality of second documents and the number of documents other than the same document; A word processing program characterized by causing a process to be executed. Get multiple documents consisting of one or more sentences,
a first identifying unit that identifies a first sentence of interest composed of a sentence that satisfies a preset condition from among the plurality of documents;
Obtaining a plurality of first documents composed of a plurality of sentences including the specified first sentence of interest from among the plurality of documents, and obtaining the specified first sentence of interest from among the plurality of acquired first documents a second identifying unit that identifies a second sentence of interest composed of a sentence other than a sentence;
Obtaining a plurality of second documents composed of a plurality of sentences including the second sentence of interest from among the plurality of documents, and acquiring the same document included in each of the plurality of first documents and the plurality of second documents and an exclusion processing unit that excludes the second sentence of interest from among the plurality of documents based on the relationship between the number of the documents and the number of documents other than the same document.

Images