JP7473245B2 - Judgment method - Google Patents

Description

The present invention relates to a judgment method, a judgment device, a recording medium, and a learning device.

Sometimes you need to create and submit documents to a client or superior. When creating such documents, a process known as rework may occur, in which the client or superior checks the documents and then makes corrections based on instructions given by the client or superior.

One example of a technique that can be used to prevent rework as described above is, for example, Patent Document 1. Patent Document 1 describes a knowledge accumulation support system that supports knowledge accumulation by classifying and storing messages posted by clients. In the case of the knowledge accumulation support system described in Patent Document 1, messages are classified and stored by topic. In addition, summary information is provided based on the similarity of summary information that consolidates groups of messages by topic.

JP 2002-342347 A

Rework-like processes can occur not only from a formal perspective, such as when necessary information is not included, but also as a result of evaluations made by clients or superiors from an implicit perspective based on the spatial arrangement and composition of the document, such as when there are not enough figures, the flow is poor, or the appearance is not to one's liking. However, in the case of technology such as that described in Patent Document 1, processing is only performed according to the literal perspective, and the above-mentioned implicit perspectives are not taken into account. As a result, it was not possible to make appropriate judgments to prevent the occurrence of rework.

Therefore, the object of the present invention is to provide a judgment method, judgment device, and program that solve the problem of the difficulty in making appropriate judgments to prevent the occurrence of rework.

In order to achieve this object, a determination method according to one aspect of the present invention comprises the steps of:
The determination device,
Obtain the verification documents,
For the acquired confirmation documents, the system estimates any missing items from a formal perspective based on whether or not specified information is included in the confirmation documents, and estimates any missing items from an implicit perspective based on the structure of the documents.

In addition, a determination device according to another aspect of the present invention includes:
The system is configured to have an estimation unit that estimates missing items from a formal perspective based on whether or not specified information is included in the acquired confirmation documents, and also estimates missing items from an implicit perspective, which is an index for evaluating the composition of the documents.

In addition, a recording medium according to another aspect of the present invention includes:
The determination device,
Obtain the verification documents,
This is a computer-readable recording medium having recorded thereon a program for implementing a process of estimating missing items from a formal perspective based on whether or not specified information is contained in the acquired confirmation documents, and estimating missing items from an implicit perspective, which is an indicator for evaluating the composition of the documents.

In addition, a learning device according to another aspect of the present invention includes:
The system is configured to have a learning unit that learns formal viewpoint data for estimating missing items in the confirmation materials from a formal viewpoint corresponding to whether or not specified information is included in the confirmation materials based on the results of the checker's check of the pre-prepared pre-confirmation materials, and also learns implicit viewpoint data for estimating missing items in the confirmation materials from an implicit viewpoint corresponding to the structure of the materials.

By being configured as described above, the present invention makes it possible to provide a judgment method, judgment device, recording medium, and learning device that solve the problem of the difficulty in making appropriate judgments to prevent the occurrence of rework.

FIG. 1 is a diagram for explaining an overview of a determination device. FIG. 2 is a block diagram showing an example of the configuration of a determination device. FIG. 11 is a diagram showing an example of check item data. FIG. 13 is a diagram for explaining an example of advance check materials and check result data. FIG. 13 is a diagram showing an example of material cluster data. FIG. 13 is a diagram for explaining an example of an implicit viewpoint. FIG. 13 is a diagram illustrating an example of implicit viewpoint data. FIG. 13 is a diagram illustrating an example of estimated parameters. FIG. 13 is a diagram for explaining an example of a formal viewpoint. FIG. 13 is a diagram illustrating an example of formal viewpoint data. FIG. 11 is a diagram showing an example of judgment word data. 4 is a flowchart showing an example of the operation of the determination device. 4 is a flowchart showing an example of the operation of the determination device. 4 is a flowchart showing an example of the operation of the determination device. FIG. 11 is a hardware configuration diagram of a determination device according to a second embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of the configuration of a determination device.

[First embodiment]
A first embodiment of the present invention will be described with reference to Figs. 1 to 14. Fig. 1 is a diagram for explaining an overview of a determination device 100. Fig. 2 is a block diagram showing an example of the configuration of the determination device 100. Fig. 3 is a diagram showing an example of check item data 141. Fig. 4 is a diagram showing an example of advance confirmation material/check result data 142. Fig. 5 is a diagram showing an example of material cluster data 143. Fig. 6 is a diagram showing an example of an implicit viewpoint. Fig. 7 is a diagram showing an example of implicit viewpoint data 144. Fig. 8 is a diagram showing an example of estimated parameters 145. Fig. 9 is a diagram showing an example of a formal viewpoint. Fig. 10 is a diagram showing an example of formal viewpoint data 146. Fig. 11 is a diagram showing an example of judgment word data. Figs. 12 and 13 are flowcharts showing an example of the operation of the determination device 100.

In the first embodiment of the present invention, a determination device 100 that determines a perspective (missing matter) that is missing in a confirmation document and outputs a determination result will be described. As shown in FIG. 1, a user who uses the determination device 100 inputs a confirmation document to the determination device 100. Then, the determination device 100 determines whether or not there is a missing perspective in the input confirmation document. For example, the determination device 100 makes a determination from a formal perspective according to whether or not necessary information, such as necessary sentences and topics, is described in the confirmation document. Also, as described later, in addition to making a determination from a formal perspective, the determination device 100 makes a determination from an implicit perspective according to the spatial arrangement such as the balance between the amount of text and the charts and the flow of the document and the configuration of the document. Then, the determination device 100 outputs the result of the determination. For example, the determination device 100 outputs information indicating a missing perspective as a result of the determination.

The judgment device 100 described in this embodiment can also learn parameters that are required when making the above-mentioned judgment. For example, the judgment device 100 accepts the results of a check on the pre-check materials from a checker such as a client or a superior. Then, based on the accepted check results, the judgment device 100 learns and generates parameters that are required when performing the above-mentioned judgment process.

Figure 2 shows an example of the configuration of the determination device 100. Referring to Figure 2, the determination device 100 has, as main components, for example, an operation input unit 110, a screen display unit 120, a communication I/F unit 130, a memory unit 140, and an arithmetic processing unit 160.

The operation input unit 110 is composed of an operation input device such as a keyboard or a mouse. The operation input unit 110 detects operations by a user or a reviewer who uses the determination device 100 and outputs the operations to the calculation processing unit 160.

The screen display unit 120 is composed of a screen display device such as an LCD (Liquid Crystal Display). The screen display unit 120 can display various information stored in the memory unit 140 on the screen in response to instructions from the calculation processing unit 160.

The communication I/F unit 130 is composed of a data communication circuit. The communication I/F unit 130 performs data communication with an external device connected via wireless communication or the like.

The storage unit 140 is a storage device such as a hard disk or memory. The storage unit 140 stores processing information and a program 150 necessary for various processes in the calculation processing unit 160. The program 150 is read into the calculation processing unit 160 and executed to realize various processing units. The program 150 is read in advance from an external device or a recording medium via a data input/output function such as the communication I/F unit 130, and is stored in the storage unit 140. Main information stored in the storage unit 140 includes, for example, check item data 141, pre-check material/check result data 142, material cluster data 143, implicit viewpoint data 144, estimated parameters 145, formal viewpoint data 146, judgment word data 147, confirmation material 148, material data 149, etc.

The check item data 141 indicates check items that are checked by a checker such as a client or a superior when checking pre-check materials prepared in advance. FIG. 3 shows an example of the check item 141 data. Referring to FIG. 3, the check item data 141 includes a plurality of check items such as Q1, Q2, Q3, .... For example, the check items are items that can be answered with a binary option such as yes or no. The check item data 141 is created in advance and stored in the memory unit 140. The check item data 141 may be created, added to, edited, etc. when the checker checks the pre-check materials.

The check item data 141 can have check items for each cluster generated by dividing each document, such as the above-mentioned pre-confirmation documents and confirmation documents 148, into multiple groups. The check item data 141 may have check items common to each cluster. The check item data 141 may also have check items common to an entire document, or may have check items for each page when the document is divided into pages, or for each group when the document is grouped by heading or content, etc.

The pre-check materials and check result data 142 indicates pre-check materials created in advance and the results of a check of the pre-check materials by a checker such as a client or a superior based on the check item data 141. The pre-check materials included in the pre-check materials and check result data 142 may, for example, have been created in advance and stored in the memory unit 140, or may be obtained from an external device operated by the checker and stored in the memory unit 140. The check results included in the pre-check materials and check result data 142 may be received from an external device operated by the checker, or obtained by the checker operating the operation input unit 110 or the like, and stored in the memory unit 140.

Figure 4 is a diagram for explaining an example of the check results included in the pre-check materials and check result data 142. Referring to Figure 4, the check results included in the pre-check materials and check result data 142 indicate the results of the checker's check of each check item included in the check item data 141. In other words, for example, the check results included in the pre-check materials and check result data 142 indicate the results of the checker's check of the multiple-choice questionnaire indicated in the check item data 141.

In addition, the pre-check material/check result data 142 includes information indicating the check result as well as information indicating the completeness of the material. Here, the completeness of the material is a value indicating the structural completeness of the material according to the spatial arrangement, etc. As an example, the completeness of the material can take any value between 0 and 1. For example, the closer the completeness of the material is to 0, the lower the completeness, and the closer it is to 1, the higher the completeness. The completeness of the material can be a value that reflects the preferences of the person checking it, such as "all the check items are there but it's difficult to read." In other words, the completeness of the material can be an index for evaluating the composition of the material and an index for evaluating the viewpoints and preferences of the person checking it.

As described above, the pre-check materials included in the pre-check materials/check result data 142 are, for example, created in advance and stored in the memory unit 140. For example, the pre-check materials are materials that have actually been created in the past. It is desirable that the pre-check materials include a variety of materials, such as materials that have been created intentionally to have a low level of completeness.

Furthermore, when the pre-check materials are checked by the checker, for example, each page is checked when the pre-check materials are divided into individual pages. Therefore, the pre-check materials/check result data 142 includes the check results for each check item for each page and the degree of completeness of the materials. The checker may check the pre-check materials as a whole, or may check each group of pre-check materials when they are grouped by heading or content. In this case, the pre-check materials/check result data 142 will include the check results and the degree of completeness of the materials according to the object of the check.

The document cluster data 143 indicates the classification results when the pre-check documents are classified into multiple clusters. For example, the pre-check documents are classified into multiple clusters by the cluster classification unit 162, which will be described later, using a document clustering method such as the group average method of hierarchical clustering. The document cluster data 143 includes information indicating the results of the classification by the cluster classification unit 162.

Figure 5 shows an example of the document cluster data 143. Referring to Figure 5, in the document cluster data 143, for example, document identification information, a cluster, and a classification degree are associated with each other. Here, the document identification information is information such as an identification code for identifying the pre-check document. The document identification information is assigned in advance to each pre-check document. Furthermore, the cluster indicates the cluster to which the pre-check document belongs. Furthermore, the classification degree is an index for evaluating the likelihood of the cluster to which the document belongs. For example, the classification degree is a value according to the distance to the center point of the cluster, and is calculated by the cluster classification unit 162, etc.

The implicit perspective data 144 indicates values corresponding to implicit perspectives, such as the balance between the amount of text and figures and tables, and the flow of materials. The values indicated by the implicit perspective data 144 can be used when the judgment estimation unit 167 described below estimates a missing perspective. The implicit perspective data 144 is generated by the implicit perspective learning unit 163 or the like, and stored in the memory unit 140.

Here, an example of an implicit viewpoint will be described with reference to Fig. 6. Fig. 6 is a diagram for explaining an example of an implicit viewpoint. As shown in Fig. 6, the implicit viewpoint includes at least one of a spatial element el _s and a compositional element el _c .

The spatial element el _s is, for example, a value indicating the balance between the amount of text and the amount of charts and diagrams. The spatial element el _s is, for example, the document completeness per _s . As an example, the document completeness per _s , which is the spatial element el _s , can be expressed by the following formula.
per _s = am + bn + c
Here, "per _s " in the above formula indicates the degree of completion of the document. The degree of completion of the document per _s is the same as the degree of completion of the document included in the advance confirmation document/check result data 142. Furthermore, "m" in the above formula is the amount of text written on one page of the document. "m" may be the amount of text written on the entire document, etc. Furthermore, "n" in the above formula is the number of figures and tables on one page of the document. "n" may be the number of figures and tables in the entire document, etc. Furthermore, "a", "b", and "c" are weight parameters determined by the results of learning by the implicit viewpoint learning unit 163, which will be described later. Details of learning by the implicit viewpoint learning unit 163 will be described later.

The constituent element el _c indicates, for example, an evaluation value of the table of contents structure of the material (for example, a value corresponding to the flow of the material). The constituent element el _c is, for example, a structure evaluation value eva. As an example, the structure evaluation value eva, which is the constituent element el _c , can be expressed by the formula shown in the following formula 1.
Here, i indicates a page in the document. Also, n is a value according to, for example, the number of pre-check documents included in the cluster. As shown in Equation 1, the composition evaluation value EVA can be calculated from a value indicating the flow of document completeness in a certain pre-check document and a value indicating whether the value indicating the flow of completeness is higher than the average. In other words, it can be said that the composition evaluation value EVA is a value according to the relationship of document completeness per _s between each page in a document such as a pre-check document. For example, the composition evaluation value EVA becomes better as the document completeness per _s is constant.

For example, the implicit viewpoint data 144 has a value indicating the spatial element el _s and a value indicating the constituent element el _c as described above. FIG. 7 is an example of the implicit viewpoint data 144. Referring to FIG. 7, in the implicit viewpoint data 144, for example, a value indicating the spatial element el _s and a value indicating the constituent element el _c are expressed in a matrix format. For example, in the case shown in FIG. 7, each row of the matrix corresponds to a cluster. Also, in the case shown in FIG. 7, each column of the matrix indicates the spatial element el _s and the constituent element el _c . For example, the second row of FIG. 7 indicates the spatial element el _s and the constituent element el _c in cluster 2. For example, the second row of FIG. 7 may be the average value of the values indicating the spatial element el _s of each advance confirmation material belonging to cluster 2 and the average value of the values indicating the constituent element el _c .

The implicit viewpoint data 144 may have a matrix for each page, for example. The implicit viewpoint data 144 may have a matrix for each grouping of the pre-check materials, such as a heading or content, or for the entire pre-check materials.

In addition, the implicit viewpoint data 144 may include the material completeness per _s and the configuration evaluation value eva as shown in FIG. 7, or may include only one of the material completeness per _s and the configuration evaluation value eva.

The estimated parameters 145 indicate the values of "a", "b", and "c" described above, which are weight parameters determined by the results of learning by the implicit viewpoint learning unit 163 described below. The estimated parameters 145 are generated by the implicit viewpoint learning unit 163 and stored in the memory unit 140.

Figure 8 shows an example of estimated parameters 145. Referring to Figure 8, estimated parameters 145 indicate values of weight parameters "a", "b", "c" etc. learned in each cluster. For example, the second line in Figure 8 indicates that the value of "a" in cluster 2 is "0.1", the value of "b" is "2", ...

The formal perspective data 146 indicates a value corresponding to a formal perspective, such as indicating whether or not predetermined information is included in the pre-confirmation materials. The value indicated by the formal perspective data 146 can be used when the judgment estimation unit 167 described later estimates a missing perspective. The formal perspective data 146 is generated by the formal perspective learning unit 164 or the like, and is stored in the memory unit 140.

Figure 9 is an example of formal viewpoint data 146. Referring to Figure 9, in formal viewpoint data 146, whether or not each check item indicated by check item 141 is necessary is expressed in a matrix format. For example, in the case shown in Figure 9, each row of the matrix corresponds to a cluster. Also, in the case shown in Figure 9, each column of the matrix corresponds to each check item indicated by check item 141. For example, the second row of Figure 9 indicates whether or not each check item (Q1, Q2, ...) is necessary in cluster 2. For example, in the case of Figure 9, when an element of the matrix is "1", it indicates that the corresponding check item is necessary. Also, in the case of Figure 9, when an element of the matrix is "0", it indicates that the corresponding check item is not necessary.

Here, each element in the matrix can be calculated using, for example, the formula shown in FIG. 10 and the following formula 2.
Note that x _Q1 corresponds to each check item indicated by the check items 141. Also, slct is "1" if it indicates that the corresponding check item is "present" in the pre-check materials and check result data 142, and is "0" if it indicates that it is not present. Also, clf is the value of the classification level of the pre-check materials. Also, per is the value of the material completeness. Also, n is a value according to, for example, the number of pre-check materials included in a cluster.

In addition, the formal viewpoint data 146 can have a matrix for each page, similar to the implicit viewpoint data 144. The formal viewpoint data 146 may also be a matrix for each group of pre-check materials grouped by heading or content, or for the entire document.

The judgment word data 147 indicates words, sentences, etc. that are necessary to determine that each check item indicated by the check item 141 is "present." The judgment word data 147 is, for example, determined in advance. The judgment word data 147 may be configured to be extracted and modified as appropriate based on the frequency of words present in the pre-check materials. In addition, the judgment word data 147 may be added or modified, for example, when the reviewer reviews the pre-check materials.

Figure 11 shows an example of the judgment word data 147. Referring to Figure 11, the judgment word data 147 shows words for determining whether each cluster and each check item is "yes". For example, the second line in Figure 11 shows that the words for determining whether cluster 2 and check item "Q1" are "forecast, sales, ...", and the words for determining whether check item "Q2" is "yes" are "budget, investment, ...", ....

The confirmation materials 148 indicate the confirmation materials entered by the user. The confirmation materials will be subject to judgment by the judgment device 100.

The material data 149 indicates the results of digitization by the material digitization unit 166, described below, based on the confirmation material indicated by the confirmation material 148. As described below, the material digitization unit 166 performs digitization from a formal perspective and from an implicit perspective. Therefore, the material data 149 includes the results of digitizing the confirmation material from a formal perspective and the results of digitizing the confirmation material from an implicit perspective. Details of the processing by the material digitization unit 166 will be described below.

The calculation processing unit 160 has a microprocessor such as an MPU and its peripheral circuits. The calculation processing unit 160 reads and executes the program 150 from the storage unit 140, thereby implementing various processing units by having the above hardware and the program 150 work together. The main processing units implemented by the calculation processing unit 160 include, for example, a data acquisition unit 161, a cluster classification unit 162, an implicit viewpoint learning unit 163, a formal viewpoint learning unit 164, a cluster classification unit 165, a material digitization unit 166, a judgment and estimation unit 167, and an output unit 168.

Among the above-mentioned processing units, the cluster classification unit 162, the implicit perspective learning unit 163, and the formal perspective learning unit 164 function as learning units used when performing learning according to the pre-confirmation materials. The cluster classification unit 165, the material data conversion unit 166, and the judgment estimation unit 167 function as judgment units used when performing judgment according to the confirmation materials.

The data acquisition unit 161 acquires data from an external device or the like. For example, the data acquisition unit 161 acquires check results indicating the results of the check on the pre-check materials from an external device or the like operated by the checker. The data acquisition unit 161 then associates the acquired check results with the corresponding pre-check materials (or identification information, etc.) and stores them in the memory unit 140 as pre-check material/check result data 142. The data acquisition unit 161 also acquires check materials from an external device or the like operated by the user. The data acquisition unit 161 then stores the acquired data in the memory unit 140 as check materials 148.

The cluster classification unit 162 performs cluster classification of the pre-check materials. For example, the cluster classification unit 162 performs cluster classification of the pre-check materials using a document clustering method such as the group average method of hierarchical clustering. The cluster classification unit 162 also performs cluster classification based on the distance between each document as described above, and calculates the classification degree based on the distance between the center point of the cluster and the document. The cluster classification unit 162 then stores the classification results and the calculated classification degree in the memory unit 140 as material cluster data 143.

Cluster classification by the cluster classification unit 162 can be performed, for example, when the pre-check materials are stored in the storage unit 140. Cluster classification by the cluster classification unit 162 may be performed in situations other than those exemplified above. Furthermore, the cluster classification unit 162 may be configured to perform cluster classification and calculate the classification degree using a known method other than those described above.

The implicit viewpoint learning unit 163 performs learning from the above-mentioned implicit viewpoint. For example, the implicit viewpoint learning unit 163 calculates the average value of the material completeness per _s included in the pre-check material and check result data 142 for each page. Furthermore, the implicit viewpoint learning unit 163 calculates a configuration evaluation value EVA based on the material completeness per _s included in the pre-check material and check result data 142, and calculates the average value of the calculated configuration evaluation values EVA. Then, the implicit viewpoint learning unit 163 generates each element of the implicit viewpoint data 144 based on the calculation result. Note that the implicit viewpoint learning unit 163 may generate each element of the implicit viewpoint data 144 by statistical processing other than the average value.

The implicit viewpoint learning unit 163 performs the above-mentioned generation process of the implicit viewpoint data 144 for each cluster. The implicit viewpoint learning unit 163 performs the generation process of each element of the implicit viewpoint data 144 corresponding to each cluster, thereby generating a matrix as shown in FIG. 7.

Furthermore, the implicit viewpoint learning unit 163 learns the values of the weight parameters "a", "b", and "c" for each cluster and for each page. For example, the implicit viewpoint learning unit 163 calculates and learns the values of the weight parameters "a", "b", and "c" by performing a multiple regression analysis for each page of each cluster based on the amount of text and the number of figures and tables included in the pre-check material and the material completeness per _s included in the pre-check material/check result data 142. Then, the implicit viewpoint learning unit 163 stores the learned results in the storage unit 140 as estimated parameters 145.

The formal viewpoint learning unit 164 performs learning from the above-mentioned formal viewpoint. For example, the formal viewpoint learning unit 164 calculates whether each check item is necessary for each page based on the check result (whether or not the checker judges that there is a check item) included in the pre-check material/check result data 142 and the formula shown in the above-mentioned formula 2. In other words, the formal viewpoint learning unit 164 calculates whether each check item is necessary by substituting the multiple-choice answer result indicated by the check result, the degree of material completeness, the classification degree indicated by the material cluster data 143, and the like into the formula shown in the above-mentioned formula 2. Then, the formal viewpoint learning unit 164 generates each element of the formal viewpoint data 146 based on the calculated result.

The formal perspective learning unit 164 performs the above-mentioned generation process of the formal perspective data 146 for each cluster. The formal perspective learning unit 164 performs the generation process of each element of the formal perspective data 146 corresponding to each cluster, thereby generating a matrix as shown in FIG. 9.

In addition, the formal perspective learning unit 164 can add or modify the judgment word data 147 based on the check results indicated by the pre-check materials and check result data 142, the pre-check materials, etc. For example, the formal perspective learning unit 164 can add or modify the judgment word data 147 based on the frequency of words in the pre-check materials, etc. The addition or modification of the judgment word data 147 by the formal perspective learning unit 164 may be achieved using known technology.

The cluster classification unit 165 performs cluster classification of the confirmation materials input by the user (confirmation materials stored in the storage unit 140 as confirmation materials 148). For example, similar to the cluster classification unit 162, the cluster classification unit 165 uses a document clustering method such as the group average method of hierarchical clustering to determine to which of the clusters included in the document cluster data 143 the confirmation materials belong.

The material digitization unit 166 digitizes the confirmation materials using the results of the classification by the cluster classification unit 165. In the present embodiment, the material digitization unit 166 digitizes the confirmation materials from a formal perspective and an implicit perspective.

For example, the material data conversion unit 166 calculates the material completeness per _s and the composition evaluation value EVA as data conversion from an implicit viewpoint. For example, the material data conversion unit 166 refers to the estimated parameters 145 and acquires the weight parameters (i.e., "a", "b", "c") of the cluster corresponding to the result of classification by the cluster classification unit 165 for each page (or each group, etc.) of the confirmation material. Then, the material data conversion unit 166 calculates the material completeness per s by substituting the acquired weight parameters and the amount of written characters and the number of figures and tables in the confirmation material into the material completeness per _s = am + bn + c. In addition, the material data conversion unit 166 calculates the composition evaluation value _EVA for each page by substituting the calculated material completeness per _s into the above-mentioned formula 1.

The document data conversion unit 166 also converts each page of the confirmation document into data from a formal perspective by checking whether the words indicated by the judgment word data 147 are present in the confirmation document. For example, the document data conversion unit 166 refers to the judgment word data 147 to obtain words for determining that each check item is "yes" that correspond to the results of classification by the cluster classification unit 165. If the words for determining that a check item is "yes" are present in the confirmation document, the document data conversion unit 166 judges the check item as being achieved as "1". On the other hand, if the words for determining that a check item is "yes" are not present in the confirmation document, the document data conversion unit 166 judges the check item as not being achieved as "0". For example, the document data conversion unit 166 converts each page and each check item into data from a formal perspective by performing the above-mentioned process.

The judgment and estimation unit 167 estimates the perspectives that are missing in the confirmation materials by comparing the results digitized by the material digitization unit 166 with the implicit perspective data 144, the formal perspective data 146, etc.

For example, the judgment estimation unit 167 refers to the implicit viewpoint data 144 and the formal viewpoint data 146 to identify each element in the matrix corresponding to the result of classification by the cluster classification unit 165. Then, the judgment estimation unit 167 compares the identified result with the result digitized by the material digitization unit 166 to estimate the viewpoint that is lacking in the confirmation material. For example, when a check item that is "1" in the formal viewpoint data 146 is "0" in the result digitized by the material digitization unit 166 from a formal viewpoint, the judgment estimation unit 167 judges that the check item is lacking. Also, for example, when the result digitized by the material digitization unit 166 from an implicit viewpoint is lower than the material completeness per _s or the configuration evaluation value EVA in the implicit viewpoint data 144, the judgment estimation unit 167 judges that the lowered material completeness per _s or the configuration evaluation value EVA is lacking. For example, as described above, the judgment and estimation unit 167 compares the results of data conversion by the material digitization unit 166 with the implicit viewpoint data 144, the formal viewpoint data 146, etc., to estimate the formal viewpoints and implicit viewpoints that are missing in the confirmation materials.

The judgment estimation unit 167 may be configured to adjust the document completeness per _s and the configuration evaluation value eva in the implicit viewpoint data 144 according to a predetermined criterion and then perform the above-mentioned comparison. For example, the criterion from the implicit viewpoint may differ between a situation where there is a relatively large margin in the delivery date and a situation where the delivery date is short. Therefore, when a predetermined condition is satisfied, such as the deadline until the delivery date being below a predetermined value, the judgment estimation unit 167 can perform the above-mentioned comparison after lowering the document completeness per _s and the configuration evaluation value eva, for example by multiplying the document completeness per _s and the configuration evaluation value eva by 0.8. Furthermore, when a predetermined condition is satisfied, such as the deadline until the delivery date being above a predetermined value, the judgment estimation unit 167 may perform the above-mentioned comparison after raising the document completeness per _s and the configuration evaluation value eva, for example by multiplying the document completeness per _s and the configuration evaluation value eva by 1.2. The judgment estimation unit 167 may perform adjustments other than those exemplified above.

The output unit 168 performs output according to the result of the estimation by the judgment estimation unit 167. For example, the output unit 168 outputs the formal perspectives and implicit perspectives that are lacking in the confirmation materials, which are identified as the result of the estimation. For example, the output unit 168 displays the formal perspectives and implicit perspectives that are lacking in the confirmation materials on the screen display unit 120, or transmits them to an external device owned by the user. The output unit 168 may also output other than the above examples, such as perspectives that are not lacking.

The above is an example of the configuration of the determination device 100. Next, an example of the operation of the determination device 100 will be described with reference to Figures 12 to 14. First, with reference to Figure 12, an example of the operation when the determination device 100 receives pre-check materials will be described.

12, the determination device 100 receives pre-check materials from an external device or the like via the communication I/F unit 130 or the like (step S101). The determination device 100 then stores the received pre-check materials in the memory unit 140 as pre-check materials/check result data 142.

The cluster classification unit 162 performs cluster classification of the pre-check materials (step S102). For example, the cluster classification unit 162 performs cluster classification of the pre-check materials using a document clustering method such as the group average method of hierarchical clustering. The cluster classification unit 162 also performs cluster classification based on the distance between each document as described above, and calculates the classification degree based on the distance between the center point of the cluster and the document. The cluster classification unit 162 then stores the classification results and the calculated classification degree in the memory unit 140 as material cluster data 143.

The above is an example of the operation when the advance confirmation materials are received. Note that the cluster classification by the cluster classification unit 162 may be performed at a timing other than the above example. Next, with reference to FIG. 13, an example of the operation when learning is performed according to the results of the confirmation by the confirmer will be described.

13, the implicit viewpoint learning unit 163 performs learning from an implicit viewpoint (step S201). For example, the implicit viewpoint learning unit 163 performs statistical processing on the material completeness per _s included in the check result for each pre-check material in the cluster, or performs statistical processing on the configuration evaluation value eva calculated based on the material completeness per _s , thereby generating each element of the implicit viewpoint data 144. In addition, the implicit viewpoint learning unit 163 performs generation processing of each element of the implicit viewpoint data 144 for each cluster. In this way, the implicit viewpoint learning unit 163 generates the implicit viewpoint data 144 expressed as a matrix. In addition, the implicit viewpoint learning unit 163 learns the values of weight parameters "a", "b", and "c" for each cluster and for each page. For example, the implicit viewpoint learning unit 163 calculates and learns the values of the weight parameters "a", "b", and "c" by performing a multiple regression analysis for each page of each cluster based on the amount of text and the number of figures and tables contained in the pre-check materials and the document completeness per _s contained in the pre-check materials/check result data 142.

The formal perspective learning unit 164 also performs learning from a formal perspective (step S202). For example, the formal perspective learning unit 164 calculates whether each check item is necessary by substituting the multiple choice answer result indicated by the check result, the degree of completeness of the material, the classification degree indicated by the material cluster data 143, and the like into the formula shown in the above-mentioned formula 2. Then, the formal perspective learning unit 164 generates each element of the formal perspective data 146 based on the calculated result. The formal perspective learning unit 164 also performs a generation process of each element of the above-mentioned formal perspective data 146 for each cluster. As a result, the formal perspective learning unit 164 generates the formal perspective data 146 represented by a matrix.

In addition, the formal perspective learning unit 164 may add or modify the judgment word data 147 based on the check results indicated by the pre-check materials/check result data 142, the pre-check materials, etc.

The above is an example of the operation when learning is performed based on the results of the verification by the verifying person. Next, with reference to FIG. 14, an example of the operation when the verification device 100 makes a judgment on the verification materials entered by the user will be described.

Referring to Figure 15, the user inputs confirmation materials to the determination device 100 via the communication I/F unit 130 or the like (step S301).

The cluster classification unit 165 performs cluster classification of the confirmation documents input by the user (step S302). For example, the cluster classification unit 165 uses a document clustering method such as the group average method of hierarchical clustering to determine to which of the clusters included in the document cluster data 143 the confirmation documents belong, in the same way as when performing cluster classification of the pre-confirmation documents.

The material digitization unit 166 digitizes the confirmation material using the results of the classification by the cluster classification unit 165 (step S303). In this embodiment, the material digitization unit 166 digitizes the confirmation material from a formal perspective and an implicit perspective. For example, the material digitization unit 166 digitizes the confirmation material from an implicit perspective using the estimated parameters 145. The material digitization unit 166 also digitizes the confirmation material from a formal perspective using the judgment word data 147.

The judgment and estimation unit 167 compares the results digitized by the material digitization unit 166 with each element in a matrix such as the implicit viewpoint data 144 or the formal viewpoint data 146 to estimate the viewpoints that are missing in the confirmation materials (step S304).

The output unit 168 performs output according to the result of the estimation by the judgment estimation unit 167 (step S305). For example, the output unit 168 outputs a formal perspective or an implicit perspective that is identified as a result of the estimation and is lacking in the confirmation materials.

The above is an example of the operation of the judgment device 100 when making a judgment on confirmation materials.

In this way, the judgment device 100 has a cluster classification unit 165, a material data conversion unit 166, and a judgment estimation unit 167. With this configuration, the material data conversion unit 166 can use the results of classification by the cluster classification unit 165 to convert data from an implicit perspective, as well as convert data from a formal perspective. As a result, the judgment estimation unit 167 can estimate the perspectives that are missing from the confirmation materials from a formal perspective, as well as the perspectives that are missing from the confirmation materials from an implicit perspective. In this way, by taking into account not only the formal perspective but also the implicit perspective, it is possible to make a more appropriate judgment to suppress the occurrence of rework.

The determination device 100 also has an implicit perspective learning unit 163 and a formal perspective learning unit 164. In this way, by having the implicit perspective learning unit 163 and the formal perspective learning unit 164, it becomes possible to learn in advance the parameters and the like required when estimating a missing perspective from an implicit perspective using the material data conversion unit 166 and the judgment estimation unit 167, etc. As a result, it becomes possible to perform the processing as described above, and it becomes possible to more appropriately make a judgment to suppress the occurrence of rework.

In this embodiment, the implicit viewpoint data 144 and the formal viewpoint data 146 are represented in a matrix format. However, the implicit viewpoint data 144 and the formal viewpoint data 146 may be represented in a format other than a matrix, such as a table format.

In addition, in this embodiment, an example is given of a case where the functions of the determination device 100 are realized by one information processing device. However, the functions of the determination device 100 may be realized by, for example, multiple information processing devices connected via a network.

Second Embodiment
Next, a second embodiment of the present disclosure will be described with reference to Fig. 15 and Fig. 16. In the second embodiment, an overview of the configuration of a determination device 200 will be described.

Fig. 15 shows an example of a hardware configuration of the determination device 200. Referring to Fig. 15, the determination device 200 has, as an example, the following hardware configuration.
CPU (Central Processing Unit) 201 (arithmetic device)
ROM (Read Only Memory) 202 (storage device)
RAM (Random Access Memory) 203 (storage device)
Program group 204 loaded into RAM 203
A storage device 205 for storing the programs 204
A drive device 206 that reads and writes data from and to a recording medium 210 outside the information processing device.
A communication interface 207 that connects to a communication network 211 outside the information processing device
Input/output interface 208 for inputting and outputting data
A bus 209 that connects each component

In addition, the determination device 200 can realize the function of the estimation unit 221 shown in Fig. 16 by the CPU 201 acquiring the program group 204 and executing it. The program group 204 is stored in advance in the storage device 205 or ROM 202, for example, and is loaded into the RAM 203 or the like by the CPU 201 as necessary for execution. The program group 204 may be supplied to the CPU 201 via the communication network 211, or may be stored in advance in the recording medium 210, and the drive device 206 may read out the programs and supply them to the CPU 201.

Note that FIG. 15 shows an example of the hardware configuration of the determination device 200. The hardware configuration of the determination device 200 is not limited to the above-described case. For example, the determination device 200 may be configured from a part of the above-described configuration, such as not having the drive device 206.

The estimation unit 221 estimates missing items from a formal perspective based on whether or not predetermined information is included in the confirmation documents acquired from an external device, etc. The estimation unit 221 also estimates missing items from an implicit perspective, which is an index for evaluating the composition of the documents, in the confirmation documents.

In this way, the judgment device 200 has an estimation unit 221. With this configuration, the estimation unit 221 can estimate the missing items from a formal perspective and also estimate the missing items from an implicit perspective. As a result, it becomes possible to make more appropriate judgments to prevent rework from occurring.

The above-mentioned determination device 200 can be realized by incorporating a predetermined program into the determination device 200. Specifically, the program, which is another form of the present invention, is a program for causing the determination device 100 to realize a process of acquiring confirmation documents, estimating missing items from a formal perspective based on whether or not predetermined information is described in the confirmation documents, and estimating missing items from an implicit perspective, which is an index for evaluating the composition of the documents.

In addition, the judgment method realized by the above-mentioned judgment device 200 is a method in which the judgment device 100 acquires confirmation documents, estimates missing items from a formal perspective based on whether or not specified information is described in the acquired confirmation documents, and estimates missing items from an implicit perspective based on the structure of the documents.

Even if the invention is a program (or recording medium) or a judgment method having the above-mentioned configuration, it has the same action and effect as the above-mentioned judgment device 200, and therefore can achieve the above-mentioned object of the present invention.

<Additional Notes>
A part or all of the above-described embodiments can be described as follows. The following provides an overview of the determination method and the like of the present invention. However, the present invention is not limited to the following configuration.

(Appendix 1)
The determination device,
Obtain the verification documents,
A determination method for estimating missing items from a formal perspective based on whether or not specified information is contained in the acquired confirmation documents, and for estimating missing items from an implicit perspective based on the structure of the documents.
(Appendix 2)
2. The determination device according to claim 1,
A method of assessment in which a document completeness level, which is a value indicating the balance between the amount of text and the number of figures and tables, is calculated from the confirmation document using pre-trained parameters, and deficiencies are estimated from an implicit perspective based on the calculated result.
(Appendix 3)
3. The determination device according to claim 2,
The degree of completion of the document is calculated using the parameters, the amount of text written in the confirmation document, and the number of figures and tables.
(Appendix 4)
The determination device according to claim 2 or 3,
A method for estimating deficiencies from an implicit perspective by comparing the calculated document completeness with a value corresponding to the results of a prior check by a checker.
(Appendix 5)
The determination device according to any one of Supplementary Note 2 to Supplementary Note 4,
A method for determining whether or not a document is complete, the method comprising: calculating a composition evaluation value, which is a value corresponding to the flow of the document, based on the calculated document completeness; and estimating deficiencies from an implicit perspective based on the calculated result.
(Appendix 6)
6. The determination device according to claim 5,
A determination method in which the calculated configuration evaluation value is compared with a value corresponding to a result of prior confirmation by a checker, thereby estimating deficiencies from an implicit perspective.
(Appendix 7)
The method according to any one of claims 1 to 6,
The method includes determining whether or not a predetermined judgment word is present in the verification material, thereby estimating deficiencies from a formal standpoint.
(Appendix 8)
The method according to any one of claims 1 to 7,
A method for determining which cluster the confirmation material belongs to, and using the result of the determination to estimate missing items from a formal perspective and also to estimate missing items from an implicit perspective, which is an index for evaluating the composition of the material.
(Appendix 9)
A determination device having an estimation unit that estimates missing items from a formal perspective based on whether or not specified information is included in the acquired confirmation documents, and that estimates missing items from an implicit perspective, which is an index for evaluating the composition of the documents.
(Appendix 10)
The determination device,
Obtain the verification documents,
A program for implementing a process of estimating missing items from a formal perspective based on whether or not specified information is contained in the acquired confirmation documents, and estimating missing items from an implicit perspective, which is an indicator for evaluating the composition of the documents.
(Appendix 11)
A learning device having a learning unit that learns formal viewpoint data for estimating missing items in the confirmation materials from a formal viewpoint according to whether or not specified information is included in the confirmation materials based on the results of a checker checking pre-prepared pre-confirmation materials, and learns implicit viewpoint data for estimating missing items in the confirmation materials from an implicit viewpoint according to the structure of the materials.

The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium. For example, the recording medium may be a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This invention claims the benefit of priority based on patent application No. 2020-109091, filed in Japan on June 24, 2020, and all contents of that patent application are incorporated herein by reference.

100 Determination device 110 Operation input unit 120 Screen display unit 130 Communication I/F unit 140 Memory unit 141 Check item data 142 Advance confirmation material/check result data 143 Material cluster data 144 Implicit viewpoint data 145 Estimated parameters 146 Formal viewpoint data 147 Determination word data 148 Confirmation material 149 Material data 150 Program 160 Calculation processing unit 161 Data acquisition unit 162 Cluster classification unit 163 Implicit viewpoint learning unit 164 Formal viewpoint learning unit 165 Cluster classification unit 166 Material data unit 167 Judgment estimation unit 168 Output unit 200 Determination device 201 CPU
202 ROM
203 RAM
204 Program group 205 Storage device 206 Drive device 207 Communication interface 208 Input/output interface 209 Bus 210 Recording medium 211 Communication network 221 Estimation unit

Claims (4)

The determination device,
Obtain the verification documents,
For the acquired confirmation materials, the previously learned parameters and the amount of text and the number of figures and tables in the acquired confirmation materials are substituted into a predetermined formula to calculate a document completeness, which is a value indicating the balance between the amount of text and the number of figures and tables, and the calculated document completeness is substituted into the formula shown in Equation 1 to calculate a composition evaluation value, which is a value according to the relationship between the document completeness, thereby converting the document into data from an implicit perspective according to the composition of the document, and by checking whether predetermined judgment words are present in the confirmation materials and judging whether each of the predetermined check items has been achieved, the data is converted into data from a formal perspective;
The system estimates deficiencies from an implicit perspective according to the structure of the document by comparing the results of the data conversion from an implicit perspective with the document completeness and structure evaluation value calculated in advance according to the results of the check made by the checker, and estimates deficiencies from a formal perspective according to whether or not predetermined information is included in the document for confirmation by comparing the results of the data conversion from a formal perspective with information indicating whether or not the check items are necessary, which is stored in advance.
Judgment method.
Note that eva is the composition evaluation value and per is the completeness of the material. The method of claim 1,
When estimating deficiencies from an implicit perspective according to the structure of the document, the result of digitization from an implicit perspective is compared with the result of adjusting the document completeness and structure evaluation value calculated in advance according to the result of the check by a checker in advance according to whether the deadline until the delivery date satisfies a predetermined condition, thereby estimating deficiencies from an implicit perspective according to the structure of the document.
Judgment method. a data conversion unit which converts the acquired confirmation materials into data from an implicit perspective according to the structure of the materials by calculating a document completeness level, which is a value indicating the balance between the document completeness level and the number of figures and charts, by substituting pre-learned parameters and the document completeness level and the number of figures and charts in the acquired confirmation materials into a predetermined formula, and converts the document into data from a formal perspective by checking whether predetermined judgment words are present in the confirmation materials and judging whether each of the predetermined check items has been achieved;
an estimation unit which estimates missing items from an implicit perspective according to the structure of the documents by comparing the results of the digitization from the implicit perspective by said digitization unit with a document completeness and structure evaluation value calculated in advance according to the results of the check by a checker, and which estimates missing items from a formal perspective according to whether or not predetermined information is included in the documents for confirmation by comparing the results of the digitization from the formal perspective by said digitization unit with information indicating whether or not the check items are necessary, which is stored in advance;
have
Judging device.
Note that eva is the composition evaluation value and per is the completeness of the material. The determination device,
Obtain the verification documents,
For the acquired confirmation materials, the previously learned parameters and the amount of text and the number of figures and tables in the acquired confirmation materials are substituted into a predetermined formula to calculate a document completeness, which is a value indicating the balance between the amount of text and the number of figures and tables, and the calculated document completeness is substituted into the formula shown in Equation 3 to calculate a composition evaluation value, which is a value according to the relationship between the document completeness, thereby converting the document into data from an implicit perspective according to the composition of the document, and by checking whether predetermined judgment words are present in the confirmation materials and judging whether each of the predetermined check items has been achieved, the data is converted into data from a formal perspective;
The system estimates deficiencies from an implicit perspective according to the structure of the document by comparing the results of the data conversion from an implicit perspective with the document completeness and structure evaluation value calculated in advance according to the results of the check made by the checker, and estimates deficiencies from a formal perspective according to whether or not predetermined information is included in the document for confirmation by comparing the results of the data conversion from a formal perspective with information indicating whether or not the check items are necessary, which is stored in advance.
A program for implementing processing.
Note that eva is the composition evaluation value and per is the completeness of the material.

Images