JP7633522B2 - Information processing program, information processing method, and information processing device - Google Patents

Description

The present invention relates to an information processing program, an information processing method, and an information processing device.

Conventionally, there are techniques for extracting characteristic words from a document that represent the characteristics of the document. For example, there is a technique called TF (Term Frequency)-IDF (Inverse Document Frequency) that extracts characteristic words of a target document among multiple documents. For example, when attempting to assign tags to documents to make them easier to search, characteristic words of the document may be extracted and used as tags.

Prior art techniques include, for example, analyzing the layout of a web page, extracting sections contained in the web page, determining the type of use of each section, and calculating the importance by weighting according to the type of use of the section in which the basic character string appears. Another technique involves, for example, generating page documents by dividing an original document, setting document information for the page documents, extracting tags from the main text of the page documents, and setting the tags to the document information.

JP 2008-262506 A JP 2018-195305 A

However, with conventional technology, it is difficult to accurately extract characteristic words from a document. For example, TF-IDF may be used to extract characteristic words from a target document among multiple documents. In this case, uncharacteristic words that commonly appear in the header or footer of each page of the target document may end up being words that appear relatively frequently in the target document, and may be erroneously extracted as characteristic words.

In one aspect, the present invention aims to accurately extract characteristic words from a document.

According to one embodiment, an information processing program, an information processing method, and an information processing device are proposed that obtain a plurality of parts obtained by dividing a target document according to specific conditions, extract feature words representing characteristics of each of the obtained plurality of parts from each of the obtained parts based on a first likelihood of each word contained in each of the obtained parts as a feature word representing a characteristic of the part, and determine feature words representing characteristics of the target document from the feature word set that includes the extracted feature words based on a second likelihood of each feature word contained in the feature word set as a feature word representing a characteristic of the target document.

According to one aspect, it becomes possible to extract characteristic words from a document with high accuracy.

FIG. 1 is a diagram illustrating an example of an information processing method according to an embodiment. FIG. 2 is an explanatory diagram illustrating an example of an information processing system 200. FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. FIG. 4 is an explanatory diagram showing an example of the contents of the overall frequency management table 400. As shown in FIG. FIG. 5 is an explanatory diagram showing an example of the contents of the partial frequency management table 500. As shown in FIG. FIG. 6 is an explanatory diagram showing an example of the contents of the important word management table 600. As shown in FIG. FIG. 7 is a block diagram showing an example of the functional configuration of the information processing device 100. FIG. 8 is an explanatory diagram showing the flow of operations of the information processing device 100. FIG. 9 is an explanatory diagram (part 1) showing an example of the operation of the information processing device 100. FIG. 10 is an explanatory diagram (part 2) showing an example of the operation of the information processing device 100. FIG. 11 is an explanatory diagram (part 3) showing an example of the operation of the information processing device 100. FIG. 12 is a flowchart illustrating an example of an overall processing procedure. FIG. 13 is a flowchart illustrating an example of a partial process procedure. FIG. 14 is a flowchart illustrating an example of a procedure of the first generation process. FIG. 15 is a flowchart illustrating an example of a procedure of the second generation process.

Below, embodiments of the information processing program, information processing method, and information processing device according to the present invention will be described in detail with reference to the drawings.

(An example of an information processing method according to an embodiment)
1 is an explanatory diagram showing an example of an information processing method according to an embodiment of the present invention. An information processing device 100 is a computer for extracting characteristic words that represent characteristics of a document from the document.

Here, for example, when adding tags to documents to make them easier to search, it may be desirable to extract characteristic words from the documents and use them as tags.

In contrast, conventionally, a technique called TF-IDF is used to extract feature words from a target document among multiple documents. Specifically, for each word that appears in the target document, the likelihood (TF x IDF) of the word being a feature word that represents the characteristics of the target document is calculated, and words with a calculated likelihood equal to or greater than a threshold are extracted as feature words of the target document.

The TF for a word is an evaluation value based on the frequency of occurrence of the word in the target document. The TF for a word indicates how important the word is in the target document. Specifically, the TF for a first word is the number of times the first word appears in the target document / the total number of times all words appear in the target document. Specifically, the TF for the first word becomes larger the more frequently the first word appears in the target document, indicating that the first word is more important in the target document. Therefore, the larger the TF for a word is and the higher the importance of the word is determined to be in the target document, the larger the likelihood of the word = TF x IDF is likely to be.

The IDF for a word is an evaluation value based on the number of documents in which the word appears among multiple documents including the target document. The IDF for a word indicates how versatile the word is. Specifically, the IDF for a first word is log(total number of documents/number of documents in which the first word appears). Specifically, the IDF for the first word becomes smaller the more documents in which the first word appears, indicating that the first word is versatile and not distinctive, and indicating that the importance of the first word in the target document is low. Therefore, the smaller the IDF for a word is and the lower the importance of the word is determined to be in the target document, the smaller the likelihood of the word = TF x IDF is likely to be.

In this way, conventionally, the accuracy of extracting characteristic words from a target document is improved by calculating the likelihood (TF x IDF) of each word that appears in the target document as a characteristic word of the target document from two different perspectives.

However, in the past, it was sometimes difficult to accurately extract characteristic words from a target document. For example, even if a certain word has a relatively low importance in the target document and is therefore inappropriate as a characteristic word for the target document, it may still be extracted as a characteristic word for the target document because it appears relatively frequently in the target document.

The inappropriate word may be, for example, contained in a template within the target document. The inappropriate word may be, for example, contained in a template document that was the basis for the target document and remains in the target document. The inappropriate word may be, for example, contained in an ancillary part of the target document that is unrelated to the subject matter of the target document.

Specifically, if a template containing a word that is inappropriate as a characteristic word for the target document appears multiple times in the target document, the word will be determined to appear relatively frequently in the target document and will be erroneously extracted as a characteristic word for the target document. The template may be included in the header or footer of the target document, for example. More specifically, if a word that is inappropriate as a characteristic word for the target document appears relatively frequently in the target document, the TF for that word will be large, and the likelihood for that word = TF x IDF will be large, so that the word will be erroneously extracted as a characteristic word for the target document.

Therefore, in this embodiment, we will explain an information processing method that can accurately extract characteristic words from a target document.

In FIG. 1, (1-1) the information processing device 100 obtains multiple parts 111 obtained by dividing a target document 110 according to specific conditions. The target document 110 includes, for example, multiple pages. The target document 110 does not need to be divided by page. The target document 110 may include a relatively large number of generic and uncharacteristic words. In the example of FIG. 1, the uncharacteristic words are specifically words such as "Copyright" included in the footer of each page of the target document 110. The specific conditions indicate, for example, dividing the target document 110 based on page breaks. The specific conditions may also indicate, for example, dividing the target document 110 based on specific keywords.

The information processing device 100, for example, divides the target document 110 according to specific conditions, thereby acquiring multiple parts 111 into which the target document 110 is divided. In the example of FIG. 1, the information processing device 100 specifically acquires multiple parts 111-i into which the target document 110 is divided. In the example of FIG. 1, i=1 to 4. This allows the information processing device 100 to acquire multiple parts 111 each containing relatively small amounts of uncharacteristic words that are relatively frequently contained in the target document 110. Even if a generic, uncharacteristic word appears relatively frequently in the target document 110, the information processing device 100 can easily correctly evaluate it as not being a characteristic word by using part 111 as a reference.

(1-2) The information processing device 100 extracts, from each of the multiple acquired parts 111, a feature word that represents the feature of the part 111. For example, the information processing device 100 extracts, from each of the parts 111, a feature word that represents the feature of the part 111, based on a first likelihood for each word contained in the part 111 as a feature word that represents the feature of the part 111. The first likelihood is, for example, TF×IDF for the words in the part 111. The information processing device 100 generates a list 121 of feature words extracted from each of the parts 111.

In the example of FIG. 1, the information processing device 100 specifically extracts characteristic words that represent the characteristics of each part 111-i from that part 111-i, and generates a list 121-i of extracted characteristic words. In the example of FIG. 1, i=1 to 4. As a result, even if a generic, non-characteristic word or phrase appears relatively frequently in the target document 110, the information processing device 100 can correctly evaluate it as not being a characteristic word or phrase based on part 111, and extract characteristic words that represent the characteristics of part 111.

(1-3) The information processing device 100 determines feature words that represent the characteristics of the target document 110 from a feature word set that is a compilation of the extracted feature words. The feature word set is obtained, for example, by collecting the extracted feature words without duplication. The information processing device 100 determines feature words that represent the characteristics of the target document 110 from the feature word set based on a second likelihood for each feature word included in the feature word set as a feature word that represents the characteristic of the target document 110. The information processing device 100 outputs, for example, a list 130 of the determined feature words. Specifically, the information processing device 100 stores the list 130 of determined feature words in association with the target document 110.

This allows the information processing device 100 to accurately determine characteristic words that represent the characteristics of the target document 110. The information processing device 100 can, for example, use the portion 111 as a reference to exclude generic and uncharacteristic words that appear relatively frequently in the target document 110 from candidates for characteristic words that represent the characteristics of the target document 110. The information processing device 100 can then determine characteristic words that represent the characteristics of the target document 110 after excluding generic and uncharacteristic words from candidates for characteristic words that represent the characteristics of the target document 110. This allows the information processing device 100 to improve the accuracy of determining characteristic words that represent the characteristics of the target document 110.

The information processing device 100 may further provide the user with a service of searching for documents that satisfy specified conditions from among multiple documents based on the stored list of characteristic words 130. Specifically, the information processing device 100 accepts the specification of conditions based on the user's operational input, and searches for documents that satisfy the specified conditions based on the specified conditions. The information processing device 100 then displays on a display the list of characteristic words 130 associated with the searched document together with the searched document, thereby providing the user with a service of searching for documents that satisfy the specified conditions from among multiple documents. This allows the information processing device 100 to easily understand the content of the searched document based on the list of characteristic words 130.

Here, a case has been described in which the information processing device 100 divides the target document 110 in accordance with specific conditions to obtain multiple parts 111 obtained by dividing the target document 110, but this is not limited to the case. For example, the information processing device 100 may receive multiple parts 111 obtained by dividing the target document 110 from another computer that divides the target document 110 in accordance with specific conditions.

Here, we have described a case where the information processing device 100 displays on a display the list 130 of characteristic words associated with the searched document together with the searched document, and provides the user with a service for searching for documents that satisfy specified conditions from among multiple documents, but this is not limited to the above. For example, the information processing device 100 may transmit the list 130 of characteristic words associated with the searched document together with the searched document to another computer, and provide the user with a service for searching for documents that satisfy specified conditions from among multiple documents.

Here, a case has been described in which the information processing device 100 provides a user with a service for searching for documents that satisfy specified conditions from among multiple documents, but this is not limited to the case. For example, the information processing device 100 may provide a user with a service for creating a summary of a document. In this case, the information processing device 100 determines characteristic words of the document and creates a summary of the document using the characteristic words of the document.

In this way, the information processing device 100 may operate independently or in cooperation with another computer. An example of the information processing device 100 in cooperation with another computer will be described in detail later with reference to FIG. 2.

(An example of the information processing system 200)
Next, an example of an information processing system 200 to which the information processing device 100 shown in FIG. 1 is applied will be described with reference to FIG.

FIG. 2 is an explanatory diagram showing an example of an information processing system 200. In FIG. 2, the information processing system 200 includes an information processing device 100, a document storage device 201, and a client device 202.

In the information processing system 200, the information processing device 100 and the document storage device 201 are connected via a wired or wireless network 210. The network 210 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, etc. In the information processing system 200, the information processing device 100 and the client device 202 are connected via the wired or wireless network 210.

The information processing device 100 is a computer that provides a service for searching documents. The documents are, for example, papers, proposals, design documents, contracts, or newspaper articles. The information processing device 100 acquires a plurality of documents from the document storage device 201. The information processing device 100 calculates the TF×IDF for each word that appears in each of the acquired documents, and stores the calculated TF×IDF in the document in an overall frequency management table 400, which will be described later in FIG. 4.

The information processing device 100 divides each of the multiple documents according to specific conditions, thereby acquiring the multiple parts into which the document is divided. The information processing device 100 may acquire the multiple parts into which each of the multiple documents is divided by receiving them from the document storage device 201. The information processing device 100 calculates TF×IDF for each word that appears in each of the multiple parts, and stores the TF×IDF in the part in the part frequency management table 500 described later in FIG. 5.

The information processing device 100 refers to various tables described later in FIG. 4 and FIG. 5 to determine characteristic words that represent the characteristics of each of a plurality of documents, associates them with the documents, and stores them in the important word management table 600 described later in FIG. 6. The characteristic words that represent the characteristics of a document can be, for example, tags of the documents.

The information processing device 100 receives conditions for searching documents from the client device 202. The condition is, for example, that the document contains a specific keyword. The condition may also be, for example, that the document is associated with a specific feature word. The information processing device 100 searches for one or more documents that satisfy the received conditions from among multiple documents. The information processing device 100 refers to the important word management table 600 and obtains the feature words associated with each of the one or more searched documents.

The information processing device 100 transmits each of the one or more retrieved documents and the characteristic words associated with the document to the client device 202. The information processing device 100 is, for example, a server or a PC (Personal Computer).

The document storage device 201 is a computer that stores documents. The document storage device 201 accepts document input based on operational input from an administrator of a service that searches for documents, and stores the accepted documents. The document storage device 201 may store documents by collecting them from other computers. The document storage device 201 provides the stored documents to the information processing device 100. The document storage device 201 may further divide each of the stored documents according to specific conditions, thereby generating multiple divided parts of the document and providing the divided parts to the information processing device 100. The document storage device 201 is, for example, a server or a PC.

The client device 202 is a computer used by a user of the document search service. The client device 202 transmits document search conditions to the information processing device 100 based on the user's operational input. The client device 202 receives each of the one or more searched documents and the characteristic words associated with the document from the information processing device 100.

The client device 202 outputs each of the one or more searched documents and the characteristic words associated with the document so that the user can refer to it. For example, the client device 202 displays each of the one or more searched documents and the characteristic words associated with the document on a display. The client device 202 is, for example, a PC, a tablet terminal, or a smartphone.

The document search service is used, for example, to search for proposals that meet the conditions, such as a time period, trend, or target demographic, specified by the user from among a number of proposals, each of which is a document.

Here, a case has been described in which the information processing device 100 and the document storage device 201 are different devices, but this is not limited thereto. For example, the information processing device 100 may have the function of the document storage device 201 and also operate as the document storage device 201. Here, a case has been described in which the information processing device 100 and the client device 202 are different devices, but this is not limited thereto. For example, the information processing device 100 may have the function of the client device 202 and also operate as the client device 202.

(Example of hardware configuration of information processing device 100)
Next, an example of the hardware configuration of the information processing device 100 will be described with reference to FIG.

Fig. 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. In Fig. 3, the information processing device 100 has a CPU (Central Processing Unit) 301, a memory 302, a network I/F (Interface) 303, a recording medium I/F 304, and a recording medium 305. In addition, each component is connected to each other by a bus 300.

Here, the CPU 301 is responsible for the overall control of the information processing device 100. The memory 302 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and a flash ROM. Specifically, for example, the flash ROM and ROM store various programs, and the RAM is used as a work area for the CPU 301. The programs stored in the memory 302 are loaded into the CPU 301, causing the CPU 301 to execute the coded processes.

The network I/F 303 is connected to the network 210 via a communication line, and is connected to other computers via the network 210. The network I/F 303 manages the internal interface with the network 210, and controls the input and output of data from other computers. The network I/F 303 is, for example, a modem or a LAN adapter.

The recording medium I/F 304 controls the reading/writing of data from/to the recording medium 305 under the control of the CPU 301. The recording medium I/F 304 is, for example, a disk drive, a solid state drive (SSD), or a universal serial bus (USB) port. The recording medium 305 is a non-volatile memory that stores data written under the control of the recording medium I/F 304. The recording medium 305 is, for example, a disk, a semiconductor memory, a USB memory, or the like. The recording medium 305 may be detachable from the information processing device 100.

In addition to the components described above, the information processing device 100 may also have, for example, a keyboard, a mouse, a display, a printer, a scanner, a microphone, a speaker, etc. The information processing device 100 may also have multiple recording medium I/Fs 304 and recording media 305. The information processing device 100 may also not have recording medium I/Fs 304 and recording media 305.

(Storage contents of overall frequency management table 400)
Next, an example of the contents of the total frequency management table 400 will be described with reference to Fig. 4. The total frequency management table 400 is realized by, for example, a storage area such as the memory 302 or the recording medium 305 of the information processing device 100 shown in Fig. 3.

Figure 4 is an explanatory diagram showing an example of the contents stored in the overall frequency management table 400. As shown in Figure 4, the overall frequency management table 400 has fields for ID, file ID, word ID, number of occurrences, and word score. The overall frequency management table 400 stores overall frequency information as record 400-a by setting information for each word in each field. a is an arbitrary integer.

The ID field is set to an ID that identifies the record. The file ID field is set to a file ID that identifies the document. The word ID field is set to a word ID that identifies a word that appears in the document. The occurrence count field is set to the number of occurrences of the word in the document. The word score field is set to the TF×IDF of the word in the document as the word score.

(Contents of Partial Frequency Management Table 500)
5, an example of the contents stored in the partial frequency management table 500 will be described. The partial frequency management table 500 is realized by, for example, a storage area such as the memory 302 or the recording medium 305 of the information processing device 100 shown in FIG.

Figure 5 is an explanatory diagram showing an example of the contents stored in the partial frequency management table 500. As shown in Figure 5, the partial frequency management table 500 has fields for ID, file ID, separator ID, word ID, occurrence count, and word score. The partial frequency management table 500 stores partial frequency information as record 500-b by setting information in each field for each word. b is an arbitrary integer.

In the ID field, an ID that identifies the record is set. In the file ID field, a file ID that identifies the document is set. In the segment ID field, a segment ID that identifies one of the multiple parts into which the document is divided is set. In the word ID field, a word ID that identifies a word that appears in one of the above parts is set. In the occurrence count field, the number of times that the word appears in one of the above parts is set. In the word score field, the TF×IDF of the word in one of the above parts is set as the word score.

(Storage contents of important word management table 600)
Next, an example of the contents stored in the important word management table 600 will be described with reference to Fig. 6. The important word management table 600 is realized, for example, by a storage area such as the memory 302 or the recording medium 305 of the information processing device 100 shown in Fig. 3.

Figure 6 is an explanatory diagram showing an example of the contents stored in the important word management table 600. As shown in Figure 6, the important word management table 600 has fields for ID, file ID, word ID, rank, and word score. The important word management table 600 stores important word information as record 600-c by setting information in each field for each word. c is an arbitrary integer.

In the ID field, an ID that identifies a record is set. In the file ID field, a file ID that identifies a document is set. In the word ID field, a word ID that identifies a word that is a characteristic word of the document is set. In the rank field, a rank is set that indicates the rank assigned to one or more words that are characteristic words of the document when the words are ranked in order of likelihood as characteristic words of the document. The smaller the word rank value, the more likely the word is to be a characteristic word of the document. In the word score field, the TF×IDF of the word in the document is set as the word score.

(Example of Hardware Configuration of Document Storage Device 201)
A specific example of the hardware configuration of the document storage device 201 is similar to the example of the hardware configuration of the information processing device 100 shown in FIG. 3, and therefore a description thereof will be omitted.

(Example of Hardware Configuration of Client Device 202)
Specifically, an example of the hardware configuration of the client device 202 is similar to the example of the hardware configuration of the information processing device 100 shown in FIG. 3, and therefore a description thereof will be omitted.

(Example of functional configuration of information processing device 100)
Next, an example of a functional configuration of the information processing device 100 will be described with reference to FIG.

FIG. 7 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 700, an acquisition unit 701, a division unit 702, a first extraction unit 703, a second extraction unit 704, and an output unit 705.

The storage unit 700 is realized, for example, by a storage area such as the memory 302 or recording medium 305 shown in FIG. 3. Below, a case where the storage unit 700 is included in the information processing device 100 will be described, but this is not limited to this. For example, the storage unit 700 may be included in a device different from the information processing device 100, and the contents stored in the storage unit 700 may be accessible from the information processing device 100.

The acquisition unit 701 to the output unit 705 function as an example of a control unit. Specifically, the acquisition unit 701 to the output unit 705 realize their functions by, for example, causing the CPU 301 to execute a program stored in a storage area such as the memory 302 or the recording medium 305 shown in FIG. 3, or by the network I/F 303. The processing results of each functional unit are stored in, for example, a storage area such as the memory 302 or the recording medium 305 shown in FIG. 3.

The storage unit 700 stores various information that is referenced or updated during processing by each functional unit. The storage unit 700 stores, for example, multiple documents. A document is data that includes multiple words. A document is data that represents a sentence that includes multiple words. A document may be, for example, data that represents a table. A document may be, for example, data that represents a combination of sentences and diagrams. A document is, for example, acquired by the acquisition unit 701 and stored by the storage unit 700. A document may be, for example, stored in advance by the storage unit 700.

The storage unit 700 stores, for example, a plurality of parts obtained by dividing a document according to a specific condition. The specific condition indicates, for example, dividing a document by page. The specific condition may indicate, for example, dividing a document by slide. The specific condition may indicate, for example, dividing a document before and after a keyword included in the document. The plurality of parts are, for example, acquired by the acquisition unit 701 and stored by the storage unit 700. The plurality of parts may be, for example, generated by the division unit 702 and stored by the storage unit 700. The plurality of parts may be, for example, stored in advance by the storage unit 700. The storage unit 700 may store, for example, a specific condition for dividing a document. The specific condition is, for example, stored in advance by the storage unit 700.

The storage unit 700 stores, for example, a first likelihood of each word contained in each of the multiple parts as a characteristic word of the part. A characteristic word of a part is a word that expresses the characteristic of the part. The first likelihood is, for example, TF×IDF. Specifically, the storage unit 700 stores the part frequency management table 500 shown in FIG. 5. The first likelihood is, for example, calculated by the first extraction unit 703 and stored by the storage unit 700. The storage unit 700 stores, for example, a characteristic word of each of the multiple parts in association with the part. A characteristic word of a part is, for example, extracted by the first extraction unit 703 and stored by the storage unit 700.

The storage unit 700 stores, for example, a second likelihood of a characteristic word of at least one of the multiple parts into which a document is divided as a characteristic word of the document. The second likelihood is, for example, TF×IDF. Specifically, the storage unit 700 stores the overall frequency management table 400 shown in FIG. 4. The second likelihood is, for example, calculated by the second extraction unit 704 and stored by the storage unit 700. The storage unit 700 stores, for example, characteristic words of a document in association with the document. A characteristic word of a document is a word or phrase that expresses a characteristic of the document. A characteristic word of a document is, for example, determined by the second extraction unit 704 and stored in the storage unit 700.

The acquisition unit 701 acquires various information used for processing by each functional unit. The acquisition unit 701 stores the acquired various information in the memory unit 700 or outputs it to each functional unit. The acquisition unit 701 may also output the various information stored in the memory unit 700 to each functional unit. The acquisition unit 701 acquires various information based on, for example, a user's operation input. The acquisition unit 701 may receive various information from, for example, a device other than the information processing device 100.

The acquisition unit 701 acquires the target document. For example, the acquisition unit 701 acquires each of the multiple documents stored in the memory unit 700 by reading them out as the target document. For example, the acquisition unit 701 may acquire any one of the multiple documents stored in the memory unit 700 by reading it out as the target document. For example, the acquisition unit 701 may acquire the target document by receiving it from another computer.

The acquisition unit 701 acquires the multiple parts into which the target document is divided. The acquisition unit 701 acquires, for example, the multiple parts into which each of the multiple documents stored in the storage unit 700 is divided, by reading them out as the multiple parts into which the target document is divided. The acquisition unit 701 may also acquire, for example, the multiple parts into which any of the multiple documents stored in the storage unit 700 is divided, by reading them out as the multiple parts into which the target document is divided. For example, if the acquisition unit 701 does not divide the target document on its own device, it may acquire the multiple parts into which the target document is divided by receiving them from another computer.

The acquisition unit 701 may receive a start trigger for starting the processing of any of the functional units. The start trigger may be, for example, a predetermined operational input by a user. The start trigger may be, for example, the receipt of predetermined information from another computer. The start trigger may be, for example, the output of predetermined information by any of the functional units. Specifically, the acquisition unit 701 may receive the acquisition of a target document as a start trigger for starting the processing of the division unit 702, the first extraction unit 703, and the second extraction unit 704. Specifically, the acquisition unit 701 may receive the acquisition of a target document and multiple parts into which the target document is divided as a start trigger for starting the processing of the first extraction unit 703 and the second extraction unit 704.

The division unit 702 obtains multiple parts obtained by dividing the target document. The division unit 702 obtains multiple parts, for example, by dividing the target document according to specific conditions. Specifically, the division unit 702 obtains multiple parts obtained by dividing the target document by page units. Specifically, the division unit 702 may obtain multiple parts obtained by dividing the target document by dividing the target document before and after a keyword included in the target document.

In this way, if there is a non-characteristic word that is contained relatively frequently in the target document, the dividing unit 702 can generate multiple parts each containing a relatively small amount of that word. Then, in addition to the target document, the dividing unit 702 can make each of the multiple parts into which the target document is divided available as a criterion for determining characteristic words for the target document.

The first extraction unit 703 extracts a characteristic word for each of the acquired parts from the plurality of parts. The first extraction unit 703 extracts a characteristic word for each of the parts from the respective parts based on, for example, a first likelihood that each word included in each of the parts is a characteristic word for the part.

Specifically, the first extraction unit 703 calculates a first likelihood for each word contained in each part based on an evaluation value based on the frequency with which the word appears in that part and an evaluation value based on the number of parts in which the word appears among the multiple parts. Then, specifically, the first extraction unit 703 extracts characteristic words for each part based on the calculated first likelihood. More specifically, the first extraction unit 703 extracts, from among the words contained in each part, words whose calculated first likelihood is equal to or greater than a threshold value as characteristic words for that part.

As a result, even if generic, non-characteristic words appear relatively frequently in the target document, the first extraction unit 703 can correctly evaluate each part as a criterion and determine that the words are not characteristic words, and extract characteristic words for each part. Therefore, the first extraction unit 703 can narrow down candidates for characteristic words when determining characteristic words for the target document, and can accurately determine characteristic words for the target document.

The first extraction unit 703 may, for example, extract characteristic words of each of the multiple acquired parts based on multiple documents including the target document. Specifically, the first extraction unit 703 calculates a first likelihood for each word included in each part based on the multiple documents. Then, specifically, the first extraction unit 703 extracts characteristic words of each part based on the calculated first likelihood.

As a result, even if generic, non-characteristic words appear relatively frequently in the target document, the first extraction unit 703 can correctly evaluate each part as a criterion and determine that the words are not characteristic words, and extract characteristic words for each part. Therefore, the first extraction unit 703 can narrow down candidates for characteristic words when determining characteristic words for the target document, and can accurately determine characteristic words for the target document.

The second extraction unit 704 determines feature words of the target document from a feature word set that is a compilation of the feature words extracted by the first extraction unit 703. The feature word set is generated by compiling the feature words extracted by the first extraction unit 703 without duplication. The second extraction unit 704 determines feature words of the target document from the feature word set, for example, based on a second likelihood of each feature word included in the feature word set as a feature word of the target document.

Specifically, the second extraction unit 704 calculates a second likelihood for each feature word based on an evaluation value based on the frequency with which the feature word appears in the target document and an evaluation value based on the number of documents in which the feature word appears among a plurality of documents including the target document. Specifically, the second extraction unit 704 then determines feature words of the target document from the feature word set based on the calculated second likelihood. More specifically, the second extraction unit 704 determines feature words of the target document from the feature word set whose calculated second likelihood is equal to or greater than a threshold value as feature words of the target document.

As a result, the second extraction unit 704 can determine the feature words of the target document from the feature word set after narrowing down the candidate feature words when determining the feature words of the target document to the feature words included in the feature word set. This allows the second extraction unit 704 to accurately determine the feature words of the target document.

Here, a case has been described in which the first extraction unit 703 extracts characteristic words of each part from the part, but this is not limiting. For example, after the first extraction unit 703 calculates the first likelihood for each word contained in each part, there may be cases in which the first extraction unit 703 does not extract characteristic words of the part from the part.

In this case, the second extraction unit 704, for example, calculates a second likelihood for each term contained in the target document as a characteristic word of the target document. Specifically, the second extraction unit 704 calculates the second likelihood for each term based on an evaluation value based on the frequency with which the term appears in the target document and an evaluation value based on the number of documents in which the term appears among multiple documents including the target document.

Then, the second extraction unit 704 determines characteristic words of the target document from among the words contained in the target document based on, for example, the first likelihood calculated by the first extraction unit 703 and the second likelihood calculated by the second extraction unit 704. Specifically, the second extraction unit 704 calculates, for each word, the product of the first likelihood calculated by the first extraction unit 703 and the second likelihood calculated by the second extraction unit 704 as a third likelihood. Specifically, the second extraction unit 704 determines, as characteristic words of the target document, words whose calculated third likelihood is equal to or greater than a threshold value.

This allows the second extraction unit 704 to determine characteristic words of the target document based on various likelihoods obtained from different perspectives of the target document unit and each of the multiple parts into which the target document is divided. This allows the second extraction unit 704 to accurately determine characteristic words of the target document.

The output unit 705 outputs the processing results of at least one of the functional units. The output format is, for example, display on a display, printout on a printer, transmission to an external device via the network I/F 303, or storage in a storage area such as the memory 302 or the recording medium 305. In this way, the output unit 705 can notify the user of the processing results of at least one of the functional units, thereby improving the convenience of the information processing device 100.

The output unit 705 outputs the target document in association with the characteristic words of the target document determined by the second extraction unit 704. The output unit 705 may, for example, associate the target document with the characteristic words of the target document determined by the second extraction unit 704 and transmit them to a search unit (not shown) that searches for documents. The search unit may, for example, be included in the information processing device 100. The search unit may, for example, be included in another computer.

As a result, the output unit 705 can make the characteristic words of the target document available when searching for the target document in the search unit, making it easier to search for the target document. The output unit 705 can make it possible for the search unit to output search results that include the characteristic words of the target document along with the searched target document, making it easier for viewers of the search results to understand the contents of the searched target document.

The output unit 705 may, for example, associate the target document with the characteristic words of the target document determined by the second extraction unit 704, and transmit them to another computer. The other computer is, for example, the client device 202. The output unit 705 may, for example, associate the target document with the characteristic words of the target document determined by the second extraction unit 704, and display them so that the user can refer to them. In this way, the output unit 705 can make it easier for the user to understand the contents of the target document.

(Operation flow of information processing device 100)
Next, the flow of operations of the information processing device 100 will be described with reference to FIG.

Figure 8 is an explanatory diagram showing the flow of operation of the information processing device 100. In Figure 8, the information processing device 100 acquires a document group 800. The document group 800 includes, for example, document A. In the following explanation, a case where the information processing device 100 processes document A will be explained. The case where the information processing device 100 processes another document is also similar to the case explained below.

(8-1) The information processing device 100 divides the document A into pages and obtains a page group 810. The page group 810 includes, for example, page 1, page 2, and page 3.

(8-2) The information processing device 100 performs morphological analysis on each page and generates a list of words that appear on each page. The words may be limited to nouns, for example. For each word included in the list of words that appear on each page, the information processing device 100 calculates a TF related to the frequency of occurrence of the word and an IDF related to the number of pages in document A that contain the word, and calculates TF x IDF as the word score.

The information processing device 100 may calculate, for each word included in the list of words that appear on each page, a TF related to the frequency of occurrence of the word and an IDF related to the number of documents that include the word in the document set 800, and calculate TF x IDF. The information processing device 100 stores the calculated word scores in the partial frequency management table 500.

The information processing device 100 refers to the partial frequency management table 500 and extracts, for each page, words whose calculated TF×IDF is equal to or greater than a first threshold value as characteristic words of the page. The information processing device 100 may also refer to the partial frequency management table 500 and extract, for each page, words up to a predetermined rank when the words are sorted in descending order of the calculated TF×IDF as characteristic words of the page.

(8-3) The information processing device 100 performs a morphological analysis on document A and generates a list of words that appear in document A. The information processing device 100 generates a word group that includes words that have been extracted as characteristic words of at least one of the pages from the generated word list.

(8-4) For each word included in the generated word group, the information processing device 100 calculates the TF related to the frequency of occurrence of the word and the IDF including the word in the document group 800, and calculates TF x IDF as the word score. The information processing device 100 stores the calculated word score in the overall frequency management table 400.

(8-5) The information processing device 100 refers to the overall frequency management table 400 and determines words whose calculated TF×IDF is equal to or greater than the second threshold as characteristic words of document A. As a result, the information processing device 100 narrows down candidates for characteristic words of the document before determining the characteristic words of the document, and therefore can determine the characteristic words of the document with high accuracy.

(Example of operation of information processing device 100)
Next, an example of the operation of the information processing device 100 will be described with reference to FIGS.

FIGS. 9 to 11 are explanatory diagrams showing an example of the operation of the information processing device 100. In FIG. 9, (9-1) the information processing device 100 acquires a document 900. The document 900 is, for example, a proposal. The document 900 includes a standard phrase such as COPYRIGHT 2021 XXXXX LIMITED in the footer of each page.

The pages are, for example, P1, P2-3, P4-5, and P6. In the example of FIG. 9, P2-3 and P4-5 are treated as one page. The information processing device 100 divides the document 900 into page units and generates multiple division parts 901.

(9-2) The information processing device 100 performs morphological analysis on each segment 901 and generates a list 911 of words that appear in each segment 901.

(9-3) For each word included in the word list 911 corresponding to each segment 901, the information processing device 100 calculates TF×IDF for that segment 901 as the word score for that segment 901. As indicated by reference numeral 921 in FIG. 9, the word score for each word included in the word list 911 is calculated. In the example of FIG. 9, the word score is the value in parentheses following the word. The information processing device 100 stores the calculated word score in the partial frequency management table 500. Next, we move on to the explanation of FIG. 10.

In FIG. 10, (10-1) the information processing device 100 extracts words whose word scores are equal to or greater than a first threshold from a word list 911 corresponding to each segment 901 as important words that represent the characteristics of the segment 901. The first threshold is, for example, 0.5. The information processing device 100 generates a list 1001 of important words that compiles the extracted important words for each segment 901.

(10-2) The information processing device 100 generates an important word group 1010 by collecting important words without duplication from the list of important words 1001. For each important word included in the important word group 1010, the information processing device 100 calculates TF×IDF in the document 900 as a word score in the document 900. As shown in FIG. 10, the word score of each word included in the important word group 1010 is calculated. In the example of FIG. 10, the word score is a value in parentheses following the word. The information processing device 100 ranks the important words included in the important word group 1010 in descending order of the calculated word score.

(10-3) As a result of the ranking, the information processing device 100 determines the top five important words, indicated by the reference symbol 1020, as characteristic words of the document 900. The information processing device 100 associates the determined characteristic words of the document 900 with the document 900 as tags and stores them in the important word management table 600. As a result, the information processing device 100 narrows down candidates for characteristic words of the document 900 before determining characteristic words of the document 900, and can therefore determine characteristic words of the document 900 with high accuracy. Next, we move on to the explanation of FIG. 11.

In FIG. 11, the information processing device 100 determines characteristic words for multiple documents 900, as in FIG. 9 and FIG. 10, and stores them as tags in the important word management table 600. The information processing device 100 accepts conditions for searching the documents 900 in an input field 1101 of a search screen 1100.

The information processing device 100 searches for documents 900 that satisfy the received conditions from among a plurality of documents 900. The information processing device 100 references the important word management table 600 and displays a thumbnail image 1110 of each of the searched documents 900, along with a list 1111 of tags for the documents 900, on the search screen 1100. This enables the information processing device 100 to make it easier for the searcher to understand the contents of each of the searched documents 900.

(Overall Processing Procedure)
Next, an example of an overall processing procedure executed by the information processing device 100 will be described with reference to Fig. 12. The overall processing is realized by, for example, the CPU 301, storage areas such as the memory 302 and the recording medium 305, and the network I/F 303 shown in Fig. 3.

Figure 12 is a flowchart showing an example of an overall processing procedure. In Figure 12, the information processing device 100 generates an overall frequency management table 400 for each document of a plurality of documents by executing a first generation process described later in Figure 14 (step S1201).

Next, the information processing device 100 selects a document to be analyzed from among the documents included in the multiple documents and that have not yet been selected as a document to be analyzed (step S1202). Then, the information processing device 100 divides the selected document to be analyzed according to specific conditions, and obtains multiple delimited portions into which the document to be analyzed is divided (step S1203).

Next, the information processing device 100 executes partial processing described later in FIG. 13 (step S1204). Then, the information processing device 100 refers to the overall frequency management table 400 for the selected document to be analyzed, and calculates the score of each of the one or more important words (step S1205).

Next, the information processing device 100 determines, based on the calculated score, from among one or more important words, an important word that will become a characteristic word of the selected document to be analyzed (step S1206). Then, the information processing device 100 stores the determined important word in the important word management table 600 in association with the selected document to be analyzed (step S1207).

Next, the information processing device 100 determines whether all documents have been selected as documents to be analyzed (step S1208). If there are documents that have not yet been selected (step S1208: No), the information processing device 100 returns to the processing of step S1202. On the other hand, if all documents have been selected (step S1208: Yes), the information processing device 100 ends the entire processing.

(Partial Processing Procedure)
Next, an example of a partial processing procedure executed by the information processing device 100 will be described with reference to Fig. 13. The partial processing is realized by, for example, the CPU 301, storage areas such as the memory 302 and the recording medium 305, and the network I/F 303 shown in Fig. 3.

Figure 13 is a flowchart showing an example of a partial processing procedure. In Figure 13, the information processing device 100 executes a second generation process described later in Figure 15 to generate a partial frequency management table 500 for each of the acquired multiple division parts (step S1301).

Next, the information processing device 100 selects a segment to be analyzed from among the segment parts that are included in the multiple segment parts and that have not yet been selected as a segment to be analyzed (step S1302). Then, the information processing device 100 performs a morphological analysis on the selected segment to be analyzed, and generates a list of words that appear in the segment to be analyzed (step S1303).

Next, the information processing device 100 refers to the part frequency management table 500 for the segment part of the selected analysis target, and calculates the score of each word included in the generated word list (step S1304). Then, the information processing device 100 extracts words that are important words for the segment part of the selected analysis target from the generated word list based on the calculated score (step S1305).

Next, the information processing device 100 stores the determined words as important words in the important word management table 600 in association with the selected segment to be analyzed (step S1306). The information processing device 100 then determines whether or not all segment parts have been selected as segment parts to be analyzed (step S1307). If there are segment parts that have not yet been selected (step S1307: No), the information processing device 100 returns to the process of step S1302. On the other hand, if all segment parts have been selected (step S1307: Yes), the information processing device 100 ends the partial processing.

(First generation process procedure)
Next, an example of a first generation process procedure executed by the information processing device 100 will be described with reference to Fig. 14. The first generation process is realized by, for example, the CPU 301, storage areas such as the memory 302 and the recording medium 305, and the network I/F 303 shown in Fig. 3.

Figure 14 is a flowchart showing an example of the first generation process procedure. In Figure 14, the information processing device 100 selects a document to be measured from among documents that are included in a plurality of documents and have not yet been selected (step S1401).

Next, the information processing device 100 performs a morphological analysis on the selected measurement target document and extracts one or more words that appear in the measurement target document (step S1402). Then, the information processing device 100 generates an overall frequency management table 400 for the selected measurement target document based on the extracted one or more words (step S1403).

Next, the information processing device 100 determines whether all documents have been selected as documents to be measured (step S1404). If there are documents that have not yet been selected (step S1404: No), the information processing device 100 returns to the process of step S1401. On the other hand, if all documents have been selected (step S1404: Yes), the information processing device 100 ends the first generation process.

(Second generation process procedure)
Next, an example of a procedure of the second generation process executed by the information processing device 100 will be described with reference to Fig. 15. The second generation process is realized by, for example, the CPU 301, storage areas such as the memory 302 and the recording medium 305, and the network I/F 303 shown in Fig. 3.

FIG. 15 is a flowchart showing an example of the second generation process procedure. In FIG. 15, the information processing device 100 selects a segment to be measured from among a plurality of segment portions that have not yet been selected (step S1501).

Next, the information processing device 100 performs a morphological analysis on the segment of the selected measurement target and extracts one or more words that appear in the segment of the measurement target (step S1502). Then, the information processing device 100 generates a part frequency management table 500 for the segment of the selected measurement target based on the extracted one or more words (step S1503).

Next, the information processing device 100 determines whether or not all of the separator parts have been selected as separator parts to be measured (step S1504). If there are separator parts that have not yet been selected (step S1504: No), the information processing device 100 returns to the process of step S1501. On the other hand, if all of the separator parts have been selected (step S1504: Yes), the information processing device 100 ends the second generation process.

Here, the information processing device 100 may change the order of the processing of some of the steps in the flowcharts of Figures 12 to 15.

As described above, the information processing device 100 can obtain multiple parts obtained by dividing a target document according to specific conditions. The information processing device 100 can extract feature words from each of the obtained multiple parts based on a first likelihood of each word contained in each part as a feature word of the part. The information processing device 100 can determine feature words of the target document from a feature word set based on a second likelihood of each feature word contained in a feature word set that summarizes the extracted feature words as a feature word of the target document. This allows the information processing device 100 to accurately determine feature words of the target document.

According to the information processing device 100, for each word contained in each part, a first likelihood can be calculated based on an evaluation value based on the frequency with which the word appears in the part and an evaluation value based on the number of parts in which the word appears among the multiple parts. According to the information processing device 100, a feature word of the part can be extracted from each part based on the calculated first likelihood. According to the information processing device 100, for each feature word, a second likelihood can be calculated based on an evaluation value based on the frequency with which the feature word appears in the target document and an evaluation value based on the number of documents in which the feature word appears among the multiple documents including the target document. According to the information processing device 100, a feature word of the target document can be determined from the feature word set based on the calculated second likelihood. As a result, the information processing device 100 can accurately calculate the first likelihood and the second likelihood, and can accurately determine the feature word of the target document.

According to the information processing device 100, among the words contained in each part, words whose calculated first likelihood is equal to or greater than a threshold value can be extracted as characteristic words of the part. This allows the information processing device 100 to narrow down candidates for characteristic words of the target document.

According to the information processing device 100, it is possible to calculate a first likelihood for each word contained in each of the multiple parts acquired. According to the information processing device 100, it is possible to calculate a second likelihood for each word contained in the target document. According to the information processing device 100, it is possible to determine characteristic words of the target document from among the words contained in the target document based on the calculated first likelihood and the calculated second likelihood. This makes it possible to omit the operation of narrowing down the characteristic words of the target document and reduce the amount of processing.

According to the information processing device 100, it is possible to obtain multiple parts obtained by dividing the target document on a page-by-page basis. This makes it easier for the information processing device 100 to obtain multiple parts each containing a relatively small amount of uncharacteristic words that are contained relatively frequently in the target document.

According to the information processing device 100, it is possible to obtain multiple parts obtained by dividing the target document around a keyword contained in the target document. This makes it easier for the information processing device 100 to obtain multiple parts each containing a relatively small amount of uncharacteristic words that are contained relatively frequently in the target document.

According to the information processing device 100, it is possible to extract characteristic words for each of the multiple parts obtained based on multiple documents including the target document. This allows the information processing device 100 to extract characteristic words for each part with high accuracy.

The information processing method described in this embodiment can be realized by executing a prepared program on a computer such as a PC or a workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and is executed by the computer reading it from the recording medium. The recording medium may be a hard disk, a flexible disk, a CD (Compact Disc)-ROM, an MO (Magneto Optical disc), a DVD (Digital Versatile Disc), or the like. The information processing program described in this embodiment may also be distributed via a network such as the Internet.

The following additional notes are provided with respect to the above-described embodiment.

(Appendix 1) Obtain multiple parts by dividing the target document according to specific conditions,
extracting, from each of the acquired plurality of parts, a feature word that represents a feature of the part, based on a first likelihood of each word contained in each of the acquired plurality of parts being a feature word that represents a feature of the part;
determining, from the feature word set, a feature word that represents a feature of the target document, based on a second likelihood, for each feature word included in a feature word set that summarizes the extracted feature words, as a feature word that represents a feature of the target document;
An information processing program that causes a computer to execute a process.

(Additional Note 2) The extraction process includes:
calculating the first likelihood for each word included in each of the portions based on an evaluation value based on a frequency at which the word appears in the portion and an evaluation value based on a number of portions in which the word appears among the plurality of portions, and extracting, from each of the portions, a characteristic word that represents a characteristic of the portion based on the calculated first likelihood;
The process of determining includes:
An information processing program as described in Appendix 1, characterized in that for each of the feature words, the second likelihood is calculated based on an evaluation value based on the frequency with which the feature word appears in the target document and an evaluation value based on the number of documents in which the feature word appears among a plurality of documents including the target document, and a feature word that represents a characteristic of the target document is determined from the set of feature words based on the calculated second likelihood.

(Additional Note 3) The extraction process includes:
An information processing program as described in Appendix 2, characterized in that, among the words contained in each of the parts, words whose calculated first likelihood is equal to or greater than a threshold value are extracted as characteristic words representing the characteristics of the part.

(Supplementary Note 4) For each word included in each of the acquired multiple parts, a first likelihood is calculated as a feature word representing a feature of the part, based on an evaluation value based on a frequency at which the word appears in the part and an evaluation value based on a number of parts in which the word appears among the multiple parts;
calculating a second likelihood for each term contained in the target document as a characteristic word representing a characteristic of the target document based on an evaluation value based on a frequency at which the term appears in the target document and an evaluation value based on a number of documents in which the term appears among a plurality of documents including the target document;
determining, based on the calculated first likelihood and the calculated second likelihood, a feature word representing a feature of the target document from among words and phrases contained in the target document;
4. The information processing program according to claim 1, wherein the information processing program causes the computer to execute a process.

(Appendix 5) The information processing program described in any one of appendices 1 to 4, characterized in that the multiple parts are obtained by dividing the target document on a page-by-page basis.

(Appendix 6) The information processing program described in any one of appendices 1 to 5, characterized in that the multiple parts are obtained by dividing the target document before and after a keyword contained in the target document.

(Additional Note 7) The extraction process includes:
An information processing program described in any one of Appendices 1 to 6, characterized in that, based on a plurality of documents including the target document, characteristic words that express characteristics of each of the acquired plurality of parts are extracted from each of the acquired plurality of parts.

(Appendix 8) Obtaining multiple parts obtained by dividing the target document according to specific conditions,
extracting, from each of the acquired plurality of parts, a feature word that represents a feature of the part, based on a first likelihood of each word contained in each of the acquired plurality of parts being a feature word that represents a feature of the part;
determining, from the feature word set, a feature word that represents a feature of the target document, based on a second likelihood of each feature word included in a feature word set that summarizes the extracted feature words as a feature word that represents a feature of the target document;
An information processing method characterized in that the processing is executed by a computer.

(Appendix 9) Obtaining multiple parts obtained by dividing the target document according to specific conditions,
extracting, from each of the acquired plurality of parts, a feature word that represents a feature of the part, based on a first likelihood of each word contained in each of the acquired plurality of parts being a feature word that represents a feature of the part;
determining, from the feature word set, a feature word that represents a feature of the target document, based on a second likelihood, for each feature word included in a feature word set that summarizes the extracted feature words, as a feature word that represents a feature of the target document;
An information processing device comprising a control unit.

REFERENCE SIGNS LIST 100 Information processing device 110 Document 111, 901 Part 121, 130, 911, 1001, 1111 List 200 Information processing system 201 Document storage device 202 Client device 210 Network 300 Bus 301 CPU
302 Memory 303 Network I/F
304 Recording medium I/F
305 Recording medium 400 Total frequency management table 500 Partial frequency management table 600 Important word management table 700 Storage unit 701 Acquisition unit 702 Dividing unit 703 First extraction unit 704 Second extraction unit 705 Output unit 800 Document group 810 Page group 900 Document 921, 1020 Code 1010 Important word group 1100 Search screen 1101 Input field 1110 Thumbnail image

Claims (6)

Obtaining multiple parts by dividing the target document according to specific conditions;
extracting, from each of the acquired plurality of parts, a feature word that represents a feature of the part, based on a first likelihood of each word contained in each of the acquired plurality of parts being a feature word that represents a feature of the part;
determining, from the feature word set, a feature word that represents a feature of the target document, based on a second likelihood, for each feature word included in a feature word set that summarizes the extracted feature words, as a feature word that represents a feature of the target document;
An information processing program that causes a computer to execute a process. The extraction process includes:
calculating the first likelihood for each word included in each of the portions based on an evaluation value based on a frequency at which the word appears in the portion and an evaluation value based on a number of portions in which the word appears among the plurality of portions, and extracting, from each of the portions, a characteristic word that represents a characteristic of the portion based on the calculated first likelihood;
The process of determining includes:
The information processing program described in claim 1, characterized in that for each of the feature words, the second likelihood is calculated based on an evaluation value based on the frequency with which the feature word appears in the target document and an evaluation value based on the number of documents in which the feature word appears among a plurality of documents including the target document, and a feature word that represents a characteristic of the target document is determined from the set of feature words based on the calculated second likelihood. The extraction process includes:
3. The information processing program according to claim 2, further comprising: extracting, from among the words contained in each of the portions, a word for which the calculated first likelihood is equal to or greater than a threshold value as a characteristic word representing a characteristic of the portion. calculating a first likelihood for each word included in each of the acquired plurality of parts as a characteristic word representing a characteristic of the part, based on an evaluation value based on a frequency at which the word appears in the part and an evaluation value based on a number of parts in which the word appears among the plurality of parts;
calculating a second likelihood for each term contained in the target document as a characteristic term representing a characteristic of the target document based on an evaluation value based on a frequency at which the term appears in the target document and an evaluation value based on a number of documents in which the term appears among a plurality of documents including the target document;
determining, based on the calculated first likelihood and the calculated second likelihood, a feature word representing a feature of the target document from among words and phrases contained in the target document;
4. The information processing program according to claim 1, which causes the computer to execute a process. Obtaining multiple parts by dividing the target document according to specific conditions;
extracting, from each of the acquired plurality of parts, a feature word that represents a feature of the part, based on a first likelihood of each word contained in each of the acquired plurality of parts being a feature word that represents a feature of the part;
determining, from the feature word set, a feature word that represents a feature of the target document, based on a second likelihood, for each feature word included in a feature word set that summarizes the extracted feature words, as a feature word that represents a feature of the target document;
An information processing method characterized in that the processing is executed by a computer. Obtaining multiple parts by dividing the target document according to specific conditions;
extracting, from each of the acquired plurality of parts, a feature word that represents a feature of the part, based on a first likelihood of each word or phrase contained in each of the acquired plurality of parts being a feature word that represents a feature of the part;
determining, from the feature word set, a feature word that represents a feature of the target document, based on a second likelihood of each feature word included in a feature word set that summarizes the extracted feature words as a feature word that represents a feature of the target document;
An information processing device comprising a control unit.

Images