JP7814892B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to information processing technology for natural language processing.

In recent years, advances in AI technology have led to advances in the field of natural language processing, which uses computers to analyze documents written in human speech and language. Natural language processing technology is expected to be applied in a variety of fields that handle document data, such as document summarization and translation, voice dialogue, and data analysis.

One of the applied technologies of natural language processing is named entity extraction, which extracts the values of pre-defined items (named entities) from document data. For example, if a corporate name and expiration date are defined as named entities, the strings corresponding to the corporate name and the expiration date can be extracted from the document data.

Currently, natural language processing models using Transformers, such as BERT, which is the mainstream in natural language processing, break down character strings contained in document data into units called tokens, and use these tokens as vectors as input data. However, because there is a limit to the number of tokens that a natural language processing model can process at one time, when inputting long document data containing tokens that exceed this limit, the document data must be divided into two or more token groups, which must be input and processed separately. If the multiple tokens contained in a single piece of document data are simply divided to fit the natural language processing model's input limit, the keywords and context (character strings surrounding the named entity) used to distinguish named entities may be lost, resulting in a decrease in the accuracy of named entity inference.

In Patent Document 1, document data is divided into sections such as chapters, sections, and paragraphs, and named entities are extracted using a natural language processing model for each token group that is expected to retain a certain context.

Japanese Patent Application Laid-Open No. 2021-64143

In Cited Document 1, tokens contained in document data are divided into sections so that the number of tokens processed at one time falls within the input limit of the natural language processing model. However, because each divided section is processed separately, there is an issue that the estimation accuracy may decrease due to the loss of context contained in adjacent sections.

The present invention therefore aims to prevent a decline in estimation accuracy in natural language processing when dividing tokens contained in document data.

The technology disclosed herein is an information processing device for extracting named entities from a plurality of tokens obtained by decomposing an input character string, the information processing device comprising: a division means for dividing the plurality of tokens obtained by decomposing the input character string into two or more token groups when the number of the plurality of tokens obtained by decomposing the input character string exceeds a predetermined upper limit, wherein each of the token groups has a predetermined number of tokens overlapping with another token group; an extraction means for extracting the named entities for each of the token groups; and a determination means for determining the extraction result of the named entities in the overlapping portion based on the extraction result of the named entities by the extraction means for the overlapping portion with the other token group, wherein the determination means determines the extraction result of the named entity in the overlapping portion to be the extraction result of the named entity in the overlapping portion, of the token group with the largest number of tokens among the extraction results of the named entities from each of the two overlapping token groups in the overlapping portion .

According to the present invention, it is possible to suppress a decrease in estimation accuracy in natural language processing when dividing tokens contained in document data.

A block diagram showing an example of the functional and hardware configuration of a named entity extraction device 100. FIG. 1 is a diagram illustrating an example of document data received by a receiving unit 102. Table showing an example of a token acquired by the control unit 101 Table showing an example of named entities and limit numbers of tokens calculated by the calculation unit 103 1 is a flowchart illustrating an example of processing executed by the control unit 101. 10 is a flowchart illustrating an example of processing executed by the control unit 101 according to the second embodiment. 10 is a table showing an example of the limit number of tokens calculated by the calculation unit 103 in the third embodiment. 10 is a flowchart illustrating an example of processing executed by the control unit 101 according to the third embodiment. 10 is a flowchart illustrating an example of processing executed by the control unit 101 according to the fourth embodiment.

The best mode for carrying out the present invention will be described below with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[Embodiment 1]
In the first embodiment, an example of a system will be described in which character strings are extracted from document data, the extracted character strings are broken down into token units, and named entities are extracted and output from the obtained tokens using a natural language processing model. If the obtained tokens can be input into the natural language processing model at once to extract named entities, the results can be estimated efficiently. However, natural language processing models have an input upper limit, meaning that there is an upper limit on the number of tokens that can be input at one time. Therefore, if the number of tokens in a long sentence exceeds the input upper limit, it is necessary to divide the token groups into multiple groups and input them into the natural language processing model. However, if the token groups are simply divided to fit the input upper limit of the natural language processing model, keywords and contexts for distinguishing named entities may be lost, which may result in a decrease in the accuracy of named entity extraction.

In embodiment 1, therefore, when the number of tokens exceeds the upper limit of input to the natural language processing model, the tokens are first divided into two or more partially overlapping token groups based on the limit number of tokens for which named entities can be correctly extracted. Named entities are then extracted for each divided token group, and the named entities to be extracted from the overlapping portions are determined, thereby preventing a decrease in the accuracy of named entity extraction.

Figure 1(a) shows an example functional block diagram of a named entity extraction device 100 according to this embodiment. The named entity extraction device 100 is an information processing device that includes a control unit 101, a receiving unit 102, a calculation unit 103, a segmentation unit 104, and an extraction unit 105.

The control unit 101 is composed of a CPU 111 and other components, and reads programs and data stored in ROM 113 into RAM 112 to perform processes such as named entity extraction.

The receiving unit 102 receives document data to be subjected to named entity extraction processing through operation of the input device 114 provided in the named entity extraction device 100. The document data may be obtained from the storage device 116, or may be obtained from the network 118 via the network interface 117.

The calculation unit 103 calculates the limit number of tokens that allows the control unit 101 to correctly extract named entities from document data.

The division unit 104 divides the token group converted from the document data into two or more token groups so that the number of tokens input at one time is less than the upper limit number of tokens that can be input to the natural language processing model.

The extraction unit 105 extracts named entities from the divided token groups using a natural language processing model. The extracted named entities are stored in the storage device 116 and displayed on an output device 115 such as a display.

Figure 2(a) shows an example of document data received by the receiving unit 102. The document data is composed of multiple elements of different types, including character strings, symbols, and ruled lines, within a page. Document data used for training and evaluation of natural language processing models is usually accompanied by ground truth data called GT (Ground Truth). The GT attached to document data 210 defines named entity types (or attributes) of corporate name, corporate name, and expiration date for each character string, in order to indicate that character strings 211, 212, and 213 within the dashed rectangles are named entities.

Document data 210 is typically composed of pages and includes symbols and ruled lines, but it may also be composed of multiple pages, or may be data that contains only text information without layout information. In other words, document data can be in any format as long as it allows for the acquisition of text information.

Figure 2(b) shows an example of document data received by the receiving unit 102. This is document data similar to that shown in Figure 2(a) and has a similar layout, except that no GT (correct answer data) is attached.

Figures 3(a) and 3(b) show an example of a token acquired by the control unit 101. The token is represented by an identifier 311, a token string 312, and a GT 313.

Figure 3(a) shows a token table 310 listing 261 tokens obtained by decomposing character strings extracted from document data 210, to which correct answer data has been assigned, into token units using morphological analysis. Each token is assigned a GT in IOB (Inside-Outside-Beginning) format. In this embodiment, three types of named entities are used: corporate name (ORG), personal name (PERSON), and expiration date (DATE), but other types may be defined and used without being limited to these three.

Because a named entity may be composed of multiple tokens, in such cases, a "B-" is added to the GT of the first token of the named entity, and an "I-" is added to the GT of each subsequent token. For example, "ABC Co., Ltd." in the character string 211 shown in Figure 2(a) is a named entity for a corporate name (ORG), but is composed of two tokens, T1_003 ("ABC") and T1_004 ("Co., Ltd."). Therefore, the GT of token T1_003 is assigned "B-ORG," indicating the first token, and the GT of token T1_004 is assigned "I-ORG," indicating the subsequent token. Tokens to which no named entity type is assigned are assigned an "O," indicating that they are not named entities. In this embodiment, a GT is assigned to each token in IOB format as described above, but other methods may be used to assign GTs to named entities that span multiple tokens.

Figure 3(b) is a token table 320 listing 264 tokens obtained by the control unit 101 breaking down the input string extracted from the document data shown in Figure 2(b) into token units using morphological analysis. Because table 320 is based on document data 220 to which no GT has been assigned, each token has an identifier 321 and a token string 322, but no GT.

Figure 4(a) shows an example of a named entity acquired by the calculation unit 103, using a table. A named entity is composed of a named entity identifier 411, a character string 412, and a type 413, and the minimum number of tokens 414 indicates the minimum number of tokens surrounding the named entity required to correctly extract the named entity. The minimum number of tokens 414 is calculated by the calculation unit 103 based on multiple tokens to which GT is assigned. The named entity NE_001 corresponds to character string 211 in document data 210, and has the character string "ABC Co., Ltd." and the type "Corporate Name (ORG)." Similarly, the named entity NE_002 corresponds to character string 212 in document data 210, and has the character string "DEF Co., Ltd." and the type "Corporate Name (ORG)." The named entity NE_003 corresponds to character string 213 in document data 210, and has the character string "October 31st" and the type "Expiration Date (DATE)." Named entities NE_004 to NE_006 are named entities defined in different document data, and the type of named entity is set to a person's name, a corporate name, or an expiration date, respectively.

Figure 4(b) shows the limit number of tokens derived by the calculation unit 103 for the example shown in Figure 4(a). The limit number of tokens is the largest value among the minimum number of tokens 414 corresponding to all named entities registered in the table 410.

Figure 5(a) is a flowchart showing an example of the limit token number derivation process executed by the control unit 101 in this embodiment. This flowchart is executed by the named entity extraction device 100 at initialization. This process may also be configured to obtain a result calculated by a device different from the named entity extraction device 100, which will be described later.

In S511, the control unit 101 acquires the document data 210 with GT attached from the receiving unit 102.

In S512, the control unit 101 extracts the input character string from the acquired document data and breaks the extracted input character string into token units using morphological analysis.

In S513, the control unit 101 uses the calculation unit 103 to identify the minimum number of tokens that will result in correct extraction of the named entity defined in the assigned GT. In this step, first, the named entity is acquired and stored based on the GT assigned to the token of each document data. For the document data 210 to which the GT is assigned, "ABC Co., Ltd.", "DEF Vacuum Co., Ltd.", and "October 31st" are each stored as named entities in table 410. At this time, an identifier 411 is assigned to each acquired named entity, and the named entity string 412 and named entity type 413 are associated and stored. Next, named entity extraction is performed on the document data, and the minimum number of tokens that will result in correct extraction of the named entity is identified for the extracted named entities. Specifically, the initial value of the number of tokens before and after the named entity is set to 128, and named entity extraction is performed using that number of surrounding tokens. If the extraction of the named entity is correct, the number of surrounding tokens is reduced by one and named entity extraction is performed again, reducing the number of surrounding tokens until extraction is impossible or an incorrect result is obtained. The identified minimum number of tokens that resulted in successful extraction is stored in minimum token number 414 of table 410. The initial value of the number of surrounding tokens when identifying the minimum number of tokens may be a fixed value, or may be determined based on the number of tokens included in the document data or the results of other document data. Methods for reducing the number of tokens when identifying the minimum number of tokens include reducing the number by one, searching using a binary search, or any other method that can determine the minimum number of tokens.

In S514, the calculation unit 103 stores the maximum value among the minimum token numbers for each identified named entity as the limit token number in table 420. Since the maximum value for the minimum token number in table 410 is "7", "7" is stored as the limit token number.

Note that here, the maximum value of the minimum number of tokens is used as the limiting number of tokens, but the smallest number of tokens that is equal to or greater than the minimum number of tokens for a specified percentage of named entities may also be used as the limiting number of tokens. For example, if the limiting number of tokens is the number of tokens that are correct for 80% of named entities, then the limiting number of tokens would be "6," which is the minimum number of tokens (414) that is correct for 80% of entities.

Figure 5(b) is a flowchart showing an example of processing executed by the control unit 101 in this embodiment. This flowchart is executed in response to an instruction to extract named entities from document data being issued by the named entity extraction device 100.

In S521, the control unit 101 extracts an input character string from the document data, decomposes the extracted input character string into token units, and proceeds to S522. In this step, an input character string is extracted from document data 220 that has not been assigned correct answer data, and the extracted input character string is decomposed into token units.

In S522, the control unit 101 compares the number of tokens with the upper limit of the number of inputs of the natural language processing model, which is the named entity extractor. If the number of tokens obtained by decomposing the extracted input character string exceeds the upper limit of the inputs, the control unit 101 proceeds to S523; if it does not, the control unit 101 proceeds to S524. When the upper limit of the inputs of the natural language processing model is set to 256, the number of tokens stored in table 410 is 264, which exceeds the upper limit of the inputs, and the control unit 101 proceeds to S523.

In S523, the division unit 104 divides the tokens contained in the document data into two or more token groups with partial overlap based on the limit number of tokens identified in S514, and then proceeds to S524. In this step, the limit number of tokens stored in table 420 is first obtained. A preset value may be used for this value. One token group is divided into two or more token groups with overlapping tokens, which is 14 tokens, double the limit number of tokens of "7". The reason for doubling the number of overlapping tokens here is to ensure seven tokens as tokens surrounding named entities. A value obtained by adding the average number of tokens for named entities to this value may also be used. In the example shown in table 320, the document data is divided into token group 1 (T2_001 to T2_256) and token group 2 (T2_243 to T2_264), so that the 14 tokens from T2_243 to T2_256 overlap. When dividing, the number "14" calculated from the limit token number can be used as the lower limit for the number of overlapping tokens, and the division can be performed so that the number of tokens is maximized without increasing the number of divisions. For example, the token group in table 320 can be divided into token group 1 from T2_001 to T2_256, and token group 2 from T2_009 to T2_264. In any case, it is sufficient that the token groups are divided so that the number of tokens after division is equal to or greater than the lower limit calculated from the limit token number.

In this embodiment, the limit number of tokens is treated as a single numerical value, but it may also be calculated separately for the number of tokens before and the number of tokens after the named entity, and the sum of these may be used to create the overlap.

In S524, the extraction unit 105 performs named entity extraction for each divided token group. In this step, named entity extraction is performed for each of token group 1 and token group 2, and named entities are obtained. Assume that "GHI Co., Ltd." and "JKL Transport Company" are extracted as the corporate name (ORG) from token group 1, and "March 5th" is extracted as the expiration date (DATE). Assume also that "March 5th" is extracted as the expiration date (DATE) from token group 2. Assume that "March 5th" is extracted from the overlapping parts of token group 1 and token group 2, respectively.

In S525, the control unit 101 determines the named entity extracted in the overlapping portion and terminates processing. Specifically, if the same result is extracted for a common token in the overlapping portion, only one of the results is output; if different results are extracted, the result with the greater number of surrounding tokens is output. Even if one of the results is not detected in the overlapping portion, the result with the greater number of tokens is given priority. In the example shown in S524, "March 5th" is extracted as the expiration date (DATE) in the overlapping portions of both token group 1 and token group 2, so only one result is output. Therefore, the extraction results for table 410 are "GHI Co., Ltd." and "JKL Transport Company" as the corporate names (ORG) and "March 5th" as the expiration date (DATE).

As described above, when extracting named entities from document data with a large number of characters, dividing the data into two or more overlapping token groups prevents the loss of keyword strings and context surrounding the named entity, and minimizes any decline in the accuracy of named entity extraction.

[Embodiment 2]
In the first embodiment, when specifying the minimum number of tokens for each named entity, the number of tokens is gradually reduced to obtain the minimum number of tokens that allows for a correct answer. In contrast, in the present embodiment, the limit number of tokens is obtained based on the degree of association with tokens that appear in the network of the natural language processing model.

Figure 6 is a flowchart showing an example of processing executed by the control unit 101 in this embodiment. Note that steps S511, S512, and S514 in the flowchart are the same as the steps with the same reference numerals in Figure 5(a), and therefore will not be described here.

In S611, the calculation unit 103 calculates the minimum number of tokens that will result in correct named entity extraction for the named entities defined in the GT, based on the degree of association with tokens that appear in the network of the natural language processing model. In this step, first, named entities are obtained based on the GT assigned to the tokens of each document data, and stored in table 410 shown in Figure 4. Table 410 stores named entity identifier 411, named entity string 412, and named entity type 413. Next, for each named entity, the minimum number of tokens 414 that will result in correct named entity extraction is calculated.

Specifically, first, the tokens contained in the document data are divided into two or more token groups so that the number of tokens before and after each named entity is maximized. Then, a natural language processing model with a self-attention mechanism, such as BERT using Transformers, is used to extract named entities from the resulting token groups. If the named entity is correctly extracted, the relationship between the named entity token and its surrounding tokens is measured by the attention strength that appears in the network of the natural language processing model. The minimum number of surrounding tokens with attention strength above a predetermined threshold is determined to be the minimum number of tokens corresponding to the named entity. The calculated minimum number of tokens is stored as minimum number of tokens 414 in table 410.

As described above, when extracting named entities from document data with a large number of characters, dividing the data into two or more overlapping token groups prevents the loss of keyword strings and context surrounding the named entity, and minimizes a decrease in the accuracy of named entity extraction. Furthermore, calculating the minimum number of tokens based on information from the network of the natural language processing model makes it easier to calculate the minimum number of tokens.

[Embodiment 3]
In the first embodiment, the shared limit number of tokens is calculated for each named entity. In contrast, in the present embodiment, the limit number of tokens is calculated for each type of named entity.

An example of the limit number of tokens determined by the calculation unit 103 will be explained using table 700 in Figure 7. The limit number of tokens is composed of a named entity type 701 and a limit number of tokens 702. A value calculated for each named entity type based on the named entity table 410 is entered in 702. In table 700, the limit numbers of tokens for corporate name (ORG), personal name (PERSON), and expiration date (DATE) are defined in 703, 704, and 705, respectively.

Figure 8(a) is a flowchart showing an example of processing executed by the control unit 101 in this embodiment. Note that steps S511, S512, and S513 in the flowchart are the same as the steps with the same reference numerals in Figure 5(a), and therefore their explanation will be omitted.

In S811, the calculation unit 103 determines the limit number of tokens for each type of named entity and terminates the process. The value stored in the minimum number of tokens 414 in the named entity table 410 is obtained, the maximum value is calculated for each type of named entity, and the limit number of tokens is saved in the table 700 for each type of named entity. While the maximum value is used here, the number of tokens that exceeds the minimum number of tokens for a specified percentage of named entities may also be used as the limit number of tokens.

Figure 8(b) is a flowchart showing an example of processing executed by the control unit 101 in this embodiment. Note that steps S521, S522, S524, and S525 in the flowchart are the same as the steps with the same names in Figure 5(b), and therefore their explanation will be omitted.

In S821, the receiving unit 102 accepts the type (or attribute) of the named entity to be extracted, and proceeds to S521. Here, it is assumed that a request has been received from the user of the named entity extraction device to extract named entities of the type of personal names contained in the document data.

In S822, the division unit 104 divides the tokens contained in the document data into two or more partially overlapping token groups based on the limit number of tokens corresponding to the type of named entity received in S821, and proceeds to S524. In this step, the limit number of tokens stored in table 700 is obtained based on the type received in S821. If multiple types are received, the maximum value is used. In this case, since a person's name was received in S821, the tokens contained in the document data are divided into two or more partially overlapping token groups based on the corresponding limit number of tokens, "2."

By limiting the token limit to the minimum number of tokens corresponding to the type of named entity accepted for extraction in this way, it is possible to reduce the number of token group divisions and the number of tokens processed during named entity extraction.

As described above, when extracting named entities from document data with a large number of characters, dividing the data into two or more overlapping token groups prevents the loss of keyword strings and context surrounding the named entity, and minimizes a decrease in the accuracy of named entity extraction. Furthermore, by setting the minimum number of tokens corresponding to the type of named entity to be extracted as the token limit, the number of times the document data needs to be divided can be reduced, thereby reducing the amount of calculations.

[Embodiment 4]
In the first embodiment, all tokens are divided into two or more token groups using a predetermined limit number of tokens. In contrast, in the present embodiment, an example will be described in which the limit number of tokens is updated based on the estimated usage status of named entities.

Figure 9 is a flowchart showing an example of processing executed by the control unit 101 in this embodiment. This flowchart is executed after named entity extraction is performed on document data in a named entity extraction device and the extracted named entities are used by the device user. Note that steps S512 and S513 in the flowchart are the same as the steps with the same reference numerals in Figure 5(a), and therefore their explanation will be omitted.

In S911, document data is acquired to which a GT has been assigned, which defines the token selected by the device user as a named entity, and the process proceeds to S512. Named entities are extracted from the document data, and when the extracted corporate name (ORG) "GHI Co., Ltd." is selected by the user, the corresponding token in table 320 is assigned a GT, which has been additionally defined as a named entity. Specifically, tokens T2_003 and T2_004 are assigned the GTs "B_ORG" and "I_ORG", respectively, and the remaining tokens are assigned the GT "O", and the resulting document data is acquired.

In S912, the calculation unit 103 updates the limit number of tokens and terminates the process. Specifically, the results of the processes in S512 and S513 on the document data acquired in S911 are reflected, and the limit number of tokens table 420 is updated based on the named entity table 410 to which the minimum number of tokens corresponding to the named entity selected by the user has been added.

As described above, when extracting named entities from document data with a large number of characters, dividing the data into two or more overlapping token groups prevents the loss of keyword strings and context surrounding the named entity, thereby minimizing a decrease in the accuracy of named entity extraction. Furthermore, by updating the limit number of tokens, it is possible to minimize a decrease in accuracy even for unknown documents.

(Other Examples)
The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

100 Named Entity Extraction Device 101 Control Unit 102 Receiving Unit 103 Calculation Unit 104 Segmentation Unit 105 Extraction Unit

Claims (11)

An information processing device for extracting named entities from a plurality of tokens obtained by decomposing an input character string, comprising:
a dividing means for dividing the plurality of tokens obtained by decomposing the input character string into two or more token groups when the number of the plurality of tokens obtained by decomposing the input character string exceeds a predetermined upper limit, wherein each of the token groups has a predetermined number of tokens overlapping with other token groups;
extraction means for extracting the named entities for each of the token groups;
a determination means for determining an extraction result of the named entity in the overlapping portion based on an extraction result of the named entity by the extraction means for the overlapping portion with the other token group;
Equipped with
the determining means determines, among the extraction results of the named entities from each of the two overlapping token groups in the overlapping portion, the extraction result of the named entity from the token group with the greater number of tokens in the overlapping token groups as the extraction result of the named entity in the overlapping portion;
1. An information processing device comprising: The predetermined upper limit number is the number of tokens that the extraction means can process at one time.
2. The information processing apparatus according to claim 1, wherein: the predetermined number is set based on a limit number of tokens required for the extraction means to extract the named entity;
3. The information processing apparatus according to claim 1 , wherein the information processing apparatus is a computer. the limit number of tokens is the smallest number of tokens among the numbers of tokens input when the extraction means correctly extracts a named entity from an input character string to which correct answer data defining the named entity has been added;
4. The information processing apparatus according to claim 3 , the limit number of tokens is the smallest number of tokens among the number of tokens whose relevance to tokens corresponding to named entities defined in the correct answer data assigned to the input character string is equal to or greater than a predetermined value in the natural language processing model used by the extraction means;
4. The information processing apparatus according to claim 3 , When the correct answer data defines a plurality of named entities, the limit number of tokens is the largest of the plurality of minimum numbers of tokens corresponding to the plurality of named entities.
6. The information processing apparatus according to claim 4, wherein: When the correct answer data defines a plurality of named entities, the limit number of tokens is the smallest number of tokens among the numbers of tokens input when a predetermined percentage of the plurality of named entities are correctly extracted.
6. The information processing apparatus according to claim 4, wherein: further comprising a receiving means for receiving a type of named entity extracted by the extracting means,
The limit number of tokens is set to the smallest number of tokens input when the extraction of the named entity of the type accepted by the accepting means is correct.
4. The information processing apparatus according to claim 3 , the correct answer data is obtained by additionally defining a named entity selected by a user from the named entities extracted from the input character string by the extraction means.
8. The information processing apparatus according to claim 4, wherein the information processing apparatus is a computer. An information processing method for extracting named entities from a plurality of tokens obtained by decomposing an input character string, comprising:
a step of dividing the plurality of tokens into two or more token groups when the number of tokens obtained by the information processing device decomposing the input character string exceeds a predetermined upper limit, wherein each of the token groups has a predetermined number of tokens overlapping with other token groups;
a step of extracting the named entities for each of the token groups by the information processing device ;
a step of determining an extraction result of the named entity in the overlapping portion based on an extraction result of the named entity in the extraction step for the overlapping portion with the other token group by the information processing device ;
and
the determining step determines, among the extraction results of the named entities from each of the two overlapping token groups in the overlapping portion, the extraction result of the named entity from a token group having a larger number of tokens in the overlapping token groups as the extraction result of the named entity in the overlapping portion;
1. An information processing method comprising: A program for causing a computer to function as the information processing device according to any one of claims 1 to 9 .