JP7700862B2 - Summary learning support device, summary learning support method and program - Google Patents

Description

The present invention relates to a summary learning support device, a summary learning support method, and a program.

The training data for a model that generates summary sentences using a neural network typically consists of pairs of source text to be summarized and summary data that is the correct summary result.

On the other hand, there are models that require input parameters (hereinafter referred to as "queries") other than the source text (for example, Non-Patent Document 1). Such models can generate summaries that correspond to the queries. In such models, a set of parameters such as the source text, the query, and the summary data is used as training data.

On the other hand, there are two types of summary generation methods: extraction and generative. Extraction is a method in which a portion of the source text is extracted as is. Generative is a method in which summary data is generated based on the words and other elements contained in the source text. In what follows, models that require a query as input and generate summary data using a generative method are referred to as "query-dependent generative models."

Gonc, alo M. Correia, Andre F. T. Martins, A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3050-3056, July 28 August 2, 2019.

Although there is a large amount of training data consisting of pairs of source text and summary data, training data that includes additional input parameters other than the source text is insufficient for training a query-dependent generative model.

The present invention has been made in consideration of the above points and aims to improve the efficiency of learning summaries that require additional input parameters.

In order to solve the above problem, the summary learning support device has a calculation unit that calculates, for a plurality of character strings, a score indicating their suitability as input parameters to be added when summarizing a first document based on a predetermined model, and a selection unit that selects a group of character strings from among the plurality of character strings based on the score as the input parameters that constitute training data for a summary generation model that generates a summary of the document, and the score is a score that is calculated for each output candidate character string so that when a second document, which is a summary of the first document, is input to a model that has learned the correspondence between the body of the document and a group of character strings that constitute the title of the document, the model selects a character string to be output from among the output candidate character strings .

It is possible to make learning of summaries more efficient where additional input parameters are required.

1 is a diagram illustrating an example of a hardware configuration of a summary generation device 10 according to a first embodiment. FIG. 1 illustrates an example of a functional configuration of a summary generation device according to a first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of a query-containing learning data generating unit 11 according to the first embodiment. 11 is a flowchart illustrating an example of a processing procedure for generating query-included learning data according to the first embodiment. FIG. 13 is a diagram illustrating an example of a configuration of a query-containing learning data generating unit 11 according to the second embodiment. 13 is a flowchart illustrating an example of a process procedure for generating query-included learning data according to the second embodiment. 13 is a diagram for explaining learning of a summary generation model and generation of a summary in the third embodiment. FIG. 13 is a diagram for explaining learning of a summary generation model and generation of a summary in the fourth embodiment. FIG.

Below, an embodiment of the present invention will be described with reference to the drawings. Figure 1 is a diagram showing an example of the hardware configuration of a summary generation device 10 in a first embodiment. The summary generation device 10 in Figure 1 has a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, an interface device 105, etc., which are all interconnected by a bus B.

The program that realizes the processing in the summary generation device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 via the drive device 100 into the auxiliary storage device 102. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

The memory device 103 reads out and stores the program from the auxiliary storage device 102 when an instruction to start the program is received. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the summary generation device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

Figure 2 is a diagram showing an example of the functional configuration of the summary generation device 10 in the first embodiment. In Figure 2, the summary generation device 10 has a query-included learning data generation unit 11, a summary learning unit 12, and a summarization unit 13. Each of these units is realized by a process in which one or more programs installed in the summary generation device 10 are executed by the processor 104.

The query-containing learning data generation unit 11 generates query-containing learning data based on each of the query-free learning data included in the query-free learning data group given as an input. One query-containing learning data is generated for one query-free learning data. Thus, for a query-free learning data group that is a collection of multiple query-free learning data, a query-containing learning data group that is a collection of multiple query-containing learning data is generated. Both query-free learning data and query-containing learning data refer to data used as learning data for a model such as a neural network that generates a summary of a document (hereinafter referred to as a "summary generation model"). Query-free learning data differs from query-containing learning data in that it does not include a query as a component. A query refers to a text (character string) that is input to a summary generation model together with a document to be summarized as additional information related to the summary. For example, the focus of the summary may be the query.

Query-free training data is training data consisting of a pair of two pieces of text data: {source text, summary text}. Source text refers to the text data of the document to be summarized. Summary text refers to the text data that indicates the correct answer resulting from summarizing the source text.

On the other hand, the query-based training data is training data composed of a set of three pieces of text data: {source text, query, summary text}.

The summary learning unit 12 trains the summary generation model using query-containing learning data.

When the summarization unit 13 receives input such as a source text to be summarized and a query for the source text, it inputs the source text and the query into a trained summary generation model, causing the summary generation model to generate a summary corresponding to the query for the source text.

The query-included learning data generation unit 11 will now be described in more detail. FIG. 3 is a diagram showing an example configuration of the query-included learning data generation unit 11 in the first embodiment. In FIG. 3, the query-included learning data generation unit 11 has an importance calculation unit 111, a query selection unit 112, and a query addition unit 113. The functions of each of these units will now be described in detail with reference to FIG. 4.

Figure 4 is a flowchart illustrating an example of a processing procedure for generating query-containing learning data in the first embodiment.

In step S101, the importance calculation unit 111 generates a document (hereinafter referred to as an "extraction source document") from which a character string that is a candidate for a query is extracted for each piece of query-free learning data (a pair of source text and summary text) included in the query-free learning data group. Therefore, N extraction source documents are generated from N pieces of query-free learning data.

For example, the importance calculation unit 111 generates any one of the following (a) to (d) of the no-query learning data as an extraction source document based on the no-query learning data.
(a) A document that combines source text and summary text (a document that contains both source text and summary text)
(b) summary text only; (c) source text only; (d) a document combining any one of (a) to (c) with other auxiliary information text (e.g., the title of the source text, etc.). Next, the importance calculation unit 111 calculates the importance of each character string (e.g., word) of a predetermined unit constituting each extraction source document in the extraction source document group as an example of a score indicating the appropriateness of the character string as a query (added input parameter) used in summarizing the document based on a predetermined model (S102). For example, the importance calculation unit 111 uses a TF-IDF calculation model as the predetermined model. In this case, the importance calculation unit 111 calculates the TF-IDF of each word as the importance. The TF-IDF of each word included in the document group can be calculated using a known method. In this embodiment, the "parameter" in the input parameter is clearly distinguished from, for example, a learning parameter of a model such as a neural network. The input parameter is data given as an input to the model, whereas the learning parameter is data whose value changes according to the learning of the model. As a general example, the input parameters are given as text data or the like, whereas the learning parameters are expressed as a set of numerical data or the like.

Next, for each source document, the query selection unit 112 selects K character strings in descending order of importance from among the character strings (words) of a predetermined unit constituting the source document as queries corresponding to the query-free learning data corresponding to the source document (S103). Note that the value of K (K>=0) may be selected randomly for each source document, or may be the same for all source documents. In addition, when selecting a query from each source document, the query selection unit 112 may select only words included in the summary text of the source document as queries. This makes it easier to train the summary generation model to include the specified query in the summary.

Next, the query adding unit 113 generates query-included learning data by adding, for each piece of query-free learning data, K words selected from the source document based on the query-free learning data as queries to the query-free learning data (S104). Therefore, the generated query-included learning data includes the source text and summary text contained in the query-free learning data, and the K queries (query strings) extracted from the query-free learning data.

As described above, according to the first embodiment, pseudo queries can be generated from training data that does not include queries. This makes it possible to efficiently train summaries that require additional input parameters.

Next, the second embodiment will be described. In the second embodiment, differences from the first embodiment will be described. Points not specifically mentioned in the second embodiment may be the same as those in the first embodiment.

In the second embodiment, the configuration of the query-included learning data generation unit 11 and the processing procedure performed by the query-included learning data generation unit 11 differ from those in the first embodiment.

Figure 5 is a diagram showing an example of the configuration of the query-containing learning data generation unit 11 in the second embodiment. In Figure 5, the same or corresponding parts as those in Figure 3 are given the same reference numerals. In the second embodiment, the query-containing learning data generation unit 11 has a query generation model learning unit 114 and a query candidate generation unit 115 instead of the importance calculation unit 111. The query generation model learning unit 114 learns a model (hereinafter referred to as a "query generation model") that generates one or more queries from query-free learning data. The query generation model is composed of, for example, a neural network. The query generation model learning unit 114 receives as input a group of learning documents that are the source of learning data for the query generation model. The group of learning documents refers to a collection of multiple learning documents. The learning documents refer to text-format document data that includes a title (heading) and a body text, such as an encyclopedia published on the Internet, such as Wikipedia, or a newspaper.

The query candidate generation unit 115 generates (outputs) query candidates based on the learned query generation model.

Figure 6 is a flowchart illustrating an example of a processing procedure for generating query-containing learning data in the second embodiment.

In step S201, the query generation model training unit 114 generates training data of the query generation model for each training document included in the training document group. Specifically, the query generation model training unit 114 decomposes (divides) the title of each training document into character strings of a predetermined unit (e.g., words). Therefore, for example, a word string (hereinafter simply referred to as a "word string") constituting the title is generated for each training document. At this time, the query generation model training unit 114 may generate a word string from which stop words have been deleted. For each training document, the query generation model training unit 114 generates, as training data, a pair of the main text (paragraph text) of the training document and a word string generated from the title corresponding to the main text.

Next, the query generation model training unit 114 trains the query generation model using the training data group generated in step S201 (S202). Specifically, the query generation model training unit 114 trains the query generation model to learn the correspondence between the body text of each training data item and the word string in the title when the body text is input and the word string in the title is output. Thus, when the body text of a document is input, the query generation model is trained to output a word string related to the title of the document. Note that the query generation model may be configured, for example, by a known encoder-decoder model or another known sentence generation model.

Next, for each piece of query-free learning data, the query candidate generation unit 115 inputs the summary text of the query-free learning data into the trained query generation model, and generates a group of strings (word strings) output by the query generation model as a query candidate string corresponding to the query-free learning data (S203).

In this case, if the query generation model is an encoder-decoder model, the query generation model sequentially outputs each word constituting the word string in response to the input of query-free training data. In sequential output of words, the query generation model calculates a score for selecting an output target from among output candidates for each of D words constituting its vocabulary (a set of words that are output candidates of the query generation model), and outputs the word with the maximum score. In the second embodiment, the score corresponds to an example of a score representing the suitability of a query (added input parameter) to be used when summarizing a document.

Next, for each piece of query-free learning data, the query selection unit 112 selects one or more words (query strings) to be used as a query from the query candidate strings generated by the query candidate generation unit 115 for the query-free learning data (S204). At this time, the query selection unit 112 may select all of the query candidate strings as query strings, or may select a part of the query candidate string as a query string. When selecting a part of the query candidate string as a query string, the query selection unit 112 may select the first to Kth words of the query candidate string as a query string. That is, among the words sequentially output by the query generation model, the Kth words may be selected as a query. Alternatively, in step S203, the number of sequential word outputs from the query generation model may be suppressed to K times. In this case, the query candidate string is composed of K words. Therefore, in this case, in step S204, all of the query candidate strings may be selected as query strings.

Next, the query addition unit 113 generates query-included learning data by adding, for each piece of query-free learning data, the query sequence selected for that piece of query-free learning data to that piece of query-free learning data (S205).

As described above, the second embodiment can achieve the same effects as the first embodiment.

Next, as a third embodiment, a first example of training a summary generation model using training data with a query and generating a summary using the trained summary generation model will be described. Note that the third embodiment can be applied to both the first and second embodiments.

Figure 7 is a diagram for explaining the learning of the summary generation model and the generation of summaries in the third embodiment. In Figure 7, the summary unit 13 includes a content selection unit 131, an encoder 132, and a decoder 133. Each of these units constitutes the summary generation model.

When training the summary generation model, the summary training unit 12 inputs the source text and query string of each piece of training data (source text, query string, summary text) included in the training data group with query to the summary unit 13.

The content selection unit 131 is a model (e.g., a neural network) that calculates the importance of each character string (e.g., word) that constitutes the text obtained by combining the source text and the query string (hereinafter referred to as the "combined text"). The content selection unit 131 may be configured by finetuning a pre-trained model such as BERT or MASS. For more information on BERT, see, for example, "https://arxiv.org/abs/1810.04805". For more information on MASS, see, for example, "https://arxiv.org/abs/1905.02450".

The content selection unit 131 extracts N word strings (keyword strings) in descending order of importance from the combined text, and inputs the key word string, the source text given as input, and the query string to the encoder 132. At this time, the content selection unit 131 combines the query string, the key word string, and the source text with a special token such as [SEP], as in "query string [SEP] key word string [SEP] source text". Note that the value of N may be input to the content selection unit 131 together with the query, etc.

The encoder 132 and the decoder 133 are, for example, known encoder-decoder models (neural networks) such as BERT or MASS.

The encoder 132 encodes the input text. The decoder 133 generates and outputs summary text based on the encoding results.

The summary learning unit 12 updates the learning parameters of the encoder 132 and the decoder 133 based on a comparison between the summary text included in the learning data and the summary text output by the decoder 133. Note that the comparison and the update of the learning parameters may be performed based on publicly known technology.

Once the learning is complete, the summarization unit 13 functions as a trained summary generation model that takes the query string and input text as input and outputs summary text.

In addition, the summary section 13 in Figure 7 may be configured using the technology disclosed in International Publication No. 2021/064907.

Next, as a fourth embodiment, a second example of training a summary generation model using training data with a query and generating a summary using the trained summary generation model will be described. Note that the fourth embodiment can be applied to both the first and second embodiments.

Figure 8 is a diagram for explaining the learning of the summary generation model and the generation of summaries in the fourth embodiment. In Figure 8, parts that are the same as or correspond to those in Figure 7 are given the same reference numerals. In Figure 8, the summary unit 13 includes an encoder 132 and a decoder 133. These parts constitute the summary generation model. In other words, the summary generation model of the fourth embodiment does not have a content selection unit 131.

When training the summary generation model, the summary learning unit 12 inputs the source text and query string of each piece of training data (source text, query string, summary text) included in the training data group with query to the summary unit 13. At this time, the summary learning unit 12 combines the query string and source text with a special token such as [SEP], such as "query string [SEP] source text".

The encoder 132 encodes the input text. The decoder 133 generates and outputs summary text based on the encoding results.

The summary learning unit 12 updates the learning parameters of the encoder 132 and the decoder 133 based on a comparison between the summary text included in the learning data and the summary text output by the decoder 133. Note that the comparison and the update of the learning parameters may be performed based on publicly known technology.

Once the learning is complete, the summarization unit 13 functions as a trained summary generation model that takes the query string and input text as input and outputs summary text.

In addition, in the fourth and fifth embodiments, the summary unit 13 may be configured based on a sentence generation model other than the encoder-decoder model.

The following notes are further disclosed with respect to the above embodiments.

(Additional note 1)
Memory,
at least one processor coupled to the memory;
Including,
The processor,
calculating, for each of the plurality of character strings, a score indicative of a suitability thereof as an input parameter to be added in summarizing the first document based on a predetermined model;
selecting a group of character strings from among the plurality of character strings based on the scores as the input parameters constituting training data for a summary generation model that generates a summary of the document;
A summary learning support device comprising:

(Additional note 2)
calculating, for each of the plurality of character strings, a score indicative of a suitability thereof as an input parameter to be added in summarizing the first document based on a predetermined model;
selecting a group of character strings from among the plurality of character strings based on the scores as the input parameters constituting training data for a summary generation model that generates a summary of the document;
A recording medium on which a program for causing a computer to execute a process is recorded.

In each of the above embodiments, the summary generation device 10 is an example of a summary learning support device. The importance calculation unit 111 or the query candidate generation unit 115 (query generation model) is an example of a calculation unit. The query selection unit 112 is an example of a selection unit. The summary learning unit 12 is an example of a learning unit.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

REFERENCE SIGNS LIST 10 Summary generation device 11 Query-containing learning data generation unit 12 Summary learning unit 13 Summarization unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 Processor 105 Interface device 111 Importance calculation unit 112 Query selection unit 113 Query addition unit 114 Query generation model learning unit 115 Query candidate generation unit 131 Content selection unit 132 Encoder 133 Decoder B Bus

Claims (6)

a calculation unit that calculates a score representing suitability of a plurality of character strings as an input parameter to be added when summarizing the first document based on a predetermined model;
a selection unit that selects a group of character strings from among the plurality of character strings based on the score as the input parameters that constitute training data for a summary generation model that generates a summary of a document;
having
the score is a score calculated for each character string of an output candidate in order for the model to select a character string to be output from among the character strings of the output candidates when a second document, which is a summary of the first document, is input to a model that has learned a correspondence relationship between a body of a document and a group of character strings constituting a title of the document.
A summary learning support device comprising: The score is a degree of importance in a third document for each of a plurality of character strings constituting the third document including either or both of the first document and a second document which is a summary of the first document.
2. The summary learning support device according to claim 1. a learning unit that learns the summary generation model using learning data including the first document, the group of character strings, and a second document that is a summary of the first document;
3. The summary learning support device according to claim 1 , further comprising: a summarization unit that inputs a certain document and a character string related to a summary of the certain document to the summary generation model trained by the training unit, and generates a summary of the certain document;
4. The summary learning support device according to claim 3, further comprising: a calculation step of calculating, for each of the plurality of character strings, a score representing a suitability thereof as an input parameter to be added in summarizing the first document based on a predetermined model;
a selection step of selecting a group of character strings from among the plurality of character strings based on the scores as the input parameters constituting training data for a summary generation model that generates a summary of the document;
The computer executes
the score is a score calculated for each character string of an output candidate in order for the model to select a character string to be output from among the character strings of the output candidates when a second document, which is a summary of the first document, is input to a model that has learned a correspondence relationship between a body of a document and a group of character strings constituting a title of the document.
A method for supporting summary learning comprising the steps of: 5. A program for causing a computer to function as the summary learning support device according to claim 1 .

Images