JP7731849B2 - Dialogue management device, dialogue management method, and dialogue management system - Google Patents

Description

The present invention relates to a dialogue management device, a dialogue management method, and a dialogue management system.

In recent years, giving machines the ability to understand language has become a central challenge in natural language processing and artificial intelligence. Machine Reading Comprehension (MRC) is a task for developing and evaluating a machine's language understanding ability, and the machine's level of understanding can be measured by whether it can correctly answer questions set about a set sentence. Furthermore, MRC technology can be applied to industry and used in dialogue systems that answer human questions.

One of the problems with current MRC is that the answer to a question may not necessarily exist in the given target text. When the answer to a question does not exist in the given target text, the MRC model will provide the best possible answer from the target text based on the clues it has learned. However, because the correct answer to the question does not exist in the text, the model may output an inappropriate response that is not a valid answer to the question.

Several proposals have been made to address the problem that MRC models output inappropriate responses when the answer to a question does not exist in the given target sentence.
For example, research by Nakanishi Mao, Kobayashi Tetsunori, and Hayashi Yoshihiko (Non-Patent Document 1) discloses the following technology: "In this study, we propose a simple method for automatically creating unanswered question data from existing datasets. Furthermore, we propose a method for automatically assigning a level of difficulty to unanswered questions to identify that the question does not have an answer in the text. This demonstrates that it is possible to control the created dataset to have a certain level of difficulty."

Mao Nakanishi, Tetsunori Kobayashi, Yoshihiko Hayashi, "Machine Reading Comprehension: Answering the Unanswerable," Faculty of Science and Engineering, Waseda University, Association for Natural Language Processing, 24th Annual Conference, Proceedings of the 2018 Annual Meeting, March 2018.

The above-mentioned non-patent document 1 describes a method for creating a dataset to train an MRC model that can respond "unable to answer" to questions about content not described in the text, and a method for assigning a level of difficulty to identify questions that do not have an answer in the text.

According to the above-mentioned Non-Patent Document 1, it is possible to train an MRC model that can respond "I cannot answer" to questions about content not described in the text. However, from the user's perspective, the response "I cannot answer" may not be perceived as sufficient, leading to dissatisfaction. For this reason, Non-Patent Document 1 does not consider a method for outputting an appropriate response even for content not described in the text, and this is an issue that needs to be explored in the future.

The present disclosure therefore aims to provide a highly flexible and robust dialogue management means that can respond to a variety of questions by training an MRC model using reference sentences generated based on natural language conversations.

To solve the above problem, one representative dialogue management device of the present invention includes an answer generation unit that receives a first question and generates a first answer to the first question based on a first reference sentence; a sentence generation unit that generates a second reference sentence including the content of the question and answer based on dialogue data including the question and answer; and an adaptation unit that trains the answer generation unit to generate an appropriate answer to the second question using the dialogue data, a second answer label indicating a correct answer to the second question, and the second reference sentence.

According to the present disclosure, by training an MRC model using reference sentences generated based on natural language conversations, it is possible to provide a highly flexible and robust dialogue management means that can respond to a variety of questions.
Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention.

FIG. 1 is a diagram illustrating a computer system for implementing an embodiment of the present disclosure. FIG. 2 is a diagram showing the flow of a question-answering process using a conventional MRC model. FIG. 3 is a diagram illustrating an example of a configuration of a dialogue management system according to an embodiment of the present disclosure. FIG. 4 is a block diagram illustrating an example of the flow of an answer generator training process for training an answer generator according to an embodiment of the present disclosure. FIG. 5 is a block diagram illustrating an example of the flow of a sentence generation unit training process according to an embodiment of the present disclosure. FIG. 6 is a diagram illustrating a specific example of a context label generation process according to an embodiment of the present disclosure. FIG. 7 is a diagram illustrating a specific example of the dialogue aggregation process according to an embodiment of the present disclosure. FIG. 8 is a block diagram showing an example of the flow of adaptation processing according to an embodiment of the present disclosure.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.
Additionally, although terms such as "first,""second," and "third" may be used to describe various elements or components in this disclosure, it will be understood that these elements or components should not be limited by these terms. These terms are used only to distinguish one element or component from another. Thus, a first element or component discussed below could also be referred to as a second element or component without departing from the teachings of the inventive concept.

Next, referring to FIG. 1, a computer system 100 for implementing embodiments of the present disclosure will be described. The mechanisms and devices of various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 100 include one or more processors 102, memory 104, a terminal interface 112, a storage interface 113, an I/O (input/output) device interface 114, and a network interface 115. These components may be interconnected via a memory bus 106, an I/O bus 108, a bus interface unit 109, and an I/O bus interface unit 110.

Computer system 100 may include one or more general-purpose programmable central processing units (CPUs) 102A and 102B, collectively referred to as processors 102. In some embodiments, computer system 100 may include multiple processors, while in other embodiments, computer system 100 may be a single CPU system. Each processor 102 executes instructions stored in memory 104 and may include an on-board cache.

In some embodiments, memory 104 may include random-access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. Memory 104 may store all or part of the programs, modules, and data structures that implement the functions described herein. For example, memory 104 may store a dialogue management application 150. In some embodiments, dialogue management application 150 may include instructions or descriptions that execute the functions described below on processor 102.

In some embodiments, the dialogue management application 150 may be implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and/or other physical hardware devices instead of or in addition to a processor-based system. In some embodiments, the dialogue management application 150 may include data other than instructions or descriptions. In some embodiments, a camera, sensor, or other data input device (not shown) may be provided to communicate directly with the bus interface unit 109, the processor 102, or other hardware of the computer system 100.

Computer system 100 may include a bus interface unit 109 that facilitates communication between processor 102, memory 104, display system 124, and I/O bus interface unit 110. I/O bus interface unit 110 may couple to I/O bus 108 for transferring data to and from various I/O units. I/O bus interface unit 110 may communicate via I/O bus 108 with multiple I/O interface units 112, 113, 114, and 115, also known as I/O processors (IOPs) or I/O adapters (IOAs).

Display system 124 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both data to display device 126. Computer system 100 may also include one or more sensors or other devices configured to collect data and provide that data to processor 102.

For example, computer system 100 may include biometric sensors that collect heart rate data, stress level data, etc., environmental sensors that collect humidity data, temperature data, pressure data, etc., and motion sensors that collect acceleration data, movement data, etc. Other types of sensors may also be used. Display system 124 may be connected to a display device 126, such as a standalone display screen, a television, a tablet, or a handheld device.

The I/O interface unit provides the ability to communicate with various storage or I/O devices. For example, the terminal interface unit 112 may be equipped with user I/O devices 116, such as user output devices such as a video display, a television with speakers, or other user input devices such as a keyboard, mouse, keypad, touchpad, trackball, buttons, light pen, or other pointing device. A user may use a user interface to enter input data or instructions into the user I/O devices 116 and the computer system 100 and receive output data from the computer system 100 by operating the user input devices. The user interface may be displayed on a display, played through speakers, or printed via a printer via the user I/O devices 116, for example.

Storage interface 113 allows attachment of one or more disk drives or direct access storage devices 117 (typically magnetic disk drive storage devices, but may also be an array of disk drives or other storage devices configured to appear as a single disk drive). In some embodiments, storage device 117 may be implemented as any secondary storage device. The contents of memory 104 may be stored in storage device 117 and retrieved from storage device 117 as needed. I/O device interface 114 may provide an interface to other I/O devices, such as printers, fax machines, etc. Network interface 115 may provide a communications path that allows computer system 100 and other devices to communicate with each other. This communications path may be, for example, network 130.

In some embodiments, computer system 100 may be a device that receives requests from other computer systems (clients) without a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer. In other embodiments, computer system 100 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

Next, we will explain the flow of question-answering processing using a conventional MRC model, with reference to Figure 2.

As mentioned above, Machine Reading Comprehension (MRC) has long been known in the fields of natural language processing and artificial intelligence as a means of giving machines language understanding capabilities. MRC is a task for developing and evaluating a machine's language understanding capabilities, and the machine's level of understanding can be measured by whether it can correctly answer questions set regarding a set sentence. The flow of question-answering processing using conventional MRC is shown in Figure 2.

First, in step S210, a question 215 such as "How do I connect my smartphone to Wi-Fi (registered trademark)?" is received. Then, in step S220, the trained MRC unit analyzes the received question 215 using natural language processing, refers to a pre-prepared reference sentence 225, and identifies an answer index in the reference sentence 225 that specifies the start and end points of the part that answers the question 215. Then, in step S230, the part of the sentence specified by the identified answer index is output as the answer to the question 215.

As an example, when receiving the question 215, "How do I connect my smartphone to Wi-Fi?", the trained MRC unit outputs as an answer the sentence "You can connect to Wi-Fi on the 'Settings' screen on your smartphone," which is specified in the answer index "60-71" in the reference sentence 225.

Such MRC models have been used, for example, in online chat support and call centers. If the answer to a question asked of a chat support or call center is present in a pre-prepared sentence, it is identified by the MRC model and output as the answer.

However, as described above, an appropriate answer to a question may not always exist in the reference sentences prepared in advance. When the answer to a question does not exist in the given reference sentences, the MRC model answers the answer that is considered to be the best answer from the reference sentences based on the learned clues. However, since the correct answer to the question does not exist in the reference sentences, there is a problem in that the MRC model outputs an inappropriate response that is not a valid answer to the question.
In such cases, for example, in online chat support or call centers, an escalation response may occur in which the issue is passed to a supervisor or other person in charge, but this is undesirable because it takes time for both the user who asked the question and the person in charge who is responding.

Therefore, in consideration of the above-mentioned problems, the present disclosure relates to a highly flexible and robust dialogue management method that can respond to a variety of questions by training an MRC model using reference sentences generated based on natural language conversations.

Next, with reference to Figure 3, we will explain the dialogue management system according to an embodiment of the present disclosure.

Figure 3 is a diagram illustrating an example of the configuration of a dialogue management system 300 according to an embodiment of the present disclosure. As shown in Figure 3, the dialogue management system 300 includes a client terminal 320 and a dialogue management device 340. In the dialogue management system 300, the client terminal 320 and the dialogue management device 340 may be connected to each other via a communication network 330.

The client terminal 320 is a device for interacting with the user 310. As an example, the client terminal 320 may be a computing device configured to provide answers to the user's 310 questions, for example, through online chat support or a call center. In some embodiments, the client terminal 320 may be configured to work with the interaction management device 340 to respond to the user 310.

As shown in FIG. 3, the client terminal 320 may include a receiving unit 322, a first communication unit 324, a response unit 326, and a storage unit 328 that stores dialogue data 329.
The reception unit 322 is a functional unit that receives questions from the user 310. The reception unit 322 may receive questions such as, for example, "What are the conditions for applying for a mortgage?" or "How do I connect my smartphone to Wi-Fi?" The reception unit 322 may be configured to receive questions uttered by voice, for example, or may be configured to receive questions inputted as text.

The first communication unit 324 is a functional unit for performing two-way data communication with the dialogue management device 340. In one embodiment, the first communication unit 324 may transmit a question received by the reception unit 322 to the dialogue management device 340 via the communication network 330 and receive an answer generated by the dialogue management device 340.

The communications network 330 may include, for example, a local area network (LAN), a wide area network (WAN), a satellite network, a cable network, a Wi-Fi network, or any combination thereof.

The response unit 326 is a functional unit for outputting a response to a question received from the user 310 by the reception unit 322. In one embodiment, the response unit 326 may respond to the user 310 using the answer received from the dialogue management device 340.

The storage unit 328 is a memory area that stores dialogue data 329 that indicates the history of dialogues conducted with the user 310.

The dialogue management device 340 is a functional unit for generating answers to various questions. In an embodiment, the dialogue management device 340 may be implemented as a server device or multiple server devices in a distributed computing environment such as a cloud, and may provide the dialogue management means according to an embodiment of the present disclosure as a service to the client terminal 320. As shown in FIG. 3 , the dialogue management device 340 includes an answer generation unit 342, an answer generation unit training unit 344, a sentence generation unit 346, a sentence generation unit training unit 348, an adaptation unit 350, and a second communication unit 352.
These functional units may be software modules constituting the dialogue management application 150 shown in Fig. 1, or may be independent dedicated hardware devices. Furthermore, the functional units may be implemented in the same computing environment or in distributed computing environments.

The answer generation unit 342 is a functional unit for generating an appropriate answer (e.g., a first answer) to a received question (e.g., a first question) based on a predetermined reference sentence (e.g., a first reference sentence). In one embodiment, the answer generation unit 342 may be an MRC model that receives a question received by the client terminal 320 from the user 310 via the communication network 330, identifies answer indexes in the predetermined sentence that specify the start and end points of a portion that serves as an answer to the question, and generates the answer.
In this disclosure, the expression "appropriate answer" means an answer that provides meaningful information to the user's question.

The questions received by the answer generation unit 342 may correspond to a specific domain. Here, a domain refers to a specific range of information, content, topic, purpose, or topic, and may relate to any matter, such as "reservation of a restaurant," "searching for real estate," "transfer procedures," "applying for a mortgage," or "connecting to Wi-Fi." In addition, in this disclosure, the expression "corresponding to a domain" means that a certain question, answer, sentence, etc. has some degree of relevance to a specific domain. As an example, the question "Are there any newly built apartments for sale near XX Station?" may be considered to correspond to the domain of "searching for real estate."

In this disclosure, terms such as "first domain" and "second domain" are used to describe embodiments of the present disclosure, but these "first domain" and "second domain" may be the same domain or different domains. In some embodiments, the "first domain" may be a source domain that the MRC model already supports, and the "second domain" may be a target domain to which the MRC model is to be adapted. In some embodiments, the second domain may cover a substantially similar topic or topic as the first domain, but may include information not included in the first domain.
Similarly, please note that terms such as "first..." and "second..." used in the following description are used to distinguish elements, not to limit them.

The answer generation unit training unit 344 is a functional unit for training the answer generation unit 342. More specifically, the answer generation unit training unit 344 may train the answer generation unit 342 so that it can generate appropriate answers to questions corresponding to a first domain (e.g., the source domain). As will be described later, training an MRC model such as the answer generation unit 342 requires three pieces of data: dialogue data corresponding to the target domain, reference sentences containing answers to questions in the dialogue data, and answer labels that serve as ground truth indicating correct answers to the questions.
In an embodiment, the answer generator training unit 344 may train the answer generator 342 to reduce the loss of a first answer to a first question corresponding to a first domain, generated by the answer generator 342, relative to a first answer label indicating a correct answer to the first question, thereby enabling the answer generator 342 to generate answers with higher accuracy to questions corresponding to the first domain.

The sentence generation unit 346 is a functional unit for generating reference sentences used by the above-mentioned answer generation unit 342 from dialogue data composed of natural language. In one embodiment, the sentence generation unit 346 may generate second reference sentences including information about the second domain based on second dialogue data including questions and answers corresponding to the second domain. Here, the sentence generation unit 346 may generate the reference sentences based on dialogue data 329 collected from a dialogue with a user and stored in the above-mentioned storage unit 328, for example. In this way, the sentence generation unit 346 can generate second reference sentences including new information (e.g., information from dialogue data 329) that is not included in the reference sentences corresponding to the first domain.

The sentence generator training unit 348 is a functional unit for training the sentence generator 346. More specifically, the sentence generator training unit 348 may train the sentence generator 346 to improve the accuracy of generating reference sentences that extract meaningful information from the dialogue data. In one embodiment, the sentence generator training unit 348 may train the sentence generator 346 to reduce the loss of the second reference sentences generated by the sentence generator 346 relative to the sentence labels that serve as ground truth containing correct content related to the dialogue data. This enables the sentence generator 346 to generate reference sentences with higher accuracy from the dialogue data.

The adaptation unit 350 is a functional unit for expanding the range of coverage of the answer generation unit 342 by training the answer generation unit 342 using the reference sentences generated by the sentence generation unit 346 described above. Here, the expression "expanding the range of coverage of the answer generation unit 342" means enabling the answer generation unit 342 to generate appropriate answers to more questions (e.g., questions relating to more domains, contents, topics, etc.). In one embodiment, the adaptation unit 350 may train the answer generation unit 342 to generate answers to questions corresponding to the second domain using second dialogue data corresponding to the second domain, answer labels for questions corresponding to the second domain generated by the answer generation unit 342, and second reference sentences generated by the sentence generation unit 346.

In principle, an MRC unit such as the answer generation unit 342 is trained by the answer generation unit training unit 344 described above using reference sentences corresponding to a specific domain (e.g., specific content and format) as training data. Therefore, answers to questions corresponding to a similar domain to the reference sentences that served as training data can be generated based on the reference sentences that serve as training data. However, performance is limited for questions corresponding to a different domain from the training data. Therefore, by using the adaptation unit 350 according to an embodiment of the present disclosure to train the answer generation unit 342 based on reference sentences corresponding to a different domain from the training data used in training by the answer generation unit training unit 344 (e.g., second reference sentences corresponding to a second domain generated by the sentence generation unit 346), the answer generation unit 342 can expand its range of coverage and generate appropriate answers to questions regarding a variety of content.

The second communication unit 352 is a functional unit for performing two-way data communication with the client terminal 320. In one embodiment, the second communication unit 352 may transmit the answer generated by the answer generation unit 342 to the client terminal 320 via the communication network 330.

The dialogue management system 300 configured as described above can provide a highly flexible and robust dialogue management means that can respond to a variety of questions by training the MRC model using reference sentences generated based on natural language conversations. In other words, in the dialogue management system 300 according to the present disclosure, the answer generation unit 342 is trained using reference sentences generated based on dialogue data that includes new information, making it possible to generate appropriate answers to questions about the new information that could not be answered previously.

Next, with reference to Figure 4, we will explain the answer generation unit training process according to an embodiment of the present disclosure.

Figure 4 is a block diagram showing an example of the flow of an answer generation unit training process 400 for training the answer generation unit 342 according to an embodiment of the present disclosure. This answer generation unit training process 400 is a process for training the answer generation unit 342 so that it can generate an appropriate first answer to a first question corresponding to a first domain (e.g., a source domain) based on a first reference sentence corresponding to the first domain, and is executed by the answer generation unit training unit 344 described above.

4 is information representing a conversation written in natural language, and includes a first question 414 and a first answer 418 to the first question 414. In an embodiment, the first dialogue data 410, and the first question 414 and first answer 418 included in the first dialogue data 410 may correspond to a first domain.

This first interaction data 410 may be, for example, a history of a conversation between a questioner seeking a first answer 418 to a first question 414 and a respondent providing the first answer 418 to the questioner's first question 414. As an example, this interaction data may be a history of a conversation between a questioner seeking a first answer 418 to a first question 414 and a representative in an online chat support or call center who becomes available to assist the questioner after an escalation occurs.

First, the answer generation unit training unit 344 inputs the first question 414 contained in the first dialogue data 410 and the first reference sentence 420 containing the first answer 418 to the first question 414 to the answer generation unit 342. The answer generation unit 342 here may be a trained MRC model such as the Bidirectional and Auto-Regressive Transformer (BART) or the Robustly Optimized BERT Pretraining Approach (ROBERTA).

Next, the answer generator 342 identifies a sentence portion in the first reference sentence 420 that is an answer to the input first question 414, and determines a first answer index 435 that specifies the start and end points of the identified sentence portion. More specifically, the encoder of the MRC model that serves as the answer generator 342 generates a contextual representation (also called an intermediate representation) that represents the first question 414 and the first reference sentence 420, and outputs it to the linear layer. The linear layer then generates a logit that indicates the probability that the input contextual representation of the encoder represents the correct sentence portion that is an answer to the first question 414.

Next, the answer generation unit training unit 344 calculates the loss of the first answer index 435 generated by the answer generation unit 342 with respect to the first answer label 440, which serves as the ground truth indicating the correct answer to the first question 414, and modifies the model parameters of the answer generation unit 342 to reduce the loss, for example, using a technique such as the so-called Cross Entropy Loss.

By repeating the above-described answer generator training process 400, the answer generator 342 improves the accuracy of outputting appropriate answers to input questions.
However, as described above, in this answer generator training process 400, an MRC unit such as the answer generator 342 is trained by the above-mentioned answer generator training unit 344 using reference sentences corresponding to a specific domain (e.g., specific content and format) as training data, so although it is possible to generate answers to questions corresponding to a similar domain to the reference sentences that became the training data based on the reference sentences that also become the training data, its performance is limited for questions corresponding to a different domain from the training data. Therefore, as will be described later, in order to expand the coverage range of the answer generator 342 and generate appropriate answers to various questions, processing by the sentence generator 346 and adaptation unit 350, which will be described later, is required.

Next, with reference to Figure 5, we will explain the sentence generation unit training process according to an embodiment of the present disclosure.

Figure 5 is a block diagram showing an example of the flow of a sentence generator training process 500 according to an embodiment of the present disclosure. This sentence generator training process 500 is a process for training the sentence generator 346 to generate reference sentences used by the answer generator 342 to generate appropriate answers to questions from dialogue data written in natural language, and is executed by the sentence generator training unit 348 described above.

First, the sentence generator training unit 348 inputs second dialogue data 510 consisting of a second question 514 and a second answer 518 to the sentence generator 346. This second dialogue data 510 and the second question 514 and second answer 518 included in the second dialogue data 510 may correspond to a second domain (e.g., a target domain). In other words, this second dialogue data 510 includes new information (e.g., information not included in the first reference sentence used to train the answer generator 342 in the answer generator training process 400 described above). As described above, this second dialogue data 510 may be a history of a conversation between a questioner seeking a second answer 518 to a second question 514 and a respondent providing the second answer 518 to the questioner's second question 514.

Next, the sentence generation unit 346 generates second reference sentences 520 including the content of the second dialogue data 510 based on the input second dialogue data 510. In other words, the sentence generation unit 346 extracts essential information (e.g., information that serves as an answer to a question) included in the second dialogue data 510 and performs a process of composing the information into a sentence. In this way, the sentence generation unit 346 can generate reference sentences including new information included in the second dialogue data 510 (e.g., information that is not included in the first reference sentences used to train the answer generation unit 342 in the above-described answer generation unit training process 400).
The sentence generation unit 346 here may be implemented as a trained sequence-to-sequence model including an encoder and a decoder, such as a BART model.

Next, the sentence generation unit training unit 348 evaluates the second reference sentence 520 generated by the sentence generation unit 346 by calculating the loss of the second reference sentence 520 against the second sentence label 540, which serves as ground truth data indicating the correct sentence for the second dialogue data 510, and changes the model parameters of the sentence generation unit 346 to reduce the loss, for example, using a technique such as the so-called Cross Entropy Loss.

The second sentence label 540 for evaluating the second reference sentence 520 generated by the sentence generation unit 346 may be an original sentence (e.g., a sentence created by a user) that includes all of the second answers 518 to the second questions 514 in the second dialogue data 510, or may be a context label generated by the context label generation process 600 described below, or may be an aggregate label generated by the dialogue aggregation process 700 described below.
More specifically, in one embodiment, the sentence generator training unit 348 may perform an initial stage of training of the sentence generator 346 using a large-scale corpus such as a CNN corpus to allow the sentence generator 346 to learn vocabulary distributions and sentence forms and structures across various domains, and then fine-tune the sentence generator 346 using a corpus of labeled dialogue data such as SAM Sum to allow the sentence generator 346 to learn auxiliary vocabulary.

By repeating the sentence generation unit training process 500 described above, the sentence generation unit 346 can improve its performance in generating reference sentences containing essential information of dialogue data corresponding to any domain (e.g., information not included in the first reference sentences used to train the answer generation unit 342 in the answer generation unit training process 400 described above). Furthermore, as described below, the adaptation unit 350 can further train the answer generation unit 342 using the reference sentences generated by the sentence generation unit 346, thereby expanding the scope of the answer generation unit 342 and obtaining an MRC model capable of generating appropriate answers to various questions (e.g., questions corresponding to the second domain).

In the above-described sentence generator training process 500, the ground truth data for evaluating the reference sentences generated by the sentence generator 346 may be not only the original sentence (e.g., a sentence created by a user) containing all of the second answers 518 to the second questions 514 in the second dialogue data 510, but also a context label extracted from the original sentence or an aggregate label aggregating multiple dialogue data. Below, with reference to Figures 6 and 7, we will explain the context label generation process 600 that extracts answers to questions in dialogue data and generates a context label, and the dialogue aggregation process 700 that aggregates multiple dialogue data to generate a dialogue aggregate.

Figure 6 is a diagram illustrating a specific example of a context label generation process 600 according to an embodiment of the present disclosure. As described above, this context label generation process 600 is a process for extracting answers to questions in dialogue data and generating context labels as ground truth for evaluating reference sentences generated by the sentence generation unit 346, and may be executed by the sentence generation unit training unit 348.

In the context label generation process 600, as shown in FIG. 6, for each question 610 included in the dialogue data used as learning data for training the sentence generation unit 346, sentence portions that provide appropriate answers are extracted from original sentences 620 that include all answers to the questions in the dialogue data, compiled, and generated as context labels 630. These original sentences 620 may be, for example, sentences created by the user.

For example, as shown in FIG. 6, if the dialogue data used as learning data for training the sentence generation unit 346 contains two questions 610, "Can I connect my smartphone to Wi-Fi?" and "Do I need a password?", the sentence generation training unit 348 extracts two sentences from the original sentence 620 as answers to these questions 610: "You can connect to Wi-Fi on the "Settings" screen of your smartphone, but the method for doing so varies depending on the smartphone's OS." and "You will need a wireless LAN password. Enter the password there to complete the connection." and generates these as context labels 630.

As described above, the context labels 630 generated in this manner may be used as ground truth data for evaluating the reference sentences generated by the sentence generation unit 346 during the training phase of the sentence generation unit 346. In this way, by training the sentence generation unit 346 using only the portion of the sentence that answers the question as ground truth, rather than the entire original sentence 620, the sentence generation unit 346 is not penalized for not including information unrelated to the answer to the question in the generated reference sentence, and is evaluated based on ground truth that includes the information that actually answers the question, and is therefore trained to generate reference sentences with higher accuracy.

In addition, in one embodiment, this context label generation process 600 may be executed when dialogue data corresponding to a specific domain (e.g., the second domain) meets a predetermined rarity criterion. This rarity criterion is based on the amount of available dialogue data, and if the available dialogue data is less than the predetermined amount, the rarity criterion is deemed to be met, and if the available dialogue data is equal to or greater than the predetermined amount, the rarity criterion is deemed not to be met. In other words, it is desirable to use this context label generation process 600 when there is little dialogue data corresponding to a specific domain (e.g., the second domain). This is because by extracting answers to questions in the dialogue data from the original text and generating context labels, it is possible to generate high-quality ground truth even when there is little dialogue data corresponding to a specific domain.

Figure 7 is a diagram illustrating a specific example of a dialogue aggregation process 700 according to an embodiment of the present disclosure. As described above, this dialogue aggregation process 700 is a process for aggregating multiple dialogue data sets and generating aggregate labels as ground truth data for evaluating reference sentences generated by the sentence generation unit 346, and may be executed by the sentence generation unit training unit 348.

In the dialogue aggregation process 700, as shown in FIG. 7, the sentence generator training unit 348 extracts and aggregates information from multiple dialogue data 710, 720 to generate aggregate labels 730 that serve as ground truth for evaluating the reference sentences generated by the sentence generator 346. These dialogue data 710, 720 may be dialogues corresponding to a common domain (e.g., a smartphone's Wi-Fi connection), but are not limited to this and may also be dialogues corresponding to different domains.

As described above, the aggregated labels 730 generated in this manner may be used as ground truth data for evaluating reference sentences generated by the sentence generation unit 346 during the training phase of the sentence generation unit 346. Furthermore, because this dialogue aggregation 730 is generated from multiple dialogue data, it contains information that provides appropriate answers to more questions than, for example, context labels generated based on a single piece of dialogue data, but requires more dialogue data to generate. Therefore, it is desirable to perform this dialogue aggregation process 700 when dialogue data corresponding to a specific domain (e.g., the second domain) does not satisfy the above-mentioned specified rarity criterion (i.e., when there is a lot of dialogue data).

Next, we will explain the adaptation process according to an embodiment of the present disclosure with reference to Figure 8.

Figure 8 is a block diagram showing an example of the flow of an adaptation process 800 according to an embodiment of the present disclosure. The adaptation process 800 shown in Figure 8 is a process for expanding the range of response of the answer generation unit 342 and training it to be able to generate appropriate answers to various questions, and is mainly performed by the answer generation unit 342, the sentence generation unit 346, and the adaptation unit 350.

As described above, according to the answer generation unit training process 400 described with reference to FIG. 4, the answer generation unit 342 is trained to generate an appropriate answer (e.g., a first answer) to a question (e.g., a first question) corresponding to a specific domain (e.g., a first domain). However, since the answer generation unit 342 trained in this manner is trained using reference sentences corresponding to a specific domain as learning data, it can generate appropriate answers to questions corresponding to a similar domain to the reference sentences that became the learning data, based on reference sentences that correspond to a similar domain to the reference sentences that became the learning data, but its performance is limited for questions corresponding to domains different from the learning data.

It is therefore desirable to improve the flexibility of the answer generation unit 342 and train it so that it can generate appropriate answers to questions that correspond to a wider range of domains. However, in principle, to train an MRC model such as the answer generation unit 342, three pieces of data are required: dialogue data corresponding to the domain to be trained (e.g., the target domain), reference sentences containing answers to questions in the dialogue data, and answer labels that serve as ground truth indicating the correct answers to the questions. For this reason, if only dialogue data corresponding to the domain to be trained is available as data, and there is no data on reference sentences containing answers to questions corresponding to the domain to be trained or answer labels indicating the correct answers to the questions, it is not possible to train the general MRC model described above.

Therefore, in the adaptation process 800 according to an embodiment of the present disclosure, the answer generation unit 342 trained by the answer generation unit training process 400 described above is used to generate second answer labels that serve as ground truth indicating correct answers to questions corresponding to the second domain, and the sentence generation unit 346 trained by the sentence generation unit training process 500 described above is used to generate second reference sentences that include answers to questions corresponding to the second domain. By training the answer generation unit 342 using the second answer labels, second reference sentences, and dialogue data corresponding to the second domain thus generated, the answer generation unit 342 has an expanded range of coverage and is able to generate appropriate answers to questions corresponding to a greater number of domains.

First, the adaptation unit 350 inputs second dialogue data 802 consisting of a second question 804 and a second answer 806 to the sentence generation unit 346, which has been trained by the sentence generation unit training process 500 described above. As described above, this second dialogue data 802 and the second question 804 and second answer 806 contained in the second dialogue data 802 may correspond to a second domain (e.g., the target domain). The sentence generation unit 346, which has input this second dialogue data 802, generates a second reference sentence 820 corresponding to the second domain, including the second answer 806 to the second question 804 in the second dialogue data 802.

Next, the adaptation unit 350 inputs to the answer generation unit 342 the second reference sentence 820 corresponding to the second domain generated by the sentence generation unit 346, the second question 804 contained in the second dialogue data 802 corresponding to the second dialogue domain, and the first question 812 contained in the first dialogue data (not shown in Figure 8) corresponding to the first domain.
It is assumed here that the answer generator 342 has already undergone the answer generator training process 400 described with reference to FIG. 4 and is capable of generating appropriate answers to questions corresponding to the first domain. However, at this point, the answer generator 342 has limited performance in a different domain (e.g., the second domain). Therefore, through the self-learning and adversarial learning processes described below, the answer generator 342 is trained to generate appropriate answers to questions corresponding to different domains.
(Self-study processing)

In the self-study process, the answer generation unit 342 first generates multiple answer candidates as answers to the input second question 804 based on the second reference sentence 820 corresponding to the second domain generated by the sentence generation unit 346. The answer candidates here are sentence sections (answer indexes) identified from the second reference sentence 820 that may indicate an appropriate answer to the second question 804. However, although the answer generation unit 342 can currently generate appropriate answers for the second domain, it has not yet been trained on the second domain, which is different from the first domain. Therefore, these answer candidates may not be appropriate answers to the second question 804 (i.e., some of these answer candidates may be incorrect). Furthermore, each of these answer candidates is associated with a confidence score that indicates the probability that it is an appropriate answer to the second question 804.

Therefore, the adaptation unit 350 extracts only those answer candidates generated by the answer generation unit 342 whose credibility meets a predetermined credibility standard and whose answer index value is not "0." The answers extracted from the answer candidates in this way have a high credibility and are likely to be appropriate answers to the second question 804, and are therefore used as second answer labels 840, which serve as ground truth for evaluating answers to questions corresponding to the second domain.

Next, the adaptation unit 350 associates the second answer label 840 extracted from the answer candidates, the second reference sentence 820 generated by the sentence generation unit 346, and the second question 804 included in the second dialogue data 802 corresponding to the second domain, and stores them in a specified storage area as a training dataset (not shown in FIG. 8) corresponding to the second domain. However, if a training dataset substantially similar to this training dataset has already been stored, the adaptation unit 350 does not need to store the newly generated training dataset.

Thereafter, the adaptation unit 350 performs the above-described answer generation unit training process 400 on the answer generation unit 342 using the learning dataset generated by the answer generation unit 342, and adjusts the model parameters of the answer generation unit 342 using a loss function such as Cross Entropy Loss, thereby improving the accuracy with which the answer generation unit 342 generates appropriate answers to questions corresponding to the second domain.
As a result, the answer generation unit 342 is trained using the second reference sentences that contain new information (i.e., information about the second domain) that is not included in the first reference sentences corresponding to the first domain used to train the answer generation unit 342 in the above-mentioned answer generation unit training process 400, and therefore becomes able to generate appropriate answers to questions regarding the new information (i.e., questions corresponding to the second domain).
(Adversarial Learning Processing)

In the adversarial learning process, the answer generator 342 is trained to distinguish between questions corresponding to a first domain and questions corresponding to a second domain.
More specifically, the adaptation unit 350 acquires first domain features (e.g., BERTfeatures if the MRC model is a BERT model) that are the output of the final layer of the MRC model that becomes the answer generation unit 342 when a first question 812 corresponding to the first domain and a reference sentence (not shown in FIG. 8 ) that includes the answer to the first question 812 and corresponds to the first domain are input to the answer generation unit 342. In addition, the adaptation unit 350 acquires second domain features (e.g., BERTfeatures if the layer model is a BERT model) that are the output of the final layer of the MRC model that becomes the answer generation unit 342 when a second question 804 corresponding to the second domain and a second reference sentence 820 corresponding to the second domain are input to the answer generation unit 342.

The adaptation unit 350 then inputs feature data 845, including the acquired first domain feature and second domain feature, to the classification network 850. The classification network 850 may be, for example, a three-layer multilayer network. In some embodiments, the classification network 850 may be part of the answer generation unit 342.
In one embodiment, the adaptation unit 350 may input the logits, which are the output of the linear layer, to the discriminant network 850 in addition to the first domain feature and the second domain feature.

The identification network 850 analyzes the input feature data 845 and calculates the probability that each feature belongs to the first domain and the probability that each feature belongs to the second domain. Based on the calculated probabilities, the identification network 850 then generates a domain label 860 indicating which domain each feature included in the feature data 845 belongs to.

The adaptation unit 350 then calculates the loss of the domain labels 860 generated by the identification network 850 relative to the ground truth (not shown in Figure 8) indicating the domain to which each feature belongs, using a loss function such as Cross Entropy Loss, and adjusts the model parameters of the answer generation unit 342 to reduce this loss, thereby improving the accuracy of the answer generation unit 342 in distinguishing between questions corresponding to the first domain and questions corresponding to the second domain.

Training when using the discriminant network 850 can take several different implementations.
First, there are implementations relating to the modification of the model parameters of the answer generation unit 342 when modifying the model parameters of the identification network 850 itself. The first implementation is a method in which, when modifying the model parameters of the identification network 850, the model parameters of the answer generation unit 342 are also modified at the same time so that it becomes easier to distinguish domains in the identification network 850. The second implementation is a method in which, when modifying the model parameters of the identification network 850, the model parameters of the answer generation unit 342 are fixed.

Furthermore, several implementations are possible for modifying the model parameters of the answer generation unit 342. The first implementation is a method of training to correctly estimate the position of the answer in the reference sentence for a given question, as described above. The second implementation is a method of training the answer generation unit 342 using, in addition to the first implementation, a criterion that makes it difficult for the identification network 850 to identify domains. The criterion here can be one that makes the output probability of the domain output by the identification network 850 as uniform as possible regardless of the domain. However, it is also possible to use a different criterion that prevents the identification network 850 from correctly identifying the domain.

The loss calculated based on the standard for the first implementation and the loss calculated based on the standard for making domain identification of the identification network 850 more difficult are then added together using a predetermined weight, and the model parameters of the answer generation unit 342 are updated to reduce this combined loss. Alternatively, the model parameter update may be divided into two stages, with the update to reduce the loss in the first implementation and the update to reduce the loss calculated based on the standard for making identification of the identification network 850 more difficult being performed in separate processes.
The choice of one of the above implementations may depend on the differences in vocabulary of questions and reference sentences used between the first and second domains.

According to the adaptation process 800 described above, the answer generation unit 342 can accurately determine the domain to which an input question corresponds and generate appropriate answers to various questions by using reference sentences corresponding to the determined domain. Furthermore, as described above, in the adaptation process 800 according to the present disclosure, the learning data for training the answer generation unit 342 in the target domain is generated by the answer generation unit 342 and the sentence generation unit 346. Therefore, even if, for example, learning data other than dialogue data corresponding to the target domain is not available, the answer generation unit 342 can be trained to generate appropriate answers to questions corresponding to the target domain.

Furthermore, the adaptation process 800 according to an embodiment of the present disclosure may be performed in real time when a question is received from a user. More specifically, when the answer generation unit 342, which has been trained to generate a first answer to a question corresponding to a first domain based on a first sentence corresponding to the first domain, receives a second question corresponding to a second domain (e.g., a question for which no answer exists in the first sentence), the above-mentioned sentence generation unit 346 may generate a second reference sentence corresponding to the second domain in real time using second dialogue data corresponding to the second domain (e.g., data corresponding to the second domain from a past dialogue history). Next, the adaptation unit 350 performs the above-mentioned adaptation process 800 using the second reference sentence generated in this manner, thereby training the answer generation unit 342 to generate an appropriate answer to the second question corresponding to the second domain. Thereafter, the answer generation unit 342 trained in this manner may generate a second answer to the received second question in real time and respond to the user on the spot.

As mentioned above, the first domain (eg, source domain) and the second domain (eg, target domain) may be different domains or may be the same domain.
When the first domain and the second domain are different, the adaptation process 800 described above enables the answer generation unit 342 to generate appropriate answers to both questions corresponding to the first domain (e.g., "mortgage application") and questions corresponding to the second domain (e.g., "fire insurance application"), thereby broadening the scope of application of the answer generation unit 342 and enabling a highly versatile MRC model to be obtained.
On the other hand, if the first domain and the second domain are the same, the adaptation process 800 described above allows the answer generator 342 to learn new information about the same domain, resulting in an MRC model with a deeper understanding of the domain that can provide appropriate answers to questions about more specific details.

According to the embodiments of the present disclosure described above, by training an MRC model using reference sentences generated based on natural language conversations, it is possible to provide a highly flexible and robust dialogue management means that can respond to a variety of questions. In other words, according to the dialogue management means of embodiments of the present disclosure, the MRC model of the answer generation unit, etc., is trained using reference sentences generated based on dialogue data that includes new information, making it possible to generate appropriate answers even to questions regarding the new information that could not be answered previously.

Examples of application scenarios for the dialogue management means according to embodiments of the present disclosure include online chat support and call centers. In online chat support and call centers, reference sentences containing the content of a conversation are created based on dialogue data, such as a conversation history between a user asking a question and the staff member assisting the user. The reference sentences are then used to train an MRC model, thereby expanding the scope of coverage of the MRC model and enabling it to generate appropriate answers to questions about a variety of topics and information.

The above describes an embodiment of the present invention, but the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention.

300 Dialogue management system 310 User 320 Client terminal 322 Reception unit 324 First communication unit 326 Response unit 328 Storage unit 329 Dialogue data 330 Communication network 340 Dialogue management device 342 Answer generation unit 344 Answer generation unit training unit 346 Sentence generation unit 348 Sentence generation unit training unit 350 Adaptation unit 352 Second communication unit

Claims (12)

A dialogue management device,
an answer generator that receives a first question and generates a first answer to the first question based on a first reference sentence;
a sentence generation unit that generates a second reference sentence including the content of the question and the answer based on dialogue data including the question and the answer;
an adaptation unit that uses the dialogue data, a second answer label indicating a correct answer to the second question, and the second reference sentence to train the answer generation unit to generate an appropriate answer to the second question;
A dialogue management device comprising: the first question corresponds to a first domain;
the first reference sentence corresponds to the first domain;
the interaction data corresponds to a second domain;
the second reference sentence corresponds to the second domain;
the second answer label corresponds to the second domain;
the second question corresponds to the second domain;
2. The dialogue management device according to claim 1, wherein: an answer generation unit training unit that trains the answer generation unit by changing model parameters of the answer generation unit so as to reduce the loss of the first answer with respect to a first answer label that is a ground truth indicating a correct answer to the first question;
The dialogue management device of claim 2 further comprising: a sentence generator training unit that trains the sentence generator by changing model parameters of the sentence generator so as to reduce loss of the second reference sentence with respect to a sentence label that is a ground truth containing correct content related to the dialogue data;
The dialogue management device of claim 2 further comprising: The sentence generation unit training unit
If the dialogue data satisfies a predetermined rarity criterion,
extracting each answer from an original sentence containing an answer to a question in the dialogue data, and setting the answer as the sentence label;
5. The dialogue management device according to claim 4, wherein: The sentence generation unit training unit
If the dialogue data does not meet a predetermined rarity criterion,
information extracted from the dialogue data and second dialogue data corresponding to a different dialogue is used as the sentence label;
5. The dialogue management device according to claim 4, wherein: The answer generation unit
generating answer candidates to the second question corresponding to the second domain using the second reference sentence corresponding to the second domain generated by the sentence generation unit;
extracting an answer that satisfies a predetermined credibility standard from the generated answer candidates as the second answer label that indicates a correct answer to the second question;
The adaptation unit
training the answer generator to reduce a loss for the second answer label of a second answer to the second question generated by the answer generator;
4. The dialogue management device according to claim 3, wherein: The adaptation unit
further comprising an identification network for distinguishing the domains;
acquiring first domain features related to the first domain from the answer generation unit based on the first question and the first reference sentence;
acquiring second domain features related to the second domain from the answer generation unit based on the second question and the second reference sentence;
training the discrimination network to distinguish between questions related to the first domain and questions related to the second domain based on the first domain features and the second domain features;
4. The dialogue management device according to claim 3, wherein: the answer generator is configured to train the discrimination network to distinguish between questions about the first domain and questions about the second domain;
9. The dialogue management device according to claim 8, wherein the dialogue management device is a dialogue management device. the answer generator is trained to make it difficult for the discrimination network to distinguish between questions about the first domain and questions about the second domain;
9. The dialogue management device according to claim 8, wherein the dialogue management device is a dialogue management device. 1. A dialogue management method, comprising:
training an answer generator to generate a first answer to a first question corresponding to a first domain based on a first reference sentence corresponding to the first domain;
training a sentence generator to generate second reference sentences including the content of the questions and answers based on second dialogue data including questions and answers corresponding to a second domain;
generating answer candidates for a second question corresponding to the second domain by the answer generation unit using the second reference sentence;
extracting, from the generated answer candidates, an answer that satisfies a predetermined reliability standard as a second answer label indicating a correct answer to the second question;
training the answer generation unit to generate appropriate answers to questions corresponding to the second domain by changing model parameters of the answer generation unit using the second dialogue data, the second answer label indicating a correct answer to the second question, and the second reference sentence so as to reduce a loss of the second answer to the second question generated by the answer generation unit for the second answer label;
A dialogue management method comprising: a client terminal that outputs appropriate responses to user questions;
a dialogue management system connected to a dialogue management device that generates the response via a communication network,
The dialogue management device
an answer generation unit that generates a first answer to a first question corresponding to a first domain based on a first reference sentence corresponding to the first domain;
a sentence generation unit that generates a second reference sentence including the content of the question and answer based on second dialogue data including the question and answer corresponding to a second domain;
an adaptation unit that trains the answer generation unit to generate appropriate answers to questions corresponding to the second domain using the second dialogue data, second answer labels indicating correct answers to second questions corresponding to the second domain, and the second reference sentences;
Dialogue management system comprising: