JP7635940B2 - Method, computer device, and computer program for providing domain-specific dialogue utilizing language models - Google Patents

Description

The following description relates to a technique for providing answers to user questions by referencing appropriate documents.

A QA (question and answer) system is a dialogue system that, when a user inputs a question, provides an answer while referring to appropriate documents.

In recent years, the performance of dialogue systems has improved rapidly with the development of language models, and has expanded beyond natural language to include multi-modal domains such as images and voice.

As an example, Patent Document 1 (published on April 25, 2002) discloses a technique for providing answers to questions using artificial intelligence and natural language processing techniques.

Korean Patent Publication No. 10-2002-0030545

We provide technology that uses large-scale language models to generate conversational bots specialized for a given domain based on documents in that domain.

We provide a search model for operating a computer that is trained to select documents that may contain answers to input user questions based on data consisting of pairs (Document, Query) of each document belonging to a specific domain and a query generated for that document, and to output answers to the user questions.

According to one aspect, a search model is provided for operating a computer to perform ranking by utilizing the scores of each document for the user's question and to select documents that may contain the answer.

According to another aspect, a search model is provided, wherein the generated query is generated using a language model that has already been trained based on the document.

A method is provided that is executed on a computer device, the computer device including at least one processor configured to execute computer-readable instructions contained in a memory, the method including the steps of: for a given document, the at least one processor generating at least one query that can be answered by the document using a language model; and the at least one processor using the document and the query to train a retrieval model for a conversational bot.

According to one aspect, the generating step may generate the query for a domain-specific search model by targeting target documents belonging to the domain.

In another aspect, the generating step may include constructing a prompt for the target document using sample data consisting of a sample document and a sample query, and using the prompt as an input to the language model to generate a query for the target document according to a pattern in the sample data.

According to another aspect, the generating step may include constructing a first prompt for the target document using first sample data consisting of a first sample document and a first sample keyword, using the first prompt as an input to the language model and generating keywords for the target document according to a pattern of the first sample data, constructing a second prompt for keywords of the target document using second sample data consisting of a second sample document, a second sample keyword, and a sample query, and using the second prompt as an input to the language model and generating a query for the target document for keywords of the target document according to a pattern of the second sample data.

In another aspect, the method may further include the at least one processor providing an answer to a question entered by a user by referring to documents selected by the search model.

According to yet another aspect, the providing step may include converting the scores of each document calculated by the search model into the form of a probability distribution for the question, and determining a type of answer to the question based on the entropy of the probability distribution.

A computer program is provided that is recorded on a computer-readable recording medium to cause the computer device to execute the above-mentioned method.

A computer device is provided that includes at least one processor configured to execute computer-readable instructions contained in a memory, the at least one processor executing, for a given document, a process of generating at least one query that can be answered by the document using a language model, and a process of using the document and the query to train a search model for a conversational bot.

According to an embodiment of the present invention, a dialogue system can be generated quickly and efficiently by using a language model to generate large-scale data specific to a given domain based on documents in that domain.

According to an embodiment of the present invention, data generated by a large-scale language model is used to train a retrieval model, and a dialogue system centered on the retrieval model is constructed, thereby making it possible to generate a highly advanced dialogue bot with a fast inference speed.

FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of a computing device in accordance with an embodiment of the present invention. FIG. 1 is a diagram for explaining a generation system of a dialogue system using a large-scale language model in one embodiment of the present invention. 1 is a flowchart illustrating an example of a method for generating dialogue data that can be executed by a computer device in an embodiment of the present invention. 1 is a flow chart illustrating an example process for generating query data using a prompt in accordance with an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a query generation method in an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a query generation method in an embodiment of the present invention. 11 is a diagram for explaining another example of a query generation method in one embodiment of the present invention. 11 is a diagram for explaining another example of a query generation method in one embodiment of the present invention. 4 is a flowchart illustrating an example of a process for providing an answer to a user's question in accordance with an embodiment of the present invention. FIG. 11 is a diagram for explaining an example of a process for classifying answer types in one embodiment of the present invention. FIG. 11 is a diagram for explaining an example of a process for classifying answer types in one embodiment of the present invention. 11A to 11C are diagrams illustrating an example of a process for distinguishing and displaying sentences related to a user's question in an embodiment of the present invention.

The following describes in detail an embodiment of the present invention with reference to the attached drawings.

An embodiment of the present invention relates to a technology that provides answers to user questions by referencing appropriate documents.

Embodiments including those specifically disclosed in this specification can generate a dialogue system quickly and efficiently by replacing the process of collecting dialogue data required to generate a domain-specific dialogue system with an automated process that uses a large-scale language model.

The dialogue data generation system according to the embodiment of the present invention may be realized by at least one computer device, and the dialogue data generation method according to the embodiment of the present invention may be executed by at least one computer device included in the dialogue data generation system. In this case, a computer program according to an embodiment of the present invention may be installed and executed in the computer device, and the computer device may execute the dialogue data generation method according to the embodiment of the present invention according to the control of the executed computer program. The above-mentioned computer program may be recorded on a computer-readable recording medium in combination with the computer device to cause the computer to execute the dialogue data generation method.

Figure 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Figure 1 shows an example including multiple electronic devices 110, 120, 130, 140, multiple servers 150, 160, and a network 170. Figure 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in Figure 1. Furthermore, the network environment in Figure 1 is merely an example of an environment applicable to this embodiment, and the environment applicable to this embodiment is not limited to the network environment in Figure 1.

The electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals realized by computer devices. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets. As an example, FIG. 1 shows a smartphone as an example of the electronic device 110, but in an embodiment of the present invention, the electronic device 110 may refer to one of various physical computer devices that can communicate with other electronic devices 120, 130, and 140 and/or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

The communication method is not limited, and may include not only a communication method using a communication network that the network 170 can include (for example, a mobile communication network, a wired Internet, a wireless Internet, or a broadcast network), but also short-range wireless communication between devices. For example, the network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Furthermore, the network 170 may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network.

Each of the servers 150, 160 may be implemented by one or more computing devices that communicate with the multiple electronic devices 110, 120, 130, 140 via the network 170 to provide instructions, code, files, content, services, and the like. For example, the server 150 may be a system that provides a service (e.g., a conversational bot service) to the multiple electronic devices 110, 120, 130, 140 connected via the network 170.

FIG. 2 is a block diagram showing an example of a computer device in one embodiment of the present invention. Each of the multiple electronic devices 110, 120, 130, and 140 and each of the servers 150 and 160 described above may be realized by the computer device 200 shown in FIG. 2. For example, the dialogue data generation system according to an embodiment of the present invention may be realized by the computer device 200 shown in FIG. 2.

Such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input/output interface 240, as shown in FIG. 2. The memory 210 is a computer-readable recording medium and may include a RAM (random access memory), a ROM (read only memory), and a permanent mass storage device such as a disk drive. Here, a permanent mass storage device such as a ROM or a disk drive may be included in the computer device 200 as a separate permanent storage device separate from the memory 210. In addition, the memory 210 may store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, or a memory card. In other embodiments, the software components may be loaded into the memory 210 through a communication interface 230 that is not a computer-readable recording medium. For example, the software components may be loaded into the memory 210 of the computer device 200 based on a computer program that is installed by a file received via the network 170.

The processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input/output operations. The instructions may be provided to the processor 220 by the memory 210 or the communication interface 230. For example, the processor 220 may be configured to execute instructions received according to program code recorded in a recording device such as the memory 210.

The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (for example, the above-mentioned recording device) via the network 170. As an example, requests, commands, data, files, etc. generated by the processor 220 of the computer device 200 according to program code recorded in a recording device such as the memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 200 through the communication interface 230 of the computer device 200 via the network 170. The signals, commands, data, etc. received through the communication interface 230 may be transmitted to the processor 220 or the memory 210, and the files, etc. may be recorded on a recording medium (the above-mentioned permanent recording device) that the computer device 200 may further include.

The input/output interface 240 may be a means for interfacing with the input/output device 250. For example, the input device may include devices such as a microphone, keyboard, or mouse, and the output device may include devices such as a display and a speaker. As another example, the input/output interface 240 may be a means for interfacing with a device that integrates functions for input and output into one, such as a touch screen. The input/output device 250 may be configured as a single device together with the computer device 200.

Also, in other embodiments, the computing device 200 may include fewer or more components than those of FIG. 2. However, most of the prior art components need not be explicitly shown in the figures. For example, the computing device 200 may be implemented to include at least some of the input/output devices 250 described above, and may further include other components such as a transceiver, a database, etc.

This embodiment utilizes a large-scale language model to generate a domain-specific conversational bot, thereby solving the problem of collecting massive amounts of data for that domain.

The large-scale language model used in this invention refers to a language model that has been trained using a huge amount of data, and is a natural language generation model that can properly execute a task when only a few-shot samples are given.

In other words, a large-scale language model is an autoregressive model that can perform inference without fine-tuning by using methods such as few-shot learning. It can have more than 10 times the parameters (e.g., more than 100 billion parameters) compared to existing general language models. For example, large-scale language models such as GPT-3 (Generative Pretrained Transformer 3) and HyperClova (registered trademark) are excellent few-shot learners that can be controlled by natural prompts, and are capable of in-context learning, which is the ability to understand patterns even with a small amount of data through prompts and solve NLP problems.

In this embodiment, a domain-specific dialogue bot is designed based on a large-scale language model and a search model. As a document-based dialogue system using a language model-based generator and a search model, a highly advanced dialogue bot with fast inference speed can be generated.

Figure 3 is a diagram for explaining the generation system of a dialogue system using a large-scale language model in one embodiment of the present invention.

Referring to FIG. 3, in this embodiment, the large-scale language model 310 is used as a backbone for generating query data 302 that is appropriate for the domain and is required to generate a domain-specific conversational bot.

Given a particular domain document 301, e.g., a web page within a home page, the processor 220 may generate query data 302 indicating questions that are anticipated from the given document 301 by using the large-scale language model 310 to generate a chatbot capable of question-and-answering (QA) on the content within the document.

The processor 220 may use the query data 302 generated by the language model 310 as training data to train a search model 320 for a dialogue bot. By using the query data 302 that matches the domain to train the search model 320, a model specialized for the domain can be constructed.

When a user inputs a question, the processor 220 may provide an answer by referencing documents that correspond to the user's question using the search model 320.

Figure 4 is a flowchart showing an example of a dialogue data generation method that can be executed by a computer device in one embodiment of the present invention.

The dialogue data generation method according to this embodiment may be executed by the computer device 200 described above. In this case, the processor 220 of the computer device 200 may be realized to execute control instructions according to the operating system code and at least one program code included in the memory 210. Here, the processor 220 may control the computer device 200 to execute steps 410 to 430 included in the dialogue data generation method of FIG. 4 according to the control instructions provided by the code recorded in the computer device 200.

Referring to FIG. 4, in step 410, the processor 220 may use a large-scale language model to generate query data from target documents of a given domain. The processor 220 may use prompts that serve as input sentences for the language model to generate predicted questions for the target documents. The processor 220 may use the excellent data generation and in-context learning capabilities of the language model to generate data for model training. Pairs of the questions generated in this way and the target documents (or correct answer phrases contained in the target documents) that generated the questions may be used as training data for the QA model.

In step 420, the processor 220 may use the target documents and the query data as training data to train a search model for QA. The processor 220 may build a dialogue model for the domain by using the target documents of the domain and the query data generated by the language model for the target documents to train the search model 320.

In step 430, when the user inputs a question, the processor 220 may use the search model 320 to provide an answer by referring to documents corresponding to the user's question. The processor 220 may perform answer formatting or answer generation according to the documents corresponding to the user's question. In other words, the processor 220 may format the documents searched for the user's question and provide them as an answer, or may directly generate and provide an answer using the documents.

Figure 5 is a flowchart illustrating an example of a process for generating query data using a prompt in one embodiment of the present invention.

Referring to FIG. 5, in step 501, the processor 220 may select original data to use for the prompt.

The processor 220 may use at least one predefined pair of documents and queries that can be answered by the documents as original data. In addition to the predefined original sample pairs, another data set may exist, and in some embodiments, an original sample pair consisting of a document and a query may be selected from within the data set.

In this case, the prompt may be composed of at least one document, a sample pair of original questions that can be answered by the document, and a target document. A detailed method for constructing the prompt will be described with reference to FIG. 6.

In step 502, the processor 220 may configure a prompt corresponding to the input of the language model. The processor 220 may configure an input prompt of the language model using the original data selected in step 501. The processor 220 may create a dedicated prompt template that appropriately reflects the characteristics of a given NLP problem, and the prompt template may include a task definition and meta information. In other words, the processor 220 uses the original data selected from the dataset to configure a prompt in a natural language form, and the prompt is created in a format that can be understood by the language model and is provided as an input of the language model. If the original data is labeled data, the processor 220 may design the prompt so that an input sentence is generated together with label information. The format of the prompt itself may be configured in various ways. As an example, the prompt may be configured to include a document (e.g., a web page) and a query that can be answered by the document as a sample.

For a given task T, the processor 220 may construct an input prompt for the language model by combining a document sample, which is original data sampled from a dataset, with a prompt template that takes into account the characteristics of the given task T.

In step 503, the processor 220 may input the prompt constructed in step 502 into a language model and generate query data in the form of natural language from the language model. In other words, the processor 220 may input the prompt input sentence into the language model and then obtain the query data as a language generation result by the generation or completion function of the language model. The processor 220 may input the prompt into the language model and obtain a new result having the pattern by analyzing a sample natural language pattern included in the prompt in the language model.

Figures 6 and 7 are diagrams illustrating an example of a query generation method in one embodiment of the present invention.

Referring to FIG. 6, the processor 220 may construct a prompt input sentence 610 using a sample data pair consisting of a document selected as the original data and at least one query that can be answered based on the contents of the document.

The processor 220 may take the prompt input sentence 610, which is composed of at least one sample data pair and the target document, as input to the language model, and cause the language model to generate a question for the target document. That is, the completion function of the language model may generate at least one query that can be answered by the target document.

As an example, the processor 220 may generate a predetermined number of queries for the target document by repeating the query generation using the prompt input sentences 610 composed of sample data pairs. In some embodiments, queries for the target document may be generated until a saturation state is reached where no new queries are generated.

The processor 220 may use a prompt 610 consisting of a sample data pair in the form of a [sample document + query] and a [target document] to generate a query for the target document according to the pattern of the sample data included in the prompt.

For example, as shown in FIG. 7, when a target document 701 is given, the processor 220 may input sample data and the target document 701 into a language model as a prompt input sentence 610, and generate query data 702 predicted from the target document 701 according to the pattern of the sample data.

In other words, the processor 220 may provide documents and queries derived from the documents as samples and generate queries for the target documents according to the patterns of the samples.

Figures 8 and 9 are diagrams illustrating another example of a query generation method in one embodiment of the present invention.

The processor 220 may apply a method for generating queries based on keywords to improve the quality of queries generated by the language model.

As an example, the processor 220 may use two prompts in a stepwise manner, first extracting keywords related to the document from the document, and then generating queries for the extracted keywords in a stepwise manner. When additional keywords are used, more diverse queries can be generated from the document, and the coverage of questions can be expanded.

Referring to FIG. 8, the processor 220 may construct a first prompt input sentence 810 using a first sample data pair consisting of a document selected as the original data and a keyword for the document.

The processor 220 may input a first prompt input sentence 810 consisting of at least one keyword-based first sample data pair and a target document into a language model, and generate at least one keyword for the target document through a generation function of the language model.

Next, the processor 220 may construct a second prompt input sentence 820 using a second sample data pair consisting of the document selected as the original data, keywords extracted from the document, and a query that can be answered based on the contents of the document.

At this time, the second prompt input sentence 820 may include keywords for the target document generated by a language model using the first prompt input sentence 810.

The processor 220 may input at least one keyword-based second sample data pair, the target document, and a second prompt input sentence 820 composed of keywords for the target document into the language model, and generate at least one query that can be answered by the target document through a completion function of the language model.

The processor 220 may generate keywords for the target document according to the pattern of the primary prompt using a first sample data pair consisting of [sample document + keyword] and a primary prompt consisting of [target document], and then generate a query for the target document for each keyword according to the pattern of the secondary prompt using a second sample data pair consisting of [sample document + keyword + query] and a secondary prompt consisting of [target document + keyword].

For example, as shown in FIG. 9, when a target document 901 is given, the processor 220 may first input the first prompt input sentence 810 into the language model and generate keywords 90 that may appear as a query in the target document 901 according to the patterns of the sample data.

The processor 220 may then input the second prompt input sentence 820 containing the keyword 90 into the language model, and generate predicted query data 902 for the target document 901 based on the keyword 90 in accordance with the pattern of the sample data.

The processor 220 may extract at least one keyword 90 for the target document 901 and generate query data 902 corresponding to each keyword 90. In this way, by using keywords, query data that is related to each keyword and has an intent corresponding to the keyword is uniformly generated. In this way, when diverse data is used for learning, the performance of the search model will be improved.

Thus, the processor 220 can provide a document, keywords extracted from the document, and a query containing the keywords as samples, and generate a query for the target document based on the keywords extracted from the target document according to the pattern of the samples.

The processor 220 may use the data consisting of the target documents and queries generated for the target documents to train a model for the dialogue bot.

In the case of the search model 320, for example, a polyencoder structure may be applied to search documents from a query to provide an answer. The search model 320 may include a reranker with a cross encoder structure to rerank documents corresponding to the query to provide an answer.

The search model 320 may be trained using pairs (Document, Query) of each document for the corresponding domain and a query generated for the document. The search model 320 may be generated by fine-tuning a model pre-trained using a labeled dataset or a model pre-trained by an unsupervised search model 320 from an unlabeled corpus of various linguistic features using data generated using an embodiment of the present invention.

In the process of fine-tuning the search model 320, a data augmentation technique using synonym and homonym information specific to the domain to which the target document belongs may be used. A certain data augmentation effect can be expected by replacing each word with a synonym or homonym with a certain probability for the questions in the training batch and training the model. For example, "postdoctoral researcher" may be replaced with "post-doc," "postdoctoral researcher," etc., and "OCR" may be replaced with "optical character recognition," OCR, etc., and training may be performed.

In addition to data augmentation techniques, a rule-based replacement method for words from the inference input can also be used to improve the learning performance of the search model 320.

Figure 10 is a flowchart illustrating an example of a process for providing an answer to a user's question in one embodiment of the present invention.

Referring to FIG. 10, in step 1001, the processor 220 may calculate a score for each document for the user's question using the search model 320. The processor 220 may use the search model 320 to search for and rank documents that contain answers to the user's question. The processor 220 may calculate a score for each document by measuring the similarity between the document and a sentence that corresponds to the user's question.

In step 1002, the processor 220 may convert the document scores calculated by the search model 320 into a form of probability distribution. As an example, the processor 220 may use a softmax function to convert the document scores of the search model 320 into a form of probability distribution.

In step 1003, the processor 220 may determine a type of answer to the user's question based on the probability distribution of the document scores. The processor 220 may format at least one document selected based on the ranking by the search model 320 and provide it as an answer to the user's question.

The number of documents provided as answers may vary depending on the question, and the number of documents may be defined for each type of answer. When document scores are converted into a probability distribution, in cases where a document is definitely ranked first among the related documents for a question, the entropy of the score probability distribution is close to 0, and the entropy increases as the number of related documents increases. In addition, questions for which related documents are not searched for tend to have a uniform distribution.

Based on this tendency, a threshold may be set based on the entropy of the score probability distribution to classify the answer type.

For example, as shown in FIG. 11, when the entropy of the score probability distribution is equal to or less than a certain threshold close to 0 (e.g., equal to or less than 0.3), the answer type may be classified into three cases: case 1 where the top 1 (top1) is certain; case 2 where the entropy is greater than the threshold and less than 1 (e.g., 0.4 to 1), where the documents ranked 1st to 3rd are all useful; and case 3 where the entropy is 2 or greater, where there are no related documents. As another example, the threshold for classifying the answer type may be set to 0.1 and 1.75 as the references. When the entropy (E) of the score probability distribution is 0≦E<0.1, the answer may be classified into case 1; when it is 0.1≦E<1.75, the answer may be classified into case 2; and when it is E≧1.75, the answer may be classified into case 3.

As shown in FIG. 12, when the entropy of the document score probability distribution is indicated as case 2 for a question about "lamp color", the processor 220 may select the top three documents based on the document score to provide an answer. When detailed documents about red, yellow, and green are searched for as documents containing the content of lamp colors, with similar scores, all three documents may be used as an answer depending on a threshold value. In this case, the titles of the three documents may be displayed as an answer to the user's question, and then, if the user selects a specific document, the detailed content of the selected document may be displayed. On the other hand, when the entropy of the document score probability distribution is indicated as case 1 for a question about "internship preferential treatment conditions", the processor 220 may select the top one document based on the document score to provide an answer. The highest-scoring document containing the content of internship preferential treatment conditions may be displayed as an answer to the question.

The processor 220 may distinguish sentences in the document provided as an answer that are relevant to the user's question from other sentences. For example, as shown in FIG. 13, highlighting 1301 may be used to indicate which part of the content of the document 1310 provided as an answer corresponds to the answer to the user's question.

To highlight sentences in a document that correspond to a user's question, a question may be generated for each sentence in the document that has the sentence as an answer, and this may be used as data for training a search model. In other words, the processor 220 may use a prompt consisting of [sample sentence + query] and [target sentence] for each sentence in the target document to generate a query for the target sentence according to the pattern of the prompt, and may use a data pair consisting of [target sentence + query] to train a search model, and search for sentences that answer the user's question within the document and display them as highlighted sentences.

When the processor 220 generates a query for a target sentence using a sample sentence and a query that uses the sentence as an answer for highlighting answer sentences in a document, the processor 220 may apply logic to filter whether the target sentence is an answerable sentence prior to query generation in order to improve the quality of query generation. For example, the target sentence may be filtered taking into consideration the title of the target document and other sentences in the document. In other words, sentences that are outside the title of the document or have a different format from other sentences in the document may be excluded from query generation.

In this embodiment, by combining a document-based dialogue bot with a Task Oriented Dialog (TOD), it is possible to provide not only active conversations such as questions and queries for necessary information, but also more advanced services such as processing actual user requests through API calls. If the dialogue system described above can be combined with a TOD, it is possible to build an active dialogue system that can provide appropriate answers to users while at the same time grasping the user's implicit requirements and proactively asking about them to resolve them.

In this way, according to an embodiment of the present invention, a dialogue system can be generated quickly and efficiently by using a language model to generate large-scale data specialized for a given domain based on documents in that domain. According to an embodiment of the present invention, data generated by a large-scale language model is used to train a retrieval model, and a dialogue system centered on the retrieval model is constructed, thereby generating a highly advanced dialogue bot with a high inference speed.

The above-described devices may be realized by hardware components, software components, and/or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also respond to the execution of the software and access, record, manipulate, process, and generate data. For ease of understanding, the description may be given as if one processing device is used, but one skilled in the art will understand that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing unit may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure or instruct the processing device to operate as desired, either independently or collectively. The software and/or data may be embodied in any type of machine, component, physical device, computer storage medium, or device to be interpreted by or provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. Here, the medium may be one that continuously records the computer-executable program or one that temporarily records it for execution or download. The medium may be one of various recording means or storage means in the form of a single or multiple hardware combined, and is not limited to a medium directly connected to a certain computer system, but may be one that is distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, etc., configured to record program instructions. Other examples of the medium include recording media or storage media managed by application stores that distribute applications, or sites, servers, etc. that supply or distribute various other software.

Although the embodiments have been described above based on limited embodiments and drawings, those skilled in the art will appreciate that various modifications and variations may be made from the above description. For example, the described techniques may be performed in an order different from that described, and/or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

Therefore, different embodiments that are equivalent to the claims are within the scope of the appended claims.

301: Domain documents 302: Query data 310: Large-scale language model 320: Search model

Claims (11)

1. A method implemented on a computer device for generating a dialogue system using a language model and a retrieval model, comprising:
The method comprises:
A processor of the computer device generates queries as expected questions for documents belonging to a particular domain by utilizing the language model;
The processor uses training data consisting of pairs of the document and the query to train a search model to construct a QA model for a conversational bot;
a step of, when a user's question is input to the search model, the processor selecting documents including an answer to the user's question and providing the answer to the user's question;
A method comprising: 2. The method of claim 1, wherein the step of providing the answer includes the processor searching for documents containing the answer content using the search model, calculating a document score for each document by measuring a similarity between a sentence corresponding to the user's question and the document containing the answer content, and selecting the document containing the answer content based on the document score . The generating step includes:
3. The method of claim 2, further comprising: constructing prompts for target documents using sample data consisting of sample documents and sample queries ; and using the prompts as input to the language model to generate queries for the target documents according to patterns in the sample data. The generating step includes:
constructing a first prompt for the target document using first sample data, the first sample document and the first sample keywords;
using the first prompt as an input to the language model and generating keywords for the target document according to patterns in the first sample data;
3. The method of claim 2, further comprising: constructing a second prompt for a keyword of the target document using second sample data consisting of a second sample document, a second sample keyword, and a sample query; and using the second prompt as an input to the language model to generate a query for the target document according to a pattern of the second sample data for the keyword of the target document. The providing step comprises:
The method of claim 2 , comprising: converting the document score of each document into a form of a probability distribution; and determining a type of answer to the user's question according to the entropy of the probability distribution. A computer program product for causing a computer device to carry out the method according to any one of claims 1 to 5 . 1. A computer device for generating a dialogue system using a language model and a retrieval model, comprising:
at least one processor configured to execute computer readable instructions contained in a memory;
The at least one processor
generating queries as anticipated questions for documents belonging to a particular domain by utilizing the language model;
training a search model using training data consisting of pairs of the document and the query to construct a QA model for a conversational bot;
When a user's question is input to the search model, selecting documents including an answer to the user's question and providing the answer to the user's question;
A computer device that executes the above-mentioned. 8. The computer device according to claim 7, wherein the step of providing the answer includes: searching for documents containing the answer content using the search model; calculating a document score for each document by measuring a similarity between a sentence corresponding to the user's question and the document containing the answer content; and selecting a document containing the answer content based on the document score . The at least one processor
Configure prompts for target documents using sample data consisting of sample documents and sample queries;
The computer apparatus of claim 7 , further comprising: a language model that uses the prompts as input to generate queries for the target documents according to patterns in the sample data. The at least one processor
constructing a first prompt for the target document using first sample data, the first sample document and the first sample keywords;
using the first prompt as an input to the language model to generate keywords for the target document according to patterns in the first sample data;
constructing second prompts for keywords of the target documents using second sample data, the second sample documents, second sample keywords, and sample queries;
8. The computer apparatus of claim 7 , further comprising: a language model that uses the second prompt as an input to generate a query for the target document according to a pattern of the second sample data for keywords of the target document. The at least one processor
Converting the document score of each document into a probability distribution;
The computer device according to claim 7 , further comprising: determining a type of answer to the user's question according to an entropy of the probability distribution.

Images