JP7807364B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

This application relates to an information processing device, an information processing method, and an information processing program.

Technologies have been proposed in the past for implementing interactive communication in conjunction with chatbots. For example, Patent Document 1 proposes technology for implementing conversational communication through a digital board in conjunction with a chatbot.

Re-table No. 2020-240838

However, the above-mentioned conventional technology leaves room for improvement in terms of efficiently extracting information from online service users through chatbot communication. For example, the conventional technology aims to prevent users from abandoning chats, and is not intended to efficiently obtain reviews from service users, so there is still considerable room for improvement.

This application has been made in light of the above, and aims to provide an information processing device, information processing method, and information processing program that can efficiently collect information from service users of online services.

The information processing device according to the present application controls processing related to a dialogue conducted with a service user of an online service through a chatbot, and includes a selection unit and an instruction unit. The selection unit selects a conversation sequence according to the state of the service user from among a plurality of conversation sequences in which conversation patterns indicating the content of a series of conversations expected in the dialogue are predefined. The instruction unit instructs an external device that executes processing related to the dialogue to use the conversation sequence selected by the selection unit.

According to one aspect of the embodiment, it is possible to efficiently collect information from service users of online services.

FIG. 1 is a diagram illustrating an overview of information processing according to an embodiment. FIG. 2 is a diagram showing an outline of a conversation sequence according to the embodiment. FIG. 3 is a diagram showing an example of a conversation sequence instruction from the second server to the first server according to the embodiment. FIG. 4 is a diagram illustrating an overview of reinforcement learning according to the embodiment. FIG. 5 is a block diagram illustrating an example of the configuration of the second server according to the embodiment. FIG. 6 is a diagram showing an outline of conversation sequence information according to the embodiment. FIG. 7 is a diagram showing an overview of information related to a selection model according to the embodiment. FIG. 8 is a diagram showing an overview of user information according to the embodiment. FIG. 9 is a flowchart illustrating an example of a processing procedure executed by the second server according to the embodiment. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the second server according to the embodiment or each modification.

Below, detailed explanations will be given of the information processing device, information processing method, and information processing program (hereinafter referred to as "embodiments") according to the present application, with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. Furthermore, the embodiments can be combined as appropriate to the extent that the processing content is not contradictory. Furthermore, the same components in the following embodiments will be assigned the same reference numerals, and redundant explanations will be omitted.

(Embodiment)
1. System configuration according to the embodiment
First, the configuration of an information processing system SYS including a second server 200, which is an example of an information processing device according to an embodiment, will be described with reference to Fig. 1. Fig. 1 shows an example of the configuration of the information processing system SYS according to an embodiment. As shown in Fig. 1, the information processing system SYS according to an embodiment includes a user terminal 10, a first server 100, and a second server 200.

The user terminal 10, the first server 100, and the second server 200 are connected to a network such as the Internet (for example, network N shown in FIG. 5). The user terminal 10 and the first server 100 can communicate with each other through the network. The first server 100 and the second server 200 can communicate with each other through the network. The user terminal 10 and the second server 200 may also communicate with each other through the network. The information processing system SYS shown in FIG. 1 may have more user terminals 10 than the example shown in FIG. 1.

The user terminal 10 is an information processing terminal used by a service user U, who is a user of various online services operated by the administrator of the first server 100 as a platform provider. For example, the user terminal 10 can be realized as a smartphone, desktop PC (Personal Computer), notebook PC, tablet terminal, mobile phone, or PDA (Personal Digital Assistant).

The user terminal 10 also has communication functions for performing wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation: fourth generation mobile communication system), and 5G (5th Generation: fifth generation mobile communication system), as well as short-range wireless communication such as Bluetooth (registered trademark) and wireless LAN (Local Area Network), and can connect to networks using these communication functions.

The user terminal 10 can also display web content from various online services provided by the first server 100, for example, using a web browser or application. When the user terminal 10 receives control information from the first server 100 or the like that enables the display processing of information, the user terminal 10 performs the display processing in accordance with the control information.

The service user U can operate the user terminal 10 to browse websites of various online services displayed by a web browser and use web content displayed by the web browser. The service user U can also download a dedicated application program (hereinafter referred to as a "dedicated app") for using the websites of various online services from the first server 100 and install it on the user terminal U. In this case, the service user U can use the content of various online services designed for the dedicated app by operating the dedicated app.

The first server 100 is an information processing device that provides various online services to each service user. The first server 100 is typically a server device, but may also be realized by a mainframe, workstation, or the like. Furthermore, when the first server 100 is realized by a server device, it may be realized by a single server device, or it may be realized by a cloud system in which multiple server devices and multiple storage devices operate in cooperation with each other.

The various online services provided by the first server 100 may include internet connection, search services, SNS (Social Networking Service), e-commerce services, electronic payment services, online games, online banking services, online trading services, hotel reservation services, ticket reservation services, video distribution services, music distribution services, news distribution services, map information services, route search services, route guidance services, line information services, traffic information services, and weather information services. The various online services may also include API (Application Programming Interface) services corresponding to various applications.

Furthermore, when providing various online services, the first server 100 creates a user account including a user ID, which is user identification information for identifying each service user (for example, service user U). The user ID included in this user account is either set arbitrarily by the service user (for example, service user U) when registering to use the various online services, or is individually assigned by the first server 100. The first server 100 links the user account of each service user (for example, service user U) to record a service usage history (an example of "user information"), which is a usage history of the online services, and manages the recorded service usage history for each service user. Furthermore, the first server 100 can distribute dedicated apps for using various online services in response to a request from the service user (for example, service user U).

The first server 100 also performs processing related to interactions with service users (e.g., service user U) of various online services through the chatbot. The processing of the first server 100 will be described later.

The second server 200 is an information processing device that controls processing related to interactions conducted through a chatbot between service users of various online services (for example, service user U) and the first server 100. The second server 200 is typically a server device, but may also be implemented as a mainframe, workstation, or the like. When the second server 200 is implemented as a server device, it may be implemented as a single server device, or as a cloud system in which multiple server devices and multiple storage devices operate in cooperation with each other. The second server 200 will be described later.

2. Overview of Information Processing According to the Embodiment
An overview of information processing according to the embodiment will be described below with reference to Figures 1 to 4. In the following description, the user terminal 10 may be referred to as the service user U. In other words, the service user U can be read as the user terminal 10.

Furthermore, in the following explanation, when there is no need to distinguish between conversation sequence SQ1-1, conversation sequence SQ2-1, etc., they will be collectively referred to as "conversation sequence SQ."

Figure 1 shows an overview of information processing according to an embodiment. As shown in Figure 1, the first server 100 provides service content CT for the online service currently being accessed by the service user U, and also performs processing related to the dialogue with the service user U through the dialogue screen of the chatbot CB displayed together with the service content CT. When providing the service content CT, the first server 100 can acquire attribute information indicating the attributes of the service user U. The first server 100 can also acquire the conversation history of the dialogue between the chatbot CB and the service user U, and information related to the service user U's reactions.

First, when executing the above-described process related to the dialogue with the service user U, the first server 100 sends a query for the initial conversation sequence SQ to the second server 200 (step S01). Figure 2 shows an overview of the conversation sequence according to the embodiment.

The conversation sequence SQ according to the embodiment is information that predefines conversation patterns that indicate the content of a series of conversations expected in a dialogue between the chatbot CB and a service user. The administrator of the first server 100 and the second server 200 identifies multiple conversation patterns that are extracted as necessary (essential) groups of the content (exchanges of utterances and responses) of a series of conversations that take place between the chatbot CB and the service user (for example, service user U shown in Figure 1). The administrator then sets each of the identified multiple conversation patterns as a conversation sequence SQ.

For example, the conversation sequence SQ1-1 shown in FIG. 2 is composed of a series of conversational contents uttered in chronological order, including utterance X1-1, utterance X1-2, utterance X1-3, and utterance (question) Q1-1. Utterance (question) Q1-1 is a question posed by chatbot CB to service user U. For example, utterance (question) Q1-1 is associated with answer options that allow service user U to select an answer to the question posed by chatbot CB. When utterance (question) Q1-1 is displayed on chatbot CB, the answer options are also displayed.

Furthermore, the conversation sequence SQ2-1 shown in Figure 2 is composed of a series of conversational contents uttered in chronological order, including utterance X2-1, utterance (question) Q2-1, utterance X2-2, and utterance (question) Q2-2. Utterance (question) Q2-1 and utterance (question) Q2-2 have the same properties as the above-mentioned utterance (question) Q1-1.

When the second server 200 receives an inquiry about the first conversation sequence from the first server 100, it selects the first conversation sequence SQ from a plurality of predefined conversation sequences SQ (step S02). Then, the second server 200 transmits an instruction for the selected first conversation sequence SQ to the first server 100 (step S03).

For example, the second server 200 may preset an initial conversation sequence common to various online services. Alternatively, the second server 200 may preset an initial conversation sequence corresponding to each online service for each online service. In this case, the second server 200 acquires, from the first server 100, an inquiry about the initial conversation sequence SQ, as well as service information for identifying the online service being used by the service user U who will be the other party in conversation with the chatbot CB. The second server 200 then selects, as the initial conversation sequence SQ, a conversation sequence SQ that is pre-associated with the acquired service information.

The second server 200 may also pre-set an initial conversation sequence SQ that corresponds to the attributes (such as demographic attributes or psychographic attributes) of the service user who will be the other party in the conversation with the chatbot CB. In this case, the second server 200 acquires, from the first server 100, an inquiry about the initial conversation sequence SQ, as well as attribute information indicating, for example, the attributes of the service user U who will be the other party in the conversation with the chatbot CB. The second server 200 then selects the conversation sequence SQ associated with the acquired attribute information as the initial conversation sequence SQ.

The second server 200 may also select the final conversation sequence SQ to be used in the dialogue between the chatbot CB and the service user U according to predetermined rules.

When the first server 100 receives the instruction for the initial conversation sequence SQ from the second server 200, it executes processing related to the dialogue with the service user U through the chatbot CB in accordance with the received initial conversation sequence SQ (step S04). According to the example of the dialogue screen of the chatbot CB shown in Figure 1, based on the information transmitted from the first server 100, information D-1 to D-3 corresponding to the utterances included in the conversation sequence SQ are displayed on the user terminal 10 from top to bottom in the order set in the conversation sequence SQ.

When the first server 100 completes the dialogue based on the first conversation sequence SQ, it sends a query for the next conversation sequence SQ to the second server 200 (step S05). At this time, the first server 100 also sends to the second server 200 information for identifying the immediately preceding conversation sequence SQ and information regarding the reaction of the service user U in the dialogue with the chatbot CB.

When the second server 200 receives an inquiry about the next conversation sequence SQ from the first server 100, it uses a selection model for selecting a conversation sequence SQ to be used in a dialogue through the chatbot CB to select the next conversation sequence SQ that corresponds to the state of the service user U from among multiple predefined conversation sequences SQ (step S06). Then, the second server 200 transmits an instruction for the selected next conversation sequence SQ to the first server 100 (step S07).

Hereinafter, an example of an instruction for a conversation sequence SQ from the second server 200 to the first server 100 will be specifically described using Figure 3. Figure 3 shows an example of an instruction for a conversation sequence SQ from the second server 200 to the first server 100 according to the embodiment.

As shown in FIG. 3, the first server 100 sends a query for the first conversation sequence SQ to the second server 200 (step S11). At this time, the first server 100 sends attribute information (attribute UA) indicating the attributes of the service user U along with the query for the first conversation sequence SQ.

When the second server 200 receives an inquiry about the first conversation sequence SQ from the first server 100, it selects the first conversation sequence SQ1-1 and transmits the sequence number "SN101" of the selected first conversation sequence SQ1-1 to the first server 100 (step S12). The second server 200 also retains attribute information indicating the attributes of the service user U received from the first server 100.

When the first server 100 completes the dialogue with the service user U using the conversation sequence SQ corresponding to the sequence number "SN101" received from the second server 200, it sends a query for the next conversation sequence SQ to the second server 200 (step S13). At this time, the first server 100 sends the query for the next conversation sequence SQ, along with the sequence number "SN101" of the previous conversation sequence SQ1-1, and information "Response R-1" indicating the response of the service user U in the dialogue with the chatbot CB.

When the second server 200 receives a query for the next conversation sequence SQ from the first server 100, it uses the selection model to select the next conversation sequence SQ2-1 that corresponds to the state of the service user U, and transmits the sequence number "SN201" of the selected next conversation sequence SQ2-1 to the first server 100 (step S14). For example, the second server 200 inputs attribute information indicating the attributes of the service user U (an example of "user information"), the sequence number "SN101" of the previous conversation sequence (an example of "conversation history"), and information "answer R-1" indicating the response result of the service user U in the dialogue with the chatbot CB (an example of "service user response") into the selection model as information indicating the state of the service user U, and then transmits the sequence number "SN201" output from the selection model to the first server 100.

Note that the attribute information indicating the attributes of the service user U is not limited to static information such as demographic attributes and psychographic attributes, but may also include dynamic information such as location information and biometric information. In this case, the second server 200 acquires dynamic information such as the location information and biometric information of the service user U each time it receives an inquiry about the next conversation sequence SQ from the first server 100, and can select the next conversation sequence SQ based on the status of the service user U updated based on the acquired dynamic information. The second server 200 can also acquire the service usage history of the service user U (such as purchase history and reservation history) from the first server 100 as information indicating the status of the service user, and use the acquired service usage history as input information when selecting a conversation sequence.

Returning to FIG. 1, the second server 200 sets a reward based on the response of the service user U in the dialogue conducted through the chatbot CB, and performs reinforcement learning of the selection model for each conversation sequence used in the dialogue (step S08). Figure 4 shows a schematic overview of reinforcement learning according to the embodiment.

As shown in FIG. 4, in reinforcement learning according to this embodiment, the selection model can be regarded as the reinforcement learning agent, and the dialogue between the chatbot CB and the service user U can be regarded as the reinforcement learning environment. In this case, reinforcement learning proceeds in the following procedure. First, the selection model selects a conversation sequence SQ corresponding to the state of the service user U, in accordance with a policy that is believed to produce the desired result. Here, the state of the service user U includes attribute information indicating the attributes of the service user U, the conversation sequence (conversation history) used in the dialogue between the chatbot CB and the service user U, and the response of the service user U in the dialogue with the chatbot CB (the reaction of the service user U). The conversation sequence SQ selected by the selection model is transmitted from the second server 200 to the first server 100.

Next, when the dialogue using the conversation sequence SQ is completed in the first server 100, the conversation history is sent from the first server 100 to the second server 200, and the state of the service user U after the dialogue using the conversation sequence SQ (the conversation sequence SQ used in the dialogue and the service user U's response in the dialogue) is fed back to the selection model. At the same time, a reward based on the service user U's response is fed back to the selection model. The selection model then reviews its policy based on the state of the service user U after the dialogue using the conversation sequence SQ and the reward based on the service user U's response.

In other words, the second server 200 acquires from the first server 100 information regarding the service user U's response (reaction) in the dialogue with the chatbot CB as a change that the action of selecting the conversation sequence SQ has brought to the environment of the dialogue between the chatbot CB and the service user U. Then, the second server 200 sets a reward based on the service user U's response (response in the dialogue) to the conversation sequence SQ selected by the selection model as an evaluation of the change that the action of selecting the conversation sequence SQ has brought to the environment of the dialogue between the chatbot CB and the service user U.

In this way, the second server 200 can perform reinforcement learning on a conversation sequence basis to optimize the selection of conversation sequences SQ using the selection model, for example, by setting a reward based on the response of the service user U in the immediately preceding conversation sequence SQ, so as to maximize the reward obtained from the dialogue between the chatbot CB and the service user U.

Furthermore, during reinforcement learning, the reward set by the second server 200 for a conversation sequence SQ is set based on at least the content of the conversation that took place using the immediately preceding conversation sequence SQ and the outcome of the conversation.

For example, the second server 200 may set a reward based on whether the service user U responded favorably in the dialogue using the immediately preceding (previous) conversation sequence SQ. Specifically, if the response obtained from the service user U in the dialogue using the immediately preceding conversation sequence SQ was favorable, the second server 200 will provide a positive (e.g., +1) reward for the immediately preceding conversation sequence SQ. On the other hand, if the response obtained from the service user U in the dialogue using the immediately preceding conversation sequence SQ was not favorable, the second server 200 will provide a negative (e.g., -1) reward for the immediately preceding conversation sequence SQ.

When setting the reward for the most recent conversation sequence SQ, the second server 200 may set the reward according to changes in the service user U's response in past conversations. For example, if the responses of the service user U in the conversation before last and the immediately preceding (previous) service user U were both favorable, a positive reward (e.g., +2) may be given for the immediately preceding conversation sequence SQ; if the service user U in the conversation before last was favorable but the immediately preceding (previous) service user U's response was unfavorable, no reward may be given for the immediately preceding conversation sequence SQ; and if the responses of the service user U in the conversation before last and the immediately preceding (previous) service user U were unfavorable, a negative reward (e.g., -2) may be given for the immediately preceding conversation sequence SQ. In this way, the second server 200 can be expected to optimize the selection model so that a conversation sequence SQ is selected according to changes in the service user U's response.

Furthermore, for example, the second server 200 may set a reward based on whether or not a predetermined conversion associated with the immediately preceding (previous) conversation sequence SQ was obtained from the service user U. Specifically, if the second server 200 is able to obtain predetermined information associated with the immediately preceding conversation sequence SQ from the service user U (for example, if the second server 200 is able to obtain information about a product that the service user U is looking for), it will provide a positive reward (for example, +1) for the immediately preceding conversation sequence SQ. On the other hand, if the second server 200 is unable to obtain predetermined information associated with the immediately preceding conversation sequence SQ from the service user U, it will provide a negative reward (for example, -1) for the immediately preceding conversation sequence SQ.

In this way, the second server 200 sets a reward based on the service user U's response to the immediately preceding conversation sequence SQ and performs learning so that the selection of the conversation sequence SQ by the selection model is optimized. The second server 200 can also perform reinforcement learning of the selection model for each attribute of the service user U. As a result, by using the selection model, the second server 200 can increase the likelihood of selecting a conversation sequence SQ that will produce desirable results depending on the attributes and state of the service user U.

The second server 200 can use any method to perform reinforcement learning of the selection model. The second server 200 may use value-based methods such as Q-learning or SARSA, or policy-based methods such as policy gradient methods.

Returning to Figure 1, when the first server 100 receives an instruction for the next conversation sequence SQ from the second server 200, it executes processing related to the dialogue with the service user U through the chatbot CB in accordance with the received next conversation sequence SQ (step S09).

3. Configuration of the second server according to the embodiment
An example configuration of the second server 200 according to the embodiment will be described with reference to Fig. 5. Fig. 5 shows an example configuration of the second server 200 according to the embodiment. As shown in Fig. 5, the second server 200 includes a communication unit 210, a storage unit 220, and a control unit 230.

(Regarding the communication unit 210)
The communication unit 210 is realized by, for example, a network interface card (NIC) or the like. The communication unit 210 is connected to the network N by wire or wirelessly. The second server 200 transmits and receives information to and from other devices such as the user terminal 10 and the first server 100 via the network N.

(Regarding the storage unit 220)
The storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 220 has a conversation sequence storage unit 221, a selection model storage unit 222, and a user information storage unit 223.

(Conversation sequence storage unit 221)
The conversation sequence storage unit 221 stores information on conversation sequences used in a dialogue with a service user (for example, the service user U shown in FIG. 1) through the chatbot CB. FIG. 6 is a diagram showing an overview of the information on conversation sequences according to the embodiment.

As shown in Figure 6, the conversation sequence information stored in the conversation sequence storage unit 221 has multiple items such as a "sequence number" item, a "conversation pattern" item, a "content for receiving answers" item, and a "supported service" item. These items in the conversation sequence information are associated with each other.

The "Sequence Number" field stores an identification number assigned individually to each conversation sequence to identify the conversation sequence. The "Conversation Pattern" field stores information about the conversation pattern included in the conversation sequence. The "Answer Receiving Content" field stores answer receiving content that is displayed in association with the utterances (questions) included in the conversation pattern. Furthermore, if the answer receiving content includes multiple answers for receiving evaluations from service users, each answer is associated in advance with an attribute value indicating whether it is a favorable answer. The "Compatible Services" field stores information indicating the various online services to which the conversation sequence applies.

(Selection model storage unit 222)
The selection model storage unit 222 stores information about selection models used when selecting conversation sequences. Fig. 7 is a diagram showing an overview of the information about selection models according to this embodiment.

As shown in FIG. 7, information about selection models stored in the selection model storage unit 222 according to the embodiment includes multiple items such as a "model ID" field, a "corresponding attribute" field, and a "model information" field. These items included in the information about selection models are associated with each other.

The "Model ID" field stores identification information for identifying the selection model. The "Corresponding Attributes" field stores information indicating the attributes of the service user (for example, service user U shown in Figure 1) corresponding to the selection model. The "Model Information" field stores information related to the selection model's policy and various information constituting the selection model, such as various parameters.

(User information storage unit 223)
The user information storage unit 223 stores information about service users who use various online services (for example, service user U shown in FIG. 1). FIG. 8 is a diagram showing an overview of user information according to the embodiment.

As shown in FIG. 8, the user information stored in the user information storage unit 223 according to the embodiment has multiple items such as a "user ID" field, an "attribute information" field, and an "interaction history" field. These items in the user information are associated with each other.

The "User ID" field stores identification information for identifying service users of various online services (for example, service user U shown in Figure 1). The "Attribute Information" field stores information related to the service user's attributes, such as demographic attributes, psychographic attributes, location information, and biometric information. The "Dialogue History" field stores dialogue history, including conversation sequences selected in dialogue with chatbot CB.

The user information storage unit 223 may also store the service usage history of the service user as user information related to the service user. For example, the second server 200 (control unit 230) can acquire the service usage history of the service user from the first server 100, associate the acquired service usage history with the service user's identification information (user ID), and register it in the user information storage unit 223.

(Regarding the control unit 230)
The control unit 230 is realized by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the second server 200 using RAM as a work area. The control unit 230 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 230 shown in FIG. 5 has a selection unit 231, an instruction unit 232, and a learning unit 233, and these units realize or execute the information processing functions and actions described below. The control unit 230 may have an internal configuration divided into multiple processing units that realize or execute the information processing functions and actions described below. The control unit 230 is not limited to the configuration shown in FIG. 5, and may have other configurations that perform the information processing described below. Functional units other than those shown in FIG. 5 may be added to the control unit 230 in response to the addition of processes to be executed by the second server 200.

(Selection unit 231)
The selection unit 231 selects a conversation sequence SQ corresponding to the state of a service user of an online service (for example, the service user U shown in FIG. 1 ) from a plurality of conversation sequences SQ in which conversation patterns indicating a series of conversation contents expected in a dialogue between the service user and the service user through a chatbot are predefined. For example, when the selection unit 231 receives an inquiry about a conversation sequence SQ from the first server 100 via the communication unit 210, the selection unit 231 selects a conversation sequence.

The selection unit 231 also selects a conversation sequence from multiple conversation sequences SQ stored in the conversation sequence storage unit 221 using a selection model (for example, the selection model shown in FIG. 2) that selects a conversation sequence SQ to be used in a conversation by setting a reward based on the response of a service user (for example, service user U shown in FIG. 1) in a conversation conducted through the chatbot. For example, the selection unit 231 acquires a selection model associated with attribute information indicating the attributes of the service user included in the inquiry for the conversation sequence SQ acquired from the first server 100. The selection unit 231 then inputs the most recent conversation sequence and the response of service user U into the acquired selection model, thereby selecting a conversation sequence linked to the sequence number output from the selection model from multiple conversation sequences stored in the conversation sequence storage unit 221.

(Instruction unit 232)
The instruction unit 232 instructs the first server 100 (an example of an “external device”) that executes processing related to the dialogue to use the conversation sequence SQ selected by the selection unit 231. For example, the instruction unit 232 transmits the sequence number of the conversation sequence to the first server 100 via the communication unit 210.

(Learning unit 233)
The learning unit 233 performs reinforcement learning of a selection model that selects a conversation sequence SQ to be used in a conversation by setting a reward based on the response of a service user (for example, service user U shown in Figure 1) in a dialogue conducted through the chatbot CB.

For example, the learning unit 233 performs reinforcement learning of the selection model by setting a reward based on whether the service user's response was favorable or not in at least the dialogue using the immediately preceding conversation sequence SQ.

Furthermore, for example, the learning unit 133 performs reinforcement learning of the selection model by setting a reward based on whether or not a specified conversion associated with the immediately preceding conversation sequence was obtained from the service user.

4. Processing Procedure According to the Embodiment
The following describes the procedure of information processing executed by the second server 200 according to the embodiment. Fig. 9 shows an example of the procedure of processing executed by the second server 200 according to the embodiment. The procedure of processing shown in Fig. 9 is executed by the control unit 230 of the second server 200. The procedure of processing shown in Fig. 9 is repeatedly executed while the second server 200 is operating.

As shown in FIG. 9, the selection unit 231 obtains a query for a conversation sequence SQ from the first server 100 (step S101).

The selection unit 231 also uses the selection model to select a conversation sequence that corresponds to the state of the service user (for example, the service user shown in Figure 1) who is having a conversation with the chatbot CB (step S102).

The instruction unit 232 also instructs the first server 100 to use the conversation sequence selected by the selection unit 231 (step S103).

The learning unit 233 also sets a reward based on the service user's response in the dialogue using the conversation sequence and performs reinforcement learning of the selection model (step S104), and then ends the processing procedure shown in Figure 9.

5. Modifications
The information processing device, the information processing method, and the information processing program according to the present application may be implemented in various different forms other than the above-described embodiment. Modifications of the above-described embodiment will be described below.

(5-1. Conversation Sequence)
The conversation sequences according to the above embodiments may be set for each predetermined topic, such as a search sequence, a today's mood sequence, or a campaign sequence. For example, a search sequence may have a conversation pattern such as "What kind of book are you looking for? → Please select a genre → ...". For example, a today's mood sequence may have a conversation pattern such as "How are you feeling today? → ...". For example, a campaign sequence may have a conversation pattern such as "We're running a great campaign today → ...".

The first server 100 may also implement controls such as not displaying a given conversation more than N times (N is a natural number).

(5-2. Learning the selection model)
In the above embodiment, the learning of the selection model executed in the second server 200 may be executed for each attribute of the service user. That is, the second server 200 sets a common selection model for each service user with the same attribute and executes reinforcement learning. In this case, the second server 200 may collect responses (contents of responses in the dialogue) of each service user for each conversation sequence at a predetermined timing and execute reinforcement learning based on the collected responses.

(5-3. About chatbots)
In the above embodiment, the first server 100 may display a virtual character image corresponding to the chatbot CB on the dialogue screen of the chatbot CB. In this case, the first server 100 may change the facial expression of the character image depending on the content of the response of the service user who is the dialogue partner. Furthermore, the first server 100 may change the appearance of the character image depending on the attributes of the service user, etc.

(6. Effects)
The second server 200 according to the embodiment is an information processing device that controls processing related to a dialogue conducted with a service user of an online service through a chatbot, and includes a selection unit 231 and an instruction unit 232. The selection unit 231 selects a conversation sequence according to the state of the service user from a plurality of conversation sequences in which conversation patterns indicating a series of conversation contents expected in the dialogue are predefined. The instruction unit 232 instructs the first server 100, which executes processing related to the dialogue, to select the conversation sequence selected by the selection unit 231.

As a result, the second server 200, which is an example of an information processing device according to the embodiment, can efficiently collect information from service users of online services. For example, the second server 200 according to the embodiment can be expected to have the effect of improving the quality of the user experience in the dialogue by conducting a dialogue with the service user in units of predefined conversation sequences. In other words, the second server 200 according to the embodiment can realize a more natural conversation with the service user U through the chatbot CB. As a result, it is possible to increase the likelihood that the dialogue with the chatbot CB will continue, and to efficiently collect information from the service user through the dialogue.

The second server 200 also has a learning unit 233 that performs reinforcement learning of a selection model on a conversation sequence basis, which selects a conversation sequence to be used in a conversation by setting a reward for the conversation sequence based on the service user's response in a conversation conducted through the chatbot. The selection unit 231 selects a conversation sequence using the selection model.

The selection unit 231 also uses a selection model to select a conversation sequence based on user information about the service user, a conversation history indicating the content of the most recent conversation, and the service user's response to the conversation.

As a result, the second server can create natural conversations for each service user that are tailored to the service user's reactions during the conversation.

The user information also includes attribute information indicating the attributes of the service user and the service usage history of the online service.

As a result, the second server can create natural conversations that are tailored to the service user's attributes and service usage situation.

In this way, the second server 200 performs reinforcement learning of the selection model on a conversation sequence basis, thereby preventing a decrease in the quality of the user experience in dialogues using conversation sequences, which is caused by the randomness of learning inherent in reinforcement learning. Furthermore, by using conversation sequences to perform learning on a conversation sequence basis, the second server 200 can reduce the amount of learning compared to normal learning using reinforcement learning for chatbot CB, thereby improving system efficiency.

The learning unit 233 also performs reinforcement learning of the selection model by setting a reward based on whether the service user's response was favorable or not in the dialogue based on at least the immediately preceding conversation sequence.

As a result, the second server 200 can optimize the selection of conversation sequences using the selection model so that the service user's reaction during a dialogue using the conversation sequences is favorable.

In addition, the learning unit 233 performs reinforcement learning of the selection model by setting a reward based on whether or not a specific conversion associated with the immediately preceding conversation sequence was obtained from the service user.

As a result, the second server 200 can optimize the selection of conversation sequences using a selection model so that desirable results are obtained from service users through dialogue using the conversation sequences.

7. Hardware Configuration
The second server 200 according to the above-described embodiment and each of the modified examples is realized, for example, by a computer 1000 having a configuration as shown in Fig. 10. Fig. 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the second server 200 according to the embodiment and each of the modified examples.

The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected via a bus 1090.

The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and secondary storage device 1050, and programs read from the input device 1020, and executes various processes. The primary storage device 1040 is a memory device, such as RAM, that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), HDD, flash memory, etc.

The output IF 1060 is an interface for transmitting information to be output to an output device 1010, such as a monitor or printer, which outputs various types of information, and is implemented, for example, by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, and scanner, and is implemented, for example, by USB.

The input device 1020 may be a device that reads information from optical recording media such as a CD (Compact Disc), DVD (Digital Versatile Disc), or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. The input device 1020 may also be an external storage medium such as a USB memory.

Network IF 1080 receives data from other devices via network N and sends it to the computing device 1030, and also transmits data generated by the computing device 1030 to other devices via network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the second server 200 according to the embodiment, the arithmetic device 1030 of the computer 1000 executes a program (e.g., an information processing program) loaded onto the primary storage device 1040, thereby realizing functions similar to those of the control unit 230. In other words, the arithmetic device 1030 works in cooperation with the program (e.g., an information processing program) loaded onto the primary storage device 1040 to realize the processing of the second server 200 according to the embodiment.

[8. Other]
Of the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

Furthermore, the components of each device shown in the figure are conceptual and functional, and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. For example, the selection unit 231 and instruction unit 232 of the control unit 230 of the second server 200 may be functionally integrated. Also, for example, the first server 100 and the second server 200 in the information processing system SYS may be a single information processing device that is functionally and physically integrated.

Furthermore, the above-described embodiments can be combined as appropriate to the extent that the processing content is not contradictory.

The above describes in detail the embodiments of the present application based on several drawings, but these are merely examples, and the present invention can be implemented in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section.

In addition, the above-mentioned "section, module, unit" can be interpreted as "means" or "circuit." For example, a control unit can be interpreted as a control means or a control circuit.

N Network SYS Information Processing System 10 User Terminal 100 First Server 200 Second Server 210 Communication Unit 220 Storage Unit 221 Conversation Sequence Storage Unit 222 Selection Model Storage Unit 223 User Information Storage Unit 230 Control Unit 231 Selection Unit 232 Instruction Unit 233 Learning Unit

Claims (8)

An information processing device that controls processing related to a dialogue conducted between a service user of an online service and a chatbot,
a selection unit that selects a conversation sequence from a plurality of conversation sequences in which conversation patterns indicating a series of conversation contents expected in the dialogue are defined in advance;
an instruction unit that instructs an external device that executes a process related to the dialogue to use the conversation sequence selected by the selection unit,
The selection unit
When receiving an inquiry about an initial conversation sequence from the external device, select, from the plurality of conversation sequences, an initial conversation sequence common to the online services, an initial conversation sequence preset for each online service, or an initial conversation sequence corresponding to an attribute of the service user who will be the other party in the conversation;
An information processing device characterized by acquiring dynamic information including location information and biometric information of the service user each time an inquiry for a next conversation sequence is received, and selecting the next conversation sequence based on the status of the service user updated based on the acquired dynamic information. a learning unit that performs reinforcement learning of a selection model that selects a conversation sequence to be used in the conversation by setting a reward for the conversation sequence based on a response of the service user in the conversation conducted through the chatbot,
The selection unit
The information processing device according to claim 1 , further comprising: selecting a conversation sequence using the selection model associated with attribute information indicating attributes of the service user included in the inquiry acquired from the external device. The selection unit
The information processing device according to claim 2, characterized in that a conversation sequence linked to a sequence number output from the selection model is selected by inputting a conversation sequence immediately before a next conversation sequence and the service user's reaction in the dialogue into the selection model. The learning unit
The information processing device according to claim 2, characterized in that reinforcement learning of the selection model is performed by setting the reward based on whether or not the service user's response to the dialogue in at least the immediately preceding conversation sequence was favorable. The learning unit
The information processing device according to claim 4 , wherein when a reward for a previous conversation sequence is set, the reward is set according to a change in the reaction of the service user in the previous conversation. The learning unit
The information processing device according to claim 2, wherein reinforcement learning of the selection model is performed by setting the reward based on whether or not a predetermined conversion associated with the immediately preceding conversation sequence has been obtained from the service user. An information processing method for controlling processing related to a dialogue conducted between a service user of an online service and a chatbot, comprising:
a selection step of selecting a conversation sequence from a plurality of conversation sequences in which conversation patterns indicating a series of conversation contents expected in the dialogue are defined in advance;
an instruction step of instructing an external device that executes processing related to the dialogue to use the conversation sequence selected in the selection step,
The selection step includes:
When receiving an inquiry about an initial conversation sequence from the external device, select, from the plurality of conversation sequences, an initial conversation sequence common to the online services, an initial conversation sequence preset for each online service, or an initial conversation sequence corresponding to an attribute of the service user who will be the other party in the conversation;
An information processing method characterized by acquiring dynamic information including location information and biometric information of a service user each time an inquiry for a next conversation sequence is received, and selecting a next conversation sequence based on the status of the service user updated based on the acquired dynamic information. A computer that controls the processing of interactions between users of an online service and the chatbot,
a selection step of selecting a conversation sequence from a plurality of conversation sequences in which conversation patterns indicating a series of conversation contents expected in the dialogue are defined in advance;
an instruction step of instructing an external device that executes processing related to the dialogue to use the conversation sequence selected by the selection step;
The selection procedure comprises:
When receiving an inquiry about an initial conversation sequence from the external device, select, from the plurality of conversation sequences, an initial conversation sequence common to the online services, an initial conversation sequence preset for each online service, or an initial conversation sequence corresponding to an attribute of the service user who will be the other party in the conversation;
An information processing program characterized by acquiring dynamic information including location information and biometric information of a service user each time an inquiry for a next conversation sequence is received, and selecting a next conversation sequence based on the status of the service user updated based on the acquired dynamic information.