JP7733329B2 - Information processing device and program - Google Patents

Description

The present invention relates to an information processing device and a program.

Various technologies are known for presenting advertisements to users of information processing devices. For example, Patent Document 1 discloses a method for selecting advertisements based on the content of chats between users.

Japanese Patent Application Publication No. 11-3348

The above-mentioned conventional technology presents advertisements to users in chats so that they are clearly visible. However, when using a machine that can interact with users, there is a problem in that if the machine's speech includes an advertisement, the conversation with the user becomes unnatural.

The present invention was made in consideration of the above-mentioned circumstances, and one of its objectives is to provide an information processing device and program that can include natural advertising text in the speech of a machine that can interact with a user.

One aspect of the present invention that solves the problems of the above-mentioned conventional examples is an information processing device that includes: storage means for storing advertising copy, which is the content of the advertisement; extraction means for extracting predetermined reserved words contained in conversational text entered by the user; selection means for selecting an advertising copy to present to the user from the advertising copy based on the extracted reserved words; and execution means for executing speech processing of a response text that includes the selected advertising copy following the conversational text based on predetermined criteria.

This invention provides an information processing device and program that can include natural advertising text in the speech of a machine that can interact with a user.

1 is a block diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of the configuration of a terminal device according to an embodiment of the present invention. 1 is an explanatory diagram illustrating an example of how advertising copy is stored in the information processing system according to the embodiment of the present invention; FIG. 3 is an explanatory diagram illustrating an example of setting information used in the information processing system according to the embodiment of the present invention. FIG. 2 is a functional block diagram illustrating an example of a server according to an embodiment of the present invention. FIG. 10 is an explanatory diagram showing an example of the contents of a conversation sentence queue used in the information processing system according to the embodiment of the present invention. FIG. 2 is a functional block diagram illustrating an example of a terminal device according to an embodiment of the present invention. 3 is a flowchart illustrating an example of the operation of the information processing system according to the embodiment of the present invention. 10 is another flowchart illustrating an example of the operation of the information processing system according to the embodiment of the present invention.

An embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, an information processing system 1 according to an embodiment of the present invention includes a server 10 and a terminal device 20 that are communicatively connected to each other via a communication means such as a network.

In one example of this embodiment, the server 10 realizes the information processing device of the present invention. As shown in FIG. 1, the server 10 in this example is configured to include a control unit 11, a memory unit 12, and a communication unit 13. The terminal device 20 is a robot, and as shown in FIG. 2, includes at least a leg unit 21 and a main body unit 22, which houses a control unit 31, a memory unit 32, a sensor unit 33, a display unit 34, an audio output unit 35, a communication unit 36, and a drive unit 37. The leg unit 21 and the main body unit 22 are connected via an actuator that can rotate around at least one axis, allowing the orientation of the main body unit 22 to be rotated relative to the leg unit 21.

The control unit 11 of the server 10 is a program-controlled device such as a CPU, and operates according to a program stored in the memory unit 12. In this embodiment, the control unit 11 accepts action request information from the terminal device 20. In response to the accepted request information, the control unit 11 transmits action information to the terminal device 20, which is the sender of the request information, the action information including an action instruction that instructs the action to be executed by the terminal device 20 and character string information representing the content of the voice to be uttered by the terminal device 20. The detailed processing content of the control unit 11 will be explained later.

The storage unit 12 is a disk device or memory device that stores programs executed by the control unit 11. The storage unit 12 also functions as a work memory for the control unit 11. In one example of this embodiment, the storage unit 12 may also store information for generating instructions to the terminal device 20. The contents of this information will be described later.

In one example of this embodiment, as shown in FIG. 3, the memory unit 12 stores, for each product (P) to be advertised, an advertising identifier (I), at least one advertising copy (E) that is the content of the advertisement to be delivered to the terminal device 20, information (A) that identifies the advertiser, and genre information (G) that identifies the advertising genre, all in association with each other. The advertising copy may also be associated with a URL (U) of a webpage that contains information about the product (P) to be advertised. In this way, in one example of this embodiment, the memory unit 12 functions as the storage means of the present invention. Note that the number of advertising copies per product (P) is not limited to one, and there may be multiple copies.

Furthermore, in one example of this embodiment, the storage unit 12 may also hold a conversation sentence queue that accumulates conversation sentence candidates to be spoken by the terminal device 20. The contents of this conversation sentence queue will be described later.

The communication unit 13 is a network interface or the like, and sends various information to the terminal device 20 via the network in accordance with instructions input from the control unit 11. The communication unit 13 also outputs information received via the network to the control unit 11.

The control unit 31 of the terminal device 20 is a program-controlled device such as a CPU, and operates according to a program stored in the memory unit 32. In this embodiment, the control unit 31 sends action request information to the server 10 at a predetermined timing. The control unit 31 also receives action information from the server 10. Upon receiving the action information, the control unit 31 synthesizes audio data based on the character string information included in the action information. The control unit 31 also plays back the synthesized audio data and performs processing such as displaying animations in accordance with the instructions in the received action information.

In one example of this embodiment, when the sensor unit 33, which will be described later, receives a user's voice input, the control unit 31 of the terminal device 20 converts the input voice into character string information. This process can use widely known voice recognition processing, and the control unit 31 may perform this process, for example, by sending input voice information to a voice recognition server that performs voice recognition processing and receiving recognized character string information.

The control unit 31 also sends action request information to the server 10 when speech input by the user is used as a trigger. This request information includes information identifying the trigger (e.g., information indicating that speech has been input by the user) and information necessary for processing by the server 10, such as character string information that is the recognition result of the speech input by the user.

In other words, when the control unit 31 determines that a predetermined trigger has occurred, it collects the information necessary for processing on the server 10 and sends request information including the collected information to the server 10, along with information identifying the trigger. This trigger can be any trigger, such as when a user inputs voice, as in the previous example, or when a predetermined time arrives. The detailed operation of this control unit 31 will be explained later.

The storage unit 32 is a memory device or the like, and stores programs executed by the control unit 31. This storage unit 32 also operates as a work memory for the control unit 31. In this embodiment, this storage unit 32 may store setting information that associates the above-mentioned trigger (N) with information (P) that specifies the information to be sent to the server 10, etc. (Figure 4). This setting information will be explained later with specific examples.

Furthermore, in one example of this embodiment, the memory unit 32 stores image data of an animation to be displayed on the display unit 34. Specifically, the memory unit 32 may store multiple image data of parts representing facial expressions, such as image data of animated eyes, such as image data of smiling eyes, image data of animated eyes with tears flowing, etc., and image data of animated mouths, such as image data of a closed mouth in a smiling state, image data of a closed mouth in a crying state, image data of an animated mouth during speech, etc.

The sensor unit 33 includes at least a microphone, which is an audio sensor. The sensor unit 33 may also include a touch sensor, an acceleration sensor, etc. The sensor unit 33 outputs audio signals detected by each sensor, information indicating the position touched by the user, acceleration information, etc. to the control unit 31.

The display unit 34 is a liquid crystal display or the like, and displays image data representing the character's facial expressions, etc., in accordance with instructions input from the control unit 31. The audio output unit 35 is a speaker or the like, and produces sound in accordance with audio signals input from the control unit 31.

The communication unit 36 includes a network interface. This communication unit 36 sends and receives information to and from the server 10 via a network, either wirelessly or via a wired connection. Specifically, the communication unit 36 sends request information and the like to the server 10 in accordance with instructions input from the control unit 31. The communication unit 36 also outputs information received from the server 10 to the control unit 31.

The drive unit 37 drives the actuator to rotate the main body unit 22 relative to the legs 21 in accordance with instructions input from the control unit 31.

Next, the operation of the control unit 11 of the server 10 in this embodiment will be described. In this embodiment, the control unit 11 of the server 10 includes a receiving unit 41, a reserved word extraction unit 42, an advertisement selection unit 43, an action information generation unit 44, and an instruction transmission unit 45, as illustrated in FIG. 5.

The receiving unit 41 receives request information from the terminal device 20. In this embodiment, the server 10 accepts request information from the terminal device 20 requesting a process (action) to be executed by the terminal device 20. This request information includes information identifying the cause (trigger) of the action request. Types of triggers will be described later, but an example would be voice input by the user. The request information based on the trigger of voice input by the user may include information identifying the trigger as well as information representing the content of the voice input by the user. Here, the information representing the content of the voice input by the user may be character string information that is the result of voice recognition.

The receiving unit 41 outputs the information contained in the received request information, such as information identifying the trigger and character string information representing the content of the voice input by the user, to the reserved word extraction unit 42 and the action information generation unit 44.

The reserved word extraction unit 42 extracts reserved words contained in the character string information output by the receiving unit 41. Here, reserved words are predetermined keywords, and in this embodiment, a plurality of keywords are listed in advance as reserved words and stored in the storage unit 12. If the listed reserved words are contained in the character string information output by the receiving unit 41, the reserved word extraction unit 42 extracts the reserved words and outputs them to the advertisement selection unit 43.

For example, if the string information representing a conversation entered by the user and included in the request information received from the terminal device 20 is "I've been sleep-deprived lately...", and the word "sleep-deprived" is listed as a reserved word, the reserved word extraction unit 42 extracts the word "sleep-deprived" and outputs it to the advertisement selection unit 43.

The advertisement selection unit 43 selects advertisement copy to be presented to the user from among the advertisement copy stored in the storage unit 12 based on the reserved words extracted by the reserved word extraction unit 42. Specifically, the advertisement selection unit 43:
(1) Select advertisement copy that contains the extracted reserved word or its synonym.
(2) Select advertising copy whose associated genre information includes the extracted reserved word.

Here, when the advertisement selection unit 43 performs the above process (1) or (2) and multiple advertisement copies are selected, the advertisement selection unit 43 further selects (narrows down and selects) one advertisement copy from among the multiple advertisement copies. This narrowing down and selection method may be performed randomly or based on other conditions.

The advertisement selection unit 43 registers the selected advertisement text as a conversational sentence in the conversational sentence queue stored in the memory unit 12. At this time, the advertisement selection unit 43 may associate the advertisement text with expiration information, which may be a predetermined time (e.g., one hour) from the time of registration. As will be explained later, conversational sentences are controlled so that they are not spoken after this expiration date, so setting expiration information in this way prevents the advertisement text from being spoken after a considerable amount of time has passed since the user uttered the reserved word.

Based on the information input from the receiving unit 41, the action information generating unit 44 determines the action to be executed by the terminal device 20 that sent the request, generates action information including information instructing the action (action instruction) and information required to execute the action (hereinafter referred to as parameter information), and outputs the action information to the instruction transmitting unit 45.

In one example of this embodiment, the memory unit 12 of the server 10 stores an action database containing at least one record that associates the following information for generating instructions to the terminal device 20: occurrence conditions, information identifying the trigger, information to be compared with information representing the content of the voice input by the user (hereinafter referred to as comparison string information; however, this comparison string information may not be necessary depending on the type of trigger), and information representing the process to be executed by the server 10 to generate action information.

The action information generation unit 44 acquires comparison string information (if any) associated with the trigger-identifying information input from the receiving unit 41 and information indicating the processing that the server 10 should perform to generate action information.

If comparison string information is acquired (if comparison string information is associated with information identifying the trigger), the action information generation unit 44 compares the comparison string information with the string information output by the receiving unit 41. If the action information generation unit 44 determines that the string information output by the receiving unit 41 matches the comparison string information, it executes the process indicated by the acquired information and generates action information.

Furthermore, if comparison string information has not been acquired, the action information generation unit 44 executes the processing indicated by the acquired information to generate action information.

As a specific example, the action database contains a record that associates information specifying a trigger that "there has been voice input by the user" with string information such as "*News [to | is] * [not | tell me | read it out loud] *" as comparison string information to be compared with information representing the content of the voice input by the user, and the information "obtain the news string information from a specified web server on the Internet and instruct to read out the string information."

Note that this comparison string information is also expressed as a regular expression. Therefore, the above string will match string information such as "Tell me today's news" or "Is there any news?"

For example, when the action information generation unit 44 receives input from the receiving unit 41, such as information identifying a trigger that "there has been voice input by the user" and character string information such as "Is there any news?" representing the content of the voice input by the user, it searches the action database for a record containing the information identifying the trigger.

Here, the action information generating unit 44 finds the above record from the action database and compares the comparison string information contained in the record with the accepted string information. In the above example, the accepted string information "Is there any news?" is determined to match the comparison string information "*News [o|ha] * [no|tell me|read it out]*", so the action information generating unit 44 compares the information contained in the above record found by the search that indicates the process to be executed by the server 10, for example, "(Step 1) Obtain news string information from a specified web server on the Internet,
(Step 2) Generate an instruction to read out the character string information, and (Step 3) Generate an instruction to display animation information to be played back when reading out the character string information. Then, the information is acquired and processing is performed according to the information.

In other words, the action information generation unit 44 obtains news string information from a specified web server on the Internet in accordance with this read information. The action information generation unit 44 may also generate a display instruction for animation information that includes information specifying the image data of the animation to be displayed in conjunction with the execution of the action processing that is performed in parallel (which may be the file name of the image data).

In this example, the action information generation unit 44 generates action information including an action instruction and parameter information and outputs it to the instruction transmission unit 45. Here, the action instruction includes an instruction to read out string information and an instruction to display animation information. Furthermore, the parameter information includes the acquired string information and information specifying the image data of the animation.

In addition, the information representing the process to be executed by the server 10 may include an instruction to "select a conversation sentence." If such an instruction is included, the action information generation unit 44 will select a conversation sentence in accordance with the instruction, for example, in the following manner.

Here, the conversation sentence queue stored in the memory unit 12 of the server 10 stores a deadline (T), a speech condition (C), and a conversation sentence (D) in association with each other, as illustrated in Figure 6.

The action information generation unit 44 extracts conversation sentences from the conversation sentences stored in the conversation sentence queue that satisfy the speech conditions associated with the conversation sentence. Here, the speech conditions are, for example, information indicating the time period during which speech is permitted. The action information generation unit 44 can obtain various information required to determine whether the speech conditions are satisfied, such as the current date and time (the date and time when processing is being performed) and weather information, from an NTP (Network Time Protocol) server or a specified web server via the network.

The action information generation unit 44 selects, for example, randomly, one of the conversation sentences extracted as satisfying the utterance conditions. At this time, the action information generation unit 44 may read information about the deadline associated with the extracted conversation sentence from the conversation sentence queue, and delete the extracted conversation sentence from the conversation sentence queue if the deadline has already passed. In this case, the action information generation unit 44 repeats the process from selecting another conversation sentence (an unselected conversation sentence) extracted as satisfying the utterance conditions.

The action information generation unit 44 determines whether the selected conversational text satisfies a predetermined criterion. For example, the action information generation unit 44 reads the conversation history relating to the user of the terminal device 20 that sent the request, which is recorded by the instruction transmission unit 45 (described later). The action information generation unit 44 then determines whether the selected conversational text will result in a natural conversation when spoken following the currently recorded conversation history. In other words, in this example, the predetermined criterion is that there is continuity between the conversational text in the history and the selected advertising text.

This determination can be made, for example, by using a neural network that has learned from machine learning the text of conversations between people to determine whether a sentence is valid as a continuation of the currently recorded conversation history, i.e., whether there is continuity in the conversation. This type of processing is known as next sentence prediction (NSP). A model known as BERT (https://arxiv.org/pdf/1706.03762.pdf) can be used as a neural network for next sentence prediction. Widely known training data and machine learning processing methods using such training data can be used as training data for machine learning to perform such next sentence prediction, such as a method that uses a pair of sentences (a first sentence and a second sentence) associated with information indicating the continuity between the first and second sentences.

When a neural network is used in this way to determine whether a selected conversational sentence is appropriate as a continuation of the currently recorded conversation history, i.e., whether there is continuity in the conversation, the output of the neural network represents that appropriateness as a numerical value. Therefore, the action information generation unit 44 determines that the conversation is natural (determines that there is continuity) when the numerical value exceeds a predetermined threshold value.

When the action information generation unit 44 determines that there is continuity as described above, it generates an action instruction including an instruction to read out the string information of the conversational sentence (this conversational sentence is also a response sentence that continues the previous conversation, and therefore will be referred to as a response sentence here when it is necessary to distinguish it from other conversational sentences) and an instruction to display animation information. The action information generation unit 44 also generates parameter information including the string information of the selected conversational sentence and information specifying image data of the animation, and instructs the instruction transmission unit 45 to send the generated action instruction and parameter information as action information to the terminal device 20.

The instruction sending unit 45 sends the action information generated by the action information generating unit 44 to the terminal device 20 that is the sender of the request information received by the receiving unit 41. At this time, the instruction sending unit 45 sequentially records, for each user, the content of the voice input by the user that is accepted by the receiving unit 41 and the content of the conversation that is instructed by the action information generating unit 44 to be spoken by the terminal device 20 of that user.

Next, the operation of the control unit 31 of the terminal device 20 will be described. In this embodiment, the control unit 31 functionally includes a request sending unit 51, an action information receiving unit 52, a voice synthesis unit 53, and an action processing execution unit 54, as illustrated in FIG. 7.

When the request sending unit 51 determines that a predetermined trigger has occurred, it collects information necessary for processing by the server 10 and sends request information including the collected information along with information identifying the trigger to the server 10. Specifically, triggers such as voice input by the user or the arrival of a specified time are listed in advance, included in the setting information, and stored in the memory unit 32.

When the request sending unit 51 determines that the occurrence condition (C) is satisfied by referencing the setting information exemplified in Figure 4, it determines that a trigger (N) associated with the satisfied occurrence condition has occurred, and refers to information (P) associated with the trigger that is necessary for processing on the server 10.

The request sending unit 51 may record, for each occurrence condition (C), the date and time when the occurrence condition was last satisfied and request information was sent. The request sending unit 51 may then perform control so as not to send request information if the difference between the date and time recorded in association with the satisfied occurrence condition (C) and the current date and time does not exceed the interval (T) recorded in the setting information in association with the occurrence condition (i.e., set for each occurrence condition, for example).

The request sending unit 51 then collects the information required for processing on the server 10, as identified by the referenced information, and sends request information to the server 10, including the collected information and information identifying the trigger that occurred (which may be the trigger name).

The action information receiving unit 52 receives action information from the server 10 and outputs the received action information to the action processing execution unit 54.

The voice synthesis unit 53 synthesizes voice data based on character string information input from the action processing execution unit 54, which will be described later. The voice synthesis unit 53 outputs the synthesized voice data to the action processing execution unit 54.

The action processing execution unit 54 extracts the action instruction and parameter information from the action information sent by the server 10 and executes processing in accordance with the action instruction. Specifically, as in the example above, we will explain the case where the action information receiving unit 52 receives action information that includes an instruction to read out the acquired character string information and an instruction to display the animation information, as well as parameter information that includes the acquired character string information and image data of the animation.

In this example, the action processing execution unit 54 outputs the acquired character string information to the voice synthesis unit 53 to acquire audio data. The action processing execution unit 54 also reads image data of the animation specified by the information included in the action information from the storage unit 22. The action processing execution unit 54 then outputs the audio data output by the voice synthesis unit 53 to the audio output unit 35 to produce audio, and outputs the image data of the read animation to the display unit 34 to play the image data of the animation.

[Operation]
The information processing system 1 of this embodiment has the above configuration and operates as follows: In the following example, it is assumed that the storage unit 12 of the server 10 stores, as an action database, information representing the process that the server 10 executes to generate action information for each trigger that is the cause of an action request.

In the following example, the information contained in this action database is: Information specifying a trigger (T): the user is having a conversation; Processing to be performed:
(Step 1) Select character string information of a conversation sentence; (Step 2) Generate an instruction to read out the character string information; (Step 3) Generate an instruction to display animation information to be played when reading out the character string information.

The storage unit 12 of the server 10 stores the following advertisement information: advertisement target (P): sleep-improving drug AA; advertisement identifier: 1; advertisement copy (E):
(1) "To improve the quality of your sleep, use the sleep aid AA."
(2) “Have you tried sleep aid AA?” ・Advertiser (A): Co., Ltd. A ・Genre (G): sleep, deep sleep, insomnia, lack of sleep, sleep aid, ... ・URL (U): http://... ・Target of advertisement (P): sleep aid BB ・Advertisement identifier: 2 ・Advertisement copy (E):
(1) "Sleep Improvement Drug BB, also used in hospitals"
(2) "Have you tried the sleep improvement drug BB?" Advertiser (A): B Co., Ltd. Genre (G): sleep, falling asleep, insomnia, sleep deprivation, sleep improvement drug, ... URL (U): http:// ... and other advertisement information is stored.

Furthermore, the storage unit 32 of the terminal device 20 stores, as setting information, for each trigger, information (C) that specifies the conditions for occurrence and information (P) that is related to the trigger and is necessary for processing on the server 10, as shown in Figure 4.

In the following example, this configuration information includes the following information: - Information identifying the trigger (trigger name N): there is voice input by the user - Occurrence condition (C): the user utters a specified wake word - Information identifying information required for processing by server 10 (P): character string information of what the user has spoken - Information identifying the trigger (trigger name N): the user is having a conversation - Occurrence condition (C): the user speaks within 10 seconds of the sound being emitted - Information identifying information required for processing by server 10 (P): character string information of what the user has spoken... Here, a wake word is a predetermined word such as "Hey, listen to me" or "Wake up," which should be recognized as the start of voice input when spoken by the user. The terminal device 20 may remove the character string portion corresponding to the wake word from the character string information of what the user has spoken that is required for processing by server 10.

Furthermore, here, the occurrence condition (C) is the user uttering the wake word, but this embodiment is not limited to this and other conditions may also be used.

The operation of the server 10, which stores such setting information, and the terminal device 20 will be explained below with reference to Figures 8 and 9.

After the terminal device 20 sounds audio data such as news information in accordance with instructions from the user, the user may respond to the audio data by saying something like, "I've been getting less sleep lately, too..." to the terminal device 20 (S11 in FIG. 8). The terminal device 20 then executes a process to recognize the user's voice (S12) and acquires character string information corresponding to the voice spoken by the user. As already mentioned, the voice recognition process does not have to be performed by the terminal device 20 itself; it can also be performed by accessing a voice recognition service via a network.

The terminal device 20 refers to the setting information to check whether any of the trigger conditions have been satisfied (S13). In this case, since the user speaks immediately after the terminal device 20 rings with audio data, it is assumed that a trigger indicating "the user is having a conversation" has occurred (S13: Yes), and character string information of the content spoken by the user is collected in accordance with the setting information. Note that if it is determined in step S13 that none of the trigger conditions have been satisfied (S103: No), the terminal device 20 terminates processing.

In this case, since the content of what the user said has already been acquired as string information in step S12, the terminal device 20 sends request information to the server 10 that includes this string information and information identifying the trigger that occurred (trigger name "User is having a conversation") (S14).

The server 10 receives the request information from the terminal device 20. The server 10 then references the trigger name "Users are having a conversation" included in the request information and retrieves information associated with the trigger name from the action database (S15). At this time, the server 10 also adds the conversational text included in the request information (string information representing what the user said) to the conversation history with the corresponding user and stores it.

In step S15, information representing the process to be executed by the server 10 is acquired. The server 10 then executes the process represented by the acquired information (S16). Here, one of the conversation sentences included in the conversation sentence queue (e.g., "Eh, are you okay?") is selected as a response sentence, an instruction to read out the response sentence is generated, and an instruction to display animation information in parallel with the reading out is generated and sent to the terminal device 20 as action information.

At this time, the server 10 adds the sent response sentence to the conversation history with the corresponding user and stores it.

Also, before or after this process, the server 10 separates the string information (conversational text entered by the user) included in the request information received from the terminal device 20, "I've been getting less sleep lately, too...", into words using, for example, morphological analysis (those words that have conjugations are taken to be in their original form), and extracts the word "not getting enough sleep" as a predetermined reserved word (reserved words are recorded in their original form) (S17). Alternatively, the server 10 may extract the word "not getting enough sleep", which is a predetermined reserved word, using pattern matching using regular expressions, etc. Specifically, the server 10 sequentially checks whether the conversational text entered by the user contains any of the predetermined reserved words, and when it is determined that any of the reserved words is included, it determines that the reserved word has been extracted.

The server 10 then selects advertising copy whose genre information includes a reserved word contained in the conversational text entered by the user (S18). In the above example, at least two advertising copy would be selected: "Sleep improvement drug AA to improve sleep quality" and "Sleep improvement drug BB, also used in hospitals." Note that if there are multiple advertising copy associated with one advertising identifier, the first advertising copy among them will be selected.

For example, in the advertisement copy (E) related to advertisement identifier: 1,
(1) "To improve the quality of your sleep, use the sleep aid AA."
(2) "Have you tried Sleep Aid AA?" are assumed to be associated with each other, but here, the first one (1) "Improve your sleep quality with Sleep Aid AA" will be selected. An example of selecting a copy other than the first one will be described later.

The server 10 further narrows down and selects one ad copy from these selected ad copies. As already mentioned, this narrowing down method may be random, for example. In this example, it is assumed that "To improve sleep quality, use sleep aid AA" has been selected. The server 10 then registers the selected ad copy in the conversation sentence queue (S19).

Meanwhile, the terminal device 20 sounds the response sentence included in the action information sent by the server 10 in step S16 (S20). When the user responds to the sounded response sentence by uttering a conversational sentence such as "I hope you can sleep a little more deeply," the terminal device 20 executes the same processes as steps S12 to S14 and transmits request information including the character string information of this conversational sentence (S21).

Now, moving on to FIG. 9, the server 10 receives the request information sent by the terminal device 20 in step S20, and adds the conversational text (character string information representing what the user said) included in this request information to the conversation history with the corresponding user and stores it.

The server 10 also references the trigger name "User is having a conversation" included in the request information and retrieves information associated with this trigger name from the action database (S22). Here, information representing the process to be executed by the server 10 is acquired. The server 10 then executes the process represented by the acquired information. Here, a process is performed to select a conversation sentence included in the conversation sentence queue as a response sentence, and first, a conversation sentence containing the advertisement copy registered in step S18, "To improve sleep quality, use sleep aid AA," is extracted (S23).

The server 10 sequentially determines whether the extracted conversation sentences satisfy predetermined criteria. Specifically, the server 10 selects one of the extracted conversation sentences (an unselected one) as a candidate (S24). The server 10 then references the conversation history related to the user of the terminal device 20 that sent the request information, and determines whether the conversation sentence selected in step S24 would result in a natural conversation when spoken following the currently stored conversation history (S25).

As an example, the server 10 performs next-sentence prediction processing using a BERT model, which is a machine-learned version of text from conversations between humans, to obtain a numerical value (the more appropriate the value, the larger the value) that indicates the appropriateness of the conversational sentence selected in step S24 as a sentence that follows the currently recorded conversation history.The server 10 then determines that the conversation is natural (there is continuity) when the obtained value exceeds a predetermined threshold value, and otherwise determines that the conversation is not natural (there is no continuity).

Specifically, in the processing up to this point, the server 10 has accumulated as a history of conversations between the user of the terminal device 20 that sent the request information in step S21 and the user, such as: User: "I've been getting less sleep lately too..." Terminal device 20 utterance: "Huh, are you okay?" User: "I wish I could sleep a little more deeply."

Then, following these conversational sentences, the server 10 determines whether the conversational sentence selected in step S24 is appropriate as a sentence that follows the currently recorded conversation history.

In step S25, if the server 10 determines that the selected conversational sentence will result in a natural conversation when spoken (S25: Yes), it generates action information including an instruction to read out the character string information of the selected conversational sentence (S26). The server 10 then sends the action information to the terminal device 20 (S27).

On the other hand, if it is determined in step S25 that the selected conversational sentence will not result in a natural conversation when spoken (S25: No), the server 10 returns to process S24 and continues the process of selecting an unselected conversational sentence. If there are no unselected conversational sentences in process S24, the server 10 executes a predetermined process, such as generating action information including an instruction to read out the character string information of a predetermined conversational sentence and sending it to the terminal device 20 (S28: Execute default process).

The terminal device 20 executes a process of reading out the character string information in accordance with the instructions in the action information sent by the server 10 in step S27 or S28 (S29).

If the server 10 determines in step S25 that the selected advertisement copy will result in a natural conversation following the conversation history up to that point, the server 10 will generate action information in step S26 that includes an instruction to read the advertisement copy as a response.

Then, the terminal device 20 will read out the advertisement text "To improve your sleep quality, use sleep aid AA" as a response text.

[URL]
The conversation sentence queue may include information other than the conversation sentence (other than the character string information to be sounded) that should be provided to the user in relation to the conversation sentence. For example, when the server 10 selects an advertisement sentence and registers it in the conversation sentence queue, if there is URL information associated with the advertisement sentence, the server 10 registers the advertisement sentence as a conversation sentence and also registers the URL information in the conversation sentence queue in association with the advertisement sentence as information to be provided to the user in relation to the conversation sentence.

Then, when the server 10 selects an advertisement text from the conversation text queue and generates action information to sound the selected advertisement text, it adds to the generated action information an instruction to sound a predetermined message such as "...I've sent you a URL for more detailed information" in addition to sounding the advertisement text, and sends the action information to the terminal device 20.

The server 10 then sends a specified email containing in its body the URL associated with the selected advertisement copy and recorded in the conversation text queue to the email address (assuming it is pre-registered) of the user of the terminal device 20 to which the action information is addressed.

As a result, the terminal device 20 will speak the advertisement copy and say, "I've sent you a URL with more detailed information," and the user will receive an email addressed to them containing a URL for obtaining information related to the advertisement.

Furthermore, while the example given here is one in which a URL is sent to the user's email address, the present embodiment is not limited to this, and the URL may also be sent to an application installed on the user's smartphone or the like that communicates with the server 10 or the terminal device 20. In this example, the server 10 sends the URL associated with the selected advertisement copy and recorded in the conversation sentence queue to the application installed on the smartphone of the user of the terminal device 20 that is the destination of the action information. Note that widely known technologies such as notification technologies can be used as the technology for displaying information in an application, so a detailed explanation will be omitted here.

[Records of past advertising provided]
In the explanation so far, when selecting ad copy to register in the conversation sentence queue, even if multiple ad copies are selected based on genre information, the ad copy to be spoken as conversation sentence is narrowed down randomly. Therefore, even if an ad copy by advertiser A is spoken in a conversation one day, "To improve the quality of your sleep, use sleep aid AA," the next day's ad copy by advertiser B may be spoken, "Sleep aid BB, which is also used in hospitals."

In other words, it is possible that advertising copy from different advertisers may be sent to a common advertising target, which would result in the terminal device 20 engaging in inconsistent conversations and prevent the user from treating the terminal device 20 as a person.

In one example of this embodiment, when the server 10 instructs a user's terminal device 20 to speak an advertisement copy, it associates the advertisement identifier of the advertisement copy with the date and time the instruction to speak was given, and records this as an advertisement history for each user. The server 10 then refers to the advertisement history when selecting an advertisement copy.

Specifically, when the server 10 selects advertising copy to present to the user from among those stored in the memory unit 12 based on reserved words extracted from conversational text spoken by the user, if there are multiple selected advertising copies, it narrows down the selection as follows: That is, the server 10 references the advertising history to check whether the advertising identifier of the selected advertising copy is recorded. If the advertising identifier of the selected advertising copy is recorded in the advertising history, it obtains information on the most recent date and time of the utterance instruction recorded in the advertising history. This provides information on the date and time when advertising copy related to the same advertising target from the same advertiser was last spoken for each of the advertising copies selected based on reserved words.

The server 10 narrows down and selects the advertising copy associated with the most recent date and time information from the obtained date and time information. This makes it easier for advertising copy by advertiser A to be spoken thereafter, maintaining the consistency of conversations on the terminal device 20.

Furthermore, as in this example, when referencing the advertising history, the server 10 may acquire an advertising identifier associated with genre information containing reserved words extracted from the conversational text spoken by the user, and select an advertising copy other than the first advertising copy from among the advertising copies associated with the acquired advertising identifier.

According to this example, for example, the advertisement copy (E) related to advertisement identifier: 1 includes the following: (1) "To improve the quality of your sleep, use sleep aid AA."
(2) When there are two options, "Have you tried Sleep Aid AA?", the first one, "To improve the quality of your sleep, try Sleep Aid AA," will be spoken the first time, but from the second time onwards (because it is recorded in the advertisement history), the other advertisement, "Have you tried Sleep Aid AA?", will be spoken, allowing for a more natural conversation.

[Setting time periods to suppress ads]
Furthermore, since times when users are busy (such as weekday mornings) are inappropriate for advertising, when the server 10 selects a conversation sentence to be spoken by the terminal device 20 from the conversation sentence queue, the server 10 may control the advertisement sentence not to be selected if the current date and time at the location of the terminal device 20 is in a time period that has been set in advance as a time period to be avoided.

In this example, the terminal device 20 may notify the server 10 of, for example, information about the current date and time in the area where the terminal device 20 is located, by including this information in the request information. Furthermore, when the server 10 registers advertising text in the conversation text queue, the server 10 may also register flag information indicating that the conversation text is advertising text, in association with the advertising text registered in the conversation text queue.

If the current date and time information included in the request information falls within a time period that has been previously set as an avoided time period, the server 10 selects the conversation sentence to be spoken from the conversation sentence queue, avoiding conversation sentences associated with flag information indicating that the conversation sentence is an advertisement sentence.

[Analysis of conversational text after advertisement speech]
When an advertising copy is uttered as a response, the server 10 may determine the user's reaction to the response. Specifically, the server 10 analyzes the content of the conversation received from the user after the advertising copy is uttered to determine whether the conversation continues with content related to the advertisement (whether words related to the subject of the advertisement are included in the subsequent conversation) and whether the conversation is positive or negative using a method such as polarity analysis.

The server 10 then stores the results of this analysis as a history of advertising effectiveness for each user and for each advertising identifier of the spoken advertising copy, and uses this information for specified processing.

The specified processing here could include obtaining statistics on the advertising effectiveness history for each advertising identifier of advertisements placed by a specific advertiser and presenting this information to the advertiser. This allows the advertiser to obtain information such as the frequency of mentions of the advertisements they have placed and user responses.

The predetermined process may also be a process of extracting, for each user, advertising identifiers that are determined to be negative or that are not likely to lead to a conversation related to the content of the advertisement after the advertisement. In this case, control may be exercised so that the user is not able to select the advertising copy of the extracted advertising identifier.

[Advertisement Targeting]
In the above description, the advertisement copy related to the advertisement identifier can be selected regardless of the user's attributes, such as age group or gender, but this embodiment is not limited to this. For example, the advertisement copy may be recorded in association with information representing the attributes of the target user. In this case, when selecting the advertisement copy to be spoken by the user's terminal device 20, the server 10 may refer to the user's attributes (which may be registered in advance), such as the user's date of birth or gender, and select the advertisement copy from among the advertisement copies associated with the information representing the attributes.

[Other examples of standards]
Here, the predetermined criterion for whether or not to speak the advertising copy is a criterion for determining whether or not there is continuity in the conversation by performing a next-sentence determination process, but this embodiment is not limited to this, and may be based on the number or frequency (number of times a keyword appears per unit time) of keywords included in the conversation history, etc.

[Processing on the terminal device side]
In the explanation so far, the server 10 functions as an example of the information processing device of the present invention, selects an advertisement, and causes the terminal device 20 to speak it, but the present embodiment is not limited to this, and the terminal device 20 may function as an example of the information processing device of the present invention and select an advertisement copy to be spoken from among advertisement copies stored in the server 10. In this example, the control unit 31 of the terminal device 20 executes the operations of the reserved word extraction unit 42 and the advertisement selection unit 43. In this example, the control unit 31 also controls the speaking of the selected advertisement copy.

[Configuration and Effects of the Embodiment]
As described above, in an embodiment of the present invention, advertising copy is stored in advance, and a determination is made as to whether it is appropriate to utter the advertising copy as a continuation of the conversation, and the advertising copy is uttered if it is determined to be appropriate. That is, the information processing device of this embodiment includes: a storage means such as a disk or server for storing the advertising copy, which is the content of the advertisement; an extraction means for extracting reserved words, which are predetermined keywords, contained in a conversational text input by a user; a selection means for selecting an advertising copy to present to the user from the advertising copy based on the extracted reserved words; and an execution means for executing a speech process for a response text including the selected advertising copy following the conversational text based on predetermined criteria. This information processing device can be realized as a terminal device (e.g., a robot) located on the server or user side.

This allows ad copy to be included in natural conversation, reducing the abruptness of the ad.

Furthermore, the predetermined criterion may be whether or not there is continuity between the conversational text and the selected advertising text, and if there is continuity, speech processing of the response text may be executed.

This allows ad copy to be included in the natural flow of conversation, making the ad feel less abrupt.

Furthermore, the storage means may store genre information indicating the genre of the advertisement represented by each of the advertising copy in association with the advertising copy, and the selection means may select advertising copy to present to the user from advertising copy associated with the genre determined as the advertisement to be presented to the user based on the reserved words.

In this way, narrowing down ad copy by genre reduces the processing burden of criteria-based judgment.

Furthermore, the execution means may suppress the utterance of the response sentence during a predetermined time period, such as a time period set as a time period when advertisements should be avoided. According to this example, it is possible to avoid presenting advertisements during times that are inappropriate for the user.

The system may further include a reaction determination means for determining the user's reaction to the response sentence utterance processing when the response sentence utterance processing is executed, and the result of the determination by the reaction determination means may be stored in association with information representing the content of the uttered response sentence, and may later be used for predetermined processing, such as control processing to repeat or suppress the advertisement. This makes it possible to perform control according to the effectiveness of the advertisement.

1 Information processing system, 10 Server, 11 Control unit, 12 Memory unit, 13 Communication unit, 20 Terminal device, 21 Leg unit, 22 Main body unit, 31 Control unit, 32 Memory unit, 33 Sensor unit, 34 Display unit, 35 Audio output unit, 36 Communication unit, 37 Drive unit, 41 Receiving unit, 42 Reserved word extraction unit, 43 Advertisement selection unit, 44 Action information generation unit, 45 Instruction transmission unit, 51 Request transmission unit, 52 Action information reception unit, 53 Audio synthesis unit, 54 Action processing execution unit.

Claims (4)

It has a processor and memory,
The advertisement content, such as the advertisement copy, advertisement identifier, and information identifying the advertiser, is stored in memory.
The processor:
extracting predetermined reserved words contained in conversational text input by a user;
Selecting an advertisement copy to be presented to the user from among a plurality of advertisement copies based on the extracted reserved words;
performing speech processing of a response sentence including the selected advertising copy following the conversation sentence based on predetermined criteria;
An advertising identifier that identifies the target of an advertisement related to the speech-processed advertising copy is recorded as an advertising history.
When the reserved word is extracted from a conversational sentence different from the conversational sentence, selecting, from among a plurality of advertisement copies, advertisement copies that are the same as the advertisement copy specified by the advertisement identifier recorded as the advertisement history,
executes speech processing of a response sentence including the selected advertisement copy;
the predetermined criterion is whether or not a response sentence including the selected advertisement copy results in a natural conversation when uttered following a conversation history;
If it is determined that the response sentence will be a natural conversation, speech processing is executed for the response sentence including the selected advertisement copy following the conversation sentence, and if it is determined that the response sentence will not be a natural conversation, another advertisement copy to be presented to the user is selected from a plurality of advertisement copies based on the extracted reserved words.
Information processing device. The processor stores in a memory an advertisement copy, an advertisement identifier, and information identifying an advertiser, which are contents of the advertisement;
A processor is caused to extract predetermined reserved words included in a conversation sentence input by a user;
causing a processor to select, from the advertising copy, an advertising copy to be presented to a user based on the extracted reserved word;
causing a processor to execute speech processing of a response sentence including a selected advertising sentence following the conversation sentence based on predetermined criteria;
causing the processor to record in the memory, as an advertisement history, an advertisement identifier that identifies a target of an advertisement related to the speech-processed advertisement copy;
a processor, when the reserved word is extracted from a conversation sentence different from the conversation sentence, selecting, from among a plurality of advertisement copies, advertisement copies that are made by the same advertiser identified based on an advertisement identifier recorded as an advertisement history;
causing a processor to execute speech processing of a response sentence including the selected advertising copy;
the predetermined criterion is whether or not a response sentence including the selected advertisement copy results in a natural conversation when uttered following a conversation history;
When it is determined that the response sentence is a natural conversation, the processor is caused to execute speech processing of the response sentence including the selected advertisement copy following the conversation sentence, and when it is determined that the response sentence is not a natural conversation, the processor is caused to select another advertisement copy to be presented to the user from a plurality of advertisement copies based on the extracted reserved words.
program. The processor stores in a memory the advertisement copy, the advertisement identifier, and information identifying the advertiser, which are the contents of the advertisement;
A processor extracts predetermined reserved words included in a conversation sentence input by a user;
The processor selects an advertising copy to be presented to the user from among a plurality of advertising copies based on the extracted reserved words;
a processor, based on predetermined criteria, performing speech processing of a response sentence including the selected advertising sentence following the conversation sentence;
The processor records, as an advertisement history, an advertisement identifier that identifies a target of an advertisement related to the speech-processed advertisement copy;
a processor, when the reserved word is extracted from a conversation sentence different from the conversation sentence, selecting, from among a plurality of advertisement copies, advertisement copies that are made by the same advertiser identified based on an advertisement identifier recorded as an advertisement history;
a processor performs speech processing of a response sentence including the selected advertisement sentence;
the predetermined criterion is whether or not a response sentence including the selected advertisement copy results in a natural conversation when uttered following a conversation history;
If it is determined that the response sentence constitutes a natural conversation, the processor executes speech processing of the response sentence including the selected advertisement sentence following the conversation sentence, and if it is determined that the response sentence does not constitute a natural conversation, the processor selects another advertisement sentence to be presented to the user from among a plurality of advertisement sentences based on the extracted reserved words.
Information processing methods. A server and a terminal device are provided,
The server
The advertisement content, such as the advertisement copy, advertisement identifier, and information identifying the advertiser, is stored in memory.
extracting predetermined reserved words contained in conversational text input by a user;
Selecting an advertisement copy to be presented to the user from among a plurality of advertisement copies based on the extracted reserved words;
causing the terminal device to execute speech processing of a response sentence including the selected advertisement sentence following the conversation sentence based on a predetermined criterion;
An advertising identifier that identifies the target of an advertisement related to the speech-processed advertising copy is recorded as an advertising history.
When the reserved word is extracted from a conversational sentence different from the conversational sentence, selecting, from among a plurality of advertisement copies, advertisement copies that are the same as the advertisement copy specified by the advertisement identifier recorded as the advertisement history,
causing the terminal device to execute speech processing of a response sentence including the selected advertisement copy;
the predetermined criterion is whether or not a response sentence including the selected advertisement copy results in a natural conversation when uttered following a conversation history;
When it is determined that the response sentence is a natural conversation, the processor is caused to execute speech processing of the response sentence including the selected advertisement copy following the conversation sentence, and when it is determined that the response sentence is not a natural conversation, the processor is caused to select another advertisement copy to be presented to the user from a plurality of advertisement copies based on the extracted reserved words.
system.