JP7836230B2 - Program, information processing method, and information processing apparatus - Google Patents

Description

This technology relates to a program, an information processing method, and an information processing device.

Conventionally, devices for engaging in conversation with users are known. For example, the dialogue control device described in Patent Document 1 selects a topic related to the topic used in the previous conversation and generates a conversation.

Japanese Patent Publication No. 2004-195636

Reactions to specific topics (conversational content) vary among users; some users react positively when conversational content is provided, while others react negatively. To ensure users can converse without feeling stressed, it's necessary to provide conversational content that elicits positive reactions from them.

This disclosure is made in light of the circumstances described above and aims to collect data for analyzing which users respond favorably to which types of conversational content.

A program according to one embodiment of this disclosure acquires the attributes of a user riding in a mobile vehicle, determines an evaluation of the conversational keywords spoken by the user when conversational content is provided to the user, and causes a computer to perform a process of storing data that associates the attributes, the conversational content, the conversational keywords, and the evaluation.

A program according to one embodiment of this disclosure acquires the location information of the moving object when the conversation content is provided, and stores the data associating the location information, the attributes, the conversation content, the conversation keywords, and the evaluation.

A program according to one embodiment of this disclosure acquires the region indicated on the license plate of the mobile vehicle and stores the data associating the region, attributes, conversation content, conversation keywords, and evaluation.

A program according to one embodiment of this disclosure acquires the location information of the mobile object when the conversation content was provided, acquires the region indicated on the license plate of the mobile object, and stores the data associating the location information, region, attributes, conversation content, conversation keywords, and evaluation.

A program according to one embodiment of this disclosure modifies the conversation content provided to the user, determines the evaluation of the conversation keywords spoken by the user when the modified conversation content was provided, and stores the data associating the attributes, the modified conversation content, the conversation keywords, and the evaluation.

A program according to one embodiment of this disclosure acquires a plurality of evaluations corresponding to the conversational content and performs statistical analysis on the acquired evaluations.

A program according to one embodiment of this disclosure acquires multiple evaluations corresponding to the attribute, location information, region, and conversation content, where the evaluations are quantified; calculates a representative value including the average of the acquired evaluations; and stores analyzed data associating the attribute, location information, region, and conversation content with the representative value.

A program according to one embodiment of this disclosure acquires the attributes of a user riding in a mobile vehicle, the location information of the mobile vehicle, and the region of the mobile vehicle. Based on the analyzed data, it determines the conversation content to be provided to the user to have the highest representative value corresponding to the attributes, location information, and region.

An information processing method according to one embodiment of this disclosure acquires the attributes of a user riding in a mobile vehicle, determines an evaluation of the conversational keywords spoken by the user when conversational content is provided to the user, and stores data associating the attributes, the conversational content, the conversational keywords, and the evaluation.

An information processing device according to one embodiment of this disclosure comprises a control unit that acquires the attributes of a user riding in a mobile vehicle and determines an evaluation of conversational keywords spoken by the user when conversational content is provided to the user, and a storage unit that stores data associating the attributes, the conversational content, the conversational keywords, and the evaluation.

In one embodiment of this disclosure, the program can collect data for analyzing which users respond favorably to which conversational content.

This is a schematic diagram showing an example of the configuration of an information processing system. This is a block diagram showing an example configuration of an information processing device, etc. This is an explanatory diagram showing keyword extraction models and keyword evaluation models, etc. This is an explanatory diagram showing the reaction table. This is an explanatory diagram showing the table after analysis. A flowchart illustrating an example of the processing procedure for an information processing device.

(Embodiment)
The present invention, according to embodiments, will be described below with reference to the drawings. Figure 1 is a schematic diagram showing an example of the configuration of an information processing system. Figure 2 is a block diagram showing an example of the configuration of an information processing device, etc. The information processing system S includes an information processing device 1 and an in-vehicle device 2, and the information processing device 1 and the in-vehicle device 2 communicate with each other via an external network N. The in-vehicle device 2 is mounted in a mobile vehicle (automobile) and provides conversation topics (conversation content) received from the information processing device 1 to the user riding in the mobile vehicle. The in-vehicle device 2 also acquires responses (statements) to the provided conversation content and transmits them to the information processing device 1. The information processing device 1 extracts keywords from the user's statements received from the in-vehicle device 2 and determines the user's evaluation of the conversation content based on these keywords. The information processing device 1 stores the conversation content, the keywords, and the evaluation, and determines the conversation content to be provided to the user next time and the conversation content to be provided to other users.

The information processing device 1 is a computer comprising a control unit 11, a storage unit 12, and a communication unit 13, and can be a server computer, a personal computer, a tablet computer, a smartphone, or multiple cloud servers connected via a network.

The control unit 11 is composed of a CPU (Central Processing Unit) or MPU (Micro Processing Unit), and performs various control processes, calculation processes, etc., by reading and executing programs and data pre-stored in the storage unit 12.

The memory unit 12 is composed of non-volatile memory elements such as ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable ROM), or flash memory, and volatile memory elements such as RAM (Random Access Memory). The memory unit 12 pre-stores programs and data referenced during processing. The program may be one that the control unit 11 reads from a program (program product) stored on the recording medium 12a. Alternatively, the program may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the memory unit 12. The memory unit 12 also stores a keyword extraction model M1, a keyword evaluation model M2, a reaction table 121, and a post-analysis table 122. The keyword extraction model M1, keyword evaluation model M2, reaction table 121, and post-analysis table 122 may be distributed and stored across multiple cloud servers. Details of the keyword extraction model M1, keyword evaluation model M2, reaction table, and post-analysis table 122 will be described later.

The communication unit 13 is a wireless communication device compatible with, for example, Wi-Fi (registered trademark), LTE (registered trademark), 4G, or 5G communication. The communication unit 13 communicates with the in-vehicle device 2 via an external network N.

The in-vehicle device 2 is a computer equipped with a control unit 21, a storage unit 22, a communication unit 23, an input/output interface 24, and a GPS module 25; for example, it is a navigation system. The in-vehicle device 2 may also be a personal computer, a tablet terminal, or a smartphone. The control unit 21, storage unit 22, and communication unit 23 of the in-vehicle device 2 may have the same configuration as those of the information processing device 1.

The storage unit 22 of the in-vehicle device 2 stores information including the region indicated on the license plate of the vehicle in which the in-vehicle device 2 is installed (vehicle information), and the attributes of the user who is the owner or user of the vehicle (e.g., age group, gender, and occupation). Note that the vehicle information and user attributes may be registered in advance by the user as member information for the information processing system S on the cloud server. Furthermore, the user attributes may be acquired by the control unit 21 of the in-vehicle device 2 based on the content of the interaction with the user.

The input/output interface 24 is a group of interfaces for communication between the in-vehicle device 2 and external devices, and is a standardized communication interface such as USB or D-SUB. A microphone 241 and a speaker 242 are connected to the input/output interface 24, enabling communication between each external device and the control unit 21 of the in-vehicle device 2.

The GPS module 25 periodically receives radio waves from GPS satellites (not shown) and calculates positional information (coordinates), including the latitude and longitude of the vehicle equipped with the in-vehicle device 2, based on the signals periodically acquired from the GPS satellites. The control unit 21 of the in-vehicle device 2 may also identify a specific area name (place name) where the vehicle is located based on the calculated positional information. In this case, the storage unit 22 of the in-vehicle device 2 stores data where the coordinate range is associated with the area name. The control unit 21 refers to this data and identifies the area name within which the coordinates calculated by the GPS module 25 fall.

Figure 3 is an explanatory diagram showing the keyword extraction model and keyword evaluation model, etc. The control unit 11 of the information processing device 1 includes a speech recognition unit 11a that performs speech recognition processing, which recognizes the user's voice data picked up and transmitted by the control unit 21 of the in-vehicle device 2 via the microphone 241, and converts it into character data (text data). This speech recognition processing is performed using processing modules such as an acoustic model, a pronunciation dictionary, and a language model. The speech recognition unit 11a may also identify the speaker (user of the voice data) based on the frequency of the converted sound waves. The control unit 11 outputs the character data (text data) converted from the voice data to the keyword extraction model M1.

The keyword extraction model M1 is a trained model that, when text data representing a sentence is input, extracts and outputs important words (keywords) from among the words contained in the sentence. For example, it is composed of BERT (Bidirectional Encoder Representations from Transformers). The keyword extraction model M1 may also be RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), or Transformer. For training data, for example, the Corpus of Spontaneous Japanese (CSJ) is used. The keyword extraction model M1 determines the importance of each word contained in the text data and extracts words with an importance value of a certain value or higher as keywords. The keyword extraction model M1 may also extract multiple words as keywords in order of importance. Furthermore, the keyword extraction model M1 may also extract important phrases contained in the sentence. The control unit 11 inputs the keywords output by the keyword extraction model M1 to the keyword evaluation model M2. Furthermore, if a word previously stored in the memory unit 12 is included in the character data, the control unit 11 may extract that word as a keyword.

The keyword evaluation model M2 is a trained model that outputs an evaluation of the input keyword when the keyword extraction model M1 has extracted the keyword. For example, it is constructed using BERT (Bidirectional Encoder Representations from Transformers). The keyword evaluation model M2 may also be RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), or Transformer. For training data, for example, the Corpus of Spontaneous Japanese (CSJ) can be used. The keyword evaluation model M2 evaluates whether the input keyword is favorable or negative using a five-point scale (evaluation value) from 1 to 5, outputting a higher value for favorable keywords. If multiple keywords have been extracted by the keyword extraction model M1 and multiple keywords are input to the keyword evaluation model M2, the keyword evaluation model M2 may output a single evaluation value based on these keywords. Alternatively, the storage unit 12 of the information processing device 1 may store data with evaluation values associated with each word, and the control unit 11 may output the evaluation value of the keyword by referring to this data. In this case, if multiple keywords are extracted by the keyword extraction model M1, the control unit 11 may output a single evaluation value by averaging the evaluation values of these multiple keywords.

This embodiment shows an example of a user's response (statement) when conversational content about "jazz" is provided at Roppongi Crossing. When the control unit 11 acquires audio data, the speech recognition unit 11a outputs the text data: "Wow. Jazz in Roppongi... it makes me feel like an adult. I'd like to go." When this text data is input to the keyword extraction model M1, keywords such as "adult" and "I'd like to go" are extracted. When the extracted keywords are input to the keyword evaluation model M2, an evaluation value is output. In this example, the evaluation value is "5".

Figure 4 is an explanatory diagram showing the response table. The management items (fields) of the response table 121 include, for example, a location information field, a conversation content ID field, a date and time field, a vehicle information field, a user attribute field, a response original text field, a keyword field, and an evaluation field. The location information field stores the vehicle's location information acquired by the GPS module 25 when the in-vehicle device 2 acquires the user's response to the conversation content. The conversation content ID field stores the ID of the conversation content provided to the user by the information processing device 1 via the in-vehicle device 2. In this embodiment, conversation content ID "0001" represents "jazz", conversation content ID "0002" represents "art", and conversation content ID "0003" represents "cherry blossoms" as conversation content. The date and time field stores the date and time when the in-vehicle device 2 acquired the user's response to the conversation content. The vehicle information field stores vehicle information stored in the storage unit 22 of the in-vehicle device 2. The user attribute field stores the attributes of the vehicle owner or user, as stored in the storage unit 22 of the in-vehicle device 2. The response source text field stores the character data converted by the speech recognition unit 11a of the control unit 11. The keyword field stores the keywords extracted by the keyword extraction model M1 of the control unit 11. The evaluation field stores the evaluation values output by the keyword evaluation model M2 of the control unit 11.

Figure 5 is an explanatory diagram showing the post-analysis table. The post-analysis table 122 stores the post-analysis data, which is the result of analyzing the data stored in the response table 121. The post-analysis data is data that aggregates each record in the response table 121 and associates the average values of keywords and evaluation values stored for specific location information, conversation content, region, age group, gender, and occupation. The management items (fields) of the post-analysis table 122 include, for example, a location information field, a conversation content ID field, a region field, an age group field, a gender field, an occupation field, a keyword field, and an average evaluation field. The location information field stores the location information of the vehicle that provided the conversation content. The conversation content ID field stores the ID of the conversation content provided to the user. The region field stores the region included in the vehicle information. The age group field stores the age group included in the user's attributes. The gender field stores the age group included in the user's attributes. The occupation field stores the occupation included in the user's attributes. The keyword field stores keywords from the keyword fields of multiple records in the response table 121 that correspond to a specific combination of location information, conversation content, region, age group, gender, and occupation. The average evaluation field stores the average evaluation value (average evaluation value) obtained by averaging the evaluation values stored in the keyword fields of multiple records that correspond to a specific combination of location information, conversation content, region, age group, gender, and occupation. Note that the post-analysis table 122 may store, in addition to or instead of the average evaluation value, representative values such as the median or mode of the evaluation values.

Figure 6 is a flowchart showing an example of the processing procedure of the information processing device. When the control unit 11 of the information processing device 1 makes a statement to the in-vehicle device 2 requesting content, for example, the control unit 11 of the information processing device 1 starts the following processing. When the control unit 11 starts processing, it acquires the vehicle's location information, vehicle information, and user attributes from the in-vehicle device 2 (S1). In this embodiment, the control unit 11 of the information processing device acquires the user's attributes from the storage unit 22 of the in-vehicle device 2, but the method of acquiring user attributes is not limited to this. For example, user attributes may be acquired based on an image of the user taken by the camera unit of the in-vehicle device 2 and data previously stored in the storage unit 12 of the information processing device 1 or the storage unit 22 of the in-vehicle device 2. In this case, data that associates the feature quantities of multiple users with the user's attributes is stored in the storage unit 12 of the information processing device 1 or the storage unit 22 of the in-vehicle device 2. The control unit 11 of the information processing device 1 extracts the user's feature quantities from the image taken by the camera unit of the in-vehicle device 2, and acquires the user's attributes corresponding to the extracted feature quantities by referring to the stored data. In this case, even when a user who is neither the owner nor a user of the mobile vehicle boards the vehicle, the control unit 11 of the information processing device 1 can acquire the user's attributes.

The control unit 11 determines the conversation content to provide to the user by referring to the post-analysis table 122 based on the acquired vehicle information and user attributes (S2). Specifically, the control unit 11 determines the conversation content to provide to the user to be the one with the highest average evaluation value among the records that match the region included in the vehicle information acquired in S1, and the age group, gender, and occupation included in the user's attributes. In the example shown in Figure 5, if the acquired vehicle location information is "Roppongi Crossing", the region included in the vehicle information is "Kashiwa", the age group included in the user's attributes is "20s", the gender is "female", and the occupation is "service industry", the average evaluation value for conversation content ID "0001" (jazz) is "4.2", and the average evaluation value for conversation content ID "0002" (art) is "3.3", so the control unit 11 determines that conversation content ID "0001" (jazz), which has the highest average evaluation value, will be provided to the user. Furthermore, the control unit 11 may determine the conversational content to be provided to the user based on a representative value such as the median or mode of the evaluation values, instead of, or in addition to, the average value of the evaluation values.

The control unit 11 transmits the determined conversation content to the in-vehicle device 2 (S3), causing the in-vehicle device 2 to provide the conversation content to the user. In other words, the control unit 11 of the information processing device 1 provides the conversation content to the user via the in-vehicle device 2. The in-vehicle device 2 provides the conversation content to the user through sound output by the speaker 242. Alternatively, the in-vehicle device 2 may include a display unit and provide the conversation content by displaying it on the display unit.

The control unit 11 acquires user voice data from the in-vehicle device 2 (S4). The control unit 11 converts the acquired voice data into text data using the voice recognition unit 11a (S5), inputs this text data into the keyword extraction model M1 (S6), and extracts keywords (S7). The control unit 11 inputs the extracted keywords into the keyword evaluation model M2 (S8) and outputs evaluation values (S9).

The control unit 11 stores records in the response table 121 that associate the acquired location information, conversation content provided to the user, vehicle information, user attributes, text data, output keywords, and output evaluation values (S10). Based on the response table 121, the control unit 11 updates the post-analysis table 122 (S11). Specifically, if a record exists in the post-analysis table 122 that matches the location information, conversation content, region included in the vehicle information, and user attributes of the record added to the response table 121 in S10, the control unit 11 updates the average evaluation value of that record based on the evaluation value of the record added in S10. If a record exists in the post-analysis table 122 that matches the location information, conversation content, region included in the vehicle information, and user attributes of the record added to the response table 121 in S10, the control unit 11 adds a new record to the post-analysis table based on the record added to the response table 121 in S10. After updating the analysis table 122, the control unit 11 returns to process S1 and retrieves location and attribute information for the user, or location and attribute information for other users, again.

Through the above process, it becomes possible to collect data for analyzing which users respond favorably to which types of conversational content. Furthermore, it becomes possible to provide users with conversational content that is likely to elicit a positive response.

(Variant)
The management items of the response table 121 may include a destination field that stores the destination of the vehicle (automobile), a waypoint field that stores the waypoints of the vehicle, a travel purpose field that stores the purpose of travel by the user using the vehicle, a weather field that stores the weather when the in-vehicle device 2 acquired the user's response to the conversation content, a season field that stores the season when the in-vehicle device 2 acquired the user's response to the conversation content, or a day of the week field that stores the day of the week when the in-vehicle device 2 acquired the user's response to the conversation content. In this case, the control unit 11 of the information processing device 1 may aggregate evaluation values for conversation content, destination, waypoints, travel purpose, weather, season, or day of the week and create an analysis table 122. Alternatively, the control unit 11 may aggregate based on some of the management items of the response table 121 and create an analysis table 122. In this case, the control unit 11 refers to the analysis table 122 and determines the conversation content to be provided to the user based on the weather, season, or day of the week at the time the conversation content is provided to the user.

The management items in the response table 121 may include an in-car audio field that stores the content of the radio broadcast or car stereo outputting in the vehicle when the in-car device 2 acquires the user's response to the conversation content. In this case, the control unit 11 of the information processing device 1 may aggregate evaluation values for each conversation content or in-car audio content and create an analysis table 122. In this case, the control unit 11 refers to the analysis table 122 and determines the conversation content to be provided to the user based on the content of the in-car audio at the time the conversation content is provided to the user.

The control unit 11 outputs evaluation values based on the extracted keywords, inputting them into the keyword evaluation model M2. However, the method of outputting evaluation values is not limited to this. For example, the control unit 11 may acquire the user's facial expression based on the image captured by the camera unit of the in-vehicle device 2 and output evaluation values based on the acquired facial expression. Alternatively, the control unit 11 may acquire the tone or voice of the user's voice based on the voice recognized by the speech recognition unit 11a and output evaluation values based on the acquired tone or voice.

If multiple users are riding in a mobile vehicle (automobile), the control unit 11 of the information processing device 1 may acquire the attributes of each user. In this case, the control unit 11 may identify the conversation content with the highest average evaluation value for each user's attributes, and provide the conversation content with the highest average evaluation value among the identified conversation content to the multiple users riding in the mobile vehicle.

The content provided to the user by the information processing device 1 is not limited to the content with the highest average rating. For example, if the same user passes through the same area twice, the control unit 11 of the information processing device 1 may provide the user with the conversation content with the second highest average rating the second time the user passes through that location. Furthermore, if the user has provided the user with the conversation content with the highest average rating in a certain area, and the moving object remains in that area for a certain period of time or longer, the control unit 11 of the information processing device 1 may provide the user with the conversation content with the second highest average rating.

The embodiments disclosed herein should be considered illustrative and not restrictive in all respects. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the claims and scopes equivalent to the claims. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. Moreover, while the claims use a multi-claim format (claims referencing two or more other claims), they are not limited to this. They may also be described using a multi-claim format (multi-multi-claims) referencing at least one multi-claim.

1. Information Processing Device 11. Control Unit 11a. Speech Recognition Unit 12. Storage Unit 12a. Recording Medium 121. Response Table 122. Post-Analysis Table 13. Communication Unit 2. In-Vehicle Device 21. Control Unit 22. Storage Unit 23. Communication Unit 24. Input/Output Interface
241 Microphone 242 Speaker 25 GPS Module M1 Keyword Extraction Model M2 Keyword Evaluation Model N External Network S Information Processing System

Claims (10)

Obtain the attributes of the user boarding the mobile vehicle.
When providing conversational content to the user, the degree to which the user is favorable towards conversational keywords whose importance is above a certain level is determined.
A program that causes a computer to perform a process of storing data that associates the aforementioned attributes, the conversation content, the conversation keywords, and the evaluation. The location information of the mobile object when the aforementioned conversation content is provided is obtained,
The program according to claim 1, which stores the data relating the location information, the attributes, the conversation content, the conversation keywords, and the evaluation. The region indicated on the license plate of the aforementioned mobile vehicle is obtained,
The program according to claim 1, which stores the data relating the region, the attribute, the conversation content, the conversation keywords, and the evaluation. The location information of the mobile object when the aforementioned conversation content is provided is obtained,
The region indicated on the license plate of the aforementioned mobile vehicle is obtained,
The program according to claim 1, which stores the data relating the location information, the region, the attributes, the conversation content, the conversation keywords, and the evaluation. The conversation content provided to the user is modified,
Determine the evaluation of the conversation keywords spoken by the user when the modified conversation content was provided.
The program according to claim 1 or 2, which stores the data associating the attributes, the modified conversation content, the conversation keywords, and the evaluation. A plurality of evaluations corresponding to the aforementioned conversation content are obtained,
A program according to claim 1 or 2, which performs statistical analysis of the acquired evaluation. The aforementioned evaluation has been quantified,
Multiple evaluations corresponding to the aforementioned attributes, location information, region, and conversation content are acquired.
A representative value is calculated that includes the average value of the multiple evaluations obtained.
The program according to claim 4, which stores analyzed data relating the aforementioned attributes, location information, region, and conversation content to the representative value. The attributes of the user riding in the mobile vehicle, the location information of the mobile vehicle, and the region of the mobile vehicle are acquired.
The program according to claim 7, which determines the conversation content to be provided to the user based on the analyzed data, wherein the conversation content with the highest representative value corresponding to the attribute, location information, and region is the conversation content to be provided to the user. A method of information processing performed by a computer,
Obtain the attributes of the user boarding the mobile vehicle.
When providing conversational content to the user, the degree to which the user is favorable towards conversational keywords whose importance is above a certain level is determined.
An information processing method for storing data that associates the aforementioned attributes, the aforementioned conversation content, the aforementioned conversation keywords, and the aforementioned evaluation. A control unit that acquires the attributes of a user riding in a mobile device and determines the degree of favorability of conversational keywords that are of a certain importance or higher among the words spoken by the user when conversational content is provided to the user,
An information processing apparatus comprising a storage unit for storing data that associates the aforementioned attributes, the aforementioned conversation content, the aforementioned conversation keywords, and the aforementioned evaluation.