JP7622840B2 - Server device, system, and server device control method and program - Google Patents

Description

The present invention relates to a server device, a system, a control method for a server device, and a storage medium.

In recent years, the practical application of self-driving cars has progressed. For example, self-driving cars have been put to practical use as a means of transportation in theme parks, tourist destinations, and the like. The technologies that can be applied to self-driving cars are described in the following Patent Documents 1 and 2. These documents disclose technologies for estimating people's interests and emotions.

For example, Patent Document 1 describes a system that detects the behavior of vehicle occupants and presents information about what the occupants are directly or potentially interested in, thereby stimulating the occupants' interest and helping to promote promotion. The cloud server in Patent Document 1 is communicatively connected to the bus. The cloud server has a communication circuit that receives attribute information of customers who plan to board the bus, which is registered in a customer interest database. The cloud server has a processor that infers the interests of customers that the customers are directly or potentially interested in based on the detection results of the customer's behavior detected by an in-vehicle camera or the like mounted on the bus while on board the bus and the customer's attribute information. The processor displays information about the inferred interests on an on-board terminal mounted on the bus via the communication circuit so that it can be viewed.

Patent Document 2 describes that it provides an emotion estimation device, emotion estimation method, emotion estimation program, and emotion counting system that can estimate the accurate emotion of an estimated target. The emotion estimation device of Patent Document 2 has a pulse analysis unit, a facial expression analysis unit, an utterance content analysis unit, an utterance tone analysis unit, and an emotion estimation unit. The pulse analysis unit analyzes the pulse of the estimated target to generate pulse information. The facial expression analysis unit analyzes the facial expression of the estimated target based on image information of the face of the estimated target to generate facial expression information. The speech content analysis unit analyzes the speech content of the estimated target based on the voice information of the estimated target to generate speech content information. The speech tone analysis unit analyzes the speech tone of the estimated target based on the voice information to generate speech tone information. The emotion estimation unit estimates the emotion of the estimated target based on at least two of the pulse information, facial expression information, utterance content information, and speech tone information.

JP 2020-057066 A JP 2018-068618 A

Users board self-driving cars and travel to tourist spots and theme parks. When they do so, the self-driving cars may provide passengers with information by voice or other means, but the information provided is predetermined and often does not correspond to the information that users truly want.

The primary objective of the present invention is to provide a server device, a system, a control method for a server device, and a storage medium that contribute to improving passenger satisfaction of transportation means used in tourist destinations, etc.

According to a first aspect of the present invention, a server device is provided that includes a communication unit that communicates with a patrol vehicle patrolling within a specified area, a conversation unit that asks passengers of the patrol vehicle about their impressions of spots introduced to them by the patrol vehicle and acquires responses from the passengers of the patrol vehicle, and a calculation unit that calculates the satisfaction level of the passengers of the patrol vehicle regarding the introduced spots based on the responses from the passengers of the patrol vehicle.

According to a second aspect of the present invention, a system is provided that includes a patrol vehicle that patrols within a specified area, and a server device connected to the patrol vehicle via wireless communication, the server device comprising: a communication unit that communicates with the patrol vehicle; a conversation unit that asks passengers of the patrol vehicle about their impressions of spots that the patrol vehicle has guided them to and obtains responses from the passengers of the patrol vehicle; and a calculation unit that calculates the satisfaction level of the passengers of the patrol vehicle regarding the guided spots based on the responses from the passengers of the patrol vehicle.

According to a third aspect of the present invention, a method for controlling a server device is provided, in which the server device communicates with a patrol vehicle circulating within a specified area, asks the passengers of the patrol vehicle about their impressions of the spots introduced to them by the patrol vehicle, obtains responses from the passengers of the patrol vehicle, and calculates the satisfaction level of the passengers of the patrol vehicle with respect to the introduced spots based on the responses from the passengers of the patrol vehicle.

According to a fourth aspect of the present invention, a computer-readable storage medium is provided that stores a program for causing a computer mounted on a server device to execute the following processes: communicating with a patrol vehicle patrolling within a specified area; asking passengers of the patrol vehicle about their impressions of spots introduced to them by the patrol vehicle and acquiring responses from the passengers of the patrol vehicle; and calculating the satisfaction level of the passengers of the patrol vehicle with respect to the introduced spots based on the responses from the passengers of the patrol vehicle.

According to each aspect of the present invention, the present invention provides a server device, a system, a control method for a server device, and a storage medium that contribute to improving the satisfaction of passengers of means of transportation used in tourist destinations and the like. Note that the effects of the present invention are not limited to the above. The present invention may achieve other effects instead of or in addition to the effects.

FIG. 1 is a diagram for explaining an overview of an embodiment. FIG. 2 is a diagram illustrating an example of a schematic configuration of an information providing system according to the first embodiment. FIG. 3 is a diagram for explaining the operation of the information providing system according to the first embodiment. FIG. 4 is a diagram for explaining a patrol vehicle according to the first embodiment. FIG. 5 is a diagram for explaining a patrol vehicle according to the first embodiment. FIG. 6 is a diagram illustrating an example of a processing configuration of the control device according to the first embodiment. FIG. 7 is a diagram illustrating an example of a processing configuration of a server device according to the first embodiment. FIG. 8 is a diagram illustrating an example of the vehicle management database according to the first embodiment. FIG. 9 is a diagram illustrating an example of a processing configuration of a communication unit according to the first embodiment. FIG. 10 is a diagram illustrating an example of the provided voice database according to the first embodiment. FIG. 11 is a sequence diagram illustrating an example of the operation of the information providing system according to the first embodiment. FIG. 12 is a diagram illustrating an example of a processing configuration of a server device according to the second embodiment. FIG. 13 is a diagram illustrating an example of a destination determination table according to the second embodiment. FIG. 14 is a diagram illustrating an example of a vehicle management database according to the second embodiment. FIG. 15 is a sequence diagram illustrating an example of an operation of the information providing system according to the second embodiment. FIG. 16 is a diagram illustrating an example of a processing configuration of a server device according to the third embodiment. FIG. 17 is a diagram illustrating an example of a user information database according to the third embodiment. FIG. 18 is a diagram illustrating an example of a vehicle management database according to the third embodiment. FIG. 19 is a diagram illustrating an example of a provided voice database according to the third embodiment. FIG. 20 is a diagram illustrating an example of a hardware configuration of a server device according to the present disclosure. FIG. 21 is a diagram showing an example of a passenger information database according to a modified example of the present disclosure. FIG. 22 is a diagram illustrating an example of a processing configuration of a server device according to a modification of the present disclosure. FIG. 23 is a diagram for explaining the operation of a server device according to a modification of the present disclosure. FIG. 24 is a diagram for explaining the operation of a server device according to a modification of the present disclosure.

First, an overview of one embodiment will be described. The reference numbers in the drawings are added to each element for convenience as an example to aid in understanding, and the description of this overview is not intended to be limiting. Furthermore, unless otherwise specified, the blocks shown in each drawing represent functional units rather than hardware units. The connection lines between blocks in each drawing include both bidirectional and unidirectional. The unidirectional arrows are used to show the main signal (data) flow diagrammatically, and do not exclude bidirectionality. In this specification and drawings, elements that can be described in the same way may be given the same reference numbers to avoid duplicated explanations.

A server device 100 according to one embodiment includes a communication unit 101, a conversation unit 102, and a calculation unit 103 (see FIG. 1). The communication unit 101 communicates with a patrol vehicle patrolling within a specified area. The conversation unit 102 asks the passengers of the patrol vehicle about their impressions of the spots that the patrol vehicle has introduced to the passengers, and acquires responses from the passengers of the patrol vehicle. The calculation unit 103 calculates the satisfaction level of the passengers of the patrol vehicle regarding the introduced spots based on the responses from the passengers of the patrol vehicle.

The server device 100 communicates with a patrol vehicle that travels within a specified area such as a tourist spot or theme park, and converses with passengers aboard the patrol vehicle. Specifically, the server device 100 asks passengers about their impressions of the tourist spots that the patrol vehicle has taken them to, and calculates the satisfaction level of the visited spots (passenger satisfaction) based on the responses. Depending on the calculated satisfaction level, the server device 100 changes the information provided to passengers in the patrol vehicle or changes the spots to which the passengers are guided. In this way, the server device 100 can change the services provided to passengers depending on their satisfaction level, thereby improving passenger satisfaction.

Specific embodiments are described in further detail below with reference to the drawings.

[First embodiment]
The first embodiment will be described in more detail with reference to the drawings.

[System Configuration]
Fig. 2 is a diagram showing an example of a schematic configuration of the information provision system according to the first embodiment. As shown in Fig. 2, the information provision system includes a server device 10 and a plurality of patrol vehicles 20-1 to 20-3.

In the following explanation, unless there is a particular reason to distinguish between patrol vehicles 20-1 to 20-3, they will simply be referred to as "patrol vehicles 20." Also, while three patrol vehicles 20 are illustrated in FIG. 2, it is of course not intended to limit the number of patrol vehicles 20 included in the information provision system to "three." It is sufficient for the information provision system to include at least one or more patrol vehicles 20.

The server device 10 and the patrol vehicle 20 are configured to be able to communicate with each other via wireless communication means such as 4G (4th Generation) or 5G (5th Generation). More specifically, the server device 10 communicates with the patrol vehicle 20 via the base station 30.

The server device 10 is a device that provides various services to passengers of the patrol vehicle 20. For example, the server device 10 provides useful information to passengers of the patrol vehicle 20. The server device 10 may be installed in a building of a system operating company, etc., or may be installed on a network (cloud).

The patrol vehicle 20 is a vehicle that travels within a predetermined area (predetermined area). The patrol vehicle 20 travels with at least one passenger on board. In the first embodiment, the patrol vehicle 20 is described on the assumption that it travels with a relatively small group of people, such as a family. However, it is of course not intended to limit the capacity of the patrol vehicle 20.

In the first embodiment, the patrol vehicle 20 is described as a vehicle (such as a cart, a microbus, or a bus) that can run automatically without a human being operating the vehicle. However, the patrol vehicle 20 may be a conventional vehicle that is driven by a human being. The patrol vehicle 20 is configured to be controllable from the server device 10.

[Operation Overview]
Next, an outline of the operation of the information providing system according to the first embodiment will be described.

The touring vehicle 20 travels (patrols) within a tourist destination as shown in FIG. 3 based on instructions from the server device 10. As shown in FIG. 3, the tourist destination includes multiple tourist spots A to F. The touring vehicle 20 is controlled to carry passengers (tourists) and tour at least one tourist spot.

For example, in Figure 3, tourists board the patrol vehicle 20 at the departure and arrival point. The patrol vehicle 20 carrying the tourists travels along the route shown by the dotted line, passing through tourist spots A to F. Note that although Figure 3 shows one patrol vehicle 20, in reality multiple patrol vehicles 20 travel around the tourist spots.

The server device 10 acquires information from inside the traveling vehicle 20 while it is moving, and analyzes the acquired information. The server device 10 provides information according to the analysis results to the passengers of the traveling vehicle 20. For example, the server device 10 provides the passengers with information about tourist spots that the tourists have been guided to, and information about tourist spots that the passengers are scheduled to be guided to.

Figure 4 is a plan view for explaining the internal configuration of the patrol vehicle 20. As shown in Figure 4, partitions 21-1 to 21-4 are installed in front of each of seats A to D of the patrol vehicle 20. Furthermore, each partition 21 is equipped with a camera 22, a microphone 23, and a speaker 24 (see Figure 5). As shown in Figure 5, messages encouraging passengers to board and disembark safely may be posted on the partitions 21.

The devices (camera 22, microphone 23, speaker 24) embedded in each partition 21 are connected to a control device 40 shown in Figure 4. The control device 40 is a device (module) that controls the entire patrol vehicle 20. More specifically, the control device 40 controls the movement of the patrol vehicle 20, controls devices such as the camera 22, and controls communications between the server device 10 and the patrol vehicle 20.

The control device 40 transmits image data obtained from the camera 22 to the server device 10 periodically or at a specified timing.

The server device 10 detects that a passenger has taken a seat by analyzing the acquired image data.

The server device 10 determines a "representative" in the patrol vehicle 20. If there is only one passenger, that passenger becomes the representative. If there are multiple passengers seated, the server device 10 determines the "representative" by any method.

For example, the server device 10 determines the representative based on the location of the seat (front row, back row, right side in the direction of travel, left side). For example, in the example of Figure 4, the server device 10 determines the person seated in seat D as the representative.

Alternatively, the server device 10 may estimate the attributes (gender, age) of the seated persons by analyzing image data, and determine a representative based on the estimated attributes. For example, in a family group, the server device 10 may designate the "father" as the representative, or the "child" as the representative.

Once the representative has been determined, the server device 10 begins movement control (patrol control) of the patrol vehicle 20. For example, the server device 10 controls the patrol vehicle 20 to move along the route shown by the dotted line in Figure 3. The server device 10 notifies the patrol vehicle 20 of the destination of the patrol vehicle 20 (the coordinates of the destination tourist spot). The control device 40 controls the movement of the patrol vehicle 20 (straight ahead, stopping, etc.) so that it reaches the destination.

In addition, the server device 10 manages the state and status of each patrol vehicle 20. For example, the server device 10 manages the state of the patrol vehicle 20, such as "guiding around tourist spots" or "traveling to tourist spots."

The server device 10 provides information according to the state of the patrol vehicle 20. For example, when the patrol vehicle 20 is guiding a tourist spot A, the server device 10 provides information about the tourist spot A. For example, if the tourist spot A is a historical building such as a "castle," the server device 10 provides passengers of the patrol vehicle 20 with information about the history and background of the construction of the building.

When the tour of the tourist spots has been completed, the server device 10 uses the microphone 23 and speaker 24 attached to the patrol vehicle 20 to converse with the representative. More specifically, the server device 10 asks the representative about his/her impressions of the tourist spots that the tour has been completed. For example, the server device 10 outputs a voice message such as "Were you satisfied with tourist spot A?" from the speaker 24.

At that time, the server device 10 acquires a facial image (image data including a facial image) and voice of the representative who answers the question. The server device 10 acquires image data of the representative who answers the question and voice data of the representative.

The server device 10 calculates the satisfaction level (representative satisfaction level) of the tourist spot that was guided by analyzing the image data and audio data.

The server device 10 determines the information to be provided to passengers of the touring vehicle 20 (passengers including the representative) according to the calculated satisfaction level. More specifically, the server device 10 changes the information (audio output from the speaker 24) provided on the way to the next tourist spot. For example, if the satisfaction level is high, the server device 10 provides more in-depth information about the tourist spot that has already been guided to. If the satisfaction level is low, the server device 10 provides information that is unrelated to the tourist spot that has been guided to (for example, a guide to the entire tourist destination).

Next, we will explain in detail each device included in the information provision system related to the first embodiment.

[Control device]
Fig. 6 is a diagram showing an example of a processing configuration (processing module) of the control device 40 according to the first embodiment. Referring to Fig. 6, the control device 40 includes a communication control unit 201, an operation control unit 202, a position information transmission unit 203, a device control unit 204, and a storage unit 205.

The communication control unit 201 is a means for controlling communication with other devices. For example, the communication control unit 201 receives data (packets) from the server device 10. The communication control unit 201 also transmits data to the server device 10. The communication control unit 201 passes the data received from the other devices to the other processing modules. The communication control unit 201 transmits data acquired from the other processing modules to the other devices. In this way, the other processing modules transmit and receive data to and from the other devices via the communication control unit 201. The communication control unit 201 has a function as a receiving unit that receives data from other devices and a function as a transmitting unit that transmits data to the other devices.

The operation control unit 202 is a means for controlling the operation of the patrol vehicle 20. The operation control unit 202 controls the drive system (engine, brakes, steering, etc.) of the patrol vehicle 20 so that the vehicle reaches the destination instructed by the server device 10. The operation control unit 202 controls the drive system in accordance with a "movement instruction" received from the server device 10. The movement instruction includes the coordinates (X coordinate, Y coordinate) of the destination, and the operation control unit 202 moves the patrol vehicle 20 to the coordinate position.

When the operation control unit 202 reaches the designated destination, it stops and waits until it receives a new movement instruction. When the operation control unit 202 receives a new movement instruction, it moves according to the instruction. Note that existing technology can be used for the automatic operation of the patrol vehicle 20, and since this is different from the purpose of the present disclosure, a detailed explanation is omitted.

The location information transmission unit 203 is a means for transmitting the current location of the device itself (the patrol vehicle 20) to the server device 10. The location information transmission unit 203 receives GPS signals from GPS (Global Positioning System) satellites to perform positioning and generate location information including the latitude, longitude, and altitude of the patrol vehicle 20. Alternatively, the location information transmission unit 203 may generate location information using radio waves received from the base station 30.

The location information transmission unit 203 periodically or at a predetermined timing notifies the server device 10 of the generated location information (coordinate information). The location information transmission unit 203 transmits a "current location notification" including the location information and the vehicle ID to the server device 10.

The vehicle ID is an ID for identifying each patrol vehicle 20 included in the information provision system. The vehicle ID may be the MAC (Media Access Control) address or IP (Internet Protocol) address of the patrol vehicle 20.

The device control unit 204 is a means for controlling each device (camera 22, microphone 23, speaker 24) embedded in the partition 21.

When there is no instruction from the server device 10 (when the state of the server device 10 is not the "conversation mode" described below), the device control unit 204 acquires image data from the camera 22 periodically or at a specified timing.

The device control unit 204 transmits the acquired image data to the server device 10. Specifically, the device control unit 204 transmits a "passenger image notification" including the vehicle ID, seat ID, and image data to the server device 10. In this manner, the passenger image notification includes image data showing the seat (the passenger seated in the seat).

The seat ID is an ID for identifying each seat in the patrol vehicle 20. The system administrator etc. determines the seat ID for each seat in the patrol vehicle and sets it in the control device 40.

The vehicle ID and seat ID are shared between the server device 10 and the patrol vehicle 20 by any method. For example, the system administrator determines the seat ID and sets the determined seat ID in the server device 10. The system administrator also sets the determined seat ID in the control device 40.

If there is no passenger seated in a seat, the device control unit 204 does not need to send a passenger image notification for that seat to the server device 10.

The device control unit 204 also receives a "guidance voice output instruction" from the server device 10. The device control unit 204 outputs the voice data included in the instruction from the speaker 24 of each seat (or a seat where an occupant is seated).

When the state of the server device 10 is in conversation mode, the device control unit 204 controls each device according to instructions from the server device 10. The device control unit 204 receives a "conversation start instruction" from the server device 10. The conversation start instruction includes the seat ID where the conversation partner (representative) will sit.

The device control unit 204 controls the microphone 23 and the speaker 24 so that the representative can converse with the server device 10. The device control unit 204 also acquires image data of the representative during the conversation.

More specifically, during a conversation, the device control unit 204 outputs voice data acquired from the server device 10 from the speaker 24 corresponding to the seat specified by the server device 10. The device control unit 204 also acquires voice data from the microphone 23 corresponding to the seat specified by the server device 10, and transmits the voice data to the server device 10.

Also, during the conversation, the device control unit 204 acquires image data from the camera 22 of the seat specified by the server device (seat ID included in the conversation start instruction) and transmits the image data to the server device 10.

When the device control unit 204 receives a "conversation end instruction" from the server device 10, it ends the conversation with the representative.

The memory unit 205 is a means for storing information necessary for the operation of the patrol vehicle 20.

[Server device]
Fig. 7 is a diagram showing an example of a processing configuration (processing module) of the server device 10 according to the first embodiment. Referring to Fig. 7, the server device 10 includes a communication control unit 301, a vehicle position management unit 302, a representative determination unit 303, a vehicle operation control unit 304, a communication unit 305, an information provision unit 306, and a storage unit 307.

The communication control unit 301 is a means (communication unit) for controlling communication with other devices. For example, the communication control unit 301 receives data (packets) from the patrol vehicle 20. The communication control unit 301 also transmits data to the patrol vehicle 20. The communication control unit 301 passes the data received from the other devices to the other processing modules. The communication control unit 301 transmits data acquired from the other processing modules to the other devices. In this way, the other processing modules transmit and receive data with the other devices via the communication control unit 301. The communication control unit 301 has a function as a receiving unit that receives data from other devices and a function as a transmitting unit that transmits data to the other devices.

The vehicle location management unit 302 is a means for managing the current location of the patrol vehicle 20. The vehicle location management unit 302 receives a "current location notification" from the patrol vehicle 20. The vehicle location management unit 302 stores the location information contained in the notification in the vehicle management database (see FIG. 8). More specifically, the vehicle location management unit 302 identifies the patrol vehicle 20 from the vehicle ID contained in the current location notification, and updates the current location field of the corresponding entry.

Note that the vehicle management database shown in Figure 8 is an example and is not intended to limit the items to be stored. Also, for ease of understanding, the vehicle ID shown in Figure 8 uses the symbol for the patrol vehicle 20, and the seat ID in which the representative sits uses the seat name (seats A to D) shown in Figure 4.

The representative determination unit 303 is a means for determining a representative from among the passengers of the patrol vehicle 20. When there are multiple passengers on board the patrol vehicle 20, the representative determination unit 303 determines a representative from among the multiple passengers. For example, the representative determination unit 303 determines a representative by analyzing image data included in a "passenger image notification" received from each patrol vehicle 20 periodically or at a specified timing.

The representative determination unit 303 analyzes image data of each seat in the patrol vehicle 20 and determines whether or not a passenger is seated (occupied) in each seat. For example, the representative determination unit 303 can determine whether or not a passenger is seated by comparing image data (template) of an empty seat with image data of an image and audio notification.

For example, in the example of Figure 4, the representative determination unit 303 recognizes that passengers are seated in each of seats B to D.

The representative determination unit 303 determines a representative from among the people seated in the patrol vehicle 20. The representative determination unit 303 determines the representative using any method. For example, the representative determination unit 303 determines the representative based on the seat position (front row, back row, right side in the direction of travel, left side). For example, in the example of Figure 4, the person seated in seat D on the right side of the front row is selected as the representative.

Alternatively, the representative determination unit 303 may estimate attributes (e.g., gender, age) of each seated person by analyzing image data, and determine a representative based on the estimated attributes, etc. The representative determination unit 303 may use a learning model prepared in advance to estimate the attributes.

The learning model can be generated by machine learning using training data to which image data labels (attributes; for example, father, child) have been added. Any algorithm, such as a support vector machine, boosting, or neural network, can be used to generate the learning model. Note that the algorithms, such as the support vector machine, can use publicly known technologies, and therefore will not be described here.

For example, in the case of a family group, the representative determination unit 303 will select the "father" as the representative or the "child" as the representative.

Alternatively, the representative determination unit 303 may determine the attributes of the seated person (father, mother, child) based on the seated person's clothing.

The representative determination unit 303 stores the seat position of the determined representative in the vehicle management database. In the example of Figure 4, the person sitting in seat D is selected as the representative and registered in the vehicle management database. Since the passenger image notification includes the seat ID of the subject of the image data capture, the representative determination unit 303 can understand the relationship between the image data and the seat ID.

Once the representative determination unit 303 has determined a representative, it notifies the vehicle operation control unit 304 of this (representative selection completed) along with the vehicle ID of the corresponding patrol vehicle 20.

The vehicle operation control unit 304 is a means for instructing (controlling) the operation (movement) of the patrol vehicle 20. When the patrol vehicle 20 is an autonomous vehicle, the vehicle operation control unit 304 controls the operation of the patrol vehicle 20 by sending a "movement instruction" including the destination of the patrol vehicle 20 to the patrol vehicle 20. The vehicle operation control unit 304 sends a movement instruction to the patrol vehicle 20 including the vehicle ID of the patrol vehicle 20 to be controlled and the destination coordinates (X coordinate, Y coordinate) of the patrol vehicle.

The vehicle operation control unit 304 starts controlling the movement of the patrol vehicle 20 for which a representative has been determined (the patrol vehicle 20 into which the passenger boarded at the departure and arrival point). The vehicle operation control unit 304 controls the patrol vehicle 20 to move to a predetermined tourist spot, stop in front of the predetermined tourist spot for a certain period of time, and move to the next predetermined tourist spot.

In the example of Figure 3, the patrol vehicle 20 travels to tourist spot A and stops there for a certain period of time (or travels around tourist spot A at a low speed). The patrol vehicle 20 then travels to the next tourist spot B. The vehicle operation control unit 304 transmits a movement instruction to the patrol vehicle 20 to be controlled so that the patrol vehicle 20 travels (travels) in this manner.

The vehicle operation control unit 304 obtains the destination (tourist spot to be guided) from the route described in the route field of the vehicle management database. In the example of FIG. 8, the vehicle operation control unit 304 controls the patrol vehicle 20 to visit tourist spots A to F in order. The vehicle operation control unit 304 can obtain the coordinates of each tourist spot by referring to table information that stores the tourist spot in association with its coordinates (X coordinate, Y coordinate).

Note that the tour route shown in FIG. 8 is an example and is not intended to limit the travel route of the patrol vehicle 20. As shown in FIG. 8, it is not necessary to tour each tourist spot included in the tourist destination. For example, a tour route that tours some of the tourist spots may be registered in the vehicle management database. Alternatively, the order of the tourist spots to be toured may differ depending on the patrol vehicle 20. For example, a tour route that guides the tourist spots in the order of "A, B, C, D, E, F" and a tour route that guides the tourist spots in the order of "A, F, E, D, C, B" may be registered.

Furthermore, the vehicle operation control unit 304 calculates the state of the patrol vehicle 20 based on the current position of the patrol vehicle 20 and the content of the movement instruction. For example, the vehicle operation control unit 304 instructs the patrol vehicle 20 to move to tourist spot A, and if the position of the patrol vehicle 20 is at the coordinate position of tourist spot A, it determines that the patrol vehicle 20 is guiding tourist spot A. Furthermore, the vehicle operation control unit 304 instructs the patrol vehicle 20 to move to tourist spot B, and if the patrol vehicle 20 has not yet reached tourist spot B, it determines that the state of the patrol vehicle 20 is moving to tourist spot B.

The vehicle operation control unit 304 reflects the determined state of the patrol vehicle 20 in the status field of the vehicle management database. For example, as shown in FIG. 8, when the patrol vehicle 20 reaches tourist spot A and stops temporarily, a setting such as "Guiding tourist spot A" is written in the status field.

Here, the operating modes (control modes) of the server device 10 for the patrol vehicle 20 include two modes: "normal mode" and "conversation mode." The default mode is "normal mode."

The operating mode while the patrol vehicle 20 is guiding passengers around the tourist spots is the normal mode. Specifically, the server device 10 is set to the normal mode while it is providing information to passengers on the patrol vehicle 20. For example, the control mode is normal mode from when the patrol vehicle 20 arrives at a tourist spot, provides passengers with information about the tourist spot, and until the patrol vehicle 20 leaves the tourist spot. In other words, the time from when the patrol vehicle 20 arrives at the tourist spot and the passengers get off the patrol vehicle 20 to tour the tourist spot and then get on the patrol vehicle 20 is also normal mode.

When the touring vehicle 20 finishes guiding the tourist spot (when it leaves the tourist spot and heads to the next tourist spot), it switches from "normal mode" to "conversation mode."

In this way, in the normal mode (guidance mode), the server device 10 (traveling vehicle 20) outputs a one-sided guidance voice, and there is no conversation between the server device 10 and the passenger (representative). In the normal mode (guidance mode), for example, the traveling vehicle 20 stops in front of a tourist spot and provides information such as tourist guidance to the passengers inside the vehicle. In addition, after the tourist guidance ends, a free time is set, and the passengers may get off the traveling vehicle 20 and tour the tourist spots. On the other hand, in the conversation mode, after the tourist guidance by the traveling vehicle 20 ends, or when the passengers who have returned from the tourist spots after the free time ends get on the traveling vehicle 20, the traveling vehicle 20 starts moving toward the next tourist spot. At the timing when the traveling vehicle 20 starts moving, the operation mode is switched from the normal mode (guidance mode) to the conversation mode, the satisfaction level of the tourist spot where the guidance has ended is analyzed, and processing according to the satisfaction level (for example, changing the content of the information provided, changing the destination) is performed.

The communication unit 305 is a means for communicating with passengers of the patrol vehicle 20, particularly the representative. FIG. 9 is a diagram showing an example of the processing configuration of the communication unit 305 according to the first embodiment. As shown in FIG. 9, the communication unit 305 includes submodules consisting of a guidance end detection unit 351, a conversation unit 352, and a satisfaction level calculation unit 353.

The guidance end detection unit 351 is a means for detecting when the touring vehicle 20 has finished guiding the tourist spots. The guidance end detection unit 351 monitors (polls) the status field of the vehicle management database, and detects a change in the setting value of the status field from "guiding" to "moving."

For example, the guidance end detection unit 351 detects that the setting value of the status field has changed from "guiding to tourist spot A" to "moving to tourist spot B." When it detects that the guidance has ended, the guidance end detection unit 351 notifies the conversation unit 352 of the tourist spot where the guidance has ended. In addition, the guidance end detection unit 351 sets the operation mode for the patrol vehicle 20 for which the guidance has ended to "conversation mode."

The conversation unit 352 is a means for conversing with passengers (representatives) of the patrol vehicle 20. The conversation unit 352 asks the passengers of the patrol vehicle 20 about their impressions of the spots that the patrol vehicle 20 has guided them to, and obtains responses from the passengers of the patrol vehicle 20. The conversation unit 352 refers to the vehicle management database and identifies the representative of the patrol vehicle 20 (the seat in which the representative sits) for which the guidance has ended.

The conversation unit 352 sends a "conversation start instruction" to the target patrol vehicle 20, which includes the vehicle ID of the patrol vehicle 20 and the seat ID of the identified seat.

After sending the instruction to start the conversation, the conversation unit 352 transmits voice data to the traveling vehicle 20, such as asking about the driver's impressions of the tourist spot that the tour has just concluded. For example, in the above example, the conversation unit 352 transmits voice data to the traveling vehicle 20, such as, "Were you satisfied with tourist spot A?"

Furthermore, in response to transmitting the above voice data (question regarding impressions of tourist spots), the conversation unit 352 acquires voice data regarding the response and image data of the representative from the patrol vehicle 20. The conversation unit 352 passes the acquired voice data and image data to the satisfaction level calculation unit 353. In the following description, the voice data and image data regarding the passenger's response to the question from the conversation unit 352 are referred to as "voice and image data at the time of response."

When the conversation ends (when the answering voice image data is acquired), the conversation unit 352 transmits a "conversation end instruction" including the vehicle ID of the patrol vehicle 20 to the patrol vehicle 20. The conversation unit 352 also sets the operation mode to the "normal mode."

In this way, the conversation unit 352 starts a conversation with the representative when the patrol vehicle 20 finishes guiding the tourist spots. At that time, the conversation unit 352 transmits voice data corresponding to a question to the representative to the patrol vehicle 20. The conversation unit 352 receives voice data and image data at the time of answering the question from the patrol vehicle 20 as a response from a passenger (representative) of the patrol vehicle 20.

The satisfaction calculation unit 353 is a means for calculating the satisfaction of passengers with respect to tourist spots guided by the patrol vehicle 20. The satisfaction calculation unit 353 calculates the satisfaction of passengers in the patrol vehicle 20 with respect to the guided locations (spots, tourist spots) based on answers from passengers in the patrol vehicle 20. The satisfaction calculation unit 353 obtains the satisfaction of the question answerer (the representative's satisfaction with the guided tourist spots) by inputting the above-acquired voice image data at the time of answering (voice data and image data when the representative answers the question) into a learning model prepared in advance.

The above learning model can be obtained by preparing a large amount of image data and audio data with labels (quantified satisfaction levels) and learning using the large amount of image data and audio data as training data. For example, image data and audio data with a bright tone and a smiling face are labeled with a high level of satisfaction, while image data and audio data with a dark tone and a smiling face that is not truly satisfied are labeled with a low level of satisfaction.

By using the learning model generated in this manner, the satisfaction calculation unit 353 can more accurately grasp the satisfaction level of passengers (representatives).

The satisfaction level calculation unit 353 sets the calculated satisfaction level in the satisfaction level field of the vehicle management database.

The information provision unit 306 is a means for providing information to passengers of the patrol vehicle 20. For example, the information provision unit 306 determines the information to be provided to passengers of the patrol vehicle 20 based on the degree of satisfaction calculated by the satisfaction calculation unit 353. The information provision unit 306 realizes the above information provision by transmitting a "guidance voice output instruction" to the patrol vehicle 20 in normal mode.

The information providing unit 306 accesses the vehicle management database and refers to the entry for the patrol vehicle 20 to which information is to be provided. The information providing unit 306 reads the satisfaction level from the satisfaction level field of the entry. If the satisfaction level field is blank (no set value), the information providing unit 306 provides information using a predetermined voice.

For example, the satisfaction level has not been calculated before the patrol vehicle 20 finishes guiding the tourist spot A, so the information provision unit 306 provides information about the tourist spot A. More specifically, the information provision unit 306 transmits to the patrol vehicle 20 a guidance voice output instruction including previously prepared voice data and the vehicle ID of the target patrol vehicle 20.

On the other hand, if a satisfaction level is set in the satisfaction level field of the identified entry, the information provision unit 306 determines the information to be provided during the movement to the next tourist spot according to the satisfaction level. More specifically, the information provision unit 306 changes the voice data to be transmitted to the patrol vehicle 20 according to the satisfaction level.

The information providing unit 306 refers to a provided voice database that stores destinations, satisfaction levels, and provided voice data (information to be provided) in association with each other, and acquires voice data to be transmitted to the patrol vehicle 20 (see FIG. 10). Note that the provided voice database shown in FIG. 10 is an example, and is not intended to limit the items to be stored.

The system administrator decides the voice data to be provided to passengers (the content to be provided as voice data) taking into consideration the tourist spots guided by the touring vehicle 20 and the level of satisfaction. The system administrator simply registers the decided voice data in the provided voice database.

For example, if the satisfaction level is high, voice data of more in-depth information on the tourist spot that has been guided to is registered. If the satisfaction level is low, voice data of information unrelated to the tourist spot that has been guided to (information unrelated to the tourist spot; for example, information on the tourist spot as a whole, information on the next tourist spot, etc.) is registered. That is, if the calculated satisfaction level is equal to or higher than a first threshold, the information providing unit 306 provides information on the tourist spot that has been guided to the passenger by the traveling vehicle 20. If the calculated satisfaction level is equal to or lower than a second threshold, the information providing unit 306 provides information unrelated to the tourist spot that has been guided to the passenger by the traveling vehicle 20.

In the example of Figure 10, if the satisfaction level with tourist spot A is high (e.g., satisfaction level is 70% or higher), the information providing unit 306 provides information using voice data B1. If the satisfaction level with tourist spot A is low (e.g., satisfaction level is 30% or lower), the information providing unit 306 provides information using voice data B3. If the satisfaction level with tourist spot A is average (medium; e.g., satisfaction level is higher than 30% and lower than 70%), the information providing unit 306 provides information using voice data B2.

The memory unit 307 is a means for storing information necessary for the operation of the server device 10. A vehicle management database and a provided voice database are constructed in the memory unit 307.

[System Operation]
Next, a description will be given of the operation of the information providing system according to the first embodiment. Fig. 11 is a sequence diagram showing an example of the operation of the information providing system according to the first embodiment.

When a passenger boards the patrol vehicle 20, the patrol vehicle 20 acquires image data of each seat. The patrol vehicle 20 transmits a passenger image notification including the captured image data to the server device 10 (step S01).

The server device 10 analyzes the image data obtained by the passenger image notification to determine a representative based on the situation when passengers board the patrol vehicle 20 (step S02).

Once the representative has been determined, the server device 10 starts the movement of the patrol vehicle 20 (movement to tourist spots; patrol) (step S03).

The server device 10 provides various information on the way to the tourist spot and after arriving at the tourist spot. The server device 10 transmits voice data to the patrol vehicle 20 in response to a voice guidance output instruction (step S04).

The patrol vehicle 20 outputs the guidance voice obtained from the server device 10 (step S05).

Once the tourist spot tour is completed, the server device 10 starts a conversation with the representative. The server device 10 sends a conversation start instruction to the patrol vehicle 20 (step S06).

The server device 10 sends a question to the representative of the touring vehicle 20 (step S07). The server device 10 sends voice data to the touring vehicle 20 asking about the representative's impressions of the tourist spot that the tour has just concluded.

The patrol vehicle 20 transmits the representative's response (step S08). The patrol vehicle 20 transmits the voice image data at the time of response to the server device 10.

The server device 10 sends a conversation end instruction to the patrol vehicle 20 (step S09).

The server device 10 calculates the representative's satisfaction level (step S10).

The server device 10 determines the information to be provided to the passengers of the traveling vehicle 20 according to the calculated satisfaction level of the representative. The server device 10 transmits a voice guidance output instruction including the determined information (voice data) to the traveling vehicle 20 (step S11).

The patrol vehicle 20 outputs the guidance voice obtained from the server device 10 (step S12).

As described above, the server device 10 according to the first embodiment asks the representative about his/her impressions of the tourist spots visited by the touring vehicle 20. The server device 10 calculates the representative's level of satisfaction with the touring tourist spots based on the representative's response. The server device 10 changes the services provided to passengers on the touring vehicle 20 according to the calculated level of satisfaction. In this way, the server device 10 can change the services provided to passengers according to their level of satisfaction, thereby improving passenger satisfaction.

In addition, the server device 10 according to the first embodiment calculates the satisfaction level of the person answering the question (representative) based on the facial expression and tone of voice of the person. That is, the server device 10 does not calculate the satisfaction level simply by image analysis (whether the representative is smiling or not), but calculates the satisfaction level by taking into account the voice data when answering the question. As a result, a highly accurate satisfaction level estimation is realized even in cases where the person is smiling on the surface but is not satisfied deep down, or where the change in facial expression indicates that the person is very satisfied deep down.

In addition, the patrol vehicle 20 according to the first embodiment provides information to passengers by outputting audio from the speaker 24. Because the information is provided by audio, the information provision system according to the first embodiment (server device 10, patrol vehicle 20) can provide useful information to passengers without interfering with their sightseeing.

Second Embodiment
Next, the second embodiment will be described in detail with reference to the drawings.

In the first embodiment, it was described that the server device 10 calculates the passenger's satisfaction level and changes the information provided to the passenger in accordance with the calculated satisfaction level. In the second embodiment, it is described that the server device 10 changes the destination of the patrol vehicle 20 in accordance with the calculated satisfaction level.

The configuration of the information provision system according to the second embodiment can be the same as that of the first embodiment, so the explanation corresponding to Fig. 2 will be omitted. Also, the processing configuration of the patrol vehicle 20 according to the second embodiment can be the same as that of the first embodiment, so the explanation will be omitted.

The following will focus on the differences between the first and second embodiments.

Figure 12 is a diagram showing an example of a processing configuration (processing module) of the server device 10 according to the second embodiment. Referring to Figure 12, a destination determination unit 308 has been added to the configuration of the server device 10 according to the first embodiment.

When the satisfaction level calculation unit 353 in the second embodiment finishes calculating the satisfaction level, it passes the vehicle ID and the generated satisfaction level to the destination determination unit 308.

The destination determination unit 308 is a means for determining the destination (a tourist spot to be the destination) according to the passenger's satisfaction level. The destination determination unit 308 determines the tourist spot to be the destination of the traveling vehicle 20 based on the satisfaction level calculated by the satisfaction level calculation unit 353. That is, when the satisfaction level is obtained from the satisfaction level calculation unit 353, the destination determination unit 308 determines the destination of the traveling vehicle 20 according to the satisfaction level.

The destination determination unit 308 refers to a destination determination table that stores the tourist spot from which the vehicle 20 travels, the degree of satisfaction with the tourist spot from which the vehicle travels, and the destination in association with each other, and determines the destination of the patrol vehicle 20 (see FIG. 13). For example, the tourist spot visited by the patrol vehicle 20 is "Tourist Spot A." In this case, if the satisfaction with Tourist Spot A is high, "Tourist Spot C" is selected as the next tourist spot. On the other hand, if the satisfaction with Tourist Spot A is low, "Tourist Spot D" is selected as the next tourist spot.

The destination determination unit 308 reflects the destination (next tourist spot to be guided) determined based on the satisfaction level in the vehicle management database (see FIG. 14). As shown in FIG. 14, the vehicle management database according to the second embodiment includes a destination field instead of or in addition to the tour route field.

The destination determination unit 308 sets the destination determined according to the destination determination table in the destination field of the vehicle management database.

The vehicle operation control unit 304 in the second embodiment instructs the patrol vehicle 20 to move to the set destination.

[System Operation]
Next, the operation of the information providing system according to the second embodiment will be described. Fig. 15 is a sequence diagram showing an example of the operation of the information providing system according to the second embodiment. Note that the operation up to the calculation of the satisfaction level can be the same as the operation of the first embodiment, and therefore the description will be omitted.

When the server device 10 calculates the satisfaction level (step S20), it determines the next destination based on the calculated satisfaction level (step S21). For example, if the satisfaction level of the introduced tourist spot is high, the server device 10 sets a tourist spot similar to the introduced tourist spot as the next destination. Alternatively, if the satisfaction level of the introduced tourist spot is low, the server device 10 sets a tourist spot that is less related to the introduced tourist spot as the next destination.

The server device 10 moves the patrol vehicle 20 to the set destination. The server device 10 transmits a movement instruction including the coordinates of the destination to the patrol vehicle 20 (step S22).

The patrol vehicle 20 moves to the instructed destination (tourist spot) (step S23).

In the second embodiment, as in the first embodiment, the information provided to passengers of the patrol vehicle 20 may be changed depending on the representative's satisfaction level.

As described above, the server device 10 according to the second embodiment changes the tourist spots to which the passenger is guided depending on the passenger's satisfaction level. Therefore, the server device 10 can guide the passenger to tourist spots that are likely to make the passenger more satisfied, or guide the passenger to tourist spots that are likely to increase the satisfaction level of a passenger with low satisfaction.

[Third embodiment]
Next, a third embodiment will be described in detail with reference to the drawings.

In the first and second embodiments, it is assumed that a passenger boards the patrol vehicle 20 without any prior preparation. In the third embodiment, a case is described in which a passenger registers their biometric information, personal information, attribute information, hobbies, interests, etc. in the server device 10 in advance. In the third embodiment, a case is described in which a passenger is identified by a matching process using biometric information, and information is provided based on the attributes, etc. of the identified passenger.

The configuration of the information provision system according to the third embodiment can be the same as that of the first embodiment, so the explanation corresponding to Fig. 2 will be omitted. Also, the processing configuration of the patrol vehicle 20 according to the third embodiment can be the same as that of the first embodiment, so the explanation will be omitted.

The following will focus on the differences between the first to third embodiments.

Figure 16 is a diagram showing an example of a processing configuration (processing module) of the server device 10 according to the third embodiment. Referring to Figure 16, a user registration unit 309 has been added to the configuration of the server device 10 according to the second embodiment.

The user registration unit 309 is a means for acquiring biometric information, personal information, etc. of a user who plans to visit a tourist spot, and registering the user. For example, the user registration unit 309 acquires the above-mentioned biometric information, etc. from a user who accesses a homepage operated by a tourist center or the like of a tourist spot.

The user registration unit 309 displays a GUI (Graphical User Interface), an input form, etc., for inputting biometric information, personal information, etc., on the user's terminal. The user operates the terminal to input biometric information (e.g., a facial image), personal information, etc., into the server device 10.

The user registration unit 309 generates features (a feature vector consisting of multiple features) from the facial image obtained from the terminal.

Since existing technology can be used for the process of generating features, a detailed description thereof will be omitted. For example, the user registration unit 309 extracts the eyes, nose, mouth, etc. as feature points from the face image. The user registration unit 309 then calculates the position of each feature point and the distance between each feature point as feature amounts, and generates a feature vector (vector information that characterizes the face image) consisting of multiple feature amounts.

The user registration unit 309 also generates a user ID for identifying the user. The user ID may be any information that can uniquely identify a registered user. For example, the user registration unit 309 may assign a unique value each time a user is registered as the user ID.

The user registration unit 309 associates the generated user ID, features (biometric information), personal information, etc., and stores them in a user information database (see FIG. 17). In other words, the storage unit 307 stores the biometric information and attribute information of each of a plurality of users in association with each other. Note that the user information database shown in FIG. 17 is an example and is not intended to limit the items to be stored. For example, a "face image" may be registered in the user information database as biometric information.

The representative determination unit 303 according to the third embodiment determines a representative by the method described in the first embodiment. After that, the representative determination unit 303 identifies the representative by a matching process using the image data of the determined representative and the biometric information registered in the user information database.

The representative determination unit 303 identifies attribute information corresponding to passengers of the patrol vehicle 20 through biometric authentication using biometric information generated from image data received from the patrol vehicle 20 and the stored biometric information. Specifically, the representative determination unit 303 extracts a facial image from the image data of the representative, and generates features from the facial image. The representative determination unit 303 sets the generated features as a comparison target, and performs a comparison process with the features registered in the user information database.

More specifically, the representative determination unit 303 calculates the similarity between the feature (feature vector) to be matched and each of the multiple feature on the registration side. The similarity can be calculated using chi-square distance, Euclidean distance, or the like. Note that the greater the distance, the lower the similarity, and the closer the distance, the higher the similarity.

The representative determination unit 303 identifies the user (entry) whose similarity is equal to or greater than a predetermined value and whose feature has the highest similarity as the representative. The representative determination unit 303 sets the user ID of the identified representative in the representative field of the vehicle management database (see FIG. 18).

When providing information to passengers of the patrol vehicle 20, the information providing unit 306 according to the third embodiment changes the information to be provided depending on the representative's satisfaction level and attributes (e.g., gender, age, hobbies and interests). For example, the information provided may differ depending on gender and satisfaction level. In this case, information such as that shown in FIG. 19 is registered in the provided voice database according to the third embodiment.

The information providing unit 306 searches the user information database using the representative's user ID as a key, and acquires the representative's attributes (gender, age, hobbies, interests, etc.). The information providing unit 306 refers to the provided voice database, and identifies the voice data to be provided as information according to the acquired attribute values and satisfaction level.

As described above, the server device 10 according to the third embodiment determines the information to be provided to the passengers of the traveling vehicle 20 based on the attributes of the passengers of the traveling vehicle 20 and the calculated satisfaction level. Since the information to be provided to the passengers is determined taking into consideration not only the satisfaction level but also the attributes of the representative, it is possible to provide information that is more useful to the passengers (which can increase their satisfaction).

Next, we will explain the hardware of each device that makes up the information provision system. Figure 20 is a diagram showing an example of the hardware configuration of the server device 10.

The server device 10 can be configured by an information processing device (so-called a computer) and has the configuration illustrated in Fig. 20. For example, the server device 10 has a processor 311, a memory 312, an input/output interface 313, a communication interface 314, etc. The components such as the processor 311 are connected by an internal bus or the like and are configured to be able to communicate with each other.

However, the configuration shown in FIG. 20 is not intended to limit the hardware configuration of the server device 10. The server device 10 may include hardware not shown, and may not include an input/output interface 313 as necessary. In addition, the number of processors 311 and the like included in the server device 10 is not intended to be limited to the example shown in FIG. 20, and for example, multiple processors 311 may be included in the server device 10.

The processor 311 is, for example, a programmable device such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). Alternatively, the processor 311 may be a device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The processor 311 executes various programs including an operating system (OS).

Memory 312 is a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), a SSD (Solid State Drive), etc. Memory 312 stores an OS program, application programs, and various data.

The input/output interface 313 is an interface for a display device and an input device (not shown). The display device is, for example, a liquid crystal display. The input device is, for example, a device that accepts user operations such as a keyboard or a mouse.

The communication interface 314 is a circuit, module, etc. that communicates with other devices. For example, the communication interface 314 includes a NIC (Network Interface Card), etc.

The functions of the server device 10 are realized by various processing modules. The processing modules are realized, for example, by the processor 311 executing a program stored in the memory 312. The program can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. That is, the present invention can also be embodied as a computer program product. The program can be downloaded via a network or updated using a storage medium storing the program. The processing modules may also be realized by a semiconductor chip.

The control device 40 can also be configured by an information processing device, just like the server device 10, and its basic hardware configuration is no different from that of the server device 10, so a description of it will be omitted. Also, the hardware configuration of the patrol vehicle 20 can be the same as that of an existing autonomous vehicle, so a description of it will be omitted.

The server device 10, which is an information processing device, is equipped with a computer, and the functions of the server device 10 can be realized by causing the computer to execute a program. In addition, the server device 10 executes a control method for the server device 10 by the program.

[Modification]
The configuration, operation, and the like of the information providing system described in the above embodiment are merely examples, and are not intended to limit the system configuration, and the like.

In the above embodiment, the specified area patrolled by the patrol vehicle 20 has been described as a tourist spot, but the specified area is not limited to a tourist spot. For example, the patrol vehicle 20 may patrol a facility such as a theme park, a zoo, or an aquarium. For example, when the patrol vehicle 20 patrols a zoo, the server device 10 may change the content of the information provided based on the impressions of the animal that was guided, or select the next animal to guide.

The server device 10 can analyze the image data included in the passenger image notification, and if a passenger is engaging in dangerous behavior (e.g., sticking a hand or face out of a window), provide information to stop the dangerous behavior. For example, the information provision unit 306 inputs the image data into a learning model and obtains a conclusion as to whether the passenger's behavior is dangerous or not. If a passenger is engaging in dangerous behavior, the information provision unit 306 instructs (warns) the passenger via the speaker 24 to stop the dangerous behavior.

In the above embodiment, it has been described that a representative is selected from among multiple passengers, and information is provided and the spots to be guided are changed based on the satisfaction level (impressions, opinions) of the representative. However, the server device 10 may calculate the satisfaction level of each passenger, calculate the consensus of the multiple passengers from the calculated satisfaction level (satisfaction of all passengers), and use this for providing information, etc. Specifically, the satisfaction level calculation unit 353 may calculate the average satisfaction level of each passenger and treat it as the satisfaction level of multiple passengers (e.g., a family). In this way, the server device 10 may calculate the satisfaction level based on conversations with multiple passengers, not limited to the representative.

In the above embodiment, the representative's response (voice image data at the time of response) to an inquiry (question) regarding his/her impressions of the tourist spot is input into a learning model to calculate the representative's satisfaction. However, the server device 10 may input the voice data and image data into different learning models to obtain a smile score and an emotion score. The smile score is information that quantifies the degree of smile. The emotion score is information that quantifies the emotion of the speaker based on the speech state of the voice. For example, even if the speaker said "it was fun," the emotion score is the data that quantifies whether the speaker really enjoyed the experience or found it boring based on the way the speaker spoke. The server device 10 may calculate the satisfaction level for the experience (the tourist spot to which the representative was guided) based on the two scores (smile score, emotion score). For example, the server device 10 may calculate the sum of the smile score and the emotion score as the satisfaction level, or may calculate the multiplication result of the smile score and the emotion score as the satisfaction level.

The server device 10 may have more conversations (complex conversations) rather than just simple conversations such as asking the passenger (representative) about their impressions of the tourist spots, and may calculate the satisfaction level based on the exchanges in the conversation. For example, the server device 10 may talk to the representative for a longer period of time by asking about detailed impressions rather than just about the overall impressions of the tourist spots. The server device 10 may calculate the satisfaction level based on the content of the long conversation, the representative's facial expression, voice, etc. In this way, the server device 10 may not only directly inquire the representative about whether or not he or she is satisfied, but may also determine whether or not the representative is truly satisfied based on the content of the conversation and facial expression. In other words, the server device 10 may estimate the satisfaction level from the content of the conversation (what the representative is saying), the timing of the conversation, or facial expression (smile score obtained by smile estimation).

The server device 10 may calculate the satisfaction level using information other than the passenger's facial expression (image) and voice. For example, the server device 10 may calculate the satisfaction level using vital data of the passenger (representative) instead of or in addition to the facial expression and voice. In this case, the passenger of the patrol vehicle 20 wears a portable terminal capable of acquiring vital data such as heart rate, pulse, and sweat. The portable terminal transmits the representative's vital data (the representative's vital data when talking with the server device 10) to the server device 10 via the control device 40. The satisfaction level calculation unit 353 of the server device 10 acquires the satisfaction level by inputting the voice, image, and vital data into a learning model prepared in advance. The satisfaction level acquired in this way more accurately reflects the representative's impressions, and the server device 10 can provide more useful information to the passengers.

In the above embodiment, a case has been described in which passengers' facial expressions, voices, etc. are acquired by devices (camera 22, microphone 23) installed in the patrol vehicle 20. However, this information may also be acquired by MR (Mixed Reality) glasses worn by the passengers. The server device 10 may calculate the satisfaction level based on the information acquired by the MR glasses.

Alternatively, the facial expressions, voice, etc. may be acquired by a terminal (e.g., a smartphone, etc.) carried by the user. The terminal acquires image data and voice data including the user's facial expressions and voice using devices such as a camera and a microphone equipped on the terminal. The terminal may transmit the acquired image data and voice data to the control device 40 via Bluetooth (registered trademark) or the like, and the control device 40 may transmit the image data, etc. to the server device 10. Alternatively, the terminal may transmit the image data, etc. directly to the server device 10. In this case, a number or the like may be attached to each seat of the patrol vehicle 20 to make each seat identifiable. In other words, by configuring the image data captured by the terminal to include seat identification information, it becomes possible to identify the seat position corresponding to the image transmitted from the terminal (smartphone).

The server device 10 may analyze the image data included in the passenger image notification and change the information provided depending on the passenger's state (e.g., standing, sitting). The scenery and views that a passenger who is standing and sightseeing at tourist spots can see are different from the scenery and views that a passenger who is sitting and sightseeing at tourist spots can see, and the server device 10 may provide information corresponding to the different scenery and views.

In addition to providing information about tourist destinations, the server device 10 may also provide passengers with information associated with sightseeing at the tourist destinations. For example, when providing information toward the end of a route tour, the server device 10 may provide passengers with recommended information about meals. For example, the server device 10 may provide passengers with information about Japanese restaurants, Chinese restaurants, etc. Alternatively, as in the third embodiment, when passenger attribute information (e.g., country of origin, etc.) is registered, the server device 10 may provide information according to the attribute information. For example, the server device 10 may provide information about restaurants preferred by foreigners. Alternatively, the server device 10 may change the content of the information provided depending on the time of day, etc., when providing the information.

Even without user registration as described in the third embodiment, the server device 10 may identify passengers using biometric authentication (face authentication). The server device 10 (representative determination unit 303) identifies passengers by biometric authentication and assigns an ID to the identified passenger. The server device 10 may store the passenger ID, biometric information (face image, feature amount), and satisfaction history assigned to the passenger in association with each other. For example, the server device 10 (satisfaction calculation unit 353) stores the passenger ID, biometric information, and satisfaction history in association with each other in the passenger information database (see FIG. 21). As shown in FIG. 21, the server device 10 stores the satisfaction history for each passenger as an element, with a pair of tourist spots introduced to the passenger (representative) and the satisfaction level for the tourist spots as elements. For example, when a passenger visits the same tourist spot on another occasion, the server device 10 may identify the passenger's history by biometric authentication and use the information to provide the passenger.

In addition, by using biometric authentication, the server device 10 can also handle seat changes among passengers. For example, it is assumed that passengers get off at a particular tourist spot and then get back on the patrol vehicle 20 at the same spot. In this case, even if the passengers do not sit in the same seat, the server device 10 can continue the conversation with the representative by identifying the representative through biometric authentication.

The server device 10 may analyze the passengers' satisfaction (satisfaction history) and provide useful information to the operator (manager) of the tourist spot or theme park. In this case, the server device 10 includes a history analysis unit 310 as shown in FIG. 22. The history analysis unit 310 generates useful information for the operator and provides the generated information to the operator. For example, the history analysis unit 310 calculates the average satisfaction level for each tourist spot. The history analysis unit 310 generates a graph as shown in FIG. 23 based on the calculated average satisfaction level and presents it to the operator. The operator (manager) who comes across the information as shown in FIG. 23 may review the route of the patrol vehicle 20 or change the information to be provided to passengers. For example, the operator considers measures to improve the satisfaction level of tourist spots with low satisfaction levels, such as tourist spots B and D. Alternatively, the server device 10 may present the passengers' satisfaction level in real time. That is, the server device 10 may display the satisfaction level of the representative of each patrol vehicle 20 in operation in real time. The server device 10 can provide the operator, etc. with information on the degree of satisfaction of passengers with each tourist spot. In this way, the operator can know the passengers' true impressions and satisfaction levels without using a questionnaire or the like to collect satisfaction with tourist spots. Alternatively, the operator can learn of the existence of tourist spots that are attracting the interest of users and that the operator itself is not aware of.

When a user registers in advance as in the third embodiment, the server device 10 may calculate the satisfaction level for each attribute of the user and present it to the operator, etc. For example, the server device 10 may calculate the average satisfaction level by generation and age for each tourist spot and present it to the operator, etc.

In the above embodiment, the server device 10 may provide the operator with information useful for the management of the tourist spot using the satisfaction level (impressions of the guided spots) obtained from the passengers of the patrol vehicle 20. The server device 10 may calculate information that can be used to review the number of patrol vehicles 20, their routes, and the content provided to the patrol vehicles 20 according to the calculated satisfaction level, and provide it to the operator. For example, the server device 10 provides the operator with satisfaction levels for each weather condition. Depending on the information, the operator may increase the number of patrol vehicles 20 on days with high satisfaction weather and reduce the number of patrol vehicles 20 on days with low satisfaction weather. Alternatively, if there is a tourist spot whose satisfaction level changes significantly depending on the weather, the server device 10 provides the operator with the information. The operator may take measures such as setting a tour course that excludes the tourist spots with low satisfaction on days with low satisfaction weather.

The server device 10 may also be capable of handling passengers transferring (changing) between patrol vehicles 20. For example, as shown in FIG. 24, a passenger gets on patrol vehicle 20-1 and travels to a transfer station. At the transfer station, the passenger transfers to either patrol vehicle 20-2, which travels around tourist spots B, C, and D, or patrol vehicle 20-3, which travels around tourist spots E, F, and D. At that time, the server device 10 uses the biometric information of the passenger who has boarded patrol vehicle 20-1 to identify the passenger, and stores the ID of the identified passenger in association with the biometric information. The server device 10 identifies whether the identified passenger has boarded patrol vehicle 20-2 or patrol vehicle 20-3. Specifically, the server device 10 identifies which patrol vehicle 20 the passenger has boarded by biometric authentication using biometric information obtained from image data acquired from patrol vehicle 20-2 or patrol vehicle 20-3 and the stored biometric information. Furthermore, the server device 10 talks with the passenger of the identified patrol vehicle 20 and calculates the satisfaction level of the passenger. The server device 10 stores the calculated satisfaction level in association with the passenger's ID. By doing so, even if the passenger changes patrol vehicles 20, the server device 10 can identify the passenger's transfer destination by biometric authentication and can manage the history of the passenger's satisfaction level.

Alternatively, as described above, when the destination of the patrol vehicle 20 is determined based on the satisfaction level, the server device 10 may provide information (information provided to passengers) to transfer to the patrol vehicle 20 heading to the determined destination. For example, in the example of FIG. 24, if the tourist spot determined based on the satisfaction level of the passenger who boarded the patrol vehicle 20-1 is tourist spot B, the server device 10 (information providing unit 306) may guide (notify) the passenger to transfer to the patrol vehicle 20-2. Also, if the tourist spot determined based on the satisfaction level of the passenger who boarded the patrol vehicle 20-1 is tourist spot E, the server device 10 (information providing unit 306) may guide the passenger to transfer to the patrol vehicle 20-3. In this way, the satisfaction level calculation unit 353 calculates the satisfaction level of the passenger of the patrol vehicle 20-1 (first patrol vehicle). The destination determination unit 308 determines the spot to which the passenger of the patrol vehicle 20-1 (first patrol vehicle) is guided based on the calculated passenger satisfaction level. The information providing unit 306 guides passengers of the patrol vehicle 20-1 (first patrol vehicle) to transfer to the patrol vehicle 20-2 or patrol vehicle 20-3 (second patrol vehicle) traveling to the determined spot.

Alternatively, the server device 10 may output the guidance regarding transfers in the patrol vehicle 20 to a display device such as an LCD monitor, rather than as audio (audio output from the speaker 24). In this case, the patrol vehicle 20 only needs to be equipped with a display device on the partition 21. Also, in this case, the server device 10 may display the above guidance on a display device corresponding to the representative's seat, and may also provide the above guidance on display devices of other seats. In this way, the server device 10 may provide the above guidance taking into account the seat position (boarding position) of the passenger.

In the above embodiment, the case where the server device 10 selects the representative has been described. However, the representative may also be determined by the passenger. For example, the patrol vehicle 20 may output a display simulating the seat arrangement and a display prompting the passenger to select a representative on an LCD monitor or the like, and the passenger in the seat selected by the passenger may be set as the representative. In this way, instead of analyzing image data obtained from the patrol vehicle 20, the server device 10 may obtain the representative using an interface that allows the user to input the representative.

In the above embodiment, a case has been described in which the server device 10 determines the patrol route of the patrol vehicle 20. However, the passenger may select the patrol route. In this case, the patrol vehicle 20 may display multiple patrol routes as "candidate patrol routes" and obtain the passenger's desired route (course). The patrol vehicle 20 transmits the obtained route to the server device 10. The server device 10 controls the patrol vehicle 20 according to the route.

The server device 10 may verify the identity of the user when registering the user. Specifically, the server device 10 acquires an identity document (e.g., a passport) containing biometric information along with the user's biometric information, personal information, etc. The server device 10 performs a one-to-one match using the biometric information on the identity document and the biometric information acquired from the user. If the match is successful, the server device 10 may register the user.

In the above embodiment, a case has been described in which the feature quantities generated from the facial image are stored in the member information database, but the feature quantities do not have to be stored in the database. A facial image may be registered in the member information database instead of the feature quantities, and the server device 10 may generate feature quantities from the registered facial image each time it processes an authentication request.

The form of data transmission and reception between each device (server device 10, patrol vehicle 20) is not particularly limited, but the data transmitted and received between these devices may be encrypted. Biometric information and the like is transmitted and received between these devices, and in order to appropriately protect this information, it is desirable to transmit and receive encrypted data.

In the flow diagrams (flowcharts, sequence diagrams) used in the above explanation, multiple steps (processes) are described in order, but the order of execution of the steps performed in the embodiments is not limited to the order described. In the embodiments, the order of the steps shown in the diagrams can be changed to the extent that does not interfere with the content, such as by executing each process in parallel.

The above embodiments have been described in detail to facilitate understanding of the present disclosure, and it is not intended that all of the configurations described above are required. Furthermore, when multiple embodiments are described, each embodiment may be used alone or in combination. For example, it is possible to replace part of the configuration of an embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of an embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of an embodiment with other configurations.

The above explanation makes clear the industrial applicability of the present invention, and the present invention is particularly applicable to information provision systems that authenticate registered users.

A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.
[Appendix 1]
A communication unit that communicates with a patrol vehicle that patrols within a predetermined area;
a conversation unit that asks the passengers of the traveling vehicle about their impressions of the spots that the traveling vehicle has guided the passengers to and acquires responses from the passengers of the traveling vehicle;
a calculation unit that calculates a degree of satisfaction of the passengers of the traveling vehicle with respect to the guided spot based on the answers from the passengers of the traveling vehicle;
A server device comprising:
[Appendix 2]
The conversation unit transmits voice data corresponding to the question to the patrol vehicle,
The server device according to claim 1, further comprising: a server for receiving, from the vehicle, voice data and image data in response to the question as a response from a passenger of the vehicle.
[Appendix 3]
The server device according to claim 2, wherein the calculation unit calculates the satisfaction level of passengers of the traveling vehicle by inputting voice data and image data of the responses to the questions into a learning model.
[Appendix 4]
A representative determination unit is further provided for determining a representative from among a plurality of passengers when the traveling vehicle has a plurality of passengers;
The server device according to any one of claims 1 to 3, wherein the conversation unit converses with the representative.
[Appendix 5]
The server device according to any one of claims 1 to 4, wherein the conversation unit starts the conversation when the patrol vehicle finishes giving guidance about the spot.
[Appendix 6]
The server device according to any one of claims 1 to 5, wherein the patrol vehicle is an autonomous vehicle, and further comprises a vehicle operation control unit that controls the operation of the patrol vehicle by sending movement instructions to the patrol vehicle, including the destination of the patrol vehicle.
[Appendix 7]
The server device according to any one of claims 1 to 6, further comprising an information providing unit that determines information to be provided to passengers of the traveling vehicle based on the calculated satisfaction level.
[Appendix 8]
The information providing unit is
When the calculated satisfaction level is equal to or greater than a first threshold, the traveling vehicle provides the passenger with information about the spots that have already been guided thereto;
The server device described in Appendix 7, wherein, if the calculated satisfaction level is equal to or lower than a second threshold, the traveling vehicle provides the passenger with information unrelated to the spots to which the passenger has already been guided.
[Appendix 9]
The server device according to any one of claims 1 to 8, further comprising a destination determination unit that determines a spot to which the traveling vehicle should move based on the calculated satisfaction level.
[Appendix 10]
A storage unit is further provided for storing biometric information and attribute information of each of a plurality of users in association with each other,
the representative determination unit identifies attribute information corresponding to a passenger of the traveling vehicle by biometric authentication using biometric information generated from image data received from the traveling vehicle and the stored biometric information;
The server device according to claim 7, which cites claim 4, wherein the information providing unit determines information to be provided to passengers of the traveling vehicle based on the specified attribute information and the calculated satisfaction level.
[Appendix 11]
A patrol vehicle that patrols within a predetermined area;
a server device connected to the patrol vehicle via wireless communication;
Including,
The server device includes:
A communication unit that communicates with the patrol vehicle;
a conversation unit that asks the passengers of the traveling vehicle about their impressions of the spots that the traveling vehicle has guided the passengers to and acquires responses from the passengers of the traveling vehicle;
a calculation unit that calculates a degree of satisfaction of the passengers of the traveling vehicle with respect to the guided spot based on the answers from the passengers of the traveling vehicle;
A system comprising:
[Appendix 12]
A first patrol vehicle and a second patrol vehicle are included.
The calculation unit calculates a satisfaction level of a passenger of the first traveling vehicle,
The server device includes:
a destination determination unit that determines a spot to which the passengers of the first traveling vehicle should be guided based on the calculated satisfaction level of the passengers of the first traveling vehicle;
an information providing unit that guides passengers of the first traveling vehicle to transfer to the second traveling vehicle that moves to the determined spot;
12. The system of claim 11, further comprising:
[Appendix 13]
In the server device,
Communicate with patrol vehicles patrolling within a specified area;
asking the passengers of the traveling vehicle about their impressions of the spots that the traveling vehicle has guided the passengers to, and acquiring responses from the passengers of the traveling vehicle;
A control method for a server device, the method comprising: calculating a degree of satisfaction of the passengers of the traveling vehicle with respect to the guided spot based on the answers from the passengers of the traveling vehicle.
[Appendix 14]
A computer installed in the server device
A process of communicating with a patrol vehicle traveling within a predetermined area;
A process of asking the passengers of the traveling vehicle about their impressions of the spots that the traveling vehicle has guided the passengers to and acquiring responses from the passengers of the traveling vehicle;
A process of calculating a satisfaction level of the passengers of the traveling vehicle regarding the guided spot based on the answers from the passengers of the traveling vehicle;
A computer-readable storage medium that stores a program for executing the above.

The disclosures of the above cited prior art documents are incorporated herein by reference. Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. Those skilled in the art will understand that these embodiments are merely illustrative and that various modifications are possible without departing from the scope and spirit of the present invention. In other words, the present invention naturally includes various modifications and amendments that a person skilled in the art can make in accordance with the entire disclosure, including the scope of the claims, and the technical ideas.

10 Server device 20 Patrol vehicle 20-1 Patrol vehicle 20-2 Patrol vehicle 20-3 Patrol vehicle 21 Partition 21-1 Partition 21-2 Partition 21-3 Partition 21-4 Partition 22 Camera 23 Microphone 24 Speaker 30 Base station 40 Control device 100 Server device 101 Communication unit 102 Conversation unit 103 Calculation unit 201 Communication control unit 202 Operation control unit 203 Position information transmission unit 204 Device control unit 205 Storage unit 301 Communication control unit 302 Vehicle position management unit 303 Representative determination unit 304 Vehicle operation control unit 305 Communication unit 306 Information provision unit 307 Storage unit 308 Destination determination unit 309 User registration unit 310 History analysis unit 311 Processor 312 Memory 313 Input/output interface 314 Communication interface 351 Guidance end detection unit 352 Conversation section 353 Satisfaction level calculation section

Claims (9)

a communication unit that communicates with the first and second patrol vehicles that patrol within a predetermined area;
a conversation unit that asks the passengers of the first traveling vehicle about their impressions of the spots that the first traveling vehicle has guided the passengers to, and acquires responses from the passengers of the first traveling vehicle;
a calculation unit that calculates a degree of satisfaction of the passengers of the first traveling vehicle with respect to the guided spot based on the answers from the passengers of the first traveling vehicle;
a destination determination unit that determines a spot to which the passengers of the first traveling vehicle should be guided based on the calculated satisfaction level of the passengers of the first traveling vehicle;
an information providing unit that guides passengers of the first traveling vehicle to transfer to the second traveling vehicle that moves to the determined spot;
A server device comprising: the conversation unit transmits voice data corresponding to the question to the first patrol vehicle;
2. The server device according to claim 1 , further comprising: a server device configured to receive, from the first vehicle, voice data and image data in response to the question as a response from a passenger of the first vehicle. The server device according to claim 2 , wherein the calculation unit calculates the satisfaction level of the passenger of the first traveling vehicle by inputting the voice data and image data of the response to the question into a learning model. a representative determination unit that, when a plurality of passengers are on board the first traveling vehicle, determines a representative from among the plurality of passengers;
The server device according to claim 1 , wherein the conversation unit converses with the representative. The server device according to claim 1 , wherein the conversation unit starts the conversation at a timing when the first patrol vehicle finishes guiding the spot. The server device according to any one of claims 1 to 5 , further comprising a vehicle operation control unit that controls the operation of the first patrol vehicle by sending a movement instruction including a destination of the first patrol vehicle to the first patrol vehicle, the first patrol vehicle being an autonomous vehicle. a first and a second patrol vehicle that patrol within a predetermined area;
a server device connected to the first and second patrol vehicles via wireless communication;
Including,
The server device includes:
A communication unit that communicates with the first and second patrol vehicles;
a conversation unit that asks the passengers of the first traveling vehicle about their impressions of the spots that the first traveling vehicle has guided the passengers to, and acquires responses from the passengers of the first traveling vehicle;
a calculation unit that calculates a degree of satisfaction of the passengers of the first traveling vehicle with respect to the guided spot based on the answers from the passengers of the first traveling vehicle;
a destination determination unit that determines a spot to which the passengers of the first traveling vehicle should be guided based on the calculated satisfaction level of the passengers of the first traveling vehicle;
an information providing unit that guides passengers of the first traveling vehicle to transfer to the second traveling vehicle that moves to the determined spot;
A system comprising: In the server device,
Communicating with first and second patrol vehicles that patrol within a predetermined area;
Asking the passengers of the first traveling vehicle about their impressions of the spots introduced to the passengers by the first traveling vehicle, and acquiring responses from the passengers of the first traveling vehicle;
Calculating a satisfaction level of the passengers of the first traveling vehicle with respect to the guided spot based on the answers from the passengers of the first traveling vehicle ;
determining a spot to which the passengers of the first traveling vehicle should be guided based on the calculated satisfaction level of the passengers of the first traveling vehicle;
A control method for a server device, which guides passengers of the first traveling vehicle to transfer to the second traveling vehicle moving to the determined spot . A computer installed in the server device
A process of communicating with first and second patrol vehicles patrolling within a predetermined area;
A process of asking the passengers of the first traveling vehicle about their impressions of the spots that the first traveling vehicle has guided the passengers to and acquiring responses from the passengers of the first traveling vehicle;
A process of calculating a satisfaction level of the passengers of the first traveling vehicle with respect to the guided spot based on the answers from the passengers of the first traveling vehicle;
A process of determining a spot to which the passengers of the first traveling vehicle should be guided based on the calculated satisfaction level of the passengers of the first traveling vehicle;
instructing passengers of the first vehicle to transfer to the second vehicle traveling to the determined spot;
A program for executing.

Images