JP7605582B2 - Object guidance control device, object guidance control method, and program - Google Patents

Description

The present invention relates to a target guidance control device, a target guidance control method, and a program.

Traditionally, guidance systems that provide route guidance and guidance to surrounding facilities in vehicles have become widespread. Car navigation devices equipped with such guidance systems are often used for route guidance in vehicles such as automobiles. Such guidance systems are also used to provide tourist information to tourists.

For example, the following Patent Document 1 discloses a guide system that can automatically provide tourist information to tourists even when there is no guide such as a bus guide on board the vehicle. In this technology, when an imaging device that is installed to be able to capture images of the outside of the vehicle captures an image of a target for tourist information, tourist information is provided by audio output based on previously prepared audio data.

In recent years, with the spread of smartphones, applications that can provide route guidance and guidance to nearby facilities on smartphones have also become widespread. As a result, car navigation devices are sometimes substituted with smartphones. Furthermore, with the development of voice recognition technology, speaker-type voice interface devices (e.g., smart speakers) that can be controlled by voice are also becoming popular. Therefore, in the future, car navigation devices may be replaced by smart speakers, which may provide route guidance and guidance to facilities around the vehicle based on the user's voice input.

JP 2017-194898 A

However, when a conventional smart speaker is installed inside a moving vehicle, the smart speaker can obtain information inside the vehicle, but has difficulty obtaining information outside the vehicle. For example, if a user specifies a landmark outside the vehicle as a guidance target in a manner that does not specify the name of the landmark, such as "What is that building?", the smart speaker can recognize that the user has requested guidance. However, it is difficult for the smart speaker to recognize which landmark outside the vehicle the user specified as the guidance target. Therefore, the smart speaker may provide erroneous guidance because it has difficulty correctly recognizing the landmark specified by the user.

In view of the above problems, the object of the present invention is to provide a target guidance control device, a target guidance control method, and a program that can improve the accuracy of guidance regarding a target designated by a user in a vehicle as a target for which guidance is requested.

In order to solve the above-mentioned problems, a target guidance control device according to one embodiment of the present invention includes a detection unit that detects candidates for a guidance target designated by a user based on the voice of a user riding in a vehicle, an in-vehicle image in which a pose of the user is captured to designate a guidance target for which the user requests guidance, and an image of the surroundings of the vehicle; an identification unit that identifies a position and orientation of the vehicle in spatial information around the vehicle based on position information of the vehicle, identifies a direction of the guidance target in the spatial information based on the identified orientation of the vehicle, determines whether the guidance target can be uniquely determined from the candidates based on the identified direction, identifies the guidance target if it can be determined, and supplies the voice spoken by the user to the detection unit by prompting the user to speak to narrow down the guidance targets if it cannot be determined ; and an output control unit that outputs output information related to the guidance target based on the spatial information corresponding to the identified guidance target.

A target guidance control method according to one aspect of the present invention includes: detecting candidates for a guidance target designated by a user based on a voice of a user riding in a vehicle, an in-vehicle image in which a pose of the user is captured to designate a guidance target for which the user requests guidance, and an image of the surroundings of the vehicle; identifying a position and orientation of the vehicle in spatial information around the vehicle based on position information of the vehicle, identifying a direction of the guidance target in the spatial information based on the identified orientation of the vehicle , determining whether the guidance target can be uniquely determined from the candidates based on the identified direction , identifying the guidance target if it is possible to determine it, and supplying the voice spoken by the user to the detection unit by prompting the user to speak to narrow down the guidance targets if it is not possible to determine the guidance target, and is executed by a processor.

A program according to one aspect of the present invention causes a computer to function as: a detection unit that detects candidates for a guidance target designated by a user based on the voice of a user riding in a vehicle, an in-vehicle image in which a pose of the user is captured to designate a guidance target for which the user requests guidance, and an image of the surroundings of the vehicle; an identification unit that identifies a position and orientation of the vehicle in spatial information around the vehicle based on position information of the vehicle, identifies a direction of the guidance target in the spatial information based on the identified orientation of the vehicle, determines whether the guidance target can be uniquely determined from the candidates based on the identified direction, identifies the guidance target if it can be determined, and supplies the voice spoken by the user to the detection unit by prompting the user to speak to narrow down the guidance targets if it cannot be determined; and an output control unit that outputs output information related to the guidance target based on the spatial information corresponding to the identified guidance target.

The present invention can improve the accuracy of guidance regarding a target specified by a user in a vehicle for which guidance is requested.

1 is a diagram showing an overview of an object guidance control system according to an embodiment of the present invention. 2 is a block diagram showing a configuration example of a target guidance control system according to the embodiment. FIG. 11 is an explanatory diagram of an example of detection of candidates for guidance targets according to the embodiment; FIG. 11 is an explanatory diagram of an example of association between vehicle position information and a three-dimensional map according to the embodiment; FIG. 10 is a flowchart showing a process flow in the object guidance control system according to the embodiment. 13 is a flowchart showing a flow of a guidance target specification process according to the embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
1 is a diagram showing an overview of an object guidance control system 1 according to an embodiment of the present invention. The object guidance control system 1 is composed of an object guidance control device 10, a vehicle 20, and a network 30 shown in FIG.

The target guidance control device 10 is a device that controls guidance (hereinafter also referred to as "guidance control") regarding a guidance target designated by a user 40 of a vehicle 20 as a target for which guidance is requested. The target guidance control device 10 is realized, for example, by a computer such as a server device.

As shown in FIG. 1, the object guidance control device 10 is connected to the vehicle 20 via a network 30, and is capable of transmitting and receiving information to and from the vehicle 20. The object guidance control device 10 according to this embodiment performs guidance control based on information received from the vehicle 20 via the network 30. The object guidance control device 10 also transmits information related to guidance control to the vehicle 20 via the network 30.

The vehicle 20 is a moving body that moves with a passenger (e.g., a user 40) on board. As an example, the vehicle 20 according to this embodiment is realized as an autonomous vehicle. When the vehicle 20 is an autonomous vehicle, the user 40 does not necessarily need to drive it. The vehicle 20 may be a non-autonomous vehicle.

The vehicle 20 is provided with a sensor device for acquiring various information. For example, as shown in FIG. 1, the vehicle 20 is provided with a microphone 212, an in-vehicle camera 214a, an in-vehicle camera 214b, an in-vehicle camera 214c, an exterior camera 216a, an exterior camera 216b, and a speaker 242. The microphone 212 is provided so as to be able to acquire sounds generated inside the vehicle 20. The in-vehicle cameras 214a to 214c (hereinafter also referred to as "in-vehicle cameras 214") are provided so as to be able to capture images of the interior of the vehicle 20. For example, as shown in FIG. 1, the in-vehicle cameras 214 are provided at the upper front, upper center, and upper rear of the vehicle 20, respectively, and capture images (still images or moving images) of the interior of the vehicle 20 from each position. The exterior camera 216a and the exterior camera 216b (hereinafter also referred to as "exterior camera 216") are provided so as to be able to capture images of the exterior of the vehicle 20. For example, as shown in FIG. 1, the exterior cameras 216 are provided at the front and rear of the vehicle 20, respectively, and capture images of the outside of the vehicle 20 from their respective positions. Note that the type, number, and positions of the sensor devices provided on the vehicle 20 are not limited to the example shown in FIG. 1.

The vehicle 20 is also provided with an output device for outputting output information. For example, as shown in FIG. 1, a speaker 242 is provided in the vehicle 20. The speaker 242 is provided so as to be able to output information relating to a guidance target (hereinafter also referred to as "guidance information") as output information toward the interior of the vehicle 20. Note that the type, number, and location of the output device provided in the vehicle 20 are not limited to the example shown in FIG. 1. In addition, the output device may be a terminal owned by the user 40. The terminal owned by the user 40 is, for example, a smartphone, a tablet terminal, a hearable terminal, etc.

The guidance target is a landmark designated by the user 40 among the landmarks existing around the vehicle 20. As shown in Fig. 1, there are buildings 50 and 51 as landmarks around the vehicle 20. As seen from the user 40 inside the vehicle 20 traveling in the direction of the arrow 22, the building 50 is in front of the user 40 to the right, and the building 51 is in front of the user 40 to the left. Note that the landmarks are not limited to buildings.
The guidance information includes introduction information of the guidance target, route information to the guidance target, and the like.

In FIG. 1, the user 40 utters "the building ahead on the right" as shown in speech bubble 42, specifying the guidance target by voice. At the same time, the user 40 specifies the guidance target by pointing with his/her finger in the direction of arrow 44 (to the front right as seen by the user 40). As a result, the guidance target in FIG. 1 is identified as building 50. Note that the user 40 may utter a question when specifying the guidance target. For example, the user 40 may utter, "What building is ahead on the right?"

The process of identifying such a guidance target (hereinafter also referred to as "guidance target identification process") is performed by the target guidance control device 10 based on the voice acquired by the vehicle 20 and the action of the user 40 designating the guidance target. The action of the user 40 designating the guidance target is, for example, a pointing action with a finger, a pointing action with a rod-shaped object (e.g., a pointer or pen), or a pointing action with the entire arm. The target guidance control device 10 may also perform the guidance target identification process using information indicating the direction of the user 40's face and the direction of their gaze. Details of the guidance target identification process will be described later.

The object guidance control device 10 may be realized as a device that can have some or all of its components installed in the vehicle 20. The components of the object guidance control device 10 installed in the vehicle 20 can be connected to each device installed in the vehicle 20 by wire or wirelessly so as to be able to send and receive information with each device. On the other hand, the components of the object guidance control device 10 that are not installed in the vehicle 20 can be realized by, for example, a server device, and can communicate with the components of the object guidance control device 10 installed in the vehicle 20 and each device installed in the vehicle 20 via the network 30.

The outline of this embodiment has been described above. Next, an example of the configuration of the object guidance control system 1 according to this embodiment will be described. FIG. 2 is a block diagram showing an example of the configuration of the object guidance control system 1 according to this embodiment. Note that, below, an example of the configuration of the object guidance control system 1 in the case where not all of the components of the object guidance control device 10 are provided in the vehicle 20, but are realized by a server device or the like, will be described.

(Configuration example of vehicle 20)
As shown in FIG. 2 , the vehicle 20 includes an acquisition unit 210 , a communication unit 220 , a control unit 230 , and a guidance information output unit 240 .

(Acquisition unit 210)
The acquisition unit 210 has a function of acquiring various information. This function can be realized by various sensor devices. The acquisition unit 210 according to this embodiment includes, for example, sensor devices such as the above-mentioned microphone 212, the in-vehicle camera 214, and the outside-vehicle camera 216. The acquisition unit 210 may also include a sensor device that acquires position information of the vehicle 20. As an example, the acquisition unit 210 includes a GPS (Global Positioning System). After acquiring the sensing information by these sensor devices, the acquisition unit 210 outputs the sensing information to the communication unit 220.

The sensor device equipped in the acquisition unit 210 is not limited to this example. For example, the acquisition unit 210 may be equipped with a distance measurement sensor such as LiDAR (Laser Imaging Detection and Ranging) as a sensor device that acquires information indicating the state inside and outside the vehicle 20.

(Communication unit 220)
The communication unit 220 has a function of communicating with an external device. The communication unit 220 outputs information received through communication with the external device to the control unit 230. As an example, the communication unit 220 outputs guidance information received from the object guidance control device 10 via the network 30 to the control unit 230. The communication unit 220 also transmits information input from the acquisition unit 210 to the object guidance control device 10. As an example, the communication unit 220 transmits sensing information input from the acquisition unit 210 to the object guidance control device 10 via the network 30.

(Control unit 230)
The control unit 230 has a function of controlling the overall operation of the vehicle 20. The control unit 230 is realized, for example, by causing a central processing unit (CPU) provided as hardware in the vehicle 20 to execute a program. For example, the control unit 230 controls the operation of each sensor device provided in the acquisition unit 210. In addition, the control unit 230 controls the output of guidance information input from the communication unit 220 to the guidance information output unit 240. In addition, when the vehicle 20 is an autonomous vehicle, the control unit 230 controls autonomous driving of the vehicle 20.

(Guidance Information Output Unit 240)
The guidance information output unit 240 has a function of outputting guidance information. The guidance information may be input from the control unit 230, for example, and output by various output devices. For example, if the guidance information output unit 240 includes an audio output device such as a speaker, the guidance information output unit 240 outputs the guidance information converted into audio from the speaker. Also, if the guidance information output unit 240 includes a display device such as a display, the guidance information output unit 240 displays the guidance information converted into an image on the display.

(Configuration example of the object guidance control device 10)
As shown in FIG. 2 , the object guidance control device 10 includes a communication unit 110, a control unit 120, and a storage unit 130.

(Communication unit 110)
The communication unit 110 has a function of communicating with an external device. The communication unit 110 outputs information received through communication with the external device to the control unit 120. As an example, the communication unit 110 outputs sensing information received from the vehicle 20 via the network 30 to the control unit 120. The communication unit 110 also transmits information input from the control unit 120 to the vehicle 20. As an example, the communication unit 110 transmits guidance information input from the control unit 120 to the vehicle 20 via the network 30.

(Control unit 120)
The control unit 120 has a function of controlling the overall operation of the object guidance control device 10. The control unit 120 is realized, for example, by causing a CPU provided as hardware in the object guidance control device 10 to execute a program. To realize this function, the control unit 120 includes a detection unit 122, an identification unit 124, and an output control unit 126.

(Detection Unit 122)
The detection unit 122 has a function of detecting candidates for guidance targets. For example, the detection unit 122 detects candidates for guidance targets based on the voice of the user 40 riding in the vehicle 20, the action of the user 40 specifying the guidance target, and a captured image of the surroundings of the vehicle 20. Specifically, the detection unit 122 detects a group of candidates for guidance targets by analyzing the captured image, and detects candidates that exist in a direction from the group of candidates based on a natural language related to the guidance target detected by analyzing the voice and a direction detected by analyzing the action of the user 40.

Here, an example of detection of a candidate for guidance will be described with reference to FIG. 3. FIG. 3 is an explanatory diagram of an example of detection of a candidate for guidance according to this embodiment. First, the detection unit 122 detects natural language from the voice by analyzing the voice acquired by the microphone 212 of the vehicle 20. When a voice command indicating that the user 40 is requesting guidance is detected from the detected natural language, the detection unit 122 further analyzes the voice and detects natural language related to the guidance target. For example, the detection unit 122 detects the voice "building in front to the right" shown in the speech bubble 42 as natural language related to the guidance target. In the example shown in FIG. 3, the user 40 is facing the direction of the arrow 22, which is the traveling direction of the vehicle 20, so the detection unit 122 can detect from the detected natural language that the user 40 is pointing to the building 50 in front to the right of the vehicle 20.

Furthermore, the detection unit 122 detects that the user 40 is pointing in the direction of the arrow 44 by analyzing the pose of the user 40 captured in the captured image inside the vehicle 20 acquired by the in-vehicle camera 214 of the vehicle 20. The detection unit 122 also detects buildings 50 and 51 shown in FIG. 3 as a group of candidates for guidance targets by analyzing the scenery outside the vehicle captured in the captured image outside the vehicle acquired by the exterior camera 216 of the vehicle 20. From the above detection results, the detection unit 122 detects building 50, which is located to the right of the vehicle 20 in the direction of the arrow 44, as a candidate for guidance targets from the group of candidates.

As described above, the detection unit 122 can link the group of candidate guidance targets detected from the scenery outside the vehicle 20 with the guidance target specified by the user 40 based on the voice of the user 40 inside the moving vehicle 20 and the actions of the user 40. With this configuration, the detection unit 122 can detect landmarks outside the moving vehicle 20 as candidate guidance targets. In addition, when detecting candidate guidance targets, the detection unit 122 uses both the voice of the user 40 inside the vehicle 20 and the actions of the user 40, and therefore can improve the accuracy of detection of candidate guidance targets. Note that the detection unit 122 may detect one candidate guidance target from the group of candidates, or may detect multiple candidate guidance targets.

(Identification unit 124)
The identification unit 124 has a function of identifying a guidance target designated by the user 40 from among the candidate guidance targets detected by the detection unit 122. For example, the identification unit 124 identifies the position and orientation of the vehicle 20 in spatial information around the vehicle 20 based on the position information of the vehicle 20, identifies the direction of the guidance target in the spatial information based on the identified orientation of the vehicle 20, and identifies the guidance target from the candidate guidance targets based on the identified direction. The spatial information around the vehicle 20 according to this embodiment is, for example, a three-dimensional map around the vehicle 20, landmark information including metadata of landmarks on the three-dimensional map, and the like. Note that the spatial information is not limited to such an example.

Here, the correspondence between the position information of the vehicle 20 and the three-dimensional map will be described with reference to FIG. 4. FIG. 4 is an explanatory diagram of an example of the correspondence between the position information of the vehicle 20 and the three-dimensional map according to this embodiment. As shown in FIG. 4, the three-dimensional map 60 has three-dimensional data of landmarks such as building 50, building 51, and mountain 52.

First, the identification unit 124 identifies the position and orientation of the vehicle 20 on a three-dimensional map from the position information acquired by the GPS of the vehicle 20, and maps the vehicle 20 on the three-dimensional map. The orientation of the vehicle 20 may be identified, for example, based on sensing information from a gyro sensor or the movement history of the vehicle 20. Next, the identification unit 124 identifies the direction of the guidance target specified by the user 40 on the three-dimensional map based on the orientation of the vehicle 20 indicated by the arrow 22. Then, the identification unit 124 identifies, from among the candidates for the guidance target, a building 50 that is a candidate located in the identified direction (arrow 44) as the guidance target.

As described above, the identification unit 124 can uniquely identify the guidance target specified by the user 40 on the 3D map from the candidate guidance targets detected by the detection unit 122 based on the position information of the vehicle 20 and the 3D map. With this configuration, when the detection unit 122 detects one candidate guidance target, the identification unit 124 can increase the certainty that the target is a guidance target. Furthermore, when the detection unit 122 detects multiple candidate guidance targets, the identification unit 124 can narrow down the guidance targets. This improves the accuracy of identifying the guidance target, and therefore makes it possible to improve the accuracy of guidance regarding the guidance target specified by the user 40.

(Output control unit 126)
The output control unit 126 has a function of controlling the output in the vehicle 20. For example, the output control unit 126 causes an output device of the vehicle 20 to output output information related to the identified guidance target based on spatial information corresponding to the identified guidance target. Specifically, the output control unit 126 first acquires metadata of the identified guidance target from the landmark information, and generates guidance information based on the metadata. Then, the output control unit 126 causes the communication unit 110 to transmit the generated guidance information to the vehicle 20, and causes an output device of the vehicle 20 to output the guidance information.

When the guidance target is a building, the metadata is, for example, the building's name, address, number of floors, tenant information for each floor, etc. Based on this metadata, the output control unit 126 generates an introduction text, for example, "That building is the AAA Building," as the introduction information. Then, the output control unit 126 causes the output device to output the introduction information according to the output device of the vehicle 20. For example, when the output device is an audio output device, the output control unit 126 converts the introduction text into audio and outputs it to the audio output device. Also, when the output device is a display device, the output control unit 126 converts the introduction text into an image and outputs it to the display device. With this configuration, flexible guidance can be provided to the user 40 according to the identified guidance target and the acquired landmark information, based on the landmark information acquired after the guidance target is identified.

When generating the guidance information, the output control unit 126 may generate the guidance information according to the status of the vehicle 20 and the user 40, and the profile information of the user 40. The status of the vehicle 20 is, for example, the position, direction, speed, and moving route of the vehicle 20. The status of the user 40 is, for example, the posture of the user 40, and the position of the user 40 in the vehicle. The profile information of the user 40 is, for example, the name, sex, height, weight, hobbies, and usage history of the target guidance control system of the user 40. Based on the profile information of the user 40, the output control unit 126 generates, for example, an introduction text such as "That building is the AAA building. Inside the building is the BBB cafe that we introduced previously." as the introduction information. With this configuration, flexible guidance can be provided to the user 40 according to the status of the vehicle 20 and the user 40, and the profile information of the user 40.

(Memory unit 130)
The storage unit 130 has a function of storing various information. The storage unit 130 is configured by a storage medium, for example, a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read/write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 130 may be, for example, a non-volatile memory.

The storage unit 130 stores spatial information about the surroundings of the vehicle 20, such as a three-dimensional map and landmark information. The storage unit 130 may also store sensing information acquired by the acquisition unit 210 of the vehicle 20, landmarks identified as guidance targets by the identification unit 124 of the target guidance control device 10, output information generated by the output control unit 126, profile information of the user 40, etc.

The above describes an example configuration of the object guidance control system 1 in the case where not all of the components of the object guidance control device 10 are provided in the vehicle 20 but are realized by a server device or the like. Note that the example configuration of the object guidance control system 1 is not limited to this example. For example, in the case where all of the components of the object guidance control device 10 are provided in the vehicle 20, at least one of the communication unit 110 or the communication unit 220 may be provided in the vehicle 20 as a component having a function of communicating with an external device.

(Processing flow in the object guidance control system 1)
Next, a process flow in the object guidance control system 1 according to the present embodiment will be described. Fig. 5 is a flowchart showing a process flow in the object guidance control system 1 according to the present embodiment. Note that, below, an example of a process flow in the object guidance control system 1 in the case where not all of the components of the object guidance control device 10 are provided in the vehicle 20 but are realized by a server device or the like will be described.

First, the microphone 212 of the vehicle 20 captures voice within the vehicle 20 (S102). The voice captured by the microphone 212 is transmitted to the target guidance control device 10 via the network 30. When the microphone 212 captures the voice, the control unit 230 of the vehicle 20 turns on the in-vehicle camera 214 of the vehicle 20 (S104). The control unit 230 also turns on the exterior camera 216 of the vehicle 20 (S106).

Next, when the user 40 speaks (S108), the microphone 212 acquires voice including the user's utterance, and the voice is transmitted to the object guidance control device 10 via the network 30. Next, the control unit 120 of the object guidance control device 10 determines whether the received voice contains a voice command requesting guidance (S110). If the voice does not contain a voice command (S110/NO), the object guidance control system 1 performs the processes of S108 and S110 until the voice contains a voice command. If the voice contains a voice command (S110/YES), the control unit 120 performs a guidance target identification process (S112). Details of the guidance target identification process will be described later.

After the guidance target identification process, the control unit 120 records the landmark identified as the guidance target (S114). Next, the control unit 120 acquires metadata of the landmark from the 3D map (S116). Next, the control unit 120 generates an introduction text according to the situation of the vehicle 20 and the user 40 and the profile information of the user 40 (S118). After generating the introduction text, the control unit 120 converts the introduction text into audio and reads out the converted audio from the speaker 242 of the vehicle 20 (S120). After reading out the introduction text, the control unit 120 determines whether there is a next dialogue (S122). If there is a next dialogue (S122/YES), the control unit 120 repeats the process from S110. If there is no next dialogue (S122/NO), the control unit 120 ends the process until the next voice is acquired.

(Flow of the guidance target specification process)
Next, the details of the guidance target specification process of S112 will be described below. Fig. 6 is a flowchart showing the flow of the guidance target specification process according to this embodiment.

First, the control unit 120 analyzes the natural language contained in the voice (S202). Next, the control unit 120 analyzes the pose of the user 40 shown in the image captured by the in-vehicle camera 214 and received from the vehicle 20 via the network 30 (S204). Next, the control unit 120 analyzes the scenery outside the vehicle shown in the image captured by the exterior camera 216 and received from the vehicle 20 via the network 30 (S206).

After analyzing the natural language, the pause, and the scenery outside the vehicle, the control unit 120 determines whether or not the guidance target has been detected (S208). If the guidance target has not been detected (S208/NO), the control unit 120 prompts the user 40 to speak again (S220).

If the guidance target is detected (S208/YES), the control unit 120 acquires the position and orientation of the vehicle 20 (S210). Next, the control unit 120 associates the position and orientation of the vehicle 20 with the three-dimensional map (S212). Next, the control unit 120 associates the orientation of the pose of the user 40 with the orientation of the vehicle 20 (S214). Next, the control unit 120 associates a captured image showing the scenery outside the vehicle 20 with the three-dimensional map (S216).

After each correspondence, the control unit 120 determines whether the guidance target can be univocally determined on the three-dimensional map (S218). If the guidance target cannot be univocally determined (S218/NO), the control unit 120 prompts the user 40 to speak to narrow down the guidance targets (S222). If the guidance target can be univocally determined (S218/YES), the control unit 120 ends the guidance target identification process.

The above describes an example of the processing flow in the object guidance control system 1 when not all of the components of the object guidance control device 10 are provided in the vehicle 20, but are realized by a server device or the like. Note that the processing flow in the object guidance control system 1 is not limited to this example. For example, when all of the components of the object guidance control device 10 are provided in the vehicle 20, the object guidance control system 1 can perform processing without going through the network 30.

As described above, the object guidance control device 10 according to this embodiment links a group of candidate guidance targets detected from the scenery outside the vehicle 20 with a guidance target designated by the user 40, based on the voice and actions of the user 40 inside the moving vehicle 20. This allows the object guidance control device 10 to detect landmarks outside the moving vehicle 20 as candidate guidance targets.
Furthermore, the object guidance control device 10 according to the present embodiment detects candidates for guidance targets using both the voice of the user 40 inside the vehicle 20 and the actions of the user 40. This allows the object guidance control device 10 to improve the accuracy of detection of guidance targets.
Furthermore, the object guidance control device 10 according to this embodiment uniquely identifies the guidance object on the 3D map based on the position information of the vehicle 20 and the 3D map. This enables the object guidance control device 10 to improve the accuracy of identifying the guidance object, and further improve the accuracy of guidance regarding the guidance object designated by the user 40.
Therefore, the target guidance control device 10 according to this embodiment can improve the accuracy of guidance regarding a guidance target designated by the user 40 in the vehicle 20 as a target for which guidance is requested.

(Modification)
The embodiments of the present invention have been described above. Next, modified examples of the embodiments of the present invention will be described. Each modified example described below may be applied alone to the embodiments of the present invention, or may be applied in combination to the embodiments of the present invention. Furthermore, each modified example may be applied in place of the configuration described in the embodiments of the present invention, or may be applied in addition to the configuration described in each embodiment of the present invention.

(First Modification)
In the above-described embodiment, an example has been described in which the target guidance control device 10 controls guidance regarding a guidance target specified by the user 40, but the present invention is not limited to such an example. If there is a landmark around the vehicle 20 that is likely to interest the user 40, the target guidance control device 10 may control guidance to the landmark as a guidance target even if it is not specified by the user 40. The landmark that is likely to interest the user 40 is detected based on, for example, position information of the vehicle 20 or profile information of the user 40. With such a configuration, flexible guidance can be provided to the user 40 according to the user 40.

(Second Modification)
In the above-described embodiment, an example has been described in which the object guidance control device 10 causes the output device of the vehicle 20 to output guidance information, but the present invention is not limited to such an example. For example, when a voice command indicating that the user 40 is requesting a route change is detected from the natural language detected by the detection unit 122, the guidance information may be output to the control unit 230 of the vehicle 20. At this time, route information indicating the changed route is input to the control unit 230 as the guidance information. Then, the control unit 230 controls the automatic driving of the vehicle 20 based on the input route information so that the vehicle 20 moves along the changed route.

(Third Modification)
In the above embodiment, an example in which the output control unit 126 generates guidance information based on metadata has been described, but the present invention is not limited to this example. The output control unit 126 may obtain information on the guidance target from a web service and generate guidance information based on the obtained information.

The above describes a modified embodiment of the present invention. The object guidance control device 10 in the above embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in such a case. Furthermore, the above program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

The above describes in detail an embodiment of the present invention with reference to the drawings, but the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the present invention.

1... object guidance control system, 10... object guidance control device, 20... vehicle, 30... network, 110... communication unit, 120... control unit, 122... detection unit, 124... identification unit, 126... output control unit, 130... storage unit, 210... acquisition unit, 212... microphone, 214... in-vehicle camera, 216... outside-vehicle camera, 220... communication unit, 230... control unit, 240... guidance information output unit

Claims (5)

a detection unit that detects candidates for a guidance target designated by a user based on a voice of the user riding in the vehicle, an image captured inside the vehicle in which a pose of the user designating a guidance target for which the user requests guidance, and an image captured around the vehicle;
an identification unit that identifies a position and a direction of the vehicle in spatial information around the vehicle based on position information of the vehicle, identifies a direction of the guidance target in the spatial information based on the identified direction of the vehicle, determines whether the guidance target can be uniquely determined from the candidates based on the identified direction, identifies the guidance target if it can be determined, and supplies a voice uttered by the user to the detection unit by prompting the user to speak to narrow down the guidance targets if it cannot be determined;
an output control unit that outputs output information related to the identified guidance target based on the spatial information corresponding to the identified guidance target;
An object guidance control device comprising: The object guidance control device according to claim 1, wherein the detection unit detects a group of candidates for the guidance target by analyzing the captured image, and detects the candidate that exists in the direction from the group of candidates based on a natural language related to the guidance target detected by analyzing the voice and the direction detected by analyzing a pose of the user included in the captured image inside the vehicle. The target guidance control device according to claim 1 or 2, wherein the output control unit acquires metadata of the identified guidance target from the spatial information and generates the output information based on the metadata. Detecting candidates for a guidance target designated by a user based on a voice of the user riding in the vehicle, an in-vehicle captured image in which a pose of the user designating a guidance target for which the user requests guidance is captured, and a captured image of the surroundings of the vehicle;
specifying a position and orientation of the vehicle in spatial information around the vehicle based on position information of the vehicle, specifying a direction of the guidance target in the spatial information based on the specified orientation of the vehicle, determining whether or not the guidance target can be uniquely determined from the candidates based on the specified direction, specifying the guidance target if it can be determined, and supplying a voice uttered by the user to the detection unit by prompting the user to speak to narrow down the guidance targets if it cannot be determined ;
outputting output information related to the identified guidance target based on the spatial information corresponding to the guidance target;
23. An object guidance control method executed by a processor, comprising: Computer,
a detection unit that detects candidates for a guidance target designated by a user based on a voice of the user riding in the vehicle, an image captured inside the vehicle in which a pose of the user designating a guidance target for which the user requests guidance, and an image captured around the vehicle;
an identification unit that identifies a position and a direction of the vehicle in spatial information around the vehicle based on position information of the vehicle, identifies a direction of the guidance target in the spatial information based on the identified direction of the vehicle, determines whether the guidance target can be uniquely determined from the candidates based on the identified direction, identifies the guidance target if it can be determined, and supplies a voice uttered by the user to the detection unit by prompting the user to speak to narrow down the guidance targets if it cannot be determined;
an output control unit that outputs output information related to the identified guidance target based on the spatial information corresponding to the identified guidance target;
A program to function as a

Images