JP7597379B2 - IMAGE DISPLAY SYSTEM, IMAGE DISPLAY METHOD, AND COMPUTER PROGRAM - Google Patents

Description

The technology disclosed in this specification relates to an image display system.

In recent years, the development of autonomous driving technology for automobiles and other vehicles has progressed rapidly. Autonomous driving technology for vehicles is a technology that recognizes the vehicle's position and surrounding environment using various sensors (cameras, radar, LIDAR, ultrasonic sensors, GPS, etc.) and high-precision three-dimensional maps, etc., and allows the vehicle to drive autonomously to a destination.

When autonomous vehicle technology has improved sufficiently to the point where the system will perform all driving operations such as acceleration, steering, and braking under certain circumstances (e.g., when driving on a highway) or under all circumstances, and the driver will no longer need to perform driving operations, the driver, as a passenger, will be able to devote his or her time to things other than driving operations. For example, the passenger will be able to watch videos or play games using an image display system (infotainment system) including an in-vehicle display used in a car navigation system or an image display device such as a smartphone, tablet device, or HMD (Head Mounted Display) brought by the passenger (see, for example, Patent Documents 1 to 3).

JP 2017-49901 A JP 2017-97921 A JP 2017-81256 A

In an image display system for passengers of an autonomous vehicle, for example, when introducing tourist attractions visible from the vehicle window, it is sometimes necessary to display on the image display device an image corresponding to the direction while indicating the direction (for example, an image showing information for introducing an object (for example, a historical building) located in the direction (for example, the name, size, year of construction, etc. of the building). In such a case, even if an image (for example, an image of an arrow pointing right) is displayed on the screen of an in-vehicle display, smartphone, or tablet terminal, or a sound (for example, a sound saying "Look to your right") is output from a speaker, it is difficult to convey the exact direction that the passenger should look in. In addition, it is possible to convey the exact direction by using a wearable image display device such as an HMD, but wearable image display devices are cumbersome to wear and have an unnatural appearance when worn. Thus, in an image display system including a conventional image display device, it is difficult to display an image corresponding to the direction while indicating the exact direction on the image display device without using a wearable image display device, and there is room for improvement in terms of convenience.

Note that this issue is not limited to image display systems installed in autonomous vehicles, but is a common issue for all image display systems that are equipped with image display devices.

This specification discloses technology that can solve the above-mentioned problems.

The technology disclosed in this specification can be realized, for example, in the following forms:

(1) The image display system disclosed in this specification is an image display system comprising an image display device, a support device that supports the image display device and is capable of changing the orientation of the image display device, and a control device that controls the image display device and the support device, and the control device has a spatial information acquisition unit that acquires spatial information including first position information indicating the position of the image display device and second position information indicating the position of an object around the image display device, an orientation control unit that causes the support device to change the orientation of the image display device based on the spatial information, and an image display control unit that causes the image display device to display an image corresponding to the orientation of the image display device. In this image display system, the orientation of the image display device is changed based on spatial information including the first position information indicating the position of the image display device and the second position information indicating the position of an object around the image display device, and an image corresponding to the orientation of the image display device is displayed on the image display device. Therefore, according to this image display system, it is possible to display an image corresponding to the direction on the image display device while indicating an accurate direction to a user based on the orientation of the image display device without using a wear-side image display device, thereby improving the convenience of the image display system.

(2) In the above image display system, the control device may further have a target setting unit that sets one of the surrounding objects as a target object based on the spatial information, and the orientation control unit may be configured to cause the support device to change the orientation of the image display device so that the image display device is oriented to point to the target object. According to this image display system, the direction of the user's line of sight can be guided to the target object, which is one of the surrounding objects, depending on the orientation of the image display device, thereby further improving the convenience of the image display system.

(3) In the above image display system, the control device may further have a target information acquisition unit that acquires target information, which is information related to the target object, and the image display control unit may be configured to cause the image display device to display an image indicating the target information. According to this image display system, the direction of the user's line of sight can be guided to a target object, which is one of the surrounding objects, depending on the orientation of the image display device, and an image indicating information related to the target object can be displayed on the image display device, thereby further improving the convenience of the image display system.

(4) In the above image display system, the target information may be configured to include information indicating at least one of the name, size, and construction year of the target object. According to this image display system, an image showing information indicating at least one of the name, size, and construction year of the target object can be displayed on the image display device, and useful information can be presented to the user.

(5) In the above image display system, the support device may support the image display device so that the orientation of the image display device can be manually changed, and the target setting unit may be configured to set an object located in the direction indicated by the image display device as the target object when the orientation of the image display device is manually changed. According to this image display system, when a user points the image display device in the direction of a desired object, the object is set as the target object, so that the target object can be set by the user's intuitive operation, and the convenience of the image display system can be further improved.

(6) In the above image display system, the image display device may have a camera that captures an image of a subject located in a direction pointed to by the image display device, and the image display control unit may be configured to cause the image display device to display an image captured by the camera when the orientation of the image display device is manually changed. According to this image display system, the current orientation of the image display device can be fed back to the user by the image displayed on the image display device, so that the user can easily manually change the orientation of the image display device so that the image display device accurately points to a desired object, thereby further improving the convenience of the image display system.

(7) In the above image display system, the image display device may have a planar image display surface, and the direction indicated by the image display device may be perpendicular to the image display surface and point in a direction opposite to the image display surface. This image display system allows a user to more accurately grasp the direction indicated by the image display device, thereby further improving the convenience of the image display system.

(8) In the above image display system, the support device may be configured to have three or more rotation axes. According to this image display system, the orientation of the image display device can be adjusted more flexibly, thereby further improving the convenience of the image display system.

(9) In the above image display system, the image display system may be mounted on an autonomous vehicle, the support device may support the image display device on the vehicle, and the spatial information acquisition unit may acquire the spatial information obtained by a sensor installed in the vehicle. According to this image display system, by changing the orientation of the image display device using information used for autonomous driving, it is possible to display an image corresponding to the direction on the image display device while indicating an accurate direction related to the traveling of the vehicle to passengers of the autonomous vehicle and people outside the vehicle, thereby improving the convenience of the image display system.

The technology disclosed in this specification can be realized in various forms, such as an image display device, an image display system, an image display method, a computer program for realizing those methods, a non-transitory recording medium on which that computer program is recorded, etc.

FIG. 1 is an explanatory diagram showing a schematic configuration of an image display system 10 according to an embodiment of the present invention. 1 is a flowchart showing the contents of an image display control process according to the present embodiment. FIG. 2 is an explanatory diagram showing a schematic state of the image display device 150 and the like during the image display control process in this embodiment. FIG. 2 is an explanatory diagram showing a schematic state of the image display device 150 and the like during the image display control process in this embodiment. FIG. 2 is an explanatory diagram showing a schematic state of the image display device 150 and the like during the image display control process in this embodiment. FIG. 1 is an explanatory diagram showing a schematic diagram of an image display device 150 according to an embodiment of the present invention;

A. Embodiments:
A-1. Configuration of image display system 10:
1 is an explanatory diagram showing a schematic configuration of an image display system 10 according to the present embodiment. The image display system 10 according to the present embodiment is an infotainment system that is mounted on an autonomous vehicle VH and can provide infotainment such as videos, games, and tourist information to a passenger PA of the vehicle VH.

As shown in FIG. 1, the image display system 10 includes an image display device 150, a support device 160, and a control device 100. The image display device 150 includes a display such as a liquid crystal display or an organic EL display, and displays images such as still images and videos. The image display device 150 is substantially flat, having a substantially planar front surface FS (i.e., an image display surface) and a surface opposite the front surface FS (hereinafter referred to as the "back surface BS"). The image display device 150 includes a camera (back surface camera) 152 with a lens arranged on the back surface BS. The camera 152 captures an object located in the direction of the back surface BS of the image display device 150 to generate an image. The image display device 150 may include other components such as a front surface camera with a lens arranged on the front surface FS and a speaker. The image display device 150 may also include a touch panel, a microphone, etc., and function as an input device.

The support device 160 is fixed to a predetermined position (the dashboard in this embodiment) of the vehicle VH and is a device that supports the image display device 150. In this embodiment, the support device 160 is configured by a robot arm having a large number of (three or more) joints and drive motors 162 that drive each joint. The support device 160 can change the position and orientation of the image display device 150 by driving each joint with each drive motor 162 (see Figures 3 and 4). Hereinafter, such a change in the position and orientation of the image display device 150 is referred to as an "automatic change" of the position and orientation of the image display device 150.

In addition, the support device 160 of this embodiment is configured such that the position and orientation of the image display device 150 can be changed by the application of an external force to the image display device 150 or the support device 160 by setting the torque of each drive motor 162 to zero, and the position and orientation of the image display device 150 are fixed when the application of the force is removed. Therefore, for example, the passenger PA can grasp the image display device 150 and move or rotate the image display device 150 to change the position and orientation of the image display device 150 to a desired position and orientation (see FIG. 5). Hereinafter, such a change in the position and orientation of the image display device 150 is referred to as a "manual change" of the position and orientation of the image display device 150.

The control device 100 is a computer for controlling each component of the vehicle VH, and includes a control unit 110 and a memory unit 170. These units are connected to each other via a bus 190 so that they can communicate with each other.

The storage unit 170 of the control device 100 is composed of, for example, a ROM, a RAM, a hard disk drive (HDD), etc., and is used to store various programs and data, and as a work area and temporary storage area for data when executing various programs. For example, the storage unit 170 stores an automatic driving control program 172, which is a computer program for executing automatic driving of the vehicle VH, and an image display control program 174 and a support control program 176, which are computer programs for executing the image display control process described below. These programs are provided in a state stored in a computer-readable recording medium (not shown), such as a CD-ROM, DVD-ROM, or USB memory, and are stored in the storage unit 170 by installing them in the control device 100.

In addition, a three-dimensional map database MD is constructed in the memory unit 170 of the control device 100. The three-dimensional map database MD stores a highly accurate three-dimensional map used for the autonomous driving of the vehicle VH. This three-dimensional map includes, for example, information on the position and shape of roads, as well as information on the position and shape of traffic lights, road signs, road trees, guardrails, buildings, etc.

An infotainment database ID is also constructed in the memory unit 170 of the control device 100. The infotainment database ID stores information for providing infotainment such as videos, games, and tourist information to the passenger PA of the vehicle VH. In this embodiment, in order to provide tourist information to the passenger PA of the vehicle VH, the infotainment database ID stores information indicating the position of an object (more specifically, a tourist attraction such as a building or natural object) (hereinafter referred to as "tourist attraction position information I2") and information indicating specifications of the tourist attraction such as its appearance, name, size, and year of construction (hereinafter referred to as "specification information I3").

The control unit 110 of the control device 100 is configured, for example, by a CPU, and controls the operation of each part of the vehicle VH by executing a computer program read from the storage unit 170. For example, the control unit 110 controls the automatic driving of the vehicle VH by reading and executing an automatic driving control program 172 from the storage unit 170. At this time, the control unit 110 functions as an automatic driving control unit 120. The automatic driving control unit 120 includes a position/orientation detection unit 122, a surrounding situation detection unit 124, and a route setting unit 126. The functions of each of these units will be explained in conjunction with the explanation of the automatic driving control process described below.

The control unit 110 also controls the image display device 150 and the support device 160 by reading and executing an image display control program 174 and a support control program 176 from the memory unit 170. At this time, the control unit 110 functions as an image display control unit 130 and a support control unit 140. The image display control unit 130 includes a spatial information acquisition unit 132, a target setting unit 134, and a target information acquisition unit 136. The support control unit 140 includes an orientation control unit 142. The functions of each of these units will be explained in conjunction with the explanation of the image display control process described below.

A-2. Automatic driving control process:
Next, an automatic driving control process by the automatic driving control unit 120 of the control device 100 will be described. The automatic driving control process is a process for controlling the automatic driving of the vehicle VH. The content of the automatic driving control process in this embodiment is similar to the known content, and is roughly as follows.

First, the route setting unit 126 of the automatic driving control unit 120 sets a travel route (hereinafter referred to as a "route") from the departure point to the destination in accordance with the instructions of the passenger PA. The route can be set by various methods, but for example, the route can be set by referring to a road network database in which roads are represented by nodes and links, and performing a route search based on this.

Once the route is set, the automatic driving control unit 120 controls the acceleration, steering and braking of the vehicle VH so that the vehicle VH travels along the set route, thereby autonomously driving the vehicle VH. At this time, the position/orientation detection unit 122 of the automatic driving control unit 120 detects the position (latitude, longitude, altitude, etc.) and orientation (direction of travel, azimuth, etc.) of the vehicle VH based on signals from various sensors 102 (GPS, geomagnetic sensor, etc.) mounted on the vehicle VH, or based on the signals and the high-precision three-dimensional map stored in the three-dimensional map database MD. In addition, the surrounding situation detection unit 124 of the automatic driving control unit 120 detects the surrounding situation of the vehicle VH (the positions and shapes of objects (other vehicles, pedestrians, road trees, lanes, etc.) surrounding the vehicle VH, etc.) based on signals from various sensors 102 (cameras, radar, LIDAR, ultrasonic sensors, GPS, etc.) mounted on the vehicle VH, or based on the signals and the high-precision three-dimensional map stored in the three-dimensional map database MD. The autonomous driving control unit 120 autonomously drives the vehicle VH to the destination along a set route based on the position and orientation of the vehicle VH detected by the position/orientation detection unit 122 and the surrounding conditions detected by the surrounding conditions detection unit 124.

A-3. Image display control process:
Next, the image display control process executed by the image display system 10 of this embodiment will be described. FIG. 2 is a flowchart showing the contents of the image display control process in this embodiment. Also, FIGS. 3 to 5 are explanatory diagrams that typically show the state of the image display device 150 and the like during the image display control process in this embodiment. The image display control process in this embodiment is a process that presents information introducing tourist attractions visible from the vehicle window to the passenger PA of the vehicle VH traveling toward a destination by automatic driving as infotainment using the image display device 150, and is started, for example, when an instruction to start the process is given by the passenger PA. FIG. 3 shows the state of the image display device 150 at the start of the image display control process. As shown in FIG. 3, at the start of the image display control process, the image display device 150 is positioned by the support device 160 in front of the passenger PA who is seated facing forward, and is positioned and oriented so that the front surface FS faces the passenger PA directly.

When the image display control process is started, the spatial information acquisition unit 132 of the image display control unit 130 acquires the vehicle position information I1 indicating the position of the vehicle VH (S110). In this embodiment, the vehicle position information I1 includes information indicating the orientation of the vehicle VH in addition to the position of the vehicle VH. In this embodiment, the spatial information acquisition unit 132 acquires the information on the position and orientation of the vehicle VH detected by the position/orientation detection unit 122 of the automatic driving control unit 120 based on signals from the various sensors 102 in the above-mentioned automatic driving control process as the vehicle position information I1. Since the image display system 10 equipped with the image display device 150 is mounted on the vehicle VH, the vehicle position information I1 indicating the position and orientation of the vehicle VH can also be said to be information indicating the position and orientation of the image display device 150. The position and orientation of the image display device 150 referred to here are not "relative" position and orientation with respect to the vehicle VH, but "absolute" position and orientation with respect to features such as roads. For example, even if the relative position and orientation of the image display device 150 with respect to the vehicle VH is fixed, if the position and orientation of the vehicle VH change as the vehicle VH travels, the absolute position and orientation of the image display device 150 with respect to the above-mentioned features also change. The vehicle position information I1 is an example of the first position information and spatial information in the claims.

Next, the image display control unit 130, in cooperation with the support control unit 140, determines whether or not the orientation of the image display device 150 described above has been manually changed (S120). Whether or not the orientation of the image display device 150 has been manually changed can be determined, for example, by detecting with a sensor whether or not each joint of the support device 160 has moved without being driven by the drive motor 162.

If it is determined that the orientation of the image display device 150 has not been manually changed (S120: NO), the goal setting unit 134 of the image display control unit 130 determines whether there is an object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S130). More specifically, as described above, the infotainment database ID stores the tourist attraction position information I2, and the tourist attraction position information I2 is acquired by the spatial information acquisition unit 132. The goal setting unit 134 determines whether there is a tourist attraction that can be seen from the vehicle window of the vehicle VH at the current position based on the relationship between the position of the vehicle VH specified by the vehicle position information I1 and the positions of each tourist attraction specified by the acquired tourist attraction position information I2. Note that a correspondence relationship between each position on a road or a parking lot and a tourist attraction that can be seen from there may be set in advance, the correspondence relationship may be stored in the infotainment database ID, and the goal setting unit 134 may make the above-mentioned determination in S130 by referring to the correspondence relationship. The tourist attraction position information I2 is an example of the second position information and spatial information in the claims.

In the above S130, if it is determined that there is an object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S130: YES), the target setting unit 134 sets one of such tourist attractions as the target object TA, and the target information acquisition unit 136 acquires specification information I3 of the target object TA from the infotainment database ID (S140). The specification information I3 of the target object TA is information that indicates the specifications of the target object TA (appearance, name, size, year of construction, etc.). The specification information I3 of the target object TA is an example of target information in the claims.

Next, the orientation control unit 142 of the support control unit 140 causes the support device 160 to change the orientation of the image display device 150 so that the image display device 150 points to the target object TA based on the vehicle position information I1 and the tourist attraction position information I2 of the target object TA (S150). Here, in this specification, the direction in which the image display device 150 points is a direction that is perpendicular to the front surface (image display surface) FS of the image display device 150 and faces the opposite side (back surface BS side) from the front surface FS. In other words, the state in which the image display device 150 is oriented to point to the target object TA is a state in which a virtual straight line VL that passes through an arbitrary point (for example, the center point P0 of the screen) on the front surface (image display surface) FS of the image display device 150 and is perpendicular to the front surface FS intersects with the target object TA, as shown in FIG. The orientation control unit 142 specifies the position and orientation of the vehicle VH (i.e., the position and orientation of the image display device 150) based on the vehicle position information I1, and specifies the position of the target object TA based on the tourist attraction position information I2, calculates the direction from the image display device 150 toward the target object TA (the direction of the virtual straight line VL), and controls the drive motor 162 to drive each joint of the support device 160 based on the direction, thereby changing the orientation of the image display device 150. FIG. 4 shows a state in which the image display device 150 has been changed to point to the first target object TA1. In response to the change in the orientation of the image display device 150, the passenger PA moves his/her line of sight to the direction pointed by the image display device 150 (i.e., the direction in which the target object TA is located). In this embodiment, the support device 160 is configured by a robot arm having many joints, and is capable of changing the position of the image display device 150 in addition to the orientation, so that the position of the image display device 150 is also changed in addition to the orientation of the image display device 150. The position of the image display device 150 after the change is a position where the virtual straight line VL also intersects with the passenger PA (i.e., a position sandwiched between the target object TA and the passenger PA). In this position, the front surface FS of the image display device 150 is approximately perpendicular to the line of sight of the passenger PA who is looking at the target object TA, improving the visibility of the image displayed on the image display device 150 by the passenger PA.

Furthermore, the image display control unit 130 causes the image display device 150 to display an image showing the specification information I3 based on the specification information I3 of the target object TA acquired in S140 (S160). FIG. 4 shows a state in which an image showing the specifications (appearance, name, size, etc.) of the first target object TA1 is displayed on the image display device 150 as the specification information I3 of the first target object TA1. Note that a part of the specification information I3 may be output as sound from a speaker (not shown) provided in the image display device 150. By displaying the image showing the specification information I3 on the image display device 150, the passenger PA can visually recognize the actual appearance of the first target object TA1 and the contents of the specification information I3 of the first target object TA1 displayed on the image display device 150 without moving his/her line of sight.

In addition, if it is determined in S130 above that there are no objects (tourist attractions) that can be introduced in the vicinity of the vehicle VH (S130: NO), the processing of S140 to S160 above is skipped.

The image display control unit 130 then determines whether the destination has been reached (S200), and if it is determined that the destination has not yet been reached (S200: NO), repeats the processing from S110 onwards described above.

Note that the position and orientation of the vehicle VH change from moment to moment as the vehicle VH travels. Therefore, if the relative position and orientation of the image display device 150 with respect to the vehicle VH is fixed, the absolute position and orientation of the image display device 150 with respect to features such as roads also change from moment to moment in response to changes in the position and orientation of the vehicle VH as the vehicle VH travels. Therefore, even after the target object TA is set in S140 and the image display device 150 is changed in orientation to point to the target object TA in S150, the relative position and orientation of the image display device 150 with respect to the vehicle VH are adjusted in response to changes in the position and orientation of the vehicle VH as the vehicle VH travels so that the image display device 150 continues to point to the target object TA.

As the vehicle VH travels, the tourist attractions that can be seen from the vehicle window also change from moment to moment. Therefore, after the target object TA is set in S140, the orientation of the image display device 150 is changed in S150, and an image showing the specification information I3 of the target object TA is displayed on the image display device 150 in S160, if the target object TA can no longer be seen from the vehicle window due to changes in the position and orientation of the vehicle VH as the vehicle VH travels, the processing from S130 onwards is executed again.

In addition, if it is determined in S130 that there are multiple objects (tourist attractions) that can be introduced around the vehicle VH, one tourist attraction is set as the target object TA in S140, and the target object TA may then be switched, for example, every time a predetermined time has elapsed or whenever an instruction is given by the passenger PA. That is, another tourist attraction may be set as a new target object TA in S140, the orientation of the image display device 150 may be changed to point to the new target object TA in S150, and an image showing the specification information I3 of the new target object TA may be displayed on the image display device 150 in S160.

During the image display control process, if the orientation of the image display device 150 is manually changed, it is determined in the above S120 that the orientation of the image display device 150 has been manually changed (S120: YES). FIG. 5 shows a state in which the passenger PA grasps the image display device 150 and changes the orientation (and position) of the image display device 150 to an orientation (and position) such that the image display device 150 points to the second target object TA2 during the image display control process. When it is determined that the orientation of the image display device 150 has been manually changed (S120: YES), the target setting unit 134 of the image display control unit 130 determines whether or not there is an object (tourist attraction) that can be introduced in the direction pointed to by the image display device 150 (S170). More specifically, as described above, the tourist attraction position information I2 is stored in the infotainment database ID, and this tourist attraction position information I2 is acquired by the spatial information acquisition unit 132. The target setting unit 134 calculates the above-mentioned virtual straight line VL (a virtual straight line that passes through a point on the front surface FS of the image display device 150 and is perpendicular to the front surface FS) from the position of the vehicle VH specified by the vehicle position information I1 and the orientation of the image display device 150 detected by the orientation control unit 142 of the support control unit 140, and determines whether or not there is a tourist attraction located on the virtual straight line VL (or a position whose distance from the virtual straight line VL is equal to or less than a predetermined threshold value). Note that it is also possible to set a correspondence relationship between the combination of each position on a road or parking lot and the position and orientation of the image display device 150, and the tourist attraction, and to store the correspondence relationship in the infotainment database ID, and the target setting unit 134 makes the determination in S170 by referring to the correspondence relationship.

In the above S170, if it is determined that there is an object (tourist attraction) that can be introduced in the direction indicated by the image display device 150 (S170: YES), the target setting unit 134 sets one of such tourist attractions as the target object TA, and the target information acquisition unit 136 acquires the specification information I3 of the target object TA from the infotainment database ID (S180). The specification information I3 of the target object TA is an image showing the specifications of the target object TA (appearance, name, size, year of construction, etc.).

The image display control unit 130 also causes the image display device 150 to display an image showing the specification information I3 of the target object TA (S190). FIG. 5 shows a state in which an image showing the specifications (appearance, name, size, etc.) of the second target object TA2 is displayed on the image display device 150 as the specification information I3 of the second target object TA2. By displaying the image showing the specification information I3 on the image display device 150, the passenger PA can visually recognize the appearance of the second target object TA2 that he or she wishes to see and the contents of the specification information I3 of the second target object TA2 displayed on the image display device 150, without moving his or her line of sight.

In this embodiment, when the orientation (and position) of the image display device 150 is manually changed, the image display control unit 130 causes the image display device 150 to display an image captured by the camera 152 provided in the image display device 150. Therefore, by referring to the image displayed on the image display device 150, the passenger PA can confirm an object located in the direction indicated by the image display device 150, and can change the position and orientation of the image display device 150 so that the image display device 150 accurately points to the desired object.

In addition, if it is determined in S170 that there is no object (tourist attraction) that can be introduced in the vicinity of the vehicle VH (S170: NO), the processing of S180 to S190 is skipped. Alternatively, the passenger PA may be notified that there is no object (tourist attraction) that can be introduced in the vicinity of the vehicle VH, and may be prompted to manually change the orientation (and position) of the image display device 150 (S120) again.

The various processes described above are repeatedly executed, and if it is determined in S200 that the destination has been reached (S200: YES), the image display control unit 130 terminates the image display control process.

A-4. Advantages of this embodiment:
As described above, the image display system 10 of this embodiment includes the image display device 150, the support device 160 that supports the image display device 150 and can change the orientation of the image display device 150, and the control device 100 that controls the image display device 150 and the support device 160. The control device 100 has a spatial information acquisition unit 132, an orientation control unit 142, and an image display control unit 130. The spatial information acquisition unit 132 acquires spatial information including vehicle position information I1 indicating the position of the vehicle VH (i.e., the position of the image display device 150) and tourist attraction position information I2 indicating the positions of objects in the vicinity of the image display device 150. The orientation control unit 142 causes the support device 160 to change the orientation of the image display device 150 based on the spatial information. The image display control unit 130 causes the image display device 150 to display an image according to the orientation of the image display device 150.

In this way, in the image display system 10 of this embodiment, the orientation of the image display device 150 is changed based on spatial information including vehicle position information I1 indicating the position of the image display device 150 and tourist attraction position information I2 indicating the positions of objects around the image display device 150, and an image corresponding to the orientation of the image display device 150 is displayed on the image display device 150. Therefore, according to the image display system 10 of this embodiment, it is possible to indicate the correct direction to the passenger PA (user) by the orientation of the image display device 150 without using a mounted image display device, while displaying an image corresponding to the direction on the image display device 150, thereby improving the convenience of the image display system 10.

Furthermore, in the image display system 10 of this embodiment, the control device 100 further has a target setting unit 134 that sets one of the surrounding objects as a target object TA based on the above spatial information, and the orientation control unit 142 causes the support device 160 to change the orientation of the image display device 150 so that the image display device 150 is oriented to point at the target object TA. Therefore, according to the image display system 10 of this embodiment, the direction of the passenger PA's line of sight can be guided to the target object TA, which is one of the surrounding objects, depending on the orientation of the image display device 150, and the convenience of the image display system 10 can be further improved.

In the image display system 10 of this embodiment, the control device 100 further has a target information acquisition unit 136 that acquires specification information I3, which is information related to the target object TA, and the image display control unit 130 displays an image showing the specification information I3 on the image display device 150. Therefore, according to the image display system 10 of this embodiment, the direction of the passenger PA's line of sight can be guided to the target object TA, which is one of the surrounding objects, depending on the orientation of the image display device 150, and an image showing the specification information I3 related to the target object TA can be displayed on the image display device 150, thereby further improving the convenience of the image display system 10.

In addition, in the image display system 10 of this embodiment, the specification information I3 of the target object TA includes information indicating at least one of the name, size, and construction year of the target object TA. Therefore, according to the image display system 10 of this embodiment, an image indicating information indicating at least one of the name, size, and construction year of the target object TA can be displayed on the image display device 150, and useful information can be presented to the passenger PA.

In addition, in the image display system 10 of this embodiment, the support device 160 supports the image display device 150 so that the orientation of the image display device 150 can be changed manually, and the target setting unit 134 of the image display control unit 130 sets, when the orientation of the image display device 150 is changed manually, the object located in the direction pointed to by the image display device 150 after the orientation is changed as the target object TA. That is, in the image display system 10 of this embodiment, the image display device 150 functions as a 3D pointing device. Therefore, according to the image display system 10 of this embodiment, when the passenger PA points the image display device 150 in the direction of a desired object, the object is set as the target object TA, so that the target object TA can be set by the passenger PA's intuitive operation, and the convenience of the image display system 10 can be further improved.

In addition, in the image display system 10 of this embodiment, the image display device 150 has a camera 152 that captures an image of a subject located in the direction pointed to by the image display device 150, and the image display control unit 130 causes the image display device 150 to display an image captured by the camera 152 when the orientation of the image display device 150 is manually changed. Therefore, according to the image display system 10 of this embodiment, the current orientation of the image display device 150 can be fed back to the passenger PA by the image displayed on the image display device 150, so that the passenger PA can easily manually change the orientation of the image display device 150 so that the image display device 150 accurately points to a desired object, thereby further improving the convenience of the image display system 10.

In addition, in the image display system 10 of this embodiment, the image display device 150 has a planar front surface (image display surface) FS, and the direction indicated by the image display device 150 is perpendicular to the front surface FS and faces the opposite direction (toward the rear surface BS) from the direction in which the front surface FS faces. Therefore, according to the image display system 10 of this embodiment, the passenger PA can more accurately grasp the direction indicated by the image display device 150, and the convenience of the image display system 10 can be further improved.

In addition, in the image display system 10 of this embodiment, the support device 160 has three or more rotation axes. Therefore, according to the image display system 10 of this embodiment, the orientation of the image display device 150 can be adjusted more flexibly, thereby further improving the convenience of the image display system 10.

In addition, the image display system 10 of this embodiment is mounted on an autonomous vehicle VH, the support device 160 supports the image display device 150 on the vehicle VH, and the spatial information acquisition unit 132 acquires the spatial information obtained by the sensor 102 installed in the vehicle VH. Therefore, according to the image display system 10 of this embodiment, by changing the orientation of the image display device 150 using information used for autonomous driving, it is possible to display an image corresponding to the direction on the image display device 150 while indicating an accurate direction related to the traveling of the vehicle VH to the passenger PA of the autonomous vehicle VH and people outside the vehicle, thereby improving the convenience of the image display system 10.

B. Variations:
The technology disclosed in this specification is not limited to the above-described embodiments, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

The configuration of the image display system 10 in the above embodiment is merely an example and can be modified in various ways. For example, in the above embodiment, the image display device 150 is supported on the dashboard of the vehicle VH by the support device 160, but as in the modified example shown in FIG. 6, the image display device 150 may be supported on the ceiling or floor of the vehicle VH by the support device 160. Also, in the above embodiment, the support device 160 is configured to allow manual change of the position and orientation of the image display device 150, but it is not necessarily required that the position and orientation of the image display device 150 be manually changed.

In the above embodiment, the support device 160 is configured as a robot arm having multiple joints, and is capable of changing the position and orientation of the image display device 150; however, the support device 160 may be configured as another device (e.g., a gimbal) as long as it is capable of changing the orientation of the image display device 150 while supporting the image display device 150. Note that if the support device 160 has three or more axes, it is preferable because, for example, the pan, tilt, and roll of the image display device 150 can be adjusted along each axis, and the orientation of the image display device 150 can be adjusted more flexibly.

In addition, in the above embodiment, in the initial state, the image display device 150 is located in front of the passenger PA, and the front surface FS of the image display device 150 is oriented so as to face the passenger PA directly, but the position and orientation of the image display device 150 in the initial state are not limited to this. For example, in the initial state, the front surface FS of the image display device 150 may face upward to form a table shape, making it easy for one or more passengers PA to view the screen. Alternatively, in the initial state, the orientation of the image display device 150 may be such that it is easy for the passenger PA to perform touch input operations on the screen.

In addition, while the above embodiment illustrates a case where one image display device 150 is used by one passenger PA (user), the technology disclosed in this specification is similarly applicable to a case where one image display device 150 is shared by multiple users. That is, since the orientation of one image display device 150 can be recognized almost accurately by each of the multiple users, it is possible to indicate an almost accurate direction to the multiple users depending on the orientation of the image display device, and to display an image corresponding to that direction on the image display device.

In addition, in the above embodiment, an in-vehicle image display device is used as the image display device 150, but other image display devices such as a smartphone or tablet terminal brought by the passenger PA may also be used as the image display device 150. In such a case, for example, a holder for holding the image display device is provided in the support device 160, and the image display device is held in the holder, thereby supporting the image display device and changing its position and orientation. In addition, a projection type image display device such as a projector may also be used as the image display device 150.

In the above embodiments, the control device 100 may be configured to be divided into multiple devices. For example, the vehicle VH may be provided with a control device for automatic driving control, a control device for image display control, and a control device for support control, which are independent of each other. In the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware.

The contents of the image display control process in the above embodiment are merely examples and can be modified in various ways. For example, in the image display control process in the above embodiment, the spatial information acquisition unit 132 acquires, as spatial information, information on the position and orientation of the vehicle VH detected based on signals from various sensors 102 installed in the vehicle VH, but the spatial information acquisition unit 132 may acquire, as spatial information, information on the position and orientation of the image display device 150 detected based on signals from a sensor other than the sensor 102 installed in the vehicle VH (for example, a sensor installed in the image display device 150). Note that the spatial information may not include information indicating the orientation of the vehicle VH or the image display device 150.

In addition, although the image display control process in this embodiment is a process of presenting information introducing tourist attractions visible from the vehicle window as infotainment to the passenger PA of the vehicle VH traveling toward a destination by automatic driving, the technology disclosed in this specification can be applied to other image display control processes. For example, the technology disclosed in this specification can be applied to image display control processes for providing other infotainment (games, videos, etc.). In addition, the technology disclosed in this specification can be applied to image display control processes in which, for example, the spatial information acquisition unit 132 acquires spatial information, the orientation control unit 142 changes the orientation of the image display device 150 so that the front surface FS of the image display device 150 is directed toward surrounding objects, and the image display control unit 130 displays an image to be shown to surrounding objects (for example, an image that calls attention to the progress of the vehicle VH, etc.) on the image display device 150.

In addition, while the above embodiment describes an example of an image display system 10 installed in an autonomously driven vehicle VH, the technology disclosed in this specification is similarly applicable to image display systems used for other purposes (e.g., an image display system installed in a vehicle that is not autonomously driven, an image display system installed on a moving body other than a vehicle, an image display system installed on a fixed object (e.g., a building), a portable image display system, etc.).

10: Image display system 100: Control device 102: Sensor 110: Control unit 120: Autonomous driving control unit 122: Position/orientation detection unit 124: Surrounding situation detection unit 126: Route setting unit 130: Image display control unit 132: Spatial information acquisition unit 134: Target setting unit 136: Target information acquisition unit 140: Support control unit 142: Orientation control unit 150: Image display device 152: Camera 160: Support device 162: Drive motor 170: Memory unit 172: Autonomous driving control program 174: Image display control program 176: Support control program 190: Bus BS: Rear FS: Front I1: Vehicle position information I2: Tourist attraction position information I3: Specifications information ID: Infotainment database MD: 3D map database PA: Passenger TA: Target object VH: Vehicle VL: Virtual line

Claims (11)

1. An image display system, comprising:
An image display device;
a support device that supports the image display device and is capable of changing the orientation of the image display device;
A control device that controls the image display device and the support device;
Equipped with
The control device includes:
a spatial information acquisition unit that acquires spatial information including first position information indicating a position of the image display device and second position information indicating positions of objects in the periphery of the image display device specified based on the first position information ;
an orientation control unit that causes the support device to change an orientation of the image display device based on the spatial information;
an image display control unit that causes the image display device to display an image according to a direction of the image display device;
having
Image display system. 2. The image display system according to claim 1,
The control device further includes a target setting unit that sets one of the surrounding objects as a target object based on the spatial information,
the orientation control unit causes the support device to change the orientation of the image display device so that the image display device is oriented to point at the target object;
Image display system. 3. The image display system according to claim 2,
The control device further includes a target information acquisition unit that acquires target information that is information related to the target object,
The image display control unit causes the image display device to display an image indicating the target information.
Image display system. 4. The image display system according to claim 3,
The target information includes information indicating at least one of the name, size, and construction year of the target object.
Image display system. The image display system according to any one of claims 2 to 4,
the support device supports the image display device so that the orientation of the image display device can be manually changed;
the target setting unit sets an object located in a direction indicated by the image display device as the target object when the orientation of the image display device is manually changed;
Image display system. 6. The image display system according to claim 5,
the image display device has a camera that captures an image of a subject located in a direction indicated by the image display device,
the image display control unit causes the image display device to display an image captured by the camera when a direction of the image display device is manually changed;
Image display system. The image display system according to any one of claims 2 to 6,
the image display device has a flat image display surface,
The direction in which the image display device points is perpendicular to the image display surface and faces the opposite side to the image display surface.
Image display system. The image display system according to any one of claims 1 to 7,
The support device has three or more rotation axes.
Image display system. The image display system according to any one of claims 1 to 8,
The image display system is mounted on an autonomously driven vehicle,
the support device supports the image display device on the vehicle;
The spatial information acquisition unit acquires the spatial information obtained by a sensor installed in the vehicle.
Image display system. 1. An image display method for displaying an image using an image display device and a support device that supports the image display device and is capable of changing an orientation of the image display device, comprising:
acquiring spatial information including first position information indicating a position of the image display device and second position information indicating positions of objects around the image display device identified based on the first position information ;
causing the support device to change the orientation of the image display device based on the spatial information;
displaying an image on the image display device according to the orientation of the image display device;
Equipped with
Image display method. A computer program for displaying an image using an image display device and a support device that supports the image display device and is capable of changing an orientation of the image display device, the computer program comprising:
On the computer,
A process of acquiring spatial information including first position information indicating a position of the image display device and second position information indicating positions of objects around the image display device identified based on the first position information ;
A process of causing the support device to change the orientation of the image display device based on the spatial information;
A process of displaying an image on the image display device according to a direction of the image display device;
Execute the
Computer program.

Images