JP7738382B2 - Vehicle display device - Google Patents

Description

This disclosure relates to a vehicle display device that is used in a vehicle and allows an image to be superimposed on the vehicle's front view for viewing.

Patent Document 1 discloses a display control method in which multiple speed limit display images are displayed from the front to the back, partially overlapping each other, and at predetermined intervals, the frontmost speed limit display image is erased and the next frontmost speed limit display image is displayed.

Japanese Patent Application Laid-Open No. 2018-31722

However, with the display control method of Patent Document 1, the display is updated at predetermined intervals, but it is assumed that the timing of the display update (the timing at which the speed limit display may change) is difficult to predict, and in this respect there is room for improvement.

In addition, because the latest speed limit image and the next speed limit image partially overlap, it is difficult to grasp the next speed limit, and it is possible that the viewer's visual attention will be drawn to the image.

A summary of certain embodiments disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with an overview of these particular embodiments, and are not intended to limit the scope of the disclosure. Indeed, the present disclosure may encompass a variety of aspects that are not set forth below.

The present disclosure relates generally to making information more effectively recognizable to vehicle drivers. More specifically, it makes it easier for drivers to recognize changes in speed limits while preventing excessive distraction of the driver's visual attention.

Therefore, the vehicle display device described in this specification comprises an image display unit that displays an image, one or more I/O interfaces, one or more processors, a memory, and one or more computer programs stored in the memory and configured to be executed by the one or more processors, wherein the one or more I/O interfaces acquire speed limit information for a driving lane in which the vehicle is traveling, and information regarding an approaching speed of a scene in front of the vehicle as the vehicle travels, or information that allows the approaching speed of the scene in front of the vehicle to be estimated, and the one or more processors cause the image display unit to display a first image including a first numeric image indicating a numerical value of a speed limit after a speed limit change scheduled ahead of the driving lane in which the vehicle is traveling, and a second image including a second numeric image indicating a numerical value of the speed limit of the driving lane in which the vehicle is currently traveling, and moves the first image toward the second image at a speed based on the approaching speed, and when the first image comes close enough to the second image, Before the first numeric image overlaps the second image, the visibility of the first image and the second image is reduced, and a third image including the first numeric image is displayed in the position where the second image was displayed.

FIG. 1 is a diagram showing an embodiment of a display device for a vehicle. FIG. 2 is a block diagram of the vehicle display device. FIG. 3A is a diagram showing the foreground viewed when looking forward from the driver's seat of a vehicle, and a virtual image of an image. FIG. 3B is a diagram showing a virtual image of an image in a situation where the vehicle has moved further forward than in FIG. 3A. FIG. 3C is a diagram showing a virtual image in a situation where the vehicle has moved further forward than in FIG. 3B. FIG. 3D is a diagram showing a virtual image in a situation where the vehicle has moved further forward than in FIG. 3C. FIG. 4 is a diagram showing an approach path (movement path) of an image on a vehicle display device. FIG. 5 is a diagram showing the positional relationship between the first image and the second image when switching between the images. FIG. 6 is a diagram showing the foreground viewed when looking forward from the driver's seat of the vehicle, and a virtual image of the image. FIG. 3A is a diagram showing an example of an image display. FIG. 3B is a diagram showing an image in a situation where the vehicle has moved further forward than in FIG. 3A. FIG. 3C is a diagram showing an image in a situation where the vehicle has moved further forward than in FIG. 3B.

Embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to the following embodiments (including the contents of the drawings). Naturally, modifications (including the omission of components) can be made to the following embodiments. In addition, in the following description, explanations of well-known technical matters will be omitted as appropriate to facilitate understanding of the present invention.

Below, Figures 1 and 2 provide a description of an exemplary vehicle display device. Figures 3A through 7C provide display examples.

Refer to Figure 1. The image display unit 11 that displays the virtual image 200 in the vehicle display device 10 of the first embodiment is a head-up display (HUD) device provided in the dashboard 5 of the vehicle 1. The HUD device emits display light 11a toward the front windshield 2 (an example of a projection target) and displays the virtual image 200 within a virtual display area 100, allowing the virtual image 200 to be viewed superimposed on the foreground 300, which is real space viewed through the front windshield 2.

In addition, the image display unit 12 that displays the virtual image 200 in the vehicle display device 10 of the second embodiment is a head-mounted display (hereinafter, HMD) device. The driver 4 wears the HMD device on his or her head and sits in a seat in the vehicle 1, and views the displayed virtual image 200 superimposed on the foreground 300 seen through the front windshield 2 of the vehicle 1. The display area 100 in which the vehicle display device 10 displays a specific virtual image 200 is fixed at a specific position based on the coordinate system of the vehicle 1, and when the driver 4 faces in that direction, the virtual image 200 displayed within the display area 100 fixed at that specific position can be viewed.

The image display units 11, 12 can also create visual augmented reality (AR) by displaying virtual images 200 in the vicinity of real objects such as the road surface 310, road signs (not shown), buildings, and obstacles (pedestrians, bicycles, motorcycles, other vehicles) that exist in the foreground 300, which is the real space visible through the front windshield 2 of the vehicle 1, or in positions that overlap the real objects.

Figure 2 is a block diagram of a vehicle display device 10 according to some embodiments. The vehicle display device 10 includes an image display unit 11 (12), an I/O interface 14, one or more processors 16, a memory 18, and an image processing circuit 20. The various functional blocks depicted in Figure 2 may be configured in hardware, software, or a combination of both. Figure 2 is merely one example implementation, and the components depicted may be combined into fewer components, or additional components may be present. For example, the image processing circuit 20 (e.g., a graphics processing unit) may be included in one or more processors 16.

As shown, the processor 16 and the image processing circuit 20 are operatively coupled to the memory 18. More specifically, the processor 16 and the image processing circuit 20 execute programs stored in the memory 18 to operate the vehicle display device 10, such as generating and/or transmitting image data. The processor 16 and/or the image processing circuit 20 may include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), at least one field-programmable gate array (FPGA), or any combination thereof. The memory 18 may include any type of magnetic media, such as a hard disk, any type of optical media, such as a CD or DVD, any type of semiconductor memory, such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

As shown, the processor 16 is operably coupled to the I/O interface 14. The I/O interface 14 communicates (also referred to as CAN communication) with other electronic devices (reference numerals 401 to 405 described below) installed in the vehicle 1, for example, according to the CAN (Controller Area Network) standard. Note that the communication standard adopted by the I/O interface 14 is not limited to CAN, and may include wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport; MOST is a registered trademark), UART, or USB, or in-vehicle communication (internal communication) interfaces, which are short-range wireless communication interfaces within a range of several tens of meters, such as personal area networks (PANs) such as Bluetooth (registered trademark) networks and local area networks (LANs) such as 802.11x Wi-Fi (registered trademark) networks. The I/O interface 14 may also include an external communication interface for communication outside the vehicle, such as a wide area communication network (e.g., an Internet communication network) based on a cellular communication standard such as a wireless wide area network (WWAN0, IEEE 802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE 802.16e (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, or 5G.

As shown in the figure, the processor 16 is interoperably connected to the I/O interface 14, enabling it to exchange information with various other electronic devices connected to the vehicle display device 10 (I/O interface 14). The I/O interface 14 is operably connected to, for example, a speed sensor 401, a road information database 402, a vehicle position detection unit 403, an external sensor 404, and an eyepoint detection unit 405, all of which are provided on the vehicle 1. The image display units 11 and 12 are operably connected to the processor 16 and the image processing circuit 20. Therefore, the images displayed by the image display units 11 and 12 may be based on image data received from the processor 16 and/or the image processing circuit 20. The processor 16 and the image processing circuit 20 control the images displayed by the image display units 11 and 12 based on information obtained from the I/O interface 14. The I/O interface 14 may also have the function of processing (converting, calculating, analyzing) information received from other electronic devices connected to the vehicle display device 10.

3A to 3D show a foreground 300 that is visible when the driver 4 faces forward, and a virtual image 200 of an image displayed on the vehicle display device 10 so as to be superimposed on the foreground 300. The virtual image 200 may be a virtual reality image that indicates the speed limit of the road on which the vehicle 1 is traveling, includes text ("60") resembling a road sign, and is floating above the right edge 312 of the road surface 310 (in the positive direction of the Y axis). The road surface 310 here refers to the area of the foreground 300 between the left edge 311 of the driving lane on which the vehicle 1 is traveling (e.g., the lane marking on the left side of the driving lane) and the right edge 312 of the driving lane (e.g., the lane marking on the right side of the driving lane). The virtual image 200 is a moving image that appears to approach the vehicle 1, but it can also be replaced by a continuously changing still image. As the vehicle 1 moves forward, the virtual image 200 (first image 210 and fourth image 240 described below) appears to gradually approach the vehicle 1 from Figures 3A to 3D within the display area 100, and is not displayed outside the display area 100. The virtual image 200 (first image 210 and fourth image 240 described below) can give the viewer the impression that it is approaching by changing its size depending on the displayed position using perspective.

3A to 3D includes a first virtual image 210, a second virtual image 220, a third virtual image 230, and a fourth virtual image 240. The first virtual image 210 is an image showing information about the speed limit after the planned speed limit change, and moves along a travel path 600 (described below) as the vehicle 1 travels. The fourth virtual image 240 is a linear image extending in the left-right direction that overlaps the road surface 310, and moves as if approaching from a distance as the first virtual image 210 moves. The second virtual image 220 is an image showing information about the speed limit before the planned speed limit change (the current speed limit), and is fixedly displayed at a specific position 101 within the display area 100. The third virtual image 230 is an image showing information about the speed limit after the speed limit change (the current speed limit), and is fixedly displayed at the same specific position 101 within the display area 100 as the second virtual image 220.

3A is a diagram showing a virtual image 200 when the planned speed limit change position is far away. FIG. 3B is a diagram showing a virtual image 200 in a situation where the vehicle 1 has traveled further than in FIG. 3A and the planned speed limit change position has approached. In FIGS. 3A and 3B, the virtual images 200 are a first virtual image 210 and a fourth virtual image 240. The first virtual image 210 shown in FIG. 3B moves along a movement path 603 (described below) and is displayed larger than the first virtual image 210 shown in FIG. 3A. The fourth virtual image 240 shown in FIG. 3B moves along a movement path 601 (described below) and is displayed larger (longer) than the fourth virtual image 240 shown in FIG. 3A. In this way, by moving and enlarging the positions of the first virtual image 210 and the fourth virtual image 240 along the movement path (described below) as the vehicle 1 travels, the viewer can more easily perceive the sense of distance.

Figure 3C shows the virtual image 200 when the planned speed limit change position is nearby. In Figure 3C, the virtual image 200 includes a first virtual image 210 and a second virtual image 220. That is, when the first virtual image 210 moves along a movement path 603 (described below) and reaches a predetermined position, the second virtual image 220 begins to be displayed at a specific position 101 within the display area 100 on an extension of the movement path 601 of the first virtual image 210, and the fourth virtual image 240 is hidden. In this way, by hiding the fourth virtual image 240 and starting to display the second virtual image 220, the viewer's visual attention can be directed to the first virtual image 210 and the second virtual image 220 displayed near the first virtual image 210. This makes it less likely for visual attention to be distracted than when the fourth virtual image 240 continues to be displayed on the road surface 310 side.

Furthermore, the second virtual image 220 begins to be displayed after the first virtual image 210 has moved along the movement path 601 described below and reached a predetermined position. In other words, in the situations of Figures 3A and 3B, the second virtual image 220 is not displayed, which makes it less likely for visual distraction to occur compared to when the second virtual image 220 is continuously displayed. Note that the fourth virtual image 240 may be hidden by fading out, and the second virtual image 220 may begin to be displayed by fading in. Note that the fourth virtual image 240 may continue to be displayed even in the situation of Figure 3C. Also, the second virtual image 220 may continue to be displayed even in the situation of Figures 3A and 3B.

Figure 3D shows the virtual image 200 after the speed limit has changed. In Figure 3D, the virtual image 200 includes a third virtual image 230. When the first virtual image 210 in Figure 3C moves along a movement path 603 (described below) and comes sufficiently close to the second virtual image 220, the first virtual image 210, the fourth virtual image 240, and the second virtual image 220 are hidden, and the third virtual image 230 is displayed at a specific position 101 instead of the second virtual image 220. The "timing at which the third virtual image 230 is displayed instead of the second virtual image 220" will be described later.

Figure 4 shows an example of a foreground 300 visible when the driver 4 faces forward, and image movement paths 601, 602, and 603 displayed on the vehicle display device 10 so as to be superimposed on the foreground 300. Multiple image movement paths are provided, including, for example, a first movement path 601 that overlaps the road surface 310 between the left edge 311 (e.g., the white line on the left side of the driving lane) and the right edge 312 (e.g., the white line on the right side of the driving lane) of the driving lane in which the vehicle 1 is traveling, a second movement path 602 above the left edge 311 of the road surface 310 (in the positive direction of the Y axis), and a third movement path 603 above the right edge 312 of the road surface 310 (in the positive direction of the Y axis). Note that the image movement paths may be selected as appropriate from those illustrated in Figure 4. Furthermore, the image movement path may be a single path.

Figure 5 shows the positional relationship between the first virtual image 210 and the second virtual image 220 just before the first virtual image 210 approaches the second virtual image 220 sufficiently and the third virtual image 230 is displayed in place of the second virtual image 220. The first virtual image 210 includes a first numerical image 211 indicating the specific numerical value of the planned speed limit, and a first background image 212 consisting of an illustration such as a graphic that forms the background of the first numerical image 211. The second virtual image 220 includes a second numerical image 221 indicating the specific numerical value of the current speed limit before the speed limit is changed, and a second background image 222 consisting of an illustration such as a graphic that forms the background of the second numerical image 221. The first background image 212 and the second background image 222 are typically filled-in images, but are shown as hollowed-out images in Figure 5 to make the positional relationship between the first virtual image 210 and the second virtual image 220 easier to understand. Specifically, before the first numerical image 211 of the first virtual image 210 overlaps with the second virtual image 220 (second background image 222), the visibility of the first virtual image 210 and the second virtual image 220 is reduced (including non-display), and the visibility of the third virtual image 230 is increased (including start of display). This reduces the redundancy of the viewer continuing to view the first numerical image 211 of the first virtual image 210 in a situation where a part of the first numerical image 211 is hidden in the shadow of the second virtual image 220 and is difficult to view, and allows the viewer to free their visual attention from the vicinity of the first virtual image 210 and the second virtual image 220 and direct their visual attention to other targets inside or outside the vehicle. While FIG. 5 shows the first background image 212 and the second background image 222 overlapping, this is not limiting. The visibility of the first virtual image 210 and the second virtual image 220 may be reduced (including not displayed) and the visibility of the third virtual image 230 may be increased (including when the display begins) before the first background image 212 and the second background image 222 overlap. The first virtual image 210 described above is a virtual reality image (hereinafter also referred to as an AR image) that appears to be floating above the right edge 312 of the road surface 310 (in the positive direction of the Y axis), and the fourth virtual image 240 is an AR image that appears to be attached to the road surface 310. However, the form of the virtual image that notifies the user of a change in the speed limit is not limited to these AR images and may also be a non-AR image.

Figure 6 shows a non-AR virtual image that notifies the driver of a change in the speed limit in some embodiments. The fifth virtual image 250 of the non-AR image is displayed in a predetermined area of the display area 100.

7A to 7C include a left-side line virtual image 251, a right-side line virtual image 252, a sixth virtual image 260, a seventh virtual image 270, an eighth virtual image 280, and a ninth virtual image 290. The left-side line virtual image 251 is located on the left side of the fifth virtual image 250 and is a perspective-enhanced image that resembles the white line on the left side of the vehicle 1's driving line. Similarly, the right-side line virtual image 252 is located on the right side of the fifth virtual image 250 and is a perspective-enhanced image that resembles the white line on the right side of the vehicle 1's driving line. The sixth virtual image 260 is an image that shows information about the speed limit after the planned speed limit change, and moves along the travel path 607 as the vehicle 1 travels. The seventh virtual image 270 is an image that shows information about the speed limit before the planned speed limit change (the current speed limit), and is fixedly displayed above the left-side line virtual image 251. The eighth virtual image 280 is an image showing information about the new speed limit (current speed limit), and is fixedly displayed in the same position as the seventh virtual image 270. The ninth virtual image 290 is a linear image extending left and right across the left line virtual image 251 and the right line virtual image 252, and moves as if approaching from a distance as the sixth virtual image 260 moves.

7A is a diagram showing a virtual image 200 when the scheduled speed limit change point is far away. FIG. 7B is a diagram showing a virtual image 200 in a situation where the vehicle 1 has traveled further than in FIG. 7A and the scheduled speed limit change point is approaching. In FIGS. 7A and 7B, the virtual images 200 are a sixth virtual image 260, a seventh virtual image 270, and a ninth virtual image 290. The sixth virtual image 260 shown in FIG. 7B moves along the travel path 607 and is displayed at the same size as the sixth virtual image 260 shown in FIG. 7A. The ninth virtual image 290 shown in FIG. 7B moves along the travel path 606 and is displayed larger (longer) than the ninth virtual image 290 shown in FIG. 7A. In this way, by moving the positions of the sixth virtual image 260 and the ninth virtual image 290 along the travel path 607 as the vehicle 1 travels, the approaching speed limit change point can be made to appear closer. Furthermore, by not changing the size of the sixth virtual image 260 as it moves, the sixth virtual image 260 can continue to be displayed at a sufficiently large size, and the visibility of the sixth virtual image 260 can be maintained at a high level for a long period of time. Because the sixth virtual image 260 moves along a movement path 607 that follows the left line virtual image 251 (or the right line virtual image 252) that resembles a white line, it is possible to recognize that the sixth virtual image 260 is approaching without gradually increasing the size of the sixth virtual image 260.

Figure 7C shows the virtual image 200 after the speed limit has changed. In Figure 7C, the virtual image 200 includes the eighth virtual image 280. When the sixth virtual image 260 in Figure 7B moves along the movement path 607 and comes sufficiently close to the seventh virtual image 270, the sixth virtual image 260, the seventh virtual image 270, and the ninth virtual image 290 are hidden, and the eighth virtual image 280 is displayed at a specific position instead of the seventh virtual image 270. The "timing at which the eighth virtual image 280 is displayed instead of the seventh virtual image 270" is the same as the "timing at which the third virtual image 230 is displayed instead of the second virtual image 220" described above using Figure 5, and therefore further explanation will be omitted.

Referring again to FIG. 2, the vehicle 1 may include a speed sensor 401 (an example of a foreground speed-related information detection unit) that detects the speed of the vehicle 1 (an example of foreground speed-related information). The processor 16 may acquire the speed of the vehicle 1 and estimate the approach speed of the foreground 300 based on this speed.

Vehicle 1 may include a road information database 402 (an example of a road shape acquisition unit) such as a navigation system, or may acquire road information from an external road information database 402. Based on the position of vehicle 1 from the vehicle position detection unit 403 (described below), road information database 402 can read road information (including lanes, white lines, stop lines, crosswalks, road width, number of lanes, the presence, location, shape, and speed limit of intersections, curves, and forks) and transmit it to processor 16. Processor 16 may acquire the road information, estimate the approach direction of foreground 300, and determine the movement path of the image. Specifically, processor 16 may determine each of movement paths 601, 602, and 603 of the image based on the road information. As described above, processor 16 can also acquire the speed limit (or a speed limit that is scheduled to change in the future) from road information database 402.

Vehicle 1 may include a vehicle position detection unit 403 (an example of a foreground speed-related information detection unit) that is configured using a GNSS (Global Navigation Satellite System) or the like. The vehicle position detection unit 403 continuously or intermittently acquires vehicle 1 position information (an example of foreground speed-related information), and can estimate the approaching speed of the foreground 300 by analyzing changes in the position information over time, and transmit the estimated speed to processor 16. Note that the function of analyzing the position information to estimate the approaching speed of the foreground 300 may be provided in processor 16.

The foreground speed-related information detection unit may be configured with a vehicle position detection unit 403 and may estimate the approaching speed of the foreground 300 using only the position information from the vehicle position detection unit 403, or may be configured with a road information database 402 and the vehicle position detection unit 403. In this case, the foreground speed-related information detection unit may more accurately estimate the approaching speed of the foreground 300 (in other words, the speed of the vehicle 1) by detecting changes in the position of the vehicle 1 along the road from the position information from the vehicle position detection unit 403, which is continuously or intermittently acquired, and the road information from the road information database 402.

Vehicle 1 may include one or more external sensors 404 that detect real objects present around vehicle 1 (particularly in front of vehicle 1 in this embodiment). External sensors 404 detect, for example, white lines (demarcation lines) 311, 312 ahead of vehicle 1. Real objects detected by external sensors 404 may include, in addition to white lines, pedestrians, bicycles, motorcycles, other vehicles (such as preceding vehicles), roadside objects, and buildings. External sensors include, for example, radar sensors such as millimeter-wave radar, ultrasonic radar, and laser radar, and camera sensors consisting of a camera and an image processing device. The system may be configured with a combination of both radar sensors and camera sensors, or with only one of them. Conventional, well-known methods are used for object detection using these radar sensors and camera sensors. Object detection using these sensors may detect the presence or absence of real objects in three-dimensional space, and, if a real object is present, the position (relative distance from the vehicle, direction (lateral position), etc.), size (lateral, vertical, etc.), and type of the real object. One or more external sensors 404 can detect real objects (at least white lines 311, 312) ahead of the vehicle 1 at each sensor's detection cycle, acquire various information, and transmit external world information signals (information on the presence or absence of real objects, and if real objects are present, information on the position, size, type, etc. of each real object) to the processor 16. Note that these external world information signals may also be transmitted to the processor 16 via other devices (for example, the ECU of the vehicle 1). Furthermore, if a camera is used as a sensor, an infrared camera or near-infrared camera is desirable so that real objects can be detected even when the surroundings are dark, such as at night. Furthermore, if a camera is used as a sensor, a stereo camera that can also acquire distance, etc. using parallax is desirable.

The external sensor 404 is composed of a detection unit such as a camera or sensor, and an image analysis unit, and is capable of capturing (detecting) an image ahead of the vehicle 1, calculating the optical flow of the captured image (in other words, a movement vector indicating the movement speed (approach speed) and direction for each area of the foreground 300) through image processing (signal processing), and transmitting the processing results to the processor 16. Furthermore, by analyzing the optical flow, it is possible to recognize the road shape, such as the width and curvature of the road surface 310 in the foreground 300 of the vehicle 1, and may function as the road shape acquisition unit.

The external sensor 404 can also calculate a first optical flow on the road surface 310 in the straight-ahead direction of the vehicle, a second optical flow a predetermined height above the left edge 311 of the road surface 310 (the dividing line on the left side of the road surface 310), and a third optical flow a predetermined height above the right edge 312 of the road surface 310 (the dividing line on the right side of the road surface 310). The processor 16 sets the first optical flow calculated by the external sensor 404 as a first movement path 601, the second optical flow as a second movement path 602, and the third optical flow as a third movement path 603. Note that in Figure 4, only the movement direction in the optical flow is indicated by the arrow, and the size of the arrow does not reflect the magnitude of the speed in the optical flow.

The vehicle 1 may include an eye point detection unit 405 consisting of an infrared camera or the like that detects the eye position of the driver 4. The processor 16 can acquire images captured by the infrared camera and identify the eye position of the driver 4 by analyzing the acquired images. The processor 16 may acquire information on the eye position of the driver 4 identified from the image captured by the infrared camera from the I/O interface 14. The method of acquiring the eye position of the driver 4 of the vehicle 1 or information that can estimate the eye position of the driver 4 is not limited to these, and may also be acquired using known eye point detection (estimation) technology. The processor 16 may at least adjust the position of the virtual image 200 based on the eye position of the driver 4, thereby allowing the virtual image 200 to be viewed superimposed at a desired position on the foreground 300.

The software components stored in memory 18 include an approach speed detection module 502, an approach speed estimation module 504, a preceding vehicle detection module 506, an eye point detection module 508, and a graphics module 510.

The approach speed detection module 502 detects the approach speed of the foreground 300 toward the vehicle 1 as the vehicle 1 moves forward. The approach speed detection module 502 includes various software components for performing various operations related to determining which of multiple speed ranges the approach speed of the foreground 300 belongs to and detecting the approach speed of the foreground 300. The approach speed detection module 502 receives, for example, the optical flow of the foreground 300 from the outside-vehicle sensor 404 and identifies the approach speed of the foreground 300 near the movement path of the virtual image 200.

The approach speed detection module 502 uses one or more speed thresholds to determine whether the approach speed of the foreground 300 is below a certain speed. In some embodiments, the speed thresholds may be adjusted according to the state of the vehicle 1, the driving scene of the vehicle 1, and the state of the driver 4.

The approach speed estimation module 504 acquires foreground speed-related information (information that allows for estimating the approach speed of the foreground 300) and estimates the approach speed of the foreground 300. The approach speed detection module 502 includes various software components for performing various operations related to detecting the speed of the vehicle 1 (foreground speed-related information) from the speed sensor 401, estimating the approach speed of the foreground 300 calculated from the speed of the vehicle 1, detecting the position information of the vehicle 1 (foreground speed-related information), and estimating the approach speed of the foreground 300 by analyzing changes over time in the position information of the vehicle 1. The approach speed estimation module 504 receives, for example, the speed of the vehicle 1 (foreground speed-related information) from the speed sensor 401 and the position information of the vehicle 1 (foreground speed-related information) from the host vehicle position detection unit 403, and estimates the approach speed of the area of the foreground 300 that overlaps with or is close to the movement path of the virtual image 200.

The preceding vehicle detection module 506 detects the position of a preceding vehicle (reference numeral 330 in FIG. 8) traveling ahead of the vehicle 1. The preceding vehicle detection module 506 includes various software components for performing various operations related to detecting the position of the preceding vehicle 330 and determining whether or not a preceding vehicle 330 is approaching within a predetermined distance of the vehicle 1. The preceding vehicle detection module 506, for example, obtains the position of a real object from the external vehicle sensor 404 and determines whether or not a preceding vehicle 330 is traveling ahead of the vehicle 1.

The eye point detection module 508 detects the eye position of the driver 4 of the vehicle 1. The eye point detection module 508 includes various software components for performing various operations related to determining where the driver 4's eyes are located within a multi-level height range, detecting the driver's 4's eye height (position in the Y-axis direction), detecting the driver's 4's eye height (position in the Y-axis direction) and depth direction position (position in the Z-axis direction), and detecting the driver's 4's eye position (position in the X, Y, and Z axes). The eye point detection module 508, for example, acquires the driver's 4's eye position from the eye point detection unit 405, or receives information from the eye point detection unit 405 that enables estimation of the driver's (4) eye position, including the eye height, and estimates the driver's 4's eye position, including the eye height.

The graphics module 510 includes various known software components for changing the visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics), size, display position, and display distance (distance from the driver 4 to the virtual image 200) of the displayed virtual image 200.

Specifically, when displaying the virtual image 200 of the AR image, the graphics module 510 changes at least the size and display position at a speed slower than the approach speed of the foreground 300, so that the virtual image 200 (first virtual image 210, third virtual image 230) appears to be approaching the vehicle 1. For example, when the virtual image 200 (third virtual image 230) is moved along the first movement path 601 that overlaps with the road surface 310 shown in FIG. 4, the display position is gradually moved downward in the vertical direction, while the size of the virtual image 200 (third virtual image 230) is gradually enlarged. Furthermore, when the virtual image 200 (first virtual image 210) is moved along the second movement path 602 shown in FIG. 4, the display position is gradually moved leftward in the horizontal direction, while the size of the virtual image 200 is gradually enlarged. If the height of the second movement path 602 from the road surface 310 is set to a position lower than the eye height of the driver 4, the display position of the virtual image 200 also gradually moves downward in the vertical direction. The graphic module 510 may change at least the size and display position of the virtual image 200 (first virtual image 210, third virtual image 230) at a speed approximately equal to the approach speed of the foreground 300.

When displaying the non-AR virtual image 200 (fifth virtual image 250), the graphic module 510 changes at least the display position according to the approach speed of the foreground 300, thereby displaying the virtual image 200 (sixth virtual image 260, seventh virtual image 270) so that it appears to be approaching the vehicle 1. For example, when moving the virtual image 200 (seventh virtual image 270) along the movement path 606 shown in FIG. 7A, the display position is gradually moved downward in the vertical direction, while the size of the virtual image 200 (seventh virtual image 270) is gradually enlarged. When moving the virtual image 200 (sixth virtual image 260) along the movement path 607 shown in FIG. 7A, the display position is gradually moved to the left in the horizontal direction and also gradually moved downward in the vertical direction.

In some embodiments, the graphics module 510 stores data representing the image elements to be used. Each image element is optionally assigned a corresponding code. The graphics module 510 receives one or more codes specifying the image elements to be displayed, along with coordinate data and other property data (size, azimuth, elevation), from one or more processors 16, and generates screen image data to be output to the image display units 11, 12. The graphics module 510 continuously changes the coordinate data and other property data (size) to make the virtual image 200 appear to be approaching the vehicle 1. Note that the graphics module 510 may also intermittently change the size of the virtual image 200.

Each of the above modules corresponds to executable instructions and methods described herein (e.g., computer-executable programs) that perform one or more of the above functions.

Some or all of the above modules send requests to the speed sensor 401, road information database 402, vehicle position detection unit 403, external sensor 404, and eye point detection unit 405 via the I/O interface 14 at predetermined intervals to transmit various types of information. In response, the I/O interface 14 acquires various types of information from the speed sensor 401, road information database 402, vehicle position detection unit 403, external sensor 404, and eye point detection unit 405 and sends it to the processor 16. In other embodiments, some or all of the above modules may request the transmission of information only when an important event occurs (for example, when it is detected that the vehicle 1 has started moving).

As described above, in this embodiment, the vehicle display device 10 comprises an image display unit 11 (12) that displays an image, one or more I/O interfaces 14, one or more processors 16, memory 18, and one or more computer programs stored in the memory 18 and configured to be executed by the one or more processors 16. The one or more I/O interfaces 14 acquire speed limit information for the driving lane in which the vehicle is traveling, information regarding the approaching speed of the foreground 300 toward the vehicle as the vehicle advances, or information that allows the approaching speed of the foreground 300 to be estimated, and the one or more processors 16 display the speed limit information for the driving lane in which the vehicle is traveling on the image display unit 11 (12). The following command is executed: a first image 210 (260) including a first numeric image 211 (261) indicating the numerical value of the speed limit after the scheduled speed limit change; and a second image 220 (270) including a second numeric image 221 (271) indicating the numerical value of the speed limit of the driving lane in which the vehicle is currently traveling; the first image 210 (260) is moved toward the second image 220 (270) at a speed based on the approach speed; the visibility of the first numeric image 211 (261) and the second image 220 (270) is reduced before the first numeric image 211 (261) overlaps with the second image 220 (270); and a third image 230; 280 including the first numeric image 211 (261) is displayed in the position where the second image 220 (270) was displayed.

In addition, in some embodiments, the image display unit 11 (12) displays an image in a display area 100 that overlaps the foreground 300 as seen by a viewer in the driver's seat of the vehicle, and one or more processors 16 execute instructions to display the first image 210 so that it appears to be approaching the vehicle at a speed slower than the approach speed of the foreground 300 based on the approach speed.

In some embodiments, the one or more processors 16 also execute instructions to further display at least one of a perspective left line image 251 simulating the lane markings on the left side of the driving line and a perspective right line image 252 simulating the lane markings on the right side of the driving line, display the first image 210 (260) above the far side of the left line image 251 or the right line image 252, display the second image 220 (270) above the near side of either the left line image 251 or the right line image 252 above which the first image 210 (260) is displayed, and move the first image 210 (260) toward the second image 220 (270) along either the left line image 251 or the right line image 252 at a speed based on the approach speed.

Also, in some embodiments, the one or more processors 16 execute instructions to maintain the size of the first image 210 (260) as the first image 210 (260) is moved toward the second image 220 (270) along either the left line image 251 or the right line image 252 at a speed based on the approach speed.

1: Vehicle 2: Front windshield 4: Driver 5: Dashboard 10: Vehicle display device 11: Image display unit 11a: Display light 12: Image display unit 14: I/O interface 16: Processor 18: Memory 20: Image processing circuit 100: Display area 200: Virtual image 210: First image (first virtual image)
211: First numerical image 212: First background image 220: Second image (second virtual image)
221: Second numerical image 222: Second background image 230: Third image (third virtual image)
240: Fourth image (fourth virtual image)
250: Fifth virtual image 251: Left line image (left line virtual image)
252: Right line image (right line virtual image)
260: Sixth virtual image 270: Seventh virtual image 280: Eighth virtual image 290: Ninth virtual image 300: Foreground 310: Road surface 401: Speed sensor 402: Road information database 403: Vehicle position detection unit 404: Vehicle exterior sensor 405: Eye point detection unit 502: Approach speed detection module 504: Approach speed estimation module 506: Leading vehicle detection module 508: Eye point detection module 510: Graphic module 600: Movement path 601: First movement path 602: Second movement path 603: Third movement path

Claims (4)

an image display unit (11, 12) for displaying an image;
one or more I/O interfaces (14);
one or more processors (16);
A memory (18);
one or more computer programs stored in the memory (18) and configured to be executed by the one or more processors (16);
The one or more I/O interfaces (14)
Speed limit information for the lane in which the vehicle is traveling;
Acquire information regarding the approaching speed of the foreground (300) to the vehicle as the vehicle travels, or information that allows the approaching speed of the foreground (300) to be estimated;
The one or more processors (16)
a first image (210; 260) including a first numeric image (211; 261) indicating the numerical value of the speed limit after the speed limit change scheduled ahead of the driving lane in which the vehicle is traveling, and a second image (220; 270) including a second numeric image (221; 271) indicating the numerical value of the speed limit of the driving lane in which the vehicle is currently traveling, on the image display unit (11, 12);
moving the first image (210; 260) toward the second image (220; 270) at a speed based on the approach speed;
The vehicle image display device (10) executes an instruction to reduce the visibility of the first image (210; 260) and the second image (220; 270) before the first image (210; 260) comes close enough to the second image (220; 270) and the first numerical image (211; 261) overlaps the second image (220; 270), and to display a third image (230; 280) including the first numerical image (211; 261) at the position where the second image (220; 270) was displayed. The image display unit (11, 12) displays the image in a display area (100) that overlaps with the foreground (300) as seen by a viewer in the driver's seat of the vehicle;
The one or more processors (16)
Execute instructions to display the first image (210) so that it appears to be approaching the vehicle at a speed slower than the speed of approach of the foreground (300) based on the approaching speed.
The vehicle display device (10) of claim 1. The one or more processors (16)
Further displaying at least one of a left-side line image (251) in perspective simulating a lane marking on the left side of the driving line and a right-side line image (252) in perspective simulating a lane marking on the right side of the driving line;
The first image (210; 260) is displayed above the far side of the left line image (251) or the right line image (252);
The second image (220; 270) is displayed on the upper side of either the left line image (251) or the right line image (252) on which the first image (210; 260) is displayed,
executes a command to move the first image (210; 260) toward the second image (220; 270) along either the left line image (251) or the right line image (252) at a speed based on the approach speed;
The vehicle display device (10) of claim 1. The one or more processors (16)
executes instructions to maintain the size of the first image (210; 260) when moving the first image (210; 260) toward the second image (220; 270) along either the left line image (251) or the right line image (252) at a speed based on the approach speed;
The vehicle display device (10) according to claim 3.