JP7818166B2 - Display control device, head-up display device, and display control method - Google Patents

Description

This disclosure relates to a display control device, a head-up display device, and a display control method that are used in a vehicle and that superimpose an image onto the vehicle's foreground for viewing.

Patent Document 1 discloses a head-up display device (HUD device) in which the upper edge of a virtual image display area in which an image (virtual image) is displayed is positioned farther from the viewer than the lower edge. This type of HUD device can change the distance (display distance) from the viewer to the image (virtual image) by changing the display position of the image (virtual image) within the virtual image display area. In other words, it is possible to add the appearance of depth to the image (virtual image).

Japanese Patent Application Laid-Open No. 2021-51231

It is known that an optical illusion (sometimes called a corridor illusion) occurs in which the retinal image of a visual object perceived (and/or judged by the human brain) at a far-sighted position appears larger than the retinal image of the visual object perceived at a near-sighted position.

Even in HUD devices where the virtual image display area is tilted from a direction perpendicular to the road surface, the above-mentioned optical illusion tends to cause virtual images seen at a far-sighted position farther from the observer to appear larger than virtual images seen at a near-sighted position.

A summary of certain embodiments disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with an overview of these particular embodiments, and are not intended to limit the scope of the disclosure. Indeed, the present disclosure may encompass a variety of aspects that are not set forth below.

The outline of this disclosure relates to preventing a virtual image viewed at a far-sighted position from being perceived as larger than a virtual image viewed at a near-sighted position. The outline of this disclosure also relates to making it difficult to notice changes in the perceived size of a virtual image due to changes in display distance.

Therefore, the display control device in the first embodiment described in this specification is a display control device that controls a head-up display device that displays a virtual image of static content in a display area that is inclined with respect to the road surface so as to have a near-vision position close to the observer and a far-vision position far from the observer, and is equipped with an image position adjustment unit that adjusts the position of the static content, and an image size adjustment unit that performs a first size adjustment process that adjusts the size of the static content so that when the size of the static content when displayed at the near-vision position is a first size, when the static content is placed at the far-vision position, the static content is perceived by the observer at a second size that is smaller than the first size so as to prevent the static content from being perceived as larger due to a longer display distance.

In the first embodiment, the size of static content displayed at a far-sighted position, where it tends to be perceived as larger, is made smaller than when displayed at a near-sighted position. In other words, this has the advantage of suppressing (or offsetting or reversing) the tendency for content to be perceived as larger as the display distance increases by reducing the size. The size of static content can be expressed as the angle (visual angle) formed by the visual object projected onto the eye. However, because the position of the observer's eyes is not constant, the size of static content here is defined as the angle (visual angle) formed by the visual object with a predetermined point within the vehicle as its vertex. The first size can be said to be the angle (visual angle) formed by the static content when displayed at a near-sighted position with the predetermined point as its vertex, and the second size can be said to be the angle (visual angle) formed by the static content when displayed at a far-sighted position with the predetermined point as its vertex.

Furthermore, in a display control device according to a second embodiment that may be dependent on the first embodiment, the image size adjustment unit adjusts the first size and the second size in the first size adjustment process so as to satisfy the following formula: Ms<AS21/AS11<1. Here, Ms is the ratio of the size of an object at a far-vision position to the size of an object at a near-vision position as viewed from a predetermined position in the natural world, AS11 is the first size, and AS21 is the second size.

In the second embodiment, the magnification (reduction ratio) AS21/AS11 of the second size at the far-vision position relative to the first size at the near-vision position is set to be greater than the ratio of the size of an object at the far-vision position relative to the size of an object at the near-vision position as viewed from a predetermined position. In other words, the first size adjustment process in the second embodiment is more gradual than the representation of size changes relative to distance due to perspective, and is expected to have the advantage of suppressing (or offsetting or reversing) the tendency for the virtual image to be perceived as larger as the display distance increases, while suppressing the amount of change in size.

Furthermore, in a display control device according to a third embodiment that may be dependent on the first or second embodiment, static content is displayed so as to appear higher in relation to the road surface than the display area. In the third embodiment, a first size adjustment process is performed, particularly on content that is displayed higher in relation to the road surface than the display area.

Furthermore, in a display control device according to a fourth embodiment that may be dependent on one or more of the first to third embodiments, the image position adjustment unit further executes a first image movement process that moves static content from one of a near-vision position or a far-vision position to the other based on an operation by the viewer, and the first image movement process includes a visibility reduction process that reduces the visibility of the static content or hides it and displays one or more frame images surrounding the static content, a manual image movement process that moves one or more frame images from one of a near-vision position or a far-vision position to the other based on an operation by the viewer, and a visibility increase process that reduces the visibility of the frame image or hides it and increases the visibility of the static content at the position where the one or more frame images are located or re-displays it.

In the fourth embodiment, when a viewer moves static content by operation, the static content is hidden and one or more frame images surrounding the static content are displayed. The viewer can adjust the position of the static content (frame image) by operation while looking at the frame image. Once the position of the static content (frame image) is determined, the frame image is hidden and the static content is redisplayed. In this way, when the static content moves between the near-vision position and the far-vision position (in other words, the display distance of the static content changes), the displayed (or highly visible) image is switched from the static content to the frame image, which is expected to have the advantage of making it less noticeable that the change in perceived size due to the change in display distance occurs.

Furthermore, in a display control device according to a fifth embodiment that can be dependent on the fourth embodiment, the image position adjustment unit, in a manual image movement process, 1) when one or more frame images are moved from a near-vision position to a far-vision position, makes one frame image smaller or narrows the spacing between the multiple frame images depending on the position of the one or more frame images, and 2) when one or more frame images are moved from a far-vision position to a near-vision position, makes one frame image larger or widens the spacing between the multiple frame images depending on the position of the one or more frame images.

In the fifth embodiment, when a frame image moves, it is possible to suppress (or cancel out or reverse) the change in the perceived size of the frame image due to a change in display distance. By making the frame image smaller (narrowing the spacing between multiple frame images) as the display distance of one or more frame images increases, it is possible to suppress, cancel out or reverse (conversely, make the frame image perceived as smaller) the change in the perceived size of the frame image (which tends to be perceived as larger). On the other hand, by making the frame image larger (increasing the spacing between multiple frame images) as the display distance of one or more frame images decreases, it is possible to suppress, cancel out or reverse (conversely, make the frame image perceived as larger) the change in the perceived size of the frame image (which tends to be perceived as smaller).

Furthermore, in a display control device according to a sixth embodiment that may be dependent on the fourth embodiment, the image position adjustment unit hides static content during visibility reduction processing, and when one or more frame images are moved from one of the near-vision position or the far-vision position to the other during manual image movement processing, maintains the size of the one or more frame images or maintains the spacing between the multiple frame images, and when the static content is redisplayed during visibility increase processing, adjusts the size of the static content depending on the position at which the static content is displayed.

In the sixth embodiment, when a frame image is moved, the size of the frame image does not change regardless of the position, which is expected to have the advantage of reducing the sense of discomfort caused by size changes that accompany movement. In addition, it is expected to have the advantage of reducing the processing load of size changes that accompany movement.

Furthermore, in a seventh embodiment of the display control device, which may be dependent on one or more of the fourth to sixth embodiments, the image position adjustment unit further performs a second image movement process that moves static content from one of the near-vision position and the far-vision position to the other based on at least one of vehicle information about the vehicle in which the head-up display is installed, environmental information about the environment in which the vehicle is traveling, and user information about the viewer, and the second image movement process performs a second size adjustment process that is different from the first size adjustment process.

Furthermore, in an eighth embodiment that can be dependent on the seventh embodiment, the display control device performs a second size adjustment process in which, when the size of static content displayed at a near-vision position is a third size, the size of the static content is adjusted so that when the static content is placed at a far-vision position, it is viewed by the viewer at a fourth size that is smaller than the third size, and the image size adjustment unit adjusts the third size and fourth size in the second size adjustment process to satisfy the following formula: AS22/AS12<AS21/AS11<1, where AS11 is the first size, AS21 is the second size, AS12 is the third size, and AS22 is the fourth size.

Furthermore, the head-up display device in the ninth embodiment includes the display control device described in any one of the first to eighth embodiments, a display that displays an image on a display surface, and one or more relay optical systems that project display light of the image displayed by the display onto an external projection unit, thereby displaying a virtual image of the image within a display area that overlaps with the foreground as viewed from the eyebox, and the display area conjugate to the display surface is inclined with respect to the road surface.

Furthermore, the display control method in the tenth embodiment is a display control method for controlling a head-up display device that displays a virtual image of static content in a display area tilted with respect to the road surface so as to have a near-vision position close to the observer and a far-vision position far from the observer, and includes adjusting the position of the static content, and performing a first size adjustment process that adjusts the size of the static content so that, when the size of the static content when displayed at the near-vision position is a first size, the static content is visually perceived by the observer at a second size smaller than the first size when placed at the far-vision position.

FIG. 1 is a diagram showing an application example of a vehicle display system. FIG. 2 is a diagram showing the configuration of the image display unit. FIG. 3 is a diagram showing an example of a virtual image displayed by the head-up display device. FIG. 4 is a block diagram of a vehicle display system. FIG. 5 is a diagram showing the viewing angle of static content displayed at the near-vision position in the virtual image display area and the viewing angle of static content displayed at the far-vision position. FIG. 6A is a graph illustrating the variation in size of static content with viewing distance in some embodiments. FIG. 6B is a graph illustrating the variation in size of static content with viewing distance in some embodiments. FIG. 6C is a graph illustrating the variation in size of static content with viewing distance in some embodiments. FIG. 6D is a graph illustrating the variation in size of static content with viewing distance in some embodiments. FIG. 7 is a diagram illustrating the resizing process in some embodiments. FIG. 8 is a diagram illustrating the resizing process in some embodiments.

The following provides an explanation of this embodiment with reference to Figures 1 to 8. Note that the present invention is not limited to the following embodiment (including the contents of the drawings). Naturally, modifications (including the omission of components) can be made to the following embodiment. In addition, in the following explanation, explanations of well-known technical matters will be omitted as appropriate to facilitate understanding of the present invention.

Refer to Figure 1. The vehicle display system 10 of this embodiment is composed of an image display unit 20, a display control device 30 that controls the image display unit 20, and electronic devices (described below) that are connected to the display control device 30.

The image display unit 20 in the vehicle display system 10 is a head-up display (HUD) device installed in the dashboard 5 of the vehicle 1. The image display unit 20 emits display light 40 toward the windshield 2 (an example of a projection target), which reflects the display light 40 of the image M displayed by the image display unit 20 toward the eye box 200. By placing the eye 4 inside the eye box 200, the observer can view a virtual image V of the image M displayed by the image display unit 20 at a position superimposed on the foreground, which is the real space viewed through the windshield 2. Note that in the drawings used in this embodiment, the left-right direction of the vehicle 1 is the X-axis direction (the left side when facing forward of the vehicle 1 is the X-axis positive direction), the up-down direction is the Y-axis direction (the upper side of the vehicle 1 traveling on the road is the Y-axis positive direction), and the fore-aft direction of the vehicle 1 is the Z-axis direction (the front of the vehicle 1 is the Z-axis positive direction).

The term "eyebox" used in the description of this embodiment refers to (1) a region within which at least a portion of the virtual image V of image M is visible, but no portion of the virtual image V of image M is visible outside the region; (2) a region within which at least a portion of the virtual image V of image M is visible at a predetermined luminance or higher, but the entire virtual image V of image M is below the predetermined luminance outside the region; or (3) a region within which, if the image display unit 20 is capable of displaying a virtual image V that can be viewed stereoscopically, at least a portion of the virtual image V can be viewed stereoscopically, but no portion of the virtual image V can be viewed stereoscopically outside the region. In other words, when the observer positions their eyes (both eyes) 4 outside the eyebox 200, the observer cannot view the entire virtual image V of image M, the entire visibility of the virtual image V of image M is very low and difficult to perceive, or the virtual image V of image M cannot be viewed stereoscopically. The predetermined luminance is, for example, approximately 1/50 of the luminance of the virtual image of image M viewed at the center of the eyebox.

The virtual image display area 100 is a flat, curved, or partially curved area where the image M generated within the image display unit 20 is focused as a virtual image V, and is also referred to as an imaging surface. The virtual image display area 100 is the position where the display surface 21a (e.g., the exit surface of a liquid crystal display panel) of the display 21 of the image display unit 20, which will be described later, is focused as a virtual image. That is, the virtual image display area 100 corresponds to the display surface 21a of the image display unit 20, which will be described later (in other words, the virtual image display area 100 is conjugate with the display surface 21a of the display 21, which will be described later). The virtual image viewed in the virtual image display area 100 can be said to correspond to the image displayed on the display surface 21a of the image display unit 20, which will be described later. It is preferable that the virtual image display area 100 itself has such low visibility that it is not actually visible to the observer's eyes 4, or is difficult to see.

The downward angle based on the front-rear and left-right directions (XZ plane) of the vehicle 1 is defined as the depression angle θv. The smaller the depression angle θv, the higher the position is perceived by the observer (positive Y-axis direction), and the larger the depression angle θv, the lower the position is perceived by the observer (negative Y-axis direction). In the virtual image display area 100 of this embodiment, the depression angle θv at the near-vision position 101, which is close to the observer 4, is greater than the depression angle θv at the far-vision position 102, which is far from the observer 4. In other words, the virtual image display area 100 is positioned so that the upper area is farther away than the lower area when viewed from the observer.

In the description of this embodiment, the virtual image display area 100 is entirely positioned below (the upper end is positioned so that one looks down on) when viewed from the eyebox 200 (more specifically, for example, when viewed from the center 205 of the eyebox 200), but it may also be partially positioned above (the upper end is positioned so that one looks up on) when viewed from the eyebox 200 (more specifically, for example, when viewed from the center 205 of the eyebox 200).

The angle (tilt angle θt in Figure 1) formed by the horizontal direction (XZ plane) with the left-right direction (X-axis direction) of the vehicle 1 as its axis is set in the virtual image display area 100. The virtual image display area 110 in Figure 1 is positioned tilted from the road surface 310 with the left-right direction (X-axis direction) of the vehicle 1 as its axis, and has a tilt angle θt of, for example, 30 degrees. However, the tilt angle θt is not limited to this and can be changed within the range of 0≦θt<45 degrees.

The virtual image display area 120 (100) may be arranged approximately parallel to the road surface 310. The virtual image display area 120 (100) may also be curved concavely toward the viewer. The virtual image display area 130 (100) may also be curved convexly toward the viewer.

Figure 2 is a diagram showing the configuration of the HUD device 20 of this embodiment. The HUD device 20 includes a display 21 having a display surface 21a that displays an image M, and a relay optical system 25.

The display 21 in Figure 2 is composed of a liquid crystal display panel 22 and a light source unit 24. The display surface 21a is the viewer-side surface of the liquid crystal display panel 22 and emits display light 40 for the image M. The angle of the virtual image display area 100 (including the tilt angle θt) can be set by setting the angle of the display surface 21a relative to the optical axis 40p of the display light 40, which travels from the center of the display surface 21a through the relay optical system 25 and the projection area toward the eyebox 200 (the center of the eyebox 200).

The relay optical system 25 is arranged on the optical path of the display light 40 emitted from the display 21 (light traveling from the display 21 toward the eyebox 200), and is composed of one or more optical components that project the display light 40 from the display 21 onto the front windshield 2 outside the HUD device 20. The relay optical system 25 in Figure 2 includes a concave first mirror 26 and a flat second mirror 27.

The first mirror 26 has, for example, a free-form surface shape with positive optical power. In other words, the first mirror 26 may have a curved surface shape with different optical powers in different regions. That is, the optical power added to the display light 40 may differ depending on the region (optical path) through which the display light 40 passes. Specifically, the optical power added by the relay optical system 25 may differ between the first image light 41, second image light 42, and third image light 43 (see Figure 2) traveling from each region of the display surface 21a toward the eyebox 200.

The second mirror 27 is, for example, a flat mirror, but is not limited to this and may also be a curved surface with optical power. In other words, the relay optical system 25 may combine multiple mirrors (for example, the first mirror 26 and the second mirror 27 in this embodiment) to add different optical power depending on the area (optical path) through which the display light 40 passes. The second mirror 27 may also be omitted. In other words, the display light 40 emitted from the display 21 may be reflected by the first mirror 26 onto the projection target (front windshield) 2.

In addition, in this embodiment, the relay optical system 25 includes two mirrors, but is not limited to this. In addition to or instead of these, it may include one or more refractive optical elements such as lenses, diffractive optical elements such as holograms, reflective optical elements, or combinations of these.

In addition, the relay optical system 25 of this embodiment has the function of setting the distance to the virtual image display area 100 and the function of generating a virtual image that is an enlarged version of the image displayed on the display surface 21a, using this curved surface shape (an example of optical power). In addition, it may also have the function of suppressing (correcting) distortion of the virtual image that may occur due to the curved shape of the front windshield 2.

The relay optical system 25 may also be rotatable and equipped with actuators 28 and 29 controlled by the display control device 30.

The liquid crystal display panel 22 receives light from the light source unit 24 and emits spatially light-modulated display light 40 toward the relay optical system 25 (second mirror 27). The liquid crystal display panel 22 is, for example, rectangular, with the shorter side aligned in the direction in which pixels corresponding to the vertical direction (Y-axis direction) of the virtual image V as seen by the observer are arranged. The observer views the light transmitted through the liquid crystal display panel 22 via the virtual image optical system 90. The virtual image optical system 90 is a combination of the relay optical system 25 and the front windshield 2 shown in Figure 2.

The light source unit 24 consists of a light source (not shown) and an illumination optical system (not shown).

The light source (not shown) is, for example, a plurality of chip-type LEDs, and emits illumination light toward the liquid crystal display panel (an example of a spatial light modulation element) 22. The light source unit 24 is composed of, for example, four light sources, arranged in a row along the long side of the liquid crystal display panel 22. The light source unit 24 emits illumination light toward the liquid crystal display panel 22 under the control of the display control device 30. The configuration of the light source unit 24 and the arrangement of the light sources are not limited to this.

The illumination optical system (not shown) is composed of, for example, one or more lenses (not shown) arranged in the direction of emission of illumination light from the light source unit 24, and a diffuser plate (not shown) arranged in the direction of emission of the one or more lenses.

The display 21 may be a self-luminous display or a projection display that projects an image onto a screen. In this case, the display surface 21a is the screen of the projection display.

The display 21 may also be equipped with an actuator (not shown) including a motor controlled by the display control device 30, allowing the display surface 21a to move and/or rotate.

The relay optical system 25 has two rotation axes (first rotation axis AX1 and second rotation axis AX2) that move the eyebox 200 in the up-down direction (Y-axis direction). The first rotation axis AX1 and second rotation axis AX2 are set so that they are not perpendicular to the left-right direction (X-axis direction) of the vehicle 1 (in other words, not parallel to the YZ plane) when the HUD device 20 is attached to the vehicle 1. Specifically, the angle between the first rotation axis AX1 and the second rotation axis AX2 and the left-right direction (X-axis direction) of the vehicle 1 is set to less than 45 degrees, and more preferably less than 20 degrees.

The HUD device 20 includes a first actuator 28 that rotates the first mirror 26 about a first rotation axis AX1, and a second actuator 29 that rotates the first mirror 26 about a second rotation axis AX2. In other words, the HUD device 20 rotates one relay optical system 25 about two axes (the first rotation axis AX1 and the second rotation axis AX2). Note that the first actuator 28 and the second actuator 29 may be integrated into one two-axis actuator.

In another embodiment, the HUD device 20 rotates the two relay optical systems 25 around two axes (a first rotation axis AX1 and a second rotation axis AX2). For example, the HUD device 20 may include a first actuator 28 that rotates the first mirror 26 around the first rotation axis AX1 and a second actuator 29 that rotates the second mirror 27 around the second rotation axis AX2.

Note that the arrangement of the first rotation axis AX1 and the second rotation axis AX2 is not limited to this, as long as rotation about the first rotation axis AX1 causes a relatively large amount of vertical movement of the eye box 200, and rotation about the second rotation axis AX2 causes a relatively large amount of vertical movement of the virtual image display area 100. Furthermore, driving by the actuator may include movement in addition to or instead of rotation.

Furthermore, in other embodiments, the HUD device 20 may not drive the relay optical system 25. In other words, the HUD device 20 may not have an actuator that rotates and/or turns the relay optical system 25. The HUD device 20 of this embodiment may have a wide eyebox 200 that covers the range of eye heights of drivers expected to use the vehicle 1.

The image display unit 20, under the control of the display control device 30 (described later), displays images near real objects 300, such as the road surface 310 of the driving lane, forks in the road, road signs, obstacles (pedestrians, bicycles, motorcycles, other vehicles, etc.), and features (buildings, bridges, etc.) that exist in the foreground, which is the real space (real scene) visible through the front windshield 2 of the vehicle 1, at positions overlapping the real objects 300, or at positions set based on the real objects 300, thereby allowing the observer (typically an observer seated in the driver's seat of the vehicle 1) to perceive visual augmented reality (AR). In the description of this embodiment, an image whose display position can be changed depending on the position of a real object 300 present in the real scene is defined as an AR image (dynamic content), and an image whose display position is set regardless of the position of the real object 300 is defined as a non-AR image (static content).

Figure 3 is a diagram showing a real object 300 present in the foreground, as viewed by an observer looking forward from the driver's seat of vehicle 1, and a virtual image V displayed by the HUD device 20 of this embodiment. The virtual image V shown in Figure 3 includes an AR (Augmented Reality) virtual image V10, whose displayed position, direction, and shape can be changed depending on the position, direction, and shape of the real object 300, and a non-AR virtual image V20, whose displayed position, direction, and shape are set regardless of the position, direction, and shape of the real object 300. The AR virtual image V10 is displayed at a position (target position PT) corresponding to the position of the real object 300 present in the real scene. The AR virtual image V10 is displayed, for example, at a position superimposed on or near the real object 300 to emphasize and notify the presence of the real object 300. In other words, the "position corresponding to the position of the real object 300 (target position PT)" is not limited to a position that is superimposed on the real object 300 as viewed by the observer, but may also be a position near the real object 300. Note that it is preferable that the AR virtual image V10 does not interfere with the visibility of the real object 300, but this is not restrictive.

The AR virtual image V10 shown in FIG. 3 includes navigation virtual images V11 and V12 that indicate the guide route, enhanced virtual images V14 and V15 that highlight and notify of objects of attention, and a POI virtual image V15 that indicates landmarks, specified buildings, etc. The position corresponding to the position of the real object 300 (target position PT) is the position of the road surface 311 (an example of real object 300) on which the navigation virtual images V11 and V12 are superimposed for these, the position around a person 313 (an example of real object 300) for the enhanced virtual image V13, the position near another vehicle 314 (an example of real object 300) for the enhanced virtual image V14, and the position around a building 315 (an example of real object 300) for the POI virtual image V15.

The non-AR virtual image (static content) V20 is positioned below the virtual image display area 100, and the area of the road surface 311 that overlaps with it, representing the real object 300, is closer to the vehicle 1 than the area of the road surface 311 that overlaps with the navigation virtual image V11 (V11, V12) in Figure 3. The non-AR virtual image (static content) V20 shown in Figure 3 includes a speed virtual image V21 indicating the speed of the vehicle 1 and a sign virtual image V22 indicating the speed limit (road sign), and these images are displayed at positions that are set regardless of the position of the real object 300. The upper end 100u of the virtual image display area 110 in Figure 3 is positioned farther from the observer than the lower end 100b. In other words, the virtual image V's imaging distance (display distance) becomes longer as the display position moves upward (positive Y-axis direction), and is perceived as being farther away. The static content V20 is rendered to appear higher relative to the road surface 310 than the virtual image display area 100. For example, even if the angle between the virtual image display area 100 in which the static content V20 is displayed and the road surface 310 is 20 degrees, the static content V20 is displayed so that the angle between the static content V20 and the road surface 310 perceived by the optical illusion is 90 degrees.

The display control device 30 in FIG. 4 includes one or more I/O interfaces 31, one or more memory units 33, and one or more processing circuits 35, and includes the functionality of the display control unit 500. FIG. 4 is merely one embodiment, and the components shown may be combined into fewer components, or additional components may be present. For example, some or all of the functionality of the display control unit 500 may be provided separately from the display control unit 500.

As shown, the processing circuit 35 is operably connected to the I/O interface 31. The I/O interface 31 communicates (also referred to as CAN communication) with a number of electronic control units (vehicle ECU 421) and a number of on-board sensors (posture sensor 410, vehicle speed sensor 422, operation unit 430, eye position detection unit 440) in accordance with the CAN (Controller Area Network) standard. The communication standard adopted by the I/O interface 31 is not limited to CAN, and may include wired communication interfaces such as CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART, or USB, or an in-vehicle communication (internal communication) interface which is a short-range wireless communication interface within a distance of several tens of meters, such as a personal area network (PAN) such as a Bluetooth (registered trademark) network or a local area network (LAN) such as an 802.11x Wi-Fi (registered trademark) network. The I/O interface 31 may also include an external communication interface, such as a wide area communication network (e.g., an Internet communication network) based on a wireless wide area network (WWAN0, IEEE 802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE 802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, 5G, or other cellular communication standards. The I/O interface 31 may also include a function to process (convert, calculate, analyze) information received from other connected electronic devices, etc.

As shown, the processing circuitry 35 is operatively coupled to the storage unit 33. More specifically, the processing circuitry 35 executes programs stored in the storage unit 33 to control the display system 10 (HUD device 20), for example, by generating and/or transmitting image data. The processing circuitry 35 may include at least one general-purpose microprocessor (e.g., a central processing unit (CPU)), at least one application-specific integrated circuit (ASIC), at least one field-programmable gate array (FPGA), or any combination thereof. The storage unit 33 may include any type of magnetic media, such as a hard disk, any type of optical media, such as a CD or DVD, any type of semiconductor memory, such as volatile memory, and non-volatile memory. Volatile memory may include DRAM and SRAM, and non-volatile memory may include ROM and NVRAM.

The HUD device 20 is operatively coupled to the processing circuitry 35. Thus, the image displayed by the light modulation element 51 may be based on image data received from the processing circuitry 35. The processing circuitry 35 controls the image displayed by the light modulation element 51 based on information obtained from the I/O interface 31.

The processing circuitry 35 constructs multiple functional blocks by executing various programs stored in the memory unit 33. Specifically, the processing circuitry 35 has an image position adjustment unit 510, an image size adjustment unit 520, and a determination unit 530 as functional blocks of the display control unit 500.

Based on the information obtained from the I/O interface 31, the image position adjustment unit 510 moves the static content V20 (1) upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102), or (2) downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101).

The image position adjustment unit 510 changes the position of the virtual image V by (1) changing the image data to be displayed on the display unit 21, (2) offsetting the position at which the image is displayed on the display surface 21a of the display unit 21, (3) driving the actuator 23 to offset the position at which it is projected onto the projection target unit 2, or a combination of these.

Based on the operation information acquired from the operation unit 430 via the I/O interface 31, the image position adjustment unit 510 executes a first image movement process to move the static content V20 (1) upward in the positive Y-axis direction (moving from the near-vision position 101 to the far-vision position 102), or (2) downward in the negative Y-axis direction (moving from the far-vision position 102 to the near-vision position 101).

In addition, the image position adjustment unit 510 may perform a second image movement process to move the static content V20 (1) upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102) or (2) downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101) based on at least one of vehicle information about the vehicle 1, environmental information about the environment in which the vehicle 1 is traveling, and user information about the observer, which are acquired from the I/O interface 31.

The vehicle information indicates the speed obtained from the vehicle speed sensor 422. The image position adjustment unit 510 may perform a second image movement process in which (1) if the speed is faster than a predetermined speed threshold, the image is moved upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102), and (2) if the speed is slower than the predetermined speed threshold, the image is moved downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101). Note that the image position adjustment unit 510 may change the image position stepwise and/or continuously depending on the speed.

The vehicle information may also be attitude information indicating the attitude of the host vehicle 1 acquired from the attitude sensor 410. The image position adjustment unit 510 may perform a second image movement process: (1) if the pitch angle (an example of attitude information) of the host vehicle 1 indicates that it is tilted forward more than a predetermined attitude threshold, the image is moved upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102); and (2) if the pitch angle (an example of attitude information) of the host vehicle 1 indicates that it is tilted backward more than a predetermined attitude threshold, the image is moved downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101). The image position adjustment unit 510 may change the position of the image stepwise and/or continuously depending on the attitude information (an example of vehicle information). The vehicle information is information related to the host vehicle 1 and is not limited to the speed information or attitude information described above. The vehicle information may also be information indicating the driving mode of the host vehicle 1.

The environmental information may be road information indicating the type of road on which the host vehicle 1 is traveling, obtained from a navigation device (not shown). The image position adjustment unit 510 may perform a second image movement process: (1) if it is estimated that the host vehicle 1 is traveling on a road on which the host vehicle 1 can travel at high speeds (e.g., a highway), the image is moved upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102); and (2) if it is estimated that the host vehicle 1 is traveling on a road on which the host vehicle 1 can travel at low speeds (e.g., an ordinary road or urban area), the image is moved downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101).

The environmental information may also be illuminance information indicating the illuminance around the host vehicle 1, obtained from an illuminance sensor (not shown). The image position adjustment unit 510 may perform a second image movement process: (1) if the illuminance around the host vehicle 1 is higher than a predetermined illuminance threshold, move the image upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102); and (2) if the illuminance around the host vehicle 1 is lower than the predetermined illuminance threshold, move the image downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101). The image position adjustment unit 510 may change the position of the image stepwise and/or continuously depending on the illuminance information (an example of environmental information). The environmental information is information related to the environment around the host vehicle 1, and is not limited to the road information or illuminance information described above.

The user information may be eye height information indicating the height of the viewer's eyes, obtained from the eye position detection unit 440. The image position adjustment unit 510 may perform a second image movement process: (1) if the viewer's eye height is higher than a predetermined height threshold, move the image upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102); and (2) if the viewer's eye height is lower than the predetermined height threshold, move the image downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101). The image position adjustment unit 510 may change the position of the image stepwise and/or continuously according to the eye height information (an example of user information).

The user information may also be biometric information including the viewer's heart rate, etc., acquired from a biometric sensor (not shown). The image position adjustment unit 510 may perform a second image movement process: (1) if the biometric information indicates that the viewer is in a mildly tense state, the image is moved upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102); and (2) if the biometric information indicates that the viewer is in a severely tense state, the image is moved downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101). The image position adjustment unit 510 may change the position of the image stepwise and/or continuously in accordance with the biometric information (an example of user information). The user information is information about the viewer and is not limited to the eye height information or biometric information described above.

The image size adjustment unit 520 performs a size adjustment process to (1) reduce the size when the image position adjustment unit 510 moves the static content V20 upward in the positive Y-axis direction (from the near-vision position 101 to the far-vision position 102), and (2) increase the size when the image position adjustment unit 510 moves the static content V20 downward in the negative Y-axis direction (from the far-vision position 102 to the near-vision position 101). Here, the size of the static content V20 is the angle formed by the visual object (static content V20) as seen from the viewpoint, and is also referred to as the visual angular or angular size. That is, as shown in FIG. 5, the image size adjustment unit 520 increases the viewing angle AS1 of the static content V20 displayed at the near-vision position 101 in the virtual image display area 100 compared to the viewing angle AS2 of the static content V20 displayed at the far-vision position 102. The image size adjustment unit 520 may include table data, calculation formulas, etc. for setting the viewing angle AS of the static content V20 from the display distance of the static content V20 (or the vertical position (display height) at which the static content V20 is displayed, or the operation information, vehicle information, environmental information, and user information that are the basis for setting the display distance and display height).

6A to 6D are graphs showing the change characteristic Q of the size of static content V20 with respect to viewing distance in some embodiments, and the change characteristic P of the size of an object in the natural world with respect to viewing distance. The horizontal axis is distance VD, and the vertical axis is the visual angle of the static content V20 or the object in the natural world viewed from a distance VD away. If the actual size of an object in the natural world is BS, the change characteristic P of the visual angle of the object viewed from a distance VD away is expressed by the relationship P = 2 arctan (BS/VD). That is, at a greater distance VD, the rate of change in the visual angle with respect to a change in distance VD is small, and at a shallower distance VD, the rate of change in the visual angle with respect to a change in distance VD is large. The image size adjustment unit 520 adjusts the visual angle AS of the static content V20 so that the change characteristic Q of the visual angle with respect to the viewing distance of the static content V20 changes more gradually with respect to distance VD than the change characteristic P of the visual angle with respect to the viewing distance of an object in the natural world.

In some embodiments, as shown in Figure 6A, the change characteristic Q1 has a small rate of change in the visual angle AS with respect to a change in the distance VD when the distance VD is deep, and a large rate of change in the visual angle AS with respect to a change in the distance VD when the distance VD is shallow. If the visual angle of an object in nature at a predetermined distance VD1 at the near-vision position 101 is M1, the visual angle of static content V20 is AS1, and the visual angle of an object in nature at a predetermined distance VD2 at the far-vision position 102 is M2, and the visual angle of static content V20 is AS2, the ratio of the visual angle at the far-vision position 102 to the visual angle at the near-vision position 101 is expressed by the relationship M2/M1<AS2/AS1<1 (in the claims, M2/M1 is referred to as Ms). For example, the ratio Ms of the size M2 when an object in nature is at 10 meters (distant vision position 102) to the size M1 when the object is at a distance VD of 4 meters (near vision position 101) is approximately 40%. In contrast, the image size adjustment unit 520 sets the ratio AS2/AS1 of the viewing angle AS2 when static content V20 is displayed at 10 meters (distant vision position 102) to the viewing angle AS1 when static content V20 is displayed at a distance VD of 4 meters (near vision position 101) to approximately 90%.

In some embodiments, the change characteristic Q2 may cause the viewing angle AS to decrease linearly as the distance VD increases, as shown in Figure 6B.

Furthermore, in the change characteristic Q3 in some embodiments, as shown in Figure 6C, the rate of change of the viewing angle AS with respect to a change in the distance VD is large when the distance VD is large, and the rate of change of the viewing angle AS with respect to a change in the distance VD is small when the distance VD is small. In particular, in the change characteristic Q3 shown in Figure 6C, the rate of change of the viewing angle AS with respect to a change in the distance VD is zero when the distance VD is small.

Furthermore, in some embodiments, the change characteristic Q4 may change gradually with respect to the change in distance VD, as shown in Figure 6D. Specifically, the change characteristic Q4 in Figure 6D is constant at a viewing angle AS2 regardless of the distance VD when the distance VD is large, and is constant at a viewing angle AS1 regardless of the distance VD when the distance VD is small.

The image position adjustment unit 510 executes a first image movement process that moves the static content V20 from one of the near-vision position 101 and the far-vision position 102 to the other based on the observer's operation of the operation unit 430. The first image movement process includes a visibility reduction process that reduces the visibility of the static content V20 or hides it and displays one or more frame images V30 surrounding the static content V20, a manual first image movement process that moves one or more frame images from one of the near-vision position 101 and the far-vision position 102 to the other based on the observer's operation, and a visibility increase process that reduces the visibility of the frame image V30 or hides it and increases the visibility of the static content V20 or re-displays it at the position where the one or more frame images V30 are located.

In the first manual image movement process, the image position adjustment unit 510: 1) when moving one or more frame images from the near-vision position 101 to the far-vision position 102, the image position adjustment unit 510 makes one frame image smaller or narrows the spacing between multiple frame images depending on the position of one or more frame images V30; and 2) when moving one or more frame images from the far-vision position 102 to the near-vision position 101, the image position adjustment unit 510 makes one frame image larger or widens the spacing between multiple frame images depending on the position of one or more frame images.

Figure 7 illustrates the size adjustment process in some embodiments, changing from (a) to (d). When an operation to move the static content V20 is detected, the image position adjustment unit 510 performs a visibility reduction process to reduce the visibility of the static content V20 (or may hide it) from the relatively high visibility state shown in Figure 7(a) to that shown in Figure 7(b), and display multiple (e.g., four) border images V30 surrounding the static content V20. The image position adjustment unit 510 narrows the spacing between the multiple border images V30, as shown in Figure 7(c), depending on the position of the static content V20 that moves based on the observer's operation. Once the movement due to the operation is complete, the image position adjustment unit 510 performs a visibility increase process to hide the border image V30 (or may reduce its visibility), and increase the visibility (brightness) of the static content V20 at the position where the multiple border images V30 are located, as shown in Figure 7(d). When the viewer operates to move the static content, the static content is hidden and one or more frame images surrounding the static content are displayed. The viewer can adjust the position of the static content (frame image) by operating while looking at the frame image. Once the position of the static content (frame image) is determined, the frame image is hidden and the static content is redisplayed. In this way, when the static content moves between the near-vision position and the far-vision position (in other words, the display distance of the static content changes), the displayed (or highly visible) image switches from the static content to the frame image, which is expected to have the advantage of making it less noticeable that the change in perceived size due to the change in display distance occurs.

In some embodiments, the image position adjustment unit 510 hides the static content V20 during the visibility reduction process, as shown in FIG. 8(b), and maintains the size of the one or more frame images or maintains the spacing between the multiple frame images during the manual first image movement process when one or more frame images are moved from one of the near-vision position 101 and the far-vision position 102 to the other, as shown in FIG. 8(c). When the static content V20 is redisplayed during the visibility increase process, the image position adjustment unit 510 adjusts the size of the static content V20 depending on the display position of the static content V20, as shown in FIG. 8(d). In this embodiment, when the frame image moves, the change in the perceived size of the frame image due to a change in display distance can be suppressed (or offset or reversed). By making the frame image smaller (narrowing the spacing between multiple frame images) as the display distance of one or more frame images increases, the change in the perceived size of the frame image (which tends to be perceived as larger) can be suppressed, offset, or reversed (conversely, made smaller). On the other hand, by making the frame images larger (increasing the spacing between multiple frame images) as the display distance of one or more frame images becomes shorter, it is possible to suppress, offset, or reverse (conversely, make the frame images appear larger) the change in perceived size of the frame images (which tend to be perceived as smaller).

In addition, in some embodiments, the image position adjustment unit 510 further performs a second image movement process to move the static content V20 from one of the near vision position 101 and the far vision position 102 to the other based on at least one of vehicle information related to the vehicle in which the HUD device 20 is installed, environmental information related to the environment in which the vehicle is traveling, and user information related to the observer. The determination unit 530 determines whether the first image movement process is a manual process based on operation information, or an automatic second image movement process based on vehicle information, environmental information, and user information. In the second image movement process, the image size adjustment unit 520 performs a second size adjustment process that is different from the first size adjustment process.

In the second size adjustment process, when the size of static content V20 when displayed at near vision position 101 is set to a third size, the size of static content V20 is adjusted so that when static content V20 is placed at far vision position 102, it is viewed by the viewer at a fourth size smaller than the third size. In the second size adjustment process, the image size adjustment unit 520 adjusts the third size and fourth size to satisfy the following formula: AS22/AS12<AS21/AS11<1, where AS11 is the first size, AS21 is the second size, AS12 is the third size, and AS22 is the fourth size.

1: Vehicle 2: Front windshield (projected area)
4: Observer (eye)
5: Dashboard 10: Display system 20: Image display unit (HUD device)
21: Display 21a: Display surface 22: Liquid crystal display panel 23: Actuator 24: Light source unit 25: Relay optical system 26: First mirror 27: Second mirror 28: First actuator 29: Second actuator 30: Display control device 31: I/O interface 33: Memory unit 35: Processing circuit 40: Display light 40p: Optical axis 41: First image light 42: Second image light 43: Third image light 51: Light modulation element 90: Virtual image optical system 100: Virtual image display area 100b: Lower end 100u: Upper end 101: Near vision position 102: Far vision position 110: Virtual image display area 120: Virtual image display area 130: Virtual image display area 200: Eye box 205: Center 300: Real object 410: Orientation sensor 421: Vehicle ECU
422: Vehicle speed sensor 430: Operation unit 440: Eye position detection unit 500: Display control unit 510: Image position adjustment unit 520: Image size adjustment unit 530: Determination unit AS: Viewing angle AS1: Viewing angle AS2: Viewing angle M: Image M1: Size M2: Size Ms: Ratio P: Change characteristic Q: Change characteristic Q1: Change characteristic Q2: Change characteristic Q3: Change characteristic Q4: Change characteristic V: Virtual image V10: AR virtual image V20: Non-AR virtual image (static content)
V30: Frame image VD: Distance

Claims (8)

A display control device (30) controls a head-up display device (20) that displays a virtual image of static content (V20) in a display area (100) that is tilted with respect to a road surface so as to have a near-vision position (101) close to a viewer and a far-vision position (102) far from the viewer,
an image position adjustment unit (510) that adjusts the position of the static content (V20);
an image size adjustment unit (520) that performs a first size adjustment process to adjust the size of the static content (V20) so that, when the size of the static content (V20) when displayed at the near-vision position (101) is a first size, the static content (V20) is visually recognized by a viewer at a second size smaller than the first size when the static content (V20) is placed at the far-vision position (102) to prevent the static content (V20) from being perceived as large due to an increase in display distance;
In the first size adjustment process, the image size adjustment unit (520) adjusts the first size and the second size so as to satisfy the following formula:
Ms<AS21/AS11<1
where Ms is the ratio of the size of the object at the far vision position (102) to the size of the object at the near vision position (101) as viewed from a predetermined position in nature, AS11 is the first size, and AS21 is the second size.
A display control device (30). The static content (V20) is expressed as being raised higher than the display area relative to the road surface.
2. A display control device (30) according to claim 1. The image position adjustment unit (510)
further performing a first image movement process of moving the static content (V20) from one of the near-vision position (101) and the far-vision position (102) to the other based on an operation by the viewer;
The first image movement process includes:
a visibility reduction process of reducing the visibility of the static content (V20) or hiding it and displaying a frame image surrounding the static content (V20);
a manual image movement process for moving the frame image from one of the near-vision position (101) and the far-vision position (102) to the other based on an operation by the observer;
a visibility increasing process of reducing the visibility of the frame image or not displaying it, and increasing the visibility of the static content (V20) at the position where the frame image is arranged or re-displaying it,
3. A display control device (30) according to claim 1 or 2. The image position adjustment unit (510) performs the following in the manual image movement process:
1) When the frame image is moved from the near-sighted position (101) to the far-sighted position (102), the frame image is made smaller or the intervals between the plurality of separated portions constituting the frame image are narrowed according to the position of the frame image;
2) When the frame image is moved from the far-sighted position (102) to the near-sighted position (101), the frame image is enlarged or the spacing between the plurality of separated portions constituting the frame image is increased depending on the position of the frame image;
4. A display control device (30) according to claim 3. The image position adjustment unit (510)
In the visibility reduction process, the static content (V20) is hidden,
In the manual image movement process, when the frame image is moved from one of the near vision position (101) and the far vision position (102) to the other, the size of the frame image is maintained, or the intervals between the plurality of parts that are spaced apart from each other and that make up the frame image are maintained;
When the static content (V20) is redisplayed by the visibility increasing process, the size of the static content (V20) is adjusted depending on the position where the static content (V20) is displayed.
4. A display control device (30) according to claim 3. The image position adjustment unit (510)
further performing a second image movement process to move the static content (V20) from one of the near-vision position (101) and the far-vision position (102) to the other based on at least one of vehicle information regarding a vehicle in which the head-up display device is installed, environmental information regarding an environment in which the vehicle is traveling, and user information regarding the viewer;
In the second image movement process, a second size adjustment process different from the first size adjustment process is executed.
A display control device (30) according to any one of claims 1 to 5. The second size adjustment process adjusts the size of the static content (V20) so that, when the size of the static content (V20) when displayed at the near-vision position (101) is a third size, the static content (V20) is visually recognized by the viewer at a fourth size smaller than the third size when the static content (V20) is placed at the far-vision position (102);
In the second size adjustment process, the image size adjustment unit (520) adjusts the third size and the fourth size so as to satisfy the following formula:
7. A display control device (30) according to claim 6.
AS22/AS12<AS21/AS11<1
However, AS11 is the first size, AS21 is the second size, AS12 is the third size, and AS22 is the fourth size. A display control device (30) according to any one of claims 1 to 7;
a display (21) for displaying an image on a display surface;
and one or more relay optical systems (25) that project the display light of the image displayed by the display (21) onto an external projection target, thereby displaying a virtual image (V) of the image within a display area (100) that overlaps with the foreground as viewed from the eyebox (200).