JP7574607B2 - Display control device, head-up display device, and image display control method - Google Patents

Description

The present invention relates to a display control device, a head-up display (HUD) device, and an image display control method that are mounted on a vehicle such as an automobile.

Patent Document 1 shows an in-vehicle stereo image display device that utilizes the principle of stereoscopic vision caused by binocular parallax to display information about the outside of the vehicle and guidance information for vehicle driving as three-dimensional stereo images (stereoscopic images). Paragraph [0031] of Patent Document 1 describes the effect of the invention as follows: "The image can be recognized with a sense of depth and distance that matches the actual scene seen by the occupant through the windshield, and has the advantage of being able to provide the occupant with accurate information about the outside of the vehicle and guidance information."

JP-A-7-144578 ([0004], [0031], FIG. 4, etc.)

The inventors have investigated a head-up display device (herein referred to as a parallax-type 3D HUD device) that uses a parallax-type 3D display as a display device and allows the driver to visually recognize a stereoscopic image by providing an arbitrary convergence angle to the image. As a result, the following problems became clear.

The parallax image is formed on an adjustment surface (imaging surface), which is a virtual plane at a point a specified distance away from a specified position on the driver's side. This specified distance is called the "adjustment distance." When the driver views the image for the left eye (parallax image for the left eye) and the image for the right eye (parallax image for the right eye) displayed on the adjustment surface, the driver's eyes are focused on the adjustment surface.

Meanwhile, the driver fuses the left eye image and the right eye image in his brain and recognizes them as a single stereoscopic virtual image (stereoscopic image). The position at which this stereoscopic virtual image is displayed (the distance as seen by the driver) is determined according to the convergence angle of the stereoscopic virtual image. The distance from a specified reference position on the driver's side to the position at which the viewer recognizes the stereoscopic virtual image is called the "convergence distance."

Because the vergence distance is greater than the accommodation distance, the two do not match. In other words, the driver's viewpoint is focused at the adjustment position, but the virtual image is perceived at a more distant convergence position. This results in a conflict in which the convergence position at which the driver perceives the virtual image does not match the adjustment position at which the driver's eyes are actually in focus due to the adjustment of the driver's eyes. This conflict is called a vergence-accommodation conflict (VAC). The vergence-accommodation conflict can cause the driver (viewer) to feel eye fatigue and annoyance.

Therefore, in the above-mentioned parallax-type 3D HUD device, for example, when a three-dimensional warning display is displayed on the road surface at the position of the vehicle ahead, which is the forward object, so that the warning display is seen to surround the vehicle ahead, problems arise, such as the driver feeling annoyed or uncomfortable, or the driver's eyes becoming fatigued, especially when the display continues for a certain period of time.

The inventor has identified this problem. The above-mentioned Patent Document 1 does not mention this problem or its solution at all.

One of the objects of the present invention is to provide a display control device, a HUD device (such as a parallax-type 3D HUD device), and an image display control method that can reduce (reduce) the viewer's discomfort and eye fatigue.

Other objects of the present invention will become apparent to those skilled in the art by reference to the following exemplary aspects and best modes, as well as the accompanying drawings.

Below, we will provide examples of embodiments according to the present invention to facilitate an understanding of the overview of the present invention.

In a first aspect of the present invention, there is provided a display control device comprising:
A display control device that performs display control in a head-up display (HUD) device that projects an image onto a projection target member, thereby allowing a viewer to view a virtual image of the image,
A control unit is provided that is capable of controlling a display mode of the virtual image based on an object distance that is a distance between a forward object and a reference position set on the viewer's side,
The control unit is
When the object distance exceeds a threshold value, a 3D display control is executed to display the virtual image as a stereoscopic virtual image by making each of the left and right eyes of the viewer view a left eye image and a right eye image having a parallax therebetween,
When the object distance is equal to or less than the threshold value, 2D display control is executed to display the virtual image as a planar virtual image by having each of the viewer's left and right eyes view the same image without parallax.

In the first aspect, when the viewer sees a distant object (such as a vehicle moving ahead) that is farther away than a position corresponding to the threshold value, the viewer can easily ensure sufficient visibility by using a good-looking, realistic 3D display. On the other hand, when the distance to the distant object (object distance) is equal to or less than the threshold value, the distant object is within an area where 3D display is possible, and the benefit of three-dimensional representation is difficult to feel. Therefore, the display is switched to 2D. This eliminates eye strain caused by convergence adjustment inconsistencies. In addition, since the display is close to the viewer, it is possible to view it at a certain size, and visibility is ensured even without 3D display. Note that the word "distant" when referring to a distant object means "visible at a distance from the viewer (farther back)."

In this way, when displaying a parallax-based 3D warning mark or the like in association with a future object (such as a vehicle ahead), if the object distance of the future object is equal to or less than a threshold (threshold distance), it is possible to eliminate convergence adjustment inconsistencies and reduce eye fatigue and discomfort.

In addition, with 2D display, virtual images are not perceived farther away than the position at which the eyes are focused, and even if the object approaches the viewer, the virtual image is unlikely to be perceived as being superimposed on the inside of the object. This also has the effect of reducing the viewer's discomfort (including annoyance) and reducing eye fatigue by suppressing it.

In addition, when a 3D display is used, if a viewpoint loss (loss of viewpoint position) occurs in either the left or right eye, it may not be possible to accurately view a stereoscopic image. This can cause discomfort and annoyance. With a 2D display, this does not need to be a concern, and the display is stable. This also reduces eye fatigue and strain on the viewer.

Note that the object in front of you includes obstacles in a broad sense. Here, obstacles can be broadly interpreted to include moving objects (such as a vehicle ahead or a person about to cross the road) as well as stationary objects (such as parked cars or motorcycles, a vehicle that has slowed down dramatically, objects that have fallen on the road, etc.).

In a second aspect dependent on the first aspect,
The control unit is
When there are a plurality of the distant objects, the distant object that appears largest to the viewer is selected, and for the selected distant object, it is determined whether or not it is necessary to switch between 3D display and 2D display;
When it is determined that switching is necessary, the display mode may be switched collectively, including the display of other preceding objects that were not selected.

In the second aspect, since a distant object that is perceived as large by the viewer also has a large impact on the viewer's vision, the size of the distant object (for example, the area occupied by the distant object, or the size of a rectangle or circle when the distant object is replaced with such a shape) is detected, and a decision is made as to whether or not to switch between 3D display and 2D display for the distant object that appears largest. When it is determined that switching is necessary, in principle, from the standpoint of ease of display control or from the standpoint of reducing the sense of discomfort felt by the viewer, it is preferable to switch the display mode all at once, including the display of other preceding objects that were not selected.

In a third aspect dependent on the second aspect,
The control unit is
When attempting to switch from the 3D display control to the 2D display control or from the 2D display control to the 3D display control for a selected destination object,
When the display priority of a destination object other than the selected destination object is higher than the display priority of the selected destination object, the switching may not be executed.

In the third aspect, when switching between 3D/2D for a selected object in the future, if there is another object in the future with a higher display priority, the display of that object in the future is prioritized and switching is not performed. For example, when a person attempting to cross the road is spotted near the vehicle and the urgency is high in terms of safety, even if the distance to the vehicle ahead of the vehicle changes and a change in 3D/2D is made, the display of the object with the higher urgency is prioritized, preventing an unexpected change in the display mode of the display with the higher urgency and preventing the viewer from feeling uncomfortable, thereby ensuring safety, etc.

In a fourth aspect dependent on any one of the first to third aspects,
The stereoscopic virtual image is
The image may be displayed so as to be superimposed on the ground or a surface equivalent to the ground in the vicinity of the forward object.

In the fourth aspect, a stereoscopic virtual image (3D display, stereoscopic display, etc.) produced by 3D display is not directly superimposed on the object ahead, but is superimposed on the ground (road surface) or the like below in the vicinity. Specifically, the display is, as an example, a warning display (or a warning display) that indirectly calls attention to the object ahead. Because this display is not directly superimposed on the object ahead, even if the object ahead (e.g., a vehicle ahead) approaches the viewer and, for example, the object ahead enters the 3D displayable range, the stereoscopic virtual image and the object ahead can be separated and distinguished, and visual discomfort is unlikely to occur. Therefore, this also reduces the strain on the viewer's eyes.

In addition, it is believed that displays superimposed on the road surface, etc. can often be enlarged in size. Therefore, it is believed that visibility can be ensured even when switching to 2D display, and no particular visual discomfort occurs when switching from 3D to 2D. This also reduces the strain on the viewer's eyes.

In a fifth aspect dependent from the fourth aspect,
The stereoscopic virtual image is
The display may be displayed so as to surround at least a portion of the forward object.

According to the fifth aspect, the stereoscopic virtual image (3D display, stereoscopic display, etc.) produced by 3D display, for example, surrounds at least a portion of the vehicle ahead, so that it becomes larger in size and easier to see. Also, as there is a portion extending from the near side to the far side as seen by the viewer, it is easy to get a sense of perspective, and therefore the sense of realism is increased when the display is in 3D. When the display is in 2D, the large size ensures sufficient visibility.

In a sixth aspect dependent on any one of the first to third aspects,
The stereoscopic virtual image may be superimposed on the forward object.

In the sixth aspect, a three-dimensional virtual image (e.g., a warning or alert display) can be directly superimposed on the object ahead using a 3D display. When a vehicle ahead approaches the vehicle, as described above, there is a concern that the three-dimensional virtual image may give the driver the impression that the vehicle is entering the interior of the vehicle ahead, creating a sense of discomfort. However, according to the present invention, the display is switched to 2D in such a situation, so the above problem does not occur.

In a seventh aspect, a head-up display device includes:
A head-up display (HUD) device that projects an image onto a projection target, thereby allowing a viewer to view a virtual image of the image,
an image generating unit that generates the image;
A display unit for displaying the image;
an optical system including an optical member that reflects display light of the image and projects the image onto the projection target;
A display control device according to any one of the first to sixth aspects;
has.

According to the seventh aspect, it is possible to effectively reduce and suppress eye fatigue and discomfort caused by inconsistencies in convergence adjustment in a parallax-type stereoscopic (3D) HUD device. Furthermore, conventional parallax-type 3D HUD devices have a visual problem in that when a preceding object approaches to a certain position, a virtual image as a 3D display becomes embedded in the preceding object, making it difficult to see. However, in the present invention, by appropriately setting a threshold value and switching to 2D display control in situations where such a problem is a concern, the above problem can be effectively solved, and it is possible to obtain the effect of reducing eye fatigue and strain while ensuring visibility, even when a preceding object approaches to a certain distance from the vehicle. Thus, the performance of the HUD device is improved.

In an eighth aspect dependent on the seventh aspect,
An image display control method, comprising:
When an object distance, which is a distance between a forward object and a reference position set on the viewer's side, exceeds a threshold, a 3D display control is executed to display a stereoscopic image by making each of the viewer's left and right eyes view a left eye image and a right eye image having a parallax,
When the object distance is equal to or less than the threshold value, 2D display control may be performed to display a planar image by having each of the viewer's left and right eyes view the same image without parallax.

According to the eighth aspect, when the object in front of the viewer is located at a certain distance from the viewer, the 3D display can provide a sense of depth, realism, and distance.

On the other hand, when the object in front of the viewer approaches a certain distance, an appropriately set threshold is used to detect this situation and the display is switched to 2D, preventing issues such as the virtual image display overlapping unnaturally with the object in front of the viewer or the viewer losing focus of one eye, making it possible to ensure a stable, easy-to-see display.

Those skilled in the art will readily appreciate that the exemplified embodiments of the present invention may be further modified without departing from the spirit of the present invention.

FIG. 1A is a diagram showing an example of the configuration of an in-vehicle system including a parallax-type 3D HUD device, and FIG. 1B is a diagram showing a 2D display without convergence. Figure 2(A) shows an example of a 3D display when the distance to the vehicle ahead (object distance) is further back than a threshold while the vehicle is traveling, and an example of a 2D display when the object distance is equal to or less than the threshold. Figure 2(B) shows details of the 3D display, and Figure 2(C) shows details of the 2D display. 3A and 3B are diagrams showing examples of setting the threshold value for switching between 3D display and 2D display. 4A and 4B are diagrams showing an example of display control when there are multiple objects in the distance. FIG. 5A is a diagram showing an example of the overall configuration of a parallax type 3D HUD device capable of switching between 3D and 2D, and FIG. 5B is a diagram showing an example of the configuration of a control unit and an image generating unit. 10 is a flowchart illustrating an example of a display control procedure. FIG. 7A is a diagram showing an example of a 3D display of an attention calling display (or a warning display) superimposed on a forward vehicle, and FIG. 7B is a diagram showing an example of a 2D display.

The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should note that the present invention is not unduly limited by the mode described below.

Please refer to Figure 1. Figure 1(A) shows an example of the configuration of an in-vehicle system including a parallax-based 3D HUD device, and Figure 1(B) shows a 2D display without congestion.

In FIG. 1(A) and (B), the direction along the line segment connecting the viewer's left and right eyes E1, E2 (in other words, the width direction of the vehicle 1) is the left-right direction (or horizontal direction: X direction), the direction along the line segment that is perpendicular to the left-right direction and perpendicular to the ground or a surface equivalent to the ground (here, road surface 6) is the up-down direction (or height direction: Y direction), and the direction along the line segment that is perpendicular to both the left-right direction and the up-down direction (directions indicating the forward and backward directions of the vehicle 1) is the front-rear direction (Z direction). The positive Z direction is the forward direction, and the negative Z direction is the backward direction. This is the same in the other drawings (FIG. 2, FIG. 3, FIG. 4, FIG. 7).

The in-vehicle system in FIG. 1(A) provided in a vehicle (own vehicle) 1 includes an pupil detection camera 43 for pupil (or face) detection that detects the line of sight and position of the driver's (viewer's) left eye EL and right eye ER, a front (broadly speaking, surroundings) imaging camera (e.g., a stereo camera) 45, an image processing unit 46 (including a distance measurement unit 47 and an object type/size detection unit 48), a HUD device 100, a communication unit (having functions such as GPS communication and vehicle-to-vehicle communication) 123, and an ECU 120 capable of collecting various information related to the vehicle 1 (e.g., lighting on/off information, vehicle speed information, engine information, etc.). If necessary, a radar unit 125 or the like may also be provided as a distance measurement means.

The distance measurement unit 47 included in the image processing unit 46, for example, refers to a pair of left and right original images captured by a stereo camera as the imaging camera 45, detects parallax for the same object (assumed to be the target object in the future) by, for example, stereo matching that searches for corresponding points in each image, and measures the distance to the target object in the future by the principle of triangulation based on this parallax. The radar unit 125 also measures the distance and direction to the target object (target object in the future) by emitting radio waves toward the target object and measuring the reflected waves. The information acquisition unit 119 of the HUD device 100 acquires the measured distance information, etc. as appropriate, and supplies it to the control unit 701 of the stereoscopic display device 111.

The HUD device 100 is installed, for example, in a dashboard (not shown in FIG. 1, reference numeral 41 in FIG. 2(A)). This HUD device 100 has a stereoscopic display device 111, an optical system 116, a light exit window 118, and an information acquisition unit 119. The information acquisition unit 119 can acquire various information from the communication unit 127, the ECU 120, the radar unit 125, the image processing unit 46, etc.

Here, the stereoscopic display device 111 is a parallax 3D display device. This stereoscopic display device (parallax 3D display device) 111 has an image generation unit 112, an image display unit (such as a liquid crystal display device, having an image display surface for displaying an image) 113, a light beam separation unit 114 that has a lenticular lens, a parallax barrier (parallax barrier), or the like, and separates the light emitted from the image display surface into light beams for the left and right eyes, and a control unit 701.

The control unit 701 can control the operation of the image generation unit (specifically, for example, image rendering) 112 and the image display unit 113, and can also switch between 2D display and 3D display. The control unit 701 may also be provided as a component of a display control device (not shown in FIG. 1(A), reference numeral 700 in FIG. 5(A)).

The optical system 116 has a curved mirror (concave mirror, etc.) 117 that reflects the light from the light beam separation unit 114 and projects the image display lights K1 and K2 onto the windshield (projected member) 2. However, it may further have other optical components (lenses, auxiliary reflectors, etc.).

In FIG. 1(A), a stereoscopic display device 111 of a HUD device 100 displays images with parallax (parallax images) for each of the left and right eyes. As shown in FIG. 1(A), each parallax image is displayed as virtual images 25L, 25R formed on an adjustment surface (imaging surface) PS. The focus of each eye of the viewer (person) is adjusted to match the position of the adjustment surface PS. The position of the adjustment surface PS is referred to as the "adjustment position," and the distance from a predetermined reference position to the adjustment surface PS (see reference symbol D1 in FIGS. 2(B) and (C)) is referred to as the adjustment distance.

However, in reality, the human brain fuses each image (virtual image), so the person perceives a three-dimensional virtual image (here, a navigation arrow graphic) 27 as being displayed on the convergence plane VS at a position further back than the adjustment position (a position determined by the convergence angle, which is called the "convergence position"). The three-dimensional virtual image 27 may be called a "three-dimensional virtual image," and if "image" is taken in a broad sense to include virtual images, it may also be called a "three-dimensional image." It may also be called a "three-dimensional image," "3D display," etc.

The control unit 701 in the HUD device 100 can switch between 2D display and 3D display by comparing the distance (referred to as the actual distance) from a reference point (reference position) set on the driver's (viewer's) side to an object (ahead object) measured by at least one of the distance measuring unit 47 and the radar unit 125 with a predetermined threshold. Examples of the reference position include the driver's viewpoint position and a reference point set on a part of the vehicle 1.

When 2D display control is executed, a planar virtual image 29 (a navigation arrow graphic) is displayed on the adjustment surface (imaging surface) PS, as shown in FIG. 1(B). The planar virtual image may also be referred to as a "planar virtual image," "planar image," "two-dimensional image," "two-dimensional display," etc.

Next, let us refer to FIG. 2. FIG. 2(A) shows an example of a 3D display when the distance to the vehicle ahead (object distance) is further back than a threshold while the vehicle is traveling, and an example of a 2D display when the object distance is equal to or less than the threshold. FIG. 2(B) shows details of the 3D display, and FIG. 2(C) shows details of the 2D display.

In FIG. 2(A), vehicle 1 is traveling on a straight road (road surface) 6. Ahead is a vehicle (forward vehicle) 200 that can be seen as an object ahead (obstacle in a broad sense). Note that when the surrounding image capturing camera 45 captures an image of the rear and detects a rear vehicle, etc., and an image (virtual image) of the rear vehicle, etc., is displayed, for example, in the lower right of the display area (HUD display area) 3, the rear vehicle can be the object ahead. When referring to an object ahead, "ahead" means "visible at a distance ahead (farther back) from the viewer."

The display area (HUD display area) 3 is typically a virtual area in which the adjustment surface (imaging surface) PS in FIG. 1(A) is displayed. The HUD device 100 is installed within the dashboard 41.

In the display example of FIG. 2(A), a warning display 300 is displayed as a three-dimensional virtual image superimposed on the road surface (surface equivalent to the ground) 6 in the vicinity of the upcoming object (vehicle ahead) 200. The display mode of the warning display 300 is switched depending on whether the object distance of the upcoming object 200 exceeds a threshold (however, depending on the driving scene, it may not be possible to switch). If it exceeds the threshold, it is displayed in 3D (see FIG. 2(B)), and if it is within the threshold, it is displayed in 2D (see FIG. 2(C)).

The object distance is the distance to the object ahead based on a reference position on the driver (viewer) 5 side (e.g., the viewpoint position of the driver 5 or a specific position of the vehicle 1) (see symbols D4a and D4b in Figures 2(B) and (C)), in other words, the distance between the reference position and the object ahead.

In addition, in FIG. 2(A), the attention-calling display 300 (3D) on the rear side, indicated by a solid line, indicates that it is a 3D display, and the attention-calling display 300 (2D) on the front side, indicated by a dashed line, indicates that it is a 2D display.

Specifically, the display area 3 (adjustment surface PS) is set at a distance of, for example, 5 m from the reference position, and the threshold value that serves as the reference for switching between 3D display and 2D display is set at, for example, 7 m. However, this is just an example and is not limiting.

If the object ahead (vehicle ahead) 200 in FIG. 2(A) is, for example, 8 m away from the reference position, the object distance exceeds the threshold (7 m) and is displayed in 3D. If the object ahead is, for example, 6 m away, the object distance satisfies the condition of being equal to or less than the threshold (7 m) and is displayed in 2D.

The 3D/2D switching will be explained using Figures 2(B) and (C). In Figures 2(B) and (C), the attention warning display 310 is a partial arc-shaped mark with a length equal to the width of the object ahead (vehicle ahead) 200 (however, this is only an example, and a mark similar to that in Figure 2(A) can also be used).

In FIG. 2B, the object distance D4a (e.g., 8 m, determined by the convergence angle θ1) is greater than the threshold Lth (e.g., 7 m). In other words, the object 200 ahead is farther away than the threshold Lth. In this case, the 3D display control causes the attention-calling display 310 (3D) to be displayed as a stereoscopic virtual image.

The adjustment distance from the reference position (here, the viewpoint position) to the adjustment surface (imaging surface) PS is D1 (e.g., 5 m), and parallax images 31L and 31R for the left and right eyes are formed on the adjustment surface PS (display area 3).

The viewer (driver) views the attention warning display 310 (3D) on the convergence plane VS at a position (convergence position) of convergence distance D2. The convergence angle of the stereoscopic virtual image 310 (3D) is θ1. Distance D3 corresponds to the difference between adjustment distance D1 and convergence distance D2a, and is the distance that causes a convergence adjustment conflict. In FIG. 2(B), R1 indicates a position (position corresponding to the threshold) that is away from the reference point by a threshold (threshold distance) Lth.

In FIG. 2(C), the object 200 ahead is located closer to the viewer (driver) than in FIG. 2(B). The object distance D4b is, for example, 5 m, which is smaller than the threshold Lth (7 m). Therefore, in the case of FIG. 2(C), the display is 2D, and the attention warning display 310 (2D) as an image without parallax is displayed in the display area 3 of the adjustment surface PS. In the example of FIG. 2(C), no convergence adjustment conflict occurs.

2(A) and (B), when the viewer 5 sees the object 200 ahead (such as a vehicle moving ahead) farther away than the position R1 corresponding to the threshold Lth (in other words, when the object distance D4a exceeds the threshold Lth), sufficient visibility can be easily ensured by a good-looking, realistic 3D (three-dimensional) display. On the other hand, when the object distance D2b of the object 200 ahead is equal to or less than the threshold Lth, the object 200 ahead is in an area where 3D display is possible, and the advantage of a three-dimensional representation is difficult to feel. Therefore, the display is switched to 2D (two-dimensional). This eliminates the strain on the eyes caused by the inconsistency in convergence adjustment.

In this way, when displaying a parallax-based 3D warning mark or the like in association with a future object (such as a vehicle ahead), if the object distance of the future object is equal to or less than a threshold (threshold distance), it is possible to eliminate convergence adjustment inconsistencies and reduce eye fatigue and discomfort.

In addition, the attention-calling display 300 (2D) shown by the dashed line in FIG. 2 (A) or the attention-calling display 310 (2D) in FIG. 2 (C) is located close to the viewer and can be seen at a certain size, so visibility is ensured even without 3D display.

In addition, with 2D display, virtual images are not perceived farther away than the position at which the eyes are focused, and even if the object approaches the viewer, the virtual image is unlikely to be perceived as being superimposed on the inside of the object. This also has the effect of reducing the viewer's discomfort (including annoyance) and reducing eye fatigue by suppressing it.

In addition, when a 3D display is used, if a viewpoint loss (loss of viewpoint position) occurs in either the left or right eye, it may not be possible to accurately view a stereoscopic image. This can cause discomfort and annoyance. With a 2D display, this does not need to be a concern, and the display is stable. This also reduces eye fatigue and strain on the viewer.

Note that the object in front of you includes obstacles in a broad sense. Here, obstacles can be broadly interpreted to include moving objects (such as a vehicle ahead or a person about to cross the road) as well as stationary objects (such as parked cars or motorcycles, a vehicle that has slowed down dramatically, objects that have fallen on the road, etc.).

The attention-calling display 300 as a three-dimensional virtual image shown in FIG. 2(A) and the attention-calling display 310 as a three-dimensional virtual image shown in FIG. 2(B) are not directly superimposed on the object 200 ahead (however, direct superimposition is not excluded: see FIG. 7), but are superimposed on the road surface 6 below in the vicinity (referred to as a road surface superimposition display). In other words, they function as an attention-calling display that indirectly calls attention to the object 200 ahead (or as a warning display that issues a warning).

Because this display is not directly superimposed on the object ahead (vehicle ahead) 200, even if the object ahead approaches the viewer 5 and, for example, the object ahead 200 enters the 3D displayable range (assuming it is farther away than the threshold Lth and 3D display is maintained), the three-dimensional virtual image and the object ahead are separated and distinguishable, making it less likely that a sense of visual discomfort will arise. Therefore, this also reduces the strain on the viewer's eyes.

In addition, it is believed that displays superimposed on the road surface, etc., can often be made larger in size. For example, the width of the attention warning display 310 in FIG. 2(B) is approximately the same as the width of the vehicle 200 ahead, making it a fairly large display. The same is true for the case of FIG. 2(A). Therefore, it is believed that visibility can be ensured even when switching to 2D display, and no particular visual discomfort occurs when switching from 3D to 2D. This also reduces the strain on the viewer's eyes.

The attention-calling display (three-dimensional virtual image) 300 in FIG. 2(A) is displayed so as to surround at least a portion of the object 200 ahead. Its shape is a partial ring shape (or a partial ellipse shape) in a plan view from a direction perpendicular to the road surface 6 (from above). However, this is not limited to this, and it may be a ring shape or an ellipse shape. Note that the shape of the attention-calling display 300 in FIG. 2(B) and (C) is a partial arc (a figure that gives a linear impression) that does not surround the object 200 ahead, but is not limited to this. For example, a portion extending from the near side to the far side as seen by the viewer 5 (in other words, a portion extending to sandwich the object 200 ahead from the left and right) may be provided.

When the three-dimensional virtual image surrounds at least a portion of the target object 200, it becomes larger in size and easier to see. In the case of a 3D display, the sense of realism is increased, and in the case of a 2D display, visibility is sufficiently ensured and visual discomfort can be sufficiently suppressed.

The above examples are cases where the three-dimensional display is not directly superimposed on the object (for example, road surface superimposed display), but this does not exclude displays that are directly superimposed on the object (object superimposed display). In other words, the three-dimensional virtual image may be superimposed on the object ahead.

Now, let us refer to FIG. 7. FIG. 7(A) is a diagram showing an example of a 3D display of a caution display (or a warning display) superimposed on the vehicle ahead, and FIG. 7(B) is a diagram showing an example of a 2D display. In the examples of FIG. 7(A) and (B), the caution display (warning display) is an image of an exclamation mark inside a triangular shape, and is displayed so as to be superimposed on the rear end surface of the vehicle ahead 202.

As shown in FIG. 7(A), it is possible to directly superimpose a three-dimensional virtual image (attention-calling display or warning display) 302 (3D) on the forward object (vehicle ahead) 202.

As described above, when the vehicle ahead 202 approaches the vehicle 1, there is a concern that the stereoscopic virtual image may appear to enter the interior of the vehicle ahead 202, creating a sense of discomfort. However, according to the present invention, in such a situation, the display is switched to 2D as shown in FIG. 2(B) (in other words, 302(2D) is displayed), so the above problem does not occur.

Next, refer to FIG. 3. FIGS. 3(A) and (B) are diagrams showing examples of setting the threshold for switching between 3D display and 2D display. In FIG. 3, parts that are common to the previous figures are given the same reference numerals (the same applies to other figures). In FIGS. 3(A) and (B), the area indicated by reference numeral 21 (area with sand markings) is an area where parallax 3D display is possible (area where parallax 3D display is possible).

Figure 3 (A) shows an example of an undesirable threshold setting. In Figure 3 (A), D1 indicates the adjustment distance, which is the distance from the position of the viewpoint E to the adjustment surface (imaging surface) PS, Lth1 indicates the threshold distance, R1 indicates the position corresponding to the threshold (threshold setting position), R2 indicates the tip position (rear position) of the area 21 where parallax-based 3D display is possible, and Z1 indicates the area that can be used for actual 3D display.

In FIG. 3A, position R1 corresponding to the threshold value is set farther away from the viewer than the tip position (rear position) R2 of area 21 where parallax 3D display is possible. The forward object, a vehicle 200 ahead, is located closer to the viewer than position R1 corresponding to the threshold value. In this state, the display is switched to 2D. In other words, although appropriate parallax 3D display is possible using area Z1, only 2D display is allowed. Therefore, this is not a preferable setting example.

To perform an appropriate parallax 3D display using region Z1, it is preferable to set threshold value Lth1 inside (which can include the position of R2) of tip position (rear position) R2 of region 21 where parallax 3D display is possible. On the other hand, since parallax 3D display (3D image) is viewed at a distance D3 behind adjustment surface PS, in order to perform an appropriate 3D display near the distant object, it is preferable that the distant object is at least a distance farther away than adjustment distance D1 plus distance D3.

Taking these points into consideration, in the preferred threshold setting example of FIG. 2(B), the position R1 corresponding to the threshold is set closer to the user than the tip position R2 of the area 21 where parallax-based 3D display is possible (which may include the position of R2), and further back than the position where a 3D image is recognized (the virtual image recognition position obtained by adding the distance D3 to the adjustment distance D1) (however, this may include the virtual image recognition position).

Next, let us refer to FIG. 4. FIGS. 4(A) and (B) are diagrams showing examples of display control when there are multiple objects ahead. In the example of FIG. 4(A), a vehicle 200 ahead and a person 250 about to cross the road are visible to the driver (viewer) 5.

In addition, a partial circular road surface overlay display 300 (3D) is displayed three-dimensionally to the vehicle 200 ahead as a warning display, and a highlighted frame 400 (3D) is displayed three-dimensionally to the person 250 as a warning display (warning display).

This can be said to be a state in which there are two objects ahead. In this case, the object ahead that is used as the basis for switching between 3D display and 2D display can be the preceding vehicle 200, which is the object ahead that appears largest to the viewer 5. In other words, when there are multiple objects ahead, the object ahead that appears largest to the viewer can be selected, and a determination can be made as to whether or not switching between 3D display and 2D display is required for the selected preceding object.

Since a distant object that is perceived as large by the viewer 5 has a large impact on the vision of the viewer 5, for example, the size of the distant objects 200, 250 (for example, the area occupied by the distant object, or the size of a rectangle or circle when the distant object is replaced with a rectangle or circle, etc.) is detected, and a determination is made as to whether or not to switch between 3D display and 2D display for the distant object that appears largest. Note that the process of determining the size of the distant object is executed by, for example, the object type/size detection unit 48 in FIG. 1 (A).

Furthermore, when it is determined that switching is necessary, it is preferable, in principle, to switch the display mode all at once, including the display of other preceding objects. For example, in FIG. 4(A), when the preceding vehicle 300 is located closer than the threshold value and as a result a switch between 3D display and 2D display is executed, it is preferable to switch the display mode from 3D to 2D for the highlight frame 400 for the person 250 as well, from the standpoint of facilitating display control and from the standpoint of minimizing the sense of discomfort felt by the viewer 5. However, this is a general rule, and there are exceptions (for example, FIG. 4(B)).

The example in FIG. 4(B) is the same as FIG. 4(A) in that there are two (multiple) objects ahead. However, in FIG. 4(B), the distance between the person 250 and the vehicle 1 is smaller than in FIG. 4(A), creating an increased risk of danger. In other words, the display priority (display urgency) of the highlighted frame 400 for the person 250 is higher than the display priority (display urgency) of the attention-calling display 300 for the vehicle ahead 200.

In such a case, forcibly switching the display mode of the highlight frame 400 in synchronization with the switching of the display mode of the attention-calling display 300 of the forward vehicle 200 may cause the viewer 5 to feel uncomfortable and may delay proper judgment. Therefore, it is preferable not to switch the display modes of the attention-calling display 300 and the highlight frame 250.

In other words, when attempting to switch from 3D display control to 2D display control or from 2D display control to 3D display control for a selected other party object, if the display priority (display urgency) of other party objects other than the selected other party object is higher than the display priority (display urgency) of the other party object, the display mode switching is not performed, thereby ensuring safety.

Next, refer to FIG. 5. FIG. 5(A) is a diagram showing an example of the overall configuration of a parallax-type 3D HUD device capable of switching between 3D and 2D, and FIG. 5(B) is a diagram showing an example of the configuration of a control unit and an image generation unit. In FIG. 5(A), parts that are common to FIG. 1(A) are given the same reference numerals.

The parallax 3D HUD device includes a stereoscopic display device 111, an optical system 116, and an information acquisition unit 119. The stereoscopic display device 111 includes a display control device 700, an image generation unit 112, a display unit 113, and an actuator 179. The actuator 179 can be used, for example, to change the orientation of a curved mirror (such as a concave mirror) 117 included in the optical system 116.

The display control device 700 has a control unit 701. The control unit 700 has an I/O interface 741, a processor 702, and a memory 743.

In the example of FIG. 5(B), a viewpoint position detection unit 44 is provided to analyze the image captured by the pupil imaging camera 43 and detect the viewpoint position (and line of sight direction).

In addition, in FIG. 5(B), the control unit 701 has a driving scene determination unit 745 and a 3D/2D switching necessity determination unit 746. The 3D/2D switching necessity determination unit 746 determines whether or not it is necessary to switch the display mode based on a comparison between the distance to the object ahead and a threshold value, and also taking into consideration various driving scenes and the display priority in the example of FIG. 5, etc.

The image generation unit 112 has an image storage unit 312, a light field rendering unit 333, a left-eye image buffer 334, a right-eye image buffer 335, and an image interface (image I/F) 336.

Next, refer to FIG. 6. FIG. 6 is a flowchart showing an example of a procedure for display control. In step S1, a process for detecting and determining a preceding object (including an obstacle) is executed. In step S2, the distance between the vehicle or the like and the preceding object (preceding object distance) is detected.

In step S3, it is determined whether the detected distance is greater than a predetermined threshold. If the result is Y, the display is in principle 3D (step S4), and if the result is N, the display is in principle 2D (step S5). However, in the driving scene shown in FIG. 5(B), the display mode is not switched.

In step S6, it is determined whether the display has ended. If the answer is Y, the display control ends, and if the answer is N, the process returns to step S1.

As described above, according to the present invention, in a parallax 3D HUD device or the like, it is possible to reduce (reduce) the viewer's discomfort and eye fatigue. In other words, when a parallax 3D warning mark or the like is displayed in association with a forward object (such as a vehicle ahead), if the object distance of the forward object is equal to or less than a threshold (threshold distance), it is possible to eliminate convergence adjustment inconsistencies and reduce eye fatigue and discomfort.

In this specification, the term "vehicle" can be broadly interpreted as a vehicle. Navigation terms (such as signs) are also broadly interpreted, taking into account the perspective of navigation information in the broad sense that is useful for driving a vehicle. HUD devices and display devices (and display devices in the broad sense) also include those used as simulators (such as aircraft simulators, simulators as game devices, etc.).

The present invention is not limited to the above-described exemplary embodiments, and a person skilled in the art could easily modify the above-described exemplary embodiments to the extent that they fall within the scope of the claims.

1: vehicle (own vehicle), 2: windshield (projection target member), 3: display area of HUD device (HUD display area), 6: ground or equivalent surface of ground (road surface, etc.), 21: area where parallax-based 3D display is possible (parallax-based 3D display possible area), 41: dashboard, 43: pupil imaging camera, 45: surrounding imaging camera, 46: image processing unit, 47: distance measurement unit, 48: object type/size detection unit, 100: HUD device, 111: stereoscopic display device, 112: image generation unit, 113: display unit (display panel, etc.), 114: light beam separation unit (lenticular lens, parallax barrier, etc.), 116: optical unit, 117: curved mirror (concave mirror, etc.), 119: information acquisition unit, 120: ECU, 123: communication unit, 125: radar unit, 200: object in front Vehicle ahead, 250... Person as the object ahead, 300, 310... Attention call display (or warning display), 400... Highlighted frame as attention call display (or warning display), 700... Display control unit, 701... Control unit, 741... I/O interface, 742... Processor, 743... Memory, 745... Driving scene determination unit, 2D/3D switching necessity determination unit, K1, K2... Display light, PS... Adjustment plane (imaging plane), VS... Convergence plane, θ1... Convergence angle, D1... Adjustment distance, D2... Convergence distance, D3... Distance between the adjustment plane and the viewing position of the stereoscopic image, D4 (D4a, D4b)... Object distance, Lth... Threshold for switching between 3D display and 2D display (predetermined threshold, or threshold distance), R1... Position corresponding to the threshold, R2... Tip position of the area where parallax-based 3D display is possible.

Claims (7)

A display control device that performs display control in a head-up display (HUD) device that projects an image onto a projection target member, thereby allowing a viewer to view a virtual image of the image,
A control unit is provided that is capable of controlling a display mode of the virtual image based on an object distance that is a distance between a forward object and a reference position set on the viewer's side ,
The control unit is
When the object distance exceeds a threshold value, a 3D display control is executed to display the virtual image as a stereoscopic virtual image by making each of the left and right eyes of the viewer view a left eye image and a right eye image having a parallax therebetween,
When the object distance is equal to or less than the threshold value, a 2D display control is executed to display the virtual image as a planar virtual image by causing each of the left and right eyes of the viewer to view an identical image without parallax,
When there are a plurality of the distant objects, the distant object that appears largest to the viewer is selected, and for the selected distant object, it is determined whether or not it is necessary to switch between 3D display and 2D display;
When it is determined that switching is necessary, the display mode is switched collectively, including the display of other preceding objects that were not selected.
Display control device. The control unit is
When attempting to switch from the 3D display control to the 2D display control or from the 2D display control to the 3D display control for a selected destination object,
When the display priority of other destination objects other than the selected destination object is higher than the display priority of the selected destination object, the switching is not executed.
The display control device according to claim 1 . The stereoscopic virtual image is
The display control device according to claim 1 or 2, wherein the display is superimposed on the ground or a surface equivalent to the ground in the vicinity of the forward object. The stereoscopic virtual image is
The display is displayed so as to surround at least a portion of the target object.
The display control device according to claim 3 . The display control device according to claim 1 or 2, wherein the stereoscopic virtual image is superimposed on the forward object. A head-up display (HUD) device that projects an image onto a projection target, thereby allowing a viewer to view a virtual image of the image,
an image generating unit that generates the image;
A display unit for displaying the image;
an optical system including an optical member that reflects display light of the image and projects the image onto the projection target;
A display control device according to any one of claims 1 to 5,
A head-up display device having the following: An image display control method, comprising:
When an object distance, which is a distance between a forward object and a reference position set on the viewer's side, exceeds a threshold, a 3D display control is executed to display a stereoscopic image by making each of the viewer's left and right eyes view a left eye image and a right eye image having a parallax,
When the object distance is equal to or less than the threshold value, a 2D display control is executed to display a planar image by making the left and right eyes of the viewer view the same image without parallax.
When there are a plurality of the distant objects, the distant object that appears largest to the viewer is selected, and for the selected distant object, it is determined whether or not it is necessary to switch between 3D display and 2D display;
When it is determined that switching is necessary, the display mode is switched collectively including the display of other preceding objects that were not selected.

Images