JP7110964B2 - VIRTUAL IMAGE DISPLAY SYSTEM, DISPLAY CONTROL DEVICE AND DISPLAY CONTROL PROGRAM - Google Patents

Description

The disclosure of this specification relates to a virtual image display system that displays a virtual image superimposed on a superimposition target in the foreground, and a display control device and a display control program that control display of the virtual image.

Conventionally, for example, Patent Literature 1 discloses a configuration in which a head-up display (hereinafter referred to as "HUD") that displays a virtual image can adjust the projection position of light according to the height of the viewpoint of a viewer. In the HUD of Patent Document 1, the tilt angle of the concave mirror in the front-rear direction and the position of the housing are adjusted based on the viewpoint position information of the viewer. As a result, even if the viewer's viewpoint position changes, the position of the virtual image viewed by the viewer can be maintained constant.

JP 2015-60180 A

In recent years, new attempts have been made to present information by superimposing a specific object in the foreground as a superimposition target and superimposing a virtual image on the superimposition target. In such a form in which a virtual image is superimposed on a superimposition target, conventional adjustment of the projection optical system may be inappropriate.

Specifically, in Patent Document 1, the display position of the virtual image is used as a reference, and adjustment is performed so that the display position of the virtual image does not change even if the viewpoint position changes. As a result, the area in the foreground where the virtual image can be superimposed as viewed from the viewer (hereinafter referred to as "displayable area") inevitably moves due to adjustment according to the change in the viewpoint position. According to such movement of the displayable area, a superimposition target on which the virtual image is to be superimposed may be out of the displayable area, and a scene may occur in which the virtual image cannot be superimposed.

An object of the present disclosure is to provide a virtual image display system, a display control device, and a display control program that adjust a projection optical system suitable for presenting information by superimposing a virtual image on a superimposition target.

In order to achieve the above object, one disclosed aspect is a virtual image display system that is used in a vehicle (A) and displays a virtual image (Vi) superimposed on a superimposed object in the foreground, the projection member of the vehicle (WS), a projection optical system (74) for projecting light formed as a virtual image, and at least one specific parameter that changes a displayable area (SA) in the foreground where the virtual image can be superimposed, as a reference a state value acquisition unit (32) that acquires a reference state value that is different from the standard state value; a projection adjustment unit (37) that adjusts the projection optical system so as to reduce a change in the displayable area caused by a change in the reference state value; A first acquisition unit (33) for acquiring posture change information in a low frequency band (LB) among posture changes occurring in the vehicle; ), and a first correction unit ( 39), a second correction unit (53) that corrects the positional deviation of the virtual image caused by the change in posture in the high frequency band by a process different from that of the first correction unit, while reflecting the adjustment result of the projection optical system; and a virtual image display system.

Moreover, one aspect disclosed is a display control device used in a vehicle (A) for controlling display of a virtual image (Vi) superimposed on a superimposition target in the foreground, wherein the virtual image can be superimposed in the foreground. a state value acquisition unit (32) for acquiring a reference state value as a reference for at least one specific parameter that changes the displayable area (SA); A projection adjustment unit (37) that controls the state of the projection optical system (74) that projects the , and adjusts the projection optical system so as to reduce changes in the displayable area caused by fluctuations in the reference state value; A first acquisition unit (33) that acquires posture change information in a low frequency band (LB) among the resulting posture changes, and a high frequency band (HB) that is higher in frequency than the low frequency band among the posture changes of the vehicle. A second acquisition unit (52) that acquires posture change information, and a first correction unit (39) that electronically corrects positional deviation of a virtual image caused by posture change in a low frequency band while reflecting the adjustment result of the projection optical system. and a second correction unit (53) that corrects the positional deviation of the virtual image caused by the posture change in the high frequency band by a process different from that of the first correction unit while reflecting the adjustment result of the projection optical system. A display control device.

Moreover, one aspect disclosed is a display control program for controlling display of a virtual image (Vi) superimposed on a superimposition target in the foreground, which is used in a vehicle (A), and comprises at least one processing unit (50a) A state value acquisition unit (32) that acquires a reference state value that serves as a reference for at least one specific parameter that changes the displayable area (SA) in which a virtual image can be superimposed in the foreground, and a vehicle projection member ( Controls the state of a projection optical system (74) that projects the light imaged as a virtual image on the WS), and adjusts the projection optical system so as to reduce changes in the displayable area caused by fluctuations in the reference state value. a projection adjustment unit (37); a first acquisition unit (33) for acquiring posture change information in a low frequency band (LB) among posture changes occurring in the vehicle; A second acquisition unit (52) that acquires posture change information in a high-frequency band (HB) with a high frequency; A first correction unit (39) that performs electronic correction; and a second correction unit (53) .

In these aspects, even if the reference state value for the specific parameter changes, the adjustment of the projection optical system by the projection adjustment section reduces the change in the displayable area. If the movement of the displayable area is suppressed in this way, the number of scenes in which the virtual image cannot be superimposed on the superimposition target on which the virtual image should be superimposed can be reduced. According to the above, it is possible to adjust the projection optical system suitable for information presentation in which a virtual image is superimposed on a superimposition target.

It should be noted that the reference numbers in parentheses above merely indicate an example of correspondence with specific configurations in the embodiments described later, and do not limit the technical scope in any way.

1 is a block diagram showing an overview of a virtual image display system according to a first embodiment of the present disclosure; FIG. FIG. 4 is a diagram showing vibration characteristics of posture changes occurring in a vehicle, showing an example of each range of a high frequency band and a low frequency band; FIG. 4 is a diagram showing details of a distortion correction table stored in a storage unit of a display control ECU, and details of processing contents of distortion correction based on the distortion correction table; It is a figure which shows the detail of the adjustment mechanism provided in HUD. FIG. 10 is a diagram schematically showing optical correction for reducing changes in the displayable area, together with a specific example of AR display. 4 is a flowchart showing details of virtual image drawing processing performed by each processing unit of the virtual image display system; FIG. 10 is a block diagram showing an overview of a virtual image display system according to a second embodiment of the present disclosure; FIG.

A plurality of embodiments of the present disclosure will be described below based on the drawings. Note that redundant description may be omitted by assigning the same reference numerals to corresponding components in each embodiment. When only a part of the configuration is described in each embodiment, the configurations of other embodiments previously described can be applied to other portions of the configuration. Moreover, not only the combinations of the configurations explicitly specified in the description of each embodiment, but also the configurations of a plurality of embodiments can be partially combined even if they are not specified unless there is a particular problem with the combination. Also, unspecified combinations of configurations described in a plurality of embodiments and modifications are also disclosed by the following description.

(First embodiment)
A virtual image display system 10 according to the first embodiment of the present disclosure shown in FIG. 1 is used in a vehicle A. As shown in FIG. The virtual image display system 10 uses a portion of the windshield WS as a projection area PA, and projects light onto the projection area PA to display a virtual image Vi superimposed on the foreground of the vehicle A so that the driver can see it. The virtual image display system 10 presents various information related to the vehicle A to the driver by augmented reality (hereinafter “AR”) display using the virtual image Vi.

The display position of the AR-displayed virtual image Vi is associated with a specific object in the foreground visually recognized through the projection area PA, for example, a superimposed target such as a vehicle in front, a pedestrian, and a road surface. The virtual image Vi follows the superimposition target and appears to the driver to move as if it is relatively fixed to the superimposition target. Therefore, even when the posture of the vehicle A changes, the state in which the virtual image Vi is superimposed and displayed on the superimposition target is maintained. As an example, a virtual image Vi indicating the range of lanes to be driven is superimposed and displayed as an AR display on the road surface between the left and right lane markings in the foreground (see FIG. 5).

In the virtual image display system 10, even if the relative positional relationship between the superimposition target, the projection area PA, and the eye point EP changes due to changes in the attitude of the vehicle A, changes in the position of the driver's eye point EP, etc., the superimposition target control is performed to maintain the superimposed state of the virtual image Vi on the . The virtual image display system 10 is capable of successively correcting the shape of light formed as the virtual image Vi, the projection position, etc. so that the display position deviation of the virtual image Vi with respect to the superimposition target is reduced, and preferably eliminated.

The virtual image display system 10 has a plurality of correction functions for correcting virtual image display. Each correction function exerts an effect of correcting a display position shift for vibrations in mutually different frequency bands. More specifically, the vibrations (for example, vehicle vibrations) that cause the display position to shift mainly include components in the frequency band up to about 2 Hz, as shown in FIG. In the virtual image display system 10, such a frequency band is divided into, for example, a low frequency band LB and a high frequency band HB, and display deviation is reduced by different methods.

The low frequency band LB is defined as a frequency band from about 0 to 0.5 Hz. In the low frequency band LB, the band on the lower frequency side is defined as the first band LB1, and the band on the higher frequency side is defined as the second band LB2. The bandwidth of the first band LB1 is set narrower than the bandwidth of the second band LB2. The first band LB1 is a band whose frequency is almost zero. For example, changes in the vehicle attitude caused by changes in the number of passengers and changes in the load weight of the vehicle A are included in the first band LB1. In addition, the change in the position of the eyepoint EP due to the physique of the driver is also included in the first band LB1. On the other hand, the second band LB2 is a band mainly including vehicle vibrations of less than 0.5 Hz among vehicle vibrations occurring during running. For example, when the vehicle A is slowly accelerated or decelerated, posture change during acceleration or deceleration based on normal driving operation (pedal operation) becomes vehicle vibration belonging to the second band LB2.

The high frequency band HB is a frequency band from 0.5 to 2 Hz higher in frequency than the low frequency band LB. For example, when driving on rough roads, when braking operations are input multiple times, when the vehicle A is suddenly started or decelerated, and so on, changes in posture in driving scenes become vehicle vibrations belonging to the high frequency band HB.

The virtual image display system 10 shown in FIG. 1 includes a drawing ECU (Electronic Control Unit) 30, a display control ECU 40, a head-up display (hereinafter referred to as "HUD") 70, and the like. Of the above-mentioned frequency bands, positional deviation of the virtual image Vi due to changes in posture and the like in the first band LB1 is reduced by electronic correction in the display control ECU 40 or optical correction in the HUD 70. Further, the positional deviation of the virtual image Vi due to the change in posture of the second band LB2 is reduced by electronic correction in the drawing ECU 30. FIG. Furthermore, the positional deviation of the virtual image Vi caused by the posture change in the high frequency band HB is reduced by electronic correction by the display control ECU 40 . Details of the drawing ECU 30, the display control ECU 40, and the HUD 70 will be described in order below. Note that the above-described electronic correction (hereinafter referred to as “electronic correction”) is correction performed on image data provided to the HUD 70 in terms of arithmetic processing, and is non-optical correction.

The drawing ECU 30 is directly or indirectly connected to a plurality of in-vehicle displays, and is a control device that integrally controls the display of each in-vehicle display. The drawing ECU 30 individually generates image data to be displayed by each vehicle-mounted display, and sequentially outputs the image data to each vehicle-mounted display. The drawing ECU 30 is electrically connected to the communication bus 29 of the in-vehicle network, and can mutually communicate with other in-vehicle components via the communication bus 29 . The drawing ECU 30 acquires data necessary for generating video data from the in-vehicle network. The communication bus 29 is electrically connected directly or indirectly to the external sensor 21, the height sensor 22, the vehicle control unit 24, the driver status monitor 25, and the like.

The external sensor 21 detects moving objects such as pedestrians and other vehicles, as well as stationary objects such as road curbs, road signs, road markings, and lane markings. At least a part of these moving and stationary objects is superimposed with the virtual image Vi. The vehicle A is equipped with an external sensor 21 such as a camera unit, a lidar, and a millimeter wave radar. The external sensor 21 sequentially outputs object information indicating relative positions and types of detected moving and stationary objects to the communication bus 29 .

The height sensor 22 is a sensor that detects vertical displacement of the vehicle A in order to measure the height from the road surface on which the vehicle A is placed to the body. The height sensor 22 is installed, for example, on either one of the left and right rear suspensions. The height sensor 22 measures the relative distance between the body and the suspension arm, and sequentially outputs the measurement result of the vehicle height to the communication bus 29 .

The vehicle control unit 24 is a control device mainly composed of a microcontroller. The vehicle control unit 24 controls the behavior of the vehicle A based on the object information detected by the external sensor 21, the driver's driving operation, and the like. Vehicle control unit 24 is electrically connected to pedal sensor 23 . The pedal sensor 23 includes an accelerator position sensor and a brake depression force sensor.

The vehicle control unit 24 controls the longitudinal acceleration generated in the vehicle A, that is, the axle torque and the braking force based on the detection signal of the pedal sensor 23 . In addition, the vehicle control unit 24 feedforward-controls the axle torque and the braking force so as to suppress the vibration of the vehicle A caused by the driver's acceleration/deceleration operation and disturbances such as unevenness of the road surface. The vehicle control unit 24 sequentially outputs each target value of the axle torque and brake force in the feedforward control to the communication bus 29 as control information.

A driver status monitor (hereinafter referred to as "DSM") 25 has a near-infrared light source, a near-infrared camera, and an image analysis section. The DSM 25 is arranged, for example, on the upper surface of the instrument panel, with the near-infrared camera directed toward the driver's seat. The DSM 25 uses a near-infrared camera to capture a face image including the driver's face by photographing the area around the driver's face or upper body irradiated with near-infrared light from the near-infrared light source. The DSM 25 analyzes the captured face image in the image analysis unit and detects the position of the driver's eye point EP. The DSM 25 sequentially outputs the position information of the eyepoint EP to the communication bus 29 .

The drawing ECU 30 is mainly composed of a computer having a processing section 30a, a RAM, a storage section 30b, and an input/output interface. The processing unit 30a is hardware for arithmetic processing coupled with a RAM. The processing unit 30a includes at least one arithmetic core such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit 30a may be configured to further include an IP core having an FPGA (Field-Programmable Gate Array) and other dedicated functions. Various programs (display control program, etc.) executed by the processing unit 30a are stored in the storage unit 30b.

The drawing ECU 30 executes a program stored in the storage section 30b by means of the processing section 30a, and includes a plurality of functional sections. Specifically, the drawing ECU 30 includes functional units such as a sensor value acquisition unit 31, a brightness setting unit 34, a content selection unit 35, a target setting unit 36, a projection adjustment unit 37, a correction amount calculation unit 38, and a drawing data generation unit 39. is constructed. The sensor value acquisition unit 31 , brightness setting unit 34 , content selection unit 35 and target setting unit 36 can acquire information from the communication bus 29 .

The sensor value acquisition unit 31 acquires from the communication bus 29 information indicating posture changes in the low frequency band LB. The sensor value acquisition unit 31 is provided with auxiliary functional units such as a DC information processing unit 32 and an AC information processing unit 33 .

The DC information processing unit 32 acquires state information of a DC (Direct Current) component belonging to the first band LB1. Specifically, the DC information processing unit 32 stores the measurement result of the vehicle height when the vehicle A is stopped (hereinafter referred to as "vehicle reference attitude") and the position information of the eye point EP when the driver is substantially stationary. (hereinafter referred to as “eyepoint reference position”) is acquired as state information of the DC component. The DC information processing unit 32 provides the vehicle reference attitude and the eyepoint reference position to the projection adjustment unit 37 and the correction amount calculation unit 38 as reference state values of specific parameters whose details will be described later.

The AC information processing unit 33 acquires the attitude change information of the AC (Alternate Current) component belonging to the second band LB2 among the attitude changes occurring in the vehicle A, using the control information of the axle torque and the braking force. Specifically, the AC information processing unit 33 extracts the vibration component of the posture change (mainly pitch angle change, etc.) belonging to the second band LB2 by passing the control information of the axle torque and the braking force through a band-pass filter. .

The luminance setting unit 34 grasps the required luminance required for the virtual image Vi. The brightness setting unit 34 acquires, for example, exposure information set by the camera unit of the external sensor 21 from the communication bus 29, and estimates the current brightness outside the vehicle (external light amount). The brightness setting unit 34 sets the required brightness of the virtual image Vi according to the estimated amount of external light. The required brightness is set higher as the outside of the vehicle becomes brighter, and is set lower as the outside of the vehicle becomes darker. The brightness setting unit 34 sequentially notifies the HUD 70 of the required brightness suitable for the current amount of outside light.

The content selection unit 35 selects and arbitrates content for virtual image display based on various information acquired from the communication bus 29 . Specifically, the content selection unit 35 comprehensively determines the display priority of each content, and selects information to be notified to the driver using the virtual image Vi. The content selection unit 35 sequentially provides the drawing data generation unit 39 with a command designating the selected content and information necessary for drawing the selected content.

The target setting unit 36 acquires the object information output by the external sensor 21 from the communication bus 29, and selects a superimposition target on which the virtual image Vi is to be superimposed, from moving objects and stationary objects. The target setting unit 36 sequentially provides the relative positions of the selected superimposition targets to the drawing data generation unit 39 . As an example, when the road surface of the lane in which the vehicle is running is set as the superimposition target and the virtual image Vi is superimposed on the road surface (see FIG. 5), the target setting unit 36 sets the relative positions of the left and right lane markings to the drawing data generation unit. 39.

The projection adjustment unit 37 adjusts the position of the projection area PA defined by the windshield WS, and thus the projection position of the light formed as the virtual image Vi, by controlling the attitude of the projection optical system 74 (described later) of the HUD 70. , is vertically displaced from the viewpoint of the driver. The projection adjustment unit 37 acquires the vehicle reference attitude and the eyepoint reference position provided by the DC information processing unit 32 . The projection adjustment unit 37 determines the target orientation of the projection optical system 74 based on these reference state values. The correlation between the reference state value and the target attitude is defined in advance at the time of design and stored in the storage section 30b in a data format such as a correlation table or correlation function. Projection adjuster 37 applies the reference state values to a correlation table or function to determine a target pose corresponding to the reference state values. The projection adjustment unit 37 notifies the HUD 70 of the determined target orientation as control information. In addition, the projection adjustment unit 37 also notifies the drawing data generation unit 39 and the display control ECU 40 of the determined target orientation as information indicating the adjustment result of the projection optical system 74 .

The correction amount calculator 38 identifies the current attitude of the vehicle A mainly using the attitude change information of the second band LB2 acquired by the AC information processor 33 . The correction amount calculator 38 calculates a correction amount (hereinafter referred to as "low-frequency correction amount") for correcting the display deviation of the virtual image Vi from the specified current attitude of the vehicle A. The correction amount calculator 38 sequentially provides the calculated value of the low-frequency correction amount to the drawing data generator 39 .

The correction amount calculator 38 takes into consideration the adjustment result of the projection optical system 74 provided by the projection adjuster 37 and the reference state value provided by the DC information processor 32 when calculating the low-frequency correction amount. For example, as will be described later, when the projection adjustment unit 37 has suspended the adjustment of the projection optical system 74 to the target orientation corresponding to the reference state value, the correction amount calculation unit 38 calculates the current orientation of the projection optical system 74 and the , a low-frequency correction amount is calculated in consideration of the vehicle reference attitude and the eyepoint reference position.

The drawing data generation unit 39 is a 3D drawing unit that generates video data. The drawing data generator 39 is electrically connected to the display control ECU 40 via a video output line or the like. The drawing data generator 39 sequentially outputs video data for displaying a virtual image, that is, input video data Vdi for the display control ECU 40 to the display control ECU 40 in a predetermined video format.

The drawing data generator 39 draws each frame image (hereinafter referred to as “input image Pi”) of the input video data Vdi. The drawing positions and drawing shapes of the individual drawing objects within the frame of the input image Pi are controlled so that the formed virtual image Vi is correctly superimposed on the superimposition target and visually recognized. More specifically, the drawing data generation unit 39 grasps the eyepoint EP, the projection area PA, and the relative positions of the superimposition target, and based on these positional relationships, determines the projection position and the projection position of the light projected in the projection area PA. The shape and the like are calculated by geometrical calculations.

In addition, the drawing data generator 39 electronically corrects the positional deviation of the virtual image Vi due to the posture change in the low frequency band LB. Based on the low-frequency correction amount obtained from the correction amount calculation unit 38, the correction amount calculation unit 38 reduces (cancels) the positional deviation of the virtual image Vi due to the posture change in the low frequency band LB (or the second band LB2). The input image Pi is successively corrected as follows. The drawing data generation unit 39 continuously transmits the input image Pi previously corrected for the attitude change in the low frequency band LB to the display control ECU 40 as the input video data Vdi.

The display control ECU 40 is a control device that relays video data between the rendering ECU 30 and the HUD 70 and controls display of the virtual image Vi by the HUD 70 . The display control ECU 40 includes an inertial sensor 41, a filter circuit 42, an LCD (Liquid Crystal Display) control board 50, and the like.

The inertial sensor 41 is a measurement unit that measures changes in the attitude of the vehicle A. As shown in FIG. The inertial sensor 41 is a combination of a gyro sensor and an acceleration sensor. The inertial sensor 41 is mounted on the vehicle A as a posture detection sensor separate from the height sensor 22, the vehicle control unit 24, and the like. The inertial sensor 41 measures the angular velocities of the vehicle A in the pitch direction and the roll direction, and the vertical acceleration of the vehicle A along the yaw axis, and transmits each measurement signal to the filter circuit 42 . The inertial sensor 41 may be fixed to the LCD control board 50, or may be fixed to the metal housing of the display control ECU 40 or the like.

The filter circuit 42 is a signal processing circuit that processes the measurement signal of the inertial sensor 41, and includes at least a high-pass filter and an integration processing unit, for example. The high-pass filter generally passes signals in the high frequency band HB and attenuates signals in the low frequency band LB and below (see FIG. 2). Additionally, the high-pass filter removes drift components contained in the measurement signal. The integration processing unit is composed mainly of a low-pass filter, for example, and generates a signal indicating the vehicle attitude (pitch angle, roll angle, etc.) by signal processing that time-integrates the measurement signal indicating the angular velocity of the attitude change. The filter circuit 42 acquires the signal that has passed through the high-pass filter and the integration processing section in order as posture change information in the high frequency band HB, and sequentially provides the LCD control board 50 with the information.

The LCD control board 50 is provided with a processing section 50a, a RAM, a storage section 50b, an input/output interface, and the like. A data bus formed on the LCD control board 50 is connected to a filter circuit 42, a RAM, a storage section 50b, a processing section 50a, and the like so as to be accessible at high speed.

The storage unit 50b is mainly composed of a non-volatile storage medium such as ROM or flash memory. The storage unit 50b holds data to be provided to the processing unit 50a. The storage unit 50b stores various programs executed by the processing unit 50a and a large number of parameters necessary for virtual image display. As one of such parameters, a distortion correction table TB (see FIG. 3), which will be described later, is stored in the storage unit 50b.

The processing unit 50a is hardware for image processing coupled with a RAM. The processing unit 50a corrects the input image Pi that forms each frame of the input video data Vdi, and draws the output image Po that forms each frame of the output video data Vdo. The processing unit 50a includes at least a memory controller that controls access to the RAM, and an arithmetic core such as a CPU and an FPGA. The processing unit 50a may be configured to further include a GPU and an IP core having other dedicated functions. Functional units such as a drawing data acquisition unit 51, a posture information acquisition unit 52, a correction execution unit 53, and the like are constructed in the processing unit 50a.

The drawing data acquisition section 51 is electrically connected to the drawing data generation section 39 . Each input image Pi of the input video data Vdi is sequentially transmitted from the drawing data generation unit 39 to the drawing data acquisition unit 51 . The drawing data acquisition unit 51 prepares the input image Pi so that it can be corrected by sequentially writing the acquired input image Pi into the RAM.

The posture information acquisition unit 52 acquires posture change information in the high frequency band HB (see FIG. 2) as information related to the posture change of the vehicle A from the filter circuit 42 .

The correction execution unit 53 is a correction unit that corrects the input image Pi by performing 2D graphics processing for two-dimensionally transforming the input image Pi. The correction executing section 53 is electrically connected to the HUD 70 via a video output line or the like. The correction execution unit 53 generates an output image Po by correcting the input image Pi, and continuously transmits it to the HUD 70 as output video data Vdo. The correction execution unit 53 electronically corrects the virtual image Vi through processing different from that of the drawing data generation unit 39 . The correction execution unit 53 performs posture correction and distortion correction as corrections on the input image Pi.

Posture correction is a correction that reduces (offsets) the positional deviation of the virtual image Vi from the superimposition target due to the posture change in the high frequency band HB. The correction execution unit 53 determines an extraction range to be used for the output image Po in the input image Pi. Based on the posture change information acquired by the posture information acquisition unit 52, the correction execution unit 53 shifts the position of the extraction range mainly in the vertical direction of the input image Pi.

Distortion correction considers deformation of the virtual image Vi due to reflection on the projection area PA and the projection optical system 74, and adjusts the shape of the output image Po so that the virtual image Vi is displayed with the distortion reduced (offset). This is a correction that deforms in advance. A distortion correction table TB shown in FIG. 3 is used as correction information for correction calculation of distortion correction.

The distortion correction table TB has a file configuration in which a set of data sets are made up of pre-correction coordinates and post-correction coordinates, and a large number of these data sets are continuous. The post-correction coordinates are information indicating the position of the pixel in the output image Po. The pre-correction coordinates are information indicating the positions of the pixels in the input image Pi that are assigned to the pixels of the post-correction coordinates using the X and Y coordinates.

Specifically, the sequence of numerical values in the distortion correction table TB is as follows: <X coordinate of post-correction coordinates>, <Y coordinate of post-correction coordinates>, <X coordinate of pre-correction coordinates>, <Y coordinate of pre-correction coordinates> are in order. For example, the numerical value group in the first row of the distortion correction table indicates that the coordinates (0, 0) of the output image Po are (-1, -1) invalid pixels (black background). The group of numerical values in the second row indicates that the pixel information (gradation value) of (0, 1) of the input image Pi is assigned to (1, 0) of the output image Po.

In addition, the distortion correction table TB shows the relationship between the orientation of the projection optical system 74 and the eyepoint EP. Therefore, the correction execution unit 53 shown in FIG. 1 acquires the adjustment result of the projection optical system 74 from the projection adjustment unit 37, and selects the distortion correction table TB having contents corresponding to the current posture of the projection optical system 74 from the storage unit 50b. can be read out.

As described above, the correction execution unit 53 acquires the distortion correction table TB having contents corresponding to the target orientation set by the projection adjustment unit 37, and converts the two-dimensional array of pixels in the extraction range based on the orientation correction into the distortion correction table TB. Sort based on table TB. Through such processing, the correction execution unit 53 generates the output image Po by deforming the extraction range of the input image Pi.

The HUD 70 shown in FIGS. 1 and 4 is an in-vehicle display that is housed in a housing space provided in the instrument panel below the windshield WS. Light emitted from the HUD 70 toward the projection area PA of the windshield WS is reflected by the projection area PA toward the eye point EP side and perceived by the driver. The driver visually recognizes a display in which the virtual image Vi is superimposed on the foreground seen through the projection area PA. The HUD 70 forms a virtual image Vi in a space about 10 to 20 m ahead of the vehicle A from the eye point EP, for example.

In the HUD 70, an eyebox EBX is set above the seat surface of the driver's seat and near the headrest. The eyebox EBX is a spatial region in which the virtual image Vi can be visually recognized with a predetermined brightness or higher. The eye box EBX is a virtual area defined for each vehicle type, and is set so as to overlap with an eye range that statistically represents the distribution of eye positions of the driver.

The HUD 70 is composed of an LCD 71 , a backlight 72 , a backlight controller 73 , a projection optical system 74 and an adjustment mechanism 75 . Each optical element is housed in a housing 79 of the HUD 70, and the housing 79 defines the relative positional relationship with high accuracy.

The LCD 71 has a display surface on which many pixels are arranged. The LCD 71 increases or decreases the light transmittance of red, green, and blue sub-pixels provided in each pixel to display various images in color on the display surface. The light transmittance of each sub-pixel is controlled by the LCD control board 50 . An output image Po based on the output video data Vdo is displayed on the display surface of the LCD 71 .

The backlight 72 is configured with a large number of light emitting diodes. A large number of light emitting diodes are arranged on the back side of the LCD 71 and are two-dimensionally arranged at predetermined intervals along the display surface of the LCD 71 . The backlight 72 is configured to be able to adjust the emission brightness for each area illuminated by each light emitting diode. The backlight 72 illuminates the LCD 71 from behind and emits light to display the output image Po drawn on the display surface. As a result, the light of the output image Po is emitted from the display surface.

The backlight control unit 73 mainly includes a drive circuit that drives each light emitting diode of the backlight 72 . The backlight control unit 73 controls the light emission luminance of the backlight 72 for each area by partial driving (local dimming) that causes each light emitting diode to emit light individually. The backlight control unit 73 acquires the output video data Vdo from the LCD control board 50, and determines the light emission brightness of each area of the backlight 72 according to the position, shape, etc. of the drawing in each output image Po. In addition, the backlight control unit 73 adjusts the brightness of each area according to the amount of outside light based on the required brightness notified from the brightness setting unit 34 .

The projection optical system 74 includes, for example, a plane mirror 74a and a concave mirror 74b, and projects the light of the output image Po formed as the virtual image Vi onto the projection area PA. Each reflecting mirror is formed by vapor-depositing a metal such as aluminum on the surface of a colorless and transparent substrate made of synthetic resin, glass, or the like. The projection optical system 74 spreads the light of the virtual image Vi emitted from the LCD 71 by reflection and projects it onto the projection area PA to display the virtual image Vi by enlarging the output image Po. For the projection optical system 74, for example, a diffractive optical element (DOE) or the like that enlarges the output image Po by diffraction may be used.

The adjustment mechanism 75 is a mechanism that cooperates with the projection adjustment section 37 and controls the state of the projection optical system 74 . The adjusting mechanism 75 is integrated with the concave mirror 74b of the projection optical system 74, and mechanically adjusts the posture of the concave mirror 74b. The adjustment mechanism 75 vertically moves the position of the projection area PA on the windshield WS by rotating the concave mirror 74b around the rotation axis Ax. The adjustment mechanism 75 is composed of a movable support portion 75a, a potentiometer 76, a stepper motor 77, a controller 78, and the like.

The movable support portion 75a supports the concave mirror 74b with respect to the housing 79 in a rotatable state about the rotation axis Ax. The potentiometer 76 is a rotation angle sensor mainly composed of a variable resistor. The potentiometer 76 detects the current rotation angles of the movable support portion 75a and the concave mirror 74b, and outputs to the controller 78 an output corresponding to the rotation angle. The stepper motor 77 is a driving portion that rotates the movable support portion 75a. The stepper motor 77 rotates the movable support portion 75a and the concave mirror 74b around the rotation axis Ax in accordance with the drive signal from the controller 78 .

A controller 78 is a drive control unit that controls the operation of the stepper motor 77 . The controller 78 sets the rotation angles of the movable support section 75a and the concave mirror 74b based on the target posture determined by the projection adjustment section 37. FIG. The controller 78 controls the driving of the stepper motor 77 while referring to the output of the potentiometer 76 to match the rotation angle of the concave mirror 74b with the target attitude.

Next, details of optical correction by cooperation between the projection adjustment unit 37 and the adjustment mechanism 75 will be described. Optical correction is performed to reduce changes in the displayable area SA of the virtual image Vi. The displayable area SA is a foreground range that overlaps the projection area PA when viewed from the eyepoint EP. The virtual image Vi can be superimposed only on the object in the displayable area SA. The displayable area SA changes mainly due to changes in the attitude of the vehicle A in the pitch direction and changes in the position of the eye point EP. Therefore, the pitch attitude of the vehicle A and the eye point EP are the main "specific parameters" that change the displayable area SA. Then, as described above, the vehicle reference attitude and the eyepoint reference position acquired by the DC information processing unit 32 serve as the "reference state values" that serve as references for the specific parameters.

The projection adjustment unit 37 acquires the reference state value of the specific parameter, and sets the target orientation of the projection optical system 74 according to the reference state value so as to reduce the change in the displayable area SA as seen from the driver. . The projection adjustment unit 37 adjusts the projection optical system 74 and moves the projection area PA by controlling the adjustment mechanism 75 based on the target attitude. By changing the posture of the projection optical system 74 in this manner, the vertical position of the projection area PA is adjusted.

As an example, when the eye point EP is higher than the standard position (EPh in FIG. 5), or when the vehicle attitude is sunken on the front side, the displayable area SA moves toward the vehicle A if the correction is not performed. do. Therefore, the projection adjustment unit 37 shifts the projection area PA on the windshield WS upward in order to reduce the change in the displayable area SA. According to the above, since the imaging position of the virtual image Vi also moves upward, the road surface range overlapping the virtual image Vi seen from the eye point EP is maintained.

On the other hand, when the eye point EP is lower than the standard position (FIG. 5 EP1), or when the vehicle attitude is such that the rear is sunken, the displayable area SA moves away from the vehicle A if the correction is not performed. . Therefore, the projection adjustment unit 37 shifts the projection area PA on the windshield WS downward in order to reduce the change in the displayable area SA. According to the above, since the imaging position of the virtual image Vi also moves downward, the road surface range overlapping with the virtual image Vi seen from the eye point EP is maintained.

Here, the movement of the projection area PA to maintain the displayable area SA may move the center of the eyebox EBX (hereinafter, "eyebox center") away from the eyepoint EP. Therefore, when performing control to maintain the optically displayable area SA, when the eye point EP deviates from the initial reference position due to a change in the driver's posture, etc., the virtual image Vi is significantly reduced in brightness and is out of sight. etc. are likely to occur.

Therefore, the projection adjustment unit 37 adjusts the projection optical system 74 while considering the above optical disadvantages. Specifically, when it is difficult to fix the displayable area SA due to the optical performance of the HUD 70 or the like, the projection adjustment unit 37 prevents the important range SAi of the displayable area SA from deviating from the displayable area SA. Second, the attitude of the projection optical system 74 is controlled. The important range SAi is defined in advance as a range essential for the driver's understanding of the meaning of the information presented using the virtual image Vi. The projection adjustment section 37 can adjust the position of the important range SAi according to the content selected by the content selection section 35 . As another example, the projection adjustment unit 37 controls the position of the projection area PA so that the depression angle from the eye point EP to the displayable area SA is constant, thereby keeping the important area SAi within the displayable area SA. may

In addition, the projection adjustment unit 37 suspends control of the projection optical system 74 that maintains the displayable area SA according to the required brightness determined by the brightness setting unit 34 . Specifically, the projection adjustment unit 37 determines whether or not the requested luminance exceeds the threshold. When the required brightness exceeds the threshold, the projection adjustment unit 37 suspends the maintenance control of the displayable area SA and adjusts the attitude of the projection optical system 74 so that the center of the eyebox coincides with the eyepoint reference position. switch to brightness priority control.

Details of a series of virtual image rendering processes performed by each component of the virtual image display system 10 in order to realize the virtual image display described above will be described based on FIG. 6 with reference to FIGS. 1, 2 and 5. do. The virtual image drawing process shown in FIG. 6 is started, for example, when the vehicle power source is turned on, and is repeatedly performed until the vehicle power source is turned off.

The processing of S101 to S103 is performed by the projection adjustment section 37 of the drawing ECU 30. FIG. In S101, the required luminance set by the luminance setting unit 34 is compared with a threshold value. If it is determined in S101 that the required luminance is higher than the threshold, the process proceeds to S102. In S102, the target orientation of the projection optical system 74 is set by luminance priority control so that the center of the eyebox coincides with the eyepoint reference position. Then, the set target posture is notified to the adjustment mechanism 75, and the process proceeds to S104. The adjustment mechanism 75 adjusts the rotation angle of the projection optical system 74 based on the target orientation notified in S102.

On the other hand, if it is determined in S101 that the required brightness is equal to or less than the threshold, the process proceeds to S103. In S103, as maintenance control of the displayable area SA, the projection optical system 74 is adjusted so as to reduce changes in the displayable area SA. In S103, while considering the distance between the eyepoint reference position and the center of the eyebox, the projection optical system 74 is adjusted so as to reduce the change in the displayable area SA within a range in which the virtual image Vi is not cut off. , S104.

Each process of S<b>104 and S<b>105 is mainly performed by the drawing data generation unit 39 of the drawing ECU 30 . In S104, the eyepoint reference position and the target orientation set by the projection adjustment unit 37 are set to be reflected in drawing of the input image Pi, and the process proceeds to S105.

In S105, posture change information indicating the posture change in the low frequency band LB is further reflected in drawing of the input image Pi, and the process proceeds to S106. If it is determined in S101 that the required brightness is equal to or less than the threshold, the change in the vehicle reference attitude caused by the number of passengers and the load weight, and the change in the eye point reference position caused by body differences are Optically corrected. On the other hand, when it is determined in S101 that the required luminance exceeds the threshold, the positional deviation correction of the optical virtual image Vi is not performed. Therefore, in S105 in this case, when the input image Pi is drawn, the amount of variation in the vehicle reference attitude and the eyepoint reference position is electronically corrected.

Each process of S106 to S108 is mainly performed by the correction executing section 53 of the display control ECU 40. FIG. In S106, the attitude change information indicating the attitude change in the high frequency band HB is reflected in the attitude correction of the output image Po, and the process proceeds to S107. In S107, the reference position of the eyepoint EP and the target orientation set by the projection adjustment unit 37 are reflected in the distortion correction of the output image Po, and the process proceeds to S108. In S108, the output image Po that has undergone posture correction and distortion correction is transmitted to the HUD 70 as output video data Vdo. As described above, the output image Po is luminescently displayed on the screen of the LCD 71 and formed as a virtual image Vi in the foreground.

In the first embodiment described so far, even if the reference state value for the specific parameter changes, the adjustment of the projection optical system 74 by the projection adjustment unit 37 reduces the change in the displayable area SA. If the movement of the displayable area SA is suppressed in this way, the number of scenes in which the virtual image Vi cannot be superimposed on the superimposition target on which the virtual image Vi should be superimposed can be reduced. According to the above, it is possible to adjust the projection optical system 74 suitable for information presentation in which the virtual image Vi is superimposed on the superimposition target.

In addition, in the first embodiment, the viewer's eye point EP and the pitch attitude of the vehicle A are employed as specific parameters. If the displayable area SA can be optically corrected when the eyepoint EP and the pitch attitude that greatly affect the displayable area SA change, scenes in which the virtual image Vi cannot be superimposed can be further reduced.

Further, in each of the ECUs 30 and 40 of the first embodiment, the AC information processing section 33 and the posture information acquisition section 52 respectively function as posture acquisition sections and acquire posture change information obtained by detecting posture changes occurring in the vehicle A. Further, in each of the ECUs 30 and 40, the drawing data generation unit 39 and the correction execution unit 53 function as display correction units, and electronically correct the positional deviation of the virtual image Vi caused by each posture change.

As described above, in the first embodiment, in addition to the optical correction for maintaining the displayable area SA, the positional deviation of the virtual image Vi caused by vehicle vibration or the like is electronically corrected. Therefore, while the virtual image Vi can be superimposed on the superimposition target, the virtual image Vi can accurately follow the superimposition target.

Furthermore, in the first embodiment, while the drawing data generation unit 39 performs electronic correction for attitude changes in the low frequency band LB, the correction execution unit 53 performs high-level It electronically corrects attitude changes in the frequency band HB. As described above, if vibrations in the high-frequency band HB are electronically corrected while the vibrations in the low-frequency band LB having a large amplitude are corrected at the time of drawing, the virtual image Vi can be superimposed on the superimposition target with higher accuracy. become.

In addition, if the projection optical system 74 that maintains the displayable area SA is adjusted as in the first embodiment, the center of the eyebox where it is easy to ensure the brightness of the virtual image Vi may move away from the eyepoint EP. In this case, the visibility of the virtual image Vi may deteriorate due to insufficient brightness. Therefore, when the required brightness of the virtual image Vi is high, the projection adjustment unit 37 suspends the maintenance control of the displayable area SA by adjusting the projection optical system 74 . Then, the drawing data generator 39 electronically corrects the positional deviation of the virtual image Vi due to changes in the vehicle reference attitude and the eyepoint reference position. According to the above, it is possible to avoid a situation in which the information presentation using the virtual image Vi becomes difficult to understand due to the lack of luminance accompanying the maintenance control of the displayable area SA.

Further, in the first embodiment, when the required brightness of the virtual image Vi is high, the projection adjustment unit 37 adjusts the adjustment mechanism 75 so that the center of the eyebox EBX overlaps the eyepoint EP. Therefore, the brightness of the virtual image Vi can be sufficiently ensured during the interruption period of the maintenance control of the displayable area SA.

Furthermore, in the LCD control board 50 of the first embodiment, a distortion correction table TB for electronic correction is stored in the storage unit 50b, and the distortion correction table TB having contents corresponding to the adjustment result of the projection optical system 74 is used for correction. It is provided to the execution unit 53 . Therefore, the correction execution unit 53 can correct the distortion of the virtual image Vi in consideration of the angle of the concave mirror 74b, the eyepoint reference position, and the like.

In the first embodiment, the drawing ECU 30 corresponds to the "display control device", the DC information processing section 32 corresponds to the "state value acquisition section", and the AC information processing section 33 corresponds to the "first acquisition section" and the " "posture acquisition unit". Further, the luminance setting unit 34 corresponds to the “luminance grasping unit”, the drawing data generation unit 39 corresponds to the “first correction unit” and the “display correction unit”, and the orientation information acquisition unit 52 corresponds to the “second acquisition unit”. and the “posture acquisition unit”, and the correction execution unit 53 corresponds to the “second correction unit” and the “display correction unit”. Further, the distortion correction table TB corresponds to "correction information", and the windshield WS corresponds to "projection member".

(Second embodiment)
The second embodiment shown in FIG. 7 is a modification of the first embodiment. A virtual image display system 210 of the second embodiment is configured by a drawing ECU 30, a HUD 270, and the like. In the virtual image display system 210, the configuration corresponding to the display control ECU 40 (see FIG. 1) of the first embodiment is omitted. In the virtual image display system 210, the drawing ECU 30 electronically corrects the vehicle vibration in the high frequency band HB (see FIG. 2) in addition to the vehicle vibration in the low frequency band LB (see FIG. 2).

A functional unit corresponding to the brightness setting unit 34 (see FIG. 1) is omitted from the drawing ECU 30 of the second embodiment. On the other hand, in the drawing ECU 30, the functions of the AC information processing section 233 and the drawing data generation section 239 are different from those of the first embodiment. The AC information processing unit 233 acquires the posture change information of the vehicle vibration in the high frequency band HB in addition to the posture change information of the vehicle vibration in the second band LB2 (see FIG. 2).

The drawing data generation unit 239 draws the output image Po transmitted to the HUD 270 while reflecting the adjustment result of the projection optical system 74 by the projection adjustment unit 37 . Based on the posture change information provided from the AC information processing unit 233, the drawing data generation unit 239 performs processing for generating the output image Po as 3D graphics. It electronically corrects display position deviation caused by vehicle vibration.

The HUD 270 further includes an external light sensor 171 and a brightness setting section 172 . The outside light sensor 171 detects the amount of outside light entering the vehicle interior through the windshield WS. The brightness setting unit 172 grasps the required brightness required for the virtual image Vi based on the detection result of the external light sensor 171 . The brightness setting unit 172 sets the required brightness of the virtual image Vi and notifies the LCD 71, the backlight control unit 73 and the controller 78 of it.

Next, details of control of the adjustment mechanism 75 in the virtual image display system 210 will be described.

The projection adjustment unit 37 cooperates with the controller 78 and switches the contents of the attitude control of the projection optical system 74 based on the required luminance notified from the luminance setting unit 172 . When the required brightness is equal to or less than the threshold (see S103 in FIG. 6), the projection adjustment unit 37 and the controller 78 match the rotation angle of the concave mirror 74b (see FIG. 4) with the target posture based on the reference state value. In this case, control is performed to reduce changes in the displayable area SA (see FIG. 5).

On the other hand, when the required brightness is higher than the threshold (see S102 in FIG. 6), the projection adjustment unit 37 and the controller 78 stop maintaining the displayable area SA as described above and switch to brightness priority control. Specifically, the distance between the eyepoint EP and the center of the eyebox is controlled according to the required brightness. Assuming that the rotation angle of the concave mirror 74b at which the eyepoint EP and the center of the eyebox match is the maximum brightness posture, the projection adjustment unit 37 and the controller 78 move the concave mirror 74b to a rotation angle between the target posture and the maximum brightness posture. Control. The projection adjustment unit 37 and the controller 78 bring the rotation angle of the concave mirror 74b closer to the maximum brightness attitude as the required brightness notified from the brightness setting unit 172 increases. As a result, the center of the eyebox comes closer to the eyepoint EP as the required luminance increases, and when the required luminance is equal to or higher than a specific required luminance, the eyebox center coincides with the eyepoint EP.

In the second embodiment described so far, the same effect as in the first embodiment is obtained, and the projection optical system 74 is adjusted suitable for presenting information by superimposing the virtual image Vi on the superimposition target. In addition, in the second embodiment, when the required brightness exceeds the threshold, the higher the required brightness, the closer the center of the eyebox is to the eye point EP. Such control makes it possible to maintain the displayable area SA while ensuring the visibility of the virtual image Vi.

Further, in the second embodiment, electronic correction for correcting display position deviation due to vehicle vibration is performed entirely by the drawing data generation unit 239 without dividing the frequency band. According to the above system configuration, the configuration corresponding to the display control ECU 40 (see FIG. 1) is omitted. As a result, simplification of the virtual image display system 210 can be achieved.

In the second embodiment, the AC information processing unit 233 corresponds to the "posture acquisition unit", the drawing data generation unit 239 corresponds to the "display correction unit", and the projection adjustment unit 37 and the controller 78 correspond to the "projection adjustment unit". , and the luminance setting unit 172 corresponds to the “luminance grasping unit”.

(Other embodiments)
A plurality of embodiments of the present disclosure have been described above, but the present disclosure is not to be construed as being limited to the above embodiments, and can be applied to various embodiments and combinations within the scope of the present disclosure. can do.

In the modified example 1 of the second embodiment, the virtual image electronic correction function in the drawing ECU is omitted. In the virtual image display system of Modified Example 1, control for causing the virtual image to follow the superimposition target by correcting vehicle vibration is not performed. In Modified Example 1, among the correction processes related to the virtual image, only optical correction for adjusting the projection area in accordance with the vehicle reference attitude and the eyepoint reference position is performed.

In Modification 2 of the above embodiment, switching to luminance priority control based on required luminance is not performed. As in Modification 2, even if the required brightness of the virtual image is high, if the virtual image brightness can be ensured at and near the eyepoint reference position, the displayable area maintenance control may be continued. Note that the functional unit that determines the required luminance of the virtual image may be provided in any configuration of the virtual image display system. Further, the specific configuration for detecting the amount of outside light can be changed as appropriate. Furthermore, the content of luminance priority control can also be changed as appropriate.

In Modification 3 of the above embodiment, only the pitch attitude of the vehicle is used as the specific parameter. The projection adjustment unit controls the state of the projection optical system in cooperation with the adjustment mechanism so as to maintain the displayable area based on the vehicle reference attitude acquired as the reference state value by the DC information processing unit. Also, in Modification 4 of the above embodiment, only the eyepoint is used as the specific parameter. The projection adjustment unit controls the state of the projection optical system in cooperation with the adjustment mechanism so as to maintain the displayable area based on the eyepoint reference position acquired as the reference state value by the DC information processing unit.

In the modified example 5 of the first embodiment, the correction execution unit uses a correction function as correction information for distortion correction instead of the distortion correction table TB (see FIG. 3). The correction function is predefined as a polynomial or monomial that takes into account the projection area and the curved shape of the optical reflection system. In the correction function, post-correction coordinates (X, Y) are input parameters, and pre-correction coordinates (X, Y) are output parameters. The correction executing unit specifies the pre-correction coordinates (X, Y) assigned to each post-correction coordinate by a process of sequentially substituting the post-correction coordinates (X, Y) into the correction function.

The drawing ECU in the above embodiment may be configured integrally with another vehicle-mounted configuration. Specifically, in Modification 6 of the above-described embodiment, the drawing ECU is configured integrally with the display control ECU. That is, in Modification 6, a function of correcting vehicle vibration in a low frequency band by 3D rendering and a function of correcting vehicle vibration in a high frequency band in 2D are implemented in one display control device. Furthermore, in Modification 7 of the above embodiment, all the functions of the drawing ECU, display control ECU, and HUD of the first embodiment are integrated into one virtual image display unit.

A specific projection configuration of the HUD that displays the virtual image in the air may be changed as appropriate. For example, the HUD of Modification 8 is provided with an EL (Electro Luminescence) panel instead of the LCD and backlight. Furthermore, instead of the EL panel, a projector using a display device such as a plasma display panel, a cathode ray tube, and an LED can be used for the HUD.

Further, the HUD of Modification 9 is provided with a laser module (hereinafter "LSM") and a screen instead of the LCD and backlight. The LSM has a configuration including, for example, a laser light source and a MEMS (Micro Electro Mechanical Systems) scanner. The screen is for example a micromirror array or a microlens array. In the HUD of Modified Example 9, an output image is drawn on the screen by scanning laser light emitted from the LSM. A HUD projects an output image projected on a screen onto a projection area using a reflective optical system to display a virtual image in the air.

Further, the HUD of Modification 10 is provided with a DLP (Digital Light Processing, registered trademark) projector. A DLP projector has a digital mirror device (hereinafter “DMD”) provided with a large number of micromirrors and a projection light source that projects light toward the DMD. A DLP projector renders an output image on a screen by coordinated control of the DMD and the projection light source. Furthermore, the HUD of Modification 11 employs a projector using LCOS (Liquid Crystal On Silicon).

In variation 12 of the above embodiment, the light of the output image is projected onto an in-vehicle configuration separate from the windshield. Specifically, in Modification 13, the light of the output image is projected onto a projection member (for example, a combiner or the like) provided separately from the windshield.

In the modification 13 of the first embodiment, the low frequency band and the high frequency band are different from those in the first embodiment. Furthermore, in the modification 14 of the first embodiment, the electronic correction for vehicle vibration in the low frequency band and the high frequency band is performed by the correction execution unit of the display control ECU. Further, the inertial sensor of Modification 15 of the first embodiment is configured to include only one of the acceleration sensor and the gyro sensor. Furthermore, in the modification 16 of the first embodiment, the orientation information acquisition unit of the display control ECU acquires orientation change information from the communication bus, similarly to the sensor value acquisition unit of the drawing ECU.

The specific configuration of each processing unit in the display control ECU and the LCD control board can be changed as appropriate. As described above, the processing unit may have a configuration in which arithmetic cores such as CPU, GPU, FPGA, and other dedicated IPs are appropriately combined. Furthermore, the processing unit may be in the form of a SoC (System-on-a-Chip) including a large number of arithmetic cores, or in a form in which a plurality of physically independent arithmetic cores are individually mounted on a circuit board. may be

Various programs (display control programs) that construct each functional unit in the processing unit may be normal software that instructs the processing content to the CPU, GPU, etc., or hardware that makes the FPGA function as an arbitrary hardware logic circuit. It may be a software program.

Various non-transitory tangible storage media such as flash memory and hard disk can be used for the storage units of the display control ECU and the LCD control board. In addition, the storage medium for storing the programs and parameters related to virtual image display is not limited to the storage medium mounted on the circuit board, but may be in the form of, for example, a memory card. It may be configured to be electrically connected to a bus. Further, the storage medium may be an optical disk, a hard disk drive of a general-purpose computer, or the like, which is the source of copying to the display control ECU, the LCD control board, or the like.

The controller and techniques described in this disclosure may be implemented by a special purpose computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and techniques described in this disclosure may be implemented by dedicated hardware logic circuitry. Alternatively, the apparatus and techniques described in this disclosure may be implemented by one or more special purpose computers configured in combination with a processor executing a computer program and one or more hardware logic circuits. The computer program may also be stored as computer-executable instructions on a computer-readable non-transitional tangible recording medium.

A vehicle, EP eye point, EBX eye box, HB high frequency band, LB low frequency band, SA displayable area, TB distortion correction table (correction information), Vi virtual image, WS windshield (projection member), 10, 210 virtual image Display system 30 Drawing ECU (display control device) 32 DC information processing unit (state value acquisition unit) 33, 233 AC information processing unit (attitude acquisition unit, first acquisition unit) 34, 172 Brightness setting unit (brightness grasping unit), 37 projection adjustment unit, 39, 239 drawing data generation unit (display correction unit, first correction unit), 50a processing unit, 50b storage unit, 52 posture information acquisition unit (posture acquisition unit, second acquisition unit) , 53 correction execution unit (display correction unit, second correction unit), 74 projection optical system

Claims (8)

A virtual image display system used in a vehicle (A) for displaying a virtual image (Vi) superimposed on a superimposition target in the foreground,
a projection optical system (74) that projects the light imaged as the virtual image onto the projection member (WS) of the vehicle;
a state value acquisition unit (32) for acquiring a reference state value as a reference for at least one specific parameter that changes a displayable area (SA) in the foreground on which the virtual image can be superimposed;
a projection adjustment unit (37) that adjusts the projection optical system so as to reduce a change in the displayable area caused by a change in the reference state value;
a first acquisition unit (33) for acquiring posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) for acquiring posture change information in a high frequency band (HB) having a higher frequency than the low frequency band among the posture changes of the vehicle;
a first correction unit (39) that electronically corrects the positional deviation of the virtual image caused by the posture change in the low frequency band while reflecting the adjustment result of the projection optical system;
a second correction unit (53) that corrects the positional deviation of the virtual image caused by the posture change in the high frequency band by a process different from that of the first correction unit while reflecting the adjustment result of the projection optical system; A virtual image display system. Further comprising a luminance grasping unit (34, 172) for grasping the required luminance required for the virtual image,
2. The first correction unit electronically corrects a displacement of the virtual image caused by a change in the reference state value instead of adjusting the projection optical system when the required luminance exceeds a threshold. The virtual image display system described. The projection adjustment unit adjusts the center of an eye box (EBX), which is a range in which the virtual image can be viewed, to overlap an eye point (EP) of the viewer of the virtual image when the required luminance exceeds the threshold. 3. The virtual image display system according to claim 2 , wherein the projection optical system is adjusted in the second step. The projection adjustment unit adjusts the projection optical system so that the center of an eye box (EBX), which is a range in which the virtual image can be viewed, approaches an eye point (EP) of a viewer of the virtual image as the required luminance increases. 3. The virtual image display system according to claim 2 , wherein is adjusted. 5. The method according to any one of claims 1 to 4 , further comprising a storage unit (50b) for storing correction information (TB) used for electronic correction of said virtual image, and for providing said correction information corresponding to the result of adjustment of said projection optical system. The virtual image display system according to any one of the items. The virtual image display system according to any one of claims 1 to 5 , wherein the specific parameter includes at least one of a viewer's eye point (EP) of the virtual image and an attitude of the vehicle in the pitch direction. A display control device used in a vehicle (A) for controlling display of a virtual image (Vi) superimposed on a superimposition target in the foreground,
a state value acquisition unit (32) for acquiring a reference state value as a reference for at least one specific parameter that changes a displayable area (SA) in the foreground on which the virtual image can be superimposed;
Controlling the state of a projection optical system (74) that projects the light imaged as the virtual image on the projection member (WS) of the vehicle to reduce changes in the displayable area caused by variations in the reference state value. a projection adjustment unit (37) for adjusting the projection optical system,
a first acquisition unit (33) for acquiring posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) for acquiring posture change information in a high frequency band (HB) having a higher frequency than the low frequency band among the posture changes of the vehicle;
a first correction unit (39) that electronically corrects the positional deviation of the virtual image caused by the posture change in the low frequency band while reflecting the adjustment result of the projection optical system;
a second correction unit (53) that corrects the positional deviation of the virtual image caused by the posture change in the high frequency band by a process different from that of the first correction unit while reflecting the adjustment result of the projection optical system; A display controller comprising: A display control program for controlling the display of a virtual image (Vi) that is used in a vehicle (A) and superimposed on a superimposition target in the foreground,
at least one processing unit (50a),
a state value acquisition unit (32) for acquiring a reference state value as a reference for at least one specific parameter that changes a displayable area (SA) in the foreground on which the virtual image can be superimposed;
Controlling the state of a projection optical system (74) that projects the light imaged as the virtual image on the projection member (WS) of the vehicle to reduce changes in the displayable area caused by variations in the reference state value. a projection adjustment unit (37) for adjusting the projection optical system,
a first acquisition unit (33) for acquiring posture change information in a low frequency band (LB) among posture changes occurring in the vehicle;
a second acquisition unit (52) for acquiring posture change information in a high frequency band (HB) having a higher frequency than the low frequency band among the posture changes of the vehicle;
a first correction unit (39) that electronically corrects the positional deviation of the virtual image caused by the posture change in the low frequency band while reflecting the adjustment result of the projection optical system;
a second correction unit (53) that corrects the positional deviation of the virtual image caused by the posture change in the high frequency band by a process different from that of the first correction unit while reflecting the adjustment result of the projection optical system; A display control program that functions to include

Images