JP7809647B2 - Vehicle Display System - Google Patents

Description

This disclosure relates to a vehicle display system.

Patent Document 1 discloses a display system that identifies the driving status of other vehicles around the vehicle and makes the driver aware of the status. The display system in Patent Document 1 includes a detection unit, an acquisition unit, and a display unit. When the detection unit detects another vehicle, the acquisition unit acquires other vehicle information indicating the driving status of the other vehicle. The display unit generates an other vehicle image representing the driving status (including driving settings and behavior) of the other vehicle based on the acquired other vehicle information, and displays the generated other vehicle image by superimposing it on the view seen through the windshield from the driver's seat of the vehicle, while associating it with the other vehicle.

Japanese Patent Application Publication No. 2017-037634

The display system in Patent Document 1 acquires and displays information about other vehicles through vehicle-to-vehicle communication when other vehicles are present around the vehicle. However, even though other vehicles do not necessarily affect the vehicle, acquiring and displaying information about other vehicles increases the display load on the display system.

The present disclosure aims to provide an image display system for vehicles that reduces the display load on the display system while alerting vehicle occupants to vehicles that they should be paying attention to.

The vehicle display system of the present disclosure is a vehicle display system provided in a vehicle,
a display unit that displays position information of at least one other vehicle on map information around the vehicle;
a designation unit that designates the other vehicle based on an instruction from an occupant of the vehicle;
a determination unit that determines whether the other vehicle is in an autonomous driving mode,
When the determination unit determines that the other vehicle is not in the autonomous driving mode, the display unit displays a driving assistance instruction to assist the driving of the vehicle.

The vehicle display system of the present disclosure displays driving assistance instructions when the determination unit determines that the other vehicle designated by the designation unit is not in autonomous driving mode. In other words, when a vehicle to watch is designated and the other vehicle is not in autonomous driving mode, and there is a possibility that the other vehicle may affect the vehicle, the driving assistance instructions are displayed. This allows the vehicle occupants to focus their attention on the other vehicle. Furthermore, since the driving assistance instructions are displayed when the determination unit determines that the other vehicle is not in autonomous driving mode, the display unit does not need to display the driving assistance instructions all the time. This reduces the display load on the vehicle display system.

FIG. 1 is a block diagram of a vehicle system including a vehicle display system according to the present disclosure. FIG. 2 is a schematic diagram of a head-up display (HUD) according to this embodiment included in the vehicle display system. FIG. 3 is a flowchart showing the processing flow of the vehicular display system. FIG. 4 shows an example of displaying map information and position information of the vehicle and other vehicles when the vehicle is traveling on a two-lane road. FIG. 5 shows a first example of a HUD display. FIG. 6 is a second example of a HUD display. FIG. 7 shows an example of displaying map information and position information of the vehicle and other vehicles when the vehicle is changing lanes. FIG. 8 shows an example of displaying map information, position information of the vehicle, and other vehicles when the other vehicle changes lanes. FIG. 9 shows an example of display of map information and position information of a vehicle and another vehicle when the vehicle and another vehicle are traveling toward a merging point. FIG. 10 shows an example of displaying map information and position information of the vehicle and other vehicles when the vehicle is traveling toward a plurality of intersections.

An embodiment of the present disclosure (hereinafter referred to as the present embodiment) will be described below with reference to the drawings. For the sake of convenience, the dimensions of each component shown in the drawings may differ from the actual dimensions of each component.

For the sake of convenience, in the description of this embodiment, the terms "left-right direction," "up-down direction," and "front-rear direction" may be referred to as appropriate. These directions are relative directions set for the HUD (head-up display) 42 shown in Figure 2. Here, the "left-right direction" refers to a direction that includes the "left direction" and the "right direction." The "up-down direction" refers to a direction that includes the "upward direction" and the "downward direction." The "front-rear direction" refers to a direction that includes the "forward direction" and the "rearward direction." Although not shown in Figure 2, the left-right direction is a direction that is perpendicular to the up-down direction and the front-rear direction.

The vehicle system 2 according to this embodiment will be described below with reference to Figure 1. Figure 1 is a block diagram of the vehicle system 2. The vehicle 1 equipped with the vehicle system 2 is a vehicle (automobile) that can run in manual driving mode or automatic driving mode.

As shown in FIG. 1, the vehicle system 2 includes a vehicle control unit 3, a vehicle display system 4 (hereinafter simply referred to as the "display system 4"), a sensor 5, a camera 6, and a radar 7. Furthermore, the vehicle system 2 includes an HMI (Human Machine Interface) 8, a GPS (Global Positioning System) 9, a wireless communication unit 10, a memory device 11, a steering actuator 12, a steering device 13, a brake actuator 14, a brake device 15, an accelerator actuator 16, and an accelerator device 17.

The vehicle control unit 3 is configured to control the driving of the vehicle. The vehicle control unit 3 is configured, for example, by at least one electronic control unit (ECU: Electronic Control Unit). The electronic control unit includes a computer system (e.g., SoC (System on a Chip)) including one or more processors and one or more memories, and an electronic circuit composed of active elements such as transistors and passive elements. The processor includes, for example, at least one of a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), and TPU (Tensor Processing Unit). The CPU may be configured by multiple CPU cores. The GPU may be configured by multiple GPU cores. The memory includes a ROM (Read Only Memory) and a RAM (Random Access Memory). A vehicle control program may be stored in the ROM. For example, the vehicle control program may include an artificial intelligence (AI) program for autonomous driving. The AI program is a program (trained model) constructed by supervised or unsupervised machine learning (particularly, deep learning) using a multi-layer neural network. The RAM may temporarily store the vehicle control program, vehicle control data, and/or surrounding environment information indicating the surrounding environment of the vehicle. The processor may be configured to load a specified program from various vehicle control programs stored in the ROM onto the RAM and execute various processes in cooperation with the RAM. The computer system may also be configured with a non-von Neumann computer such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). Furthermore, the computer system may be configured with a combination of a von Neumann computer and a non-von Neumann computer. The vehicle control unit 3 may control the operation of the display system 4. In this case, the vehicle control unit 3 constitutes a part of the display system 4. The vehicle control unit 3 is an example of a determination unit.

The display system 4 comprises a headlamp 20, a HUD 42, and a display control unit 43.

The headlamps 20 are arranged on the left and right sides of the front of the vehicle 1 and comprise low beam lamps configured to irradiate a low beam ahead of the vehicle 1 and high beam lamps configured to irradiate a high beam ahead of the vehicle 1. Each of the low beam lamps and high beam lamps has one or more light-emitting elements such as an LED (Light Emitting Diode) or an LD (Laser Diode), and optical components such as a lens and a reflector.

At least a portion of the HUD 42 is located inside the vehicle 1. Specifically, the HUD 42 is installed in a predetermined location within the interior of the vehicle 1. For example, the HUD 42 may be located within the dashboard of the vehicle 1. The HUD 42 is a visual interface between the vehicle 1 and the occupant. The HUD 42 is configured to display predetermined information (hereinafter referred to as HUD information) to the occupant so that the HUD information is superimposed on the real space outside the vehicle 1 (particularly, the surrounding environment in front of the vehicle). In this manner, the HUD 42 is an AR (Augmented Reality) display. The HUD information displayed by the HUD 42 is, for example, vehicle driving information related to the driving of the vehicle 1 and/or surrounding environment information related to the surrounding environment of the vehicle 1 (particularly, information related to objects present outside the vehicle 1). The HUD 42 is an example of a display unit. Details of the HUD 42 will be described later.

The display control unit 43 is configured to control the operation of the headlamps 20 and the HUD 42. Furthermore, the display control unit 43 is configured to determine the driving mode of at least one other vehicle around the vehicle 1. The display control unit 43 is configured by an electronic control unit (ECU). The electronic control unit includes a computer system (e.g., SoC, etc.) including one or more processors and one or more memories, and an electronic circuit composed of active elements such as transistors and passive elements. The processor includes at least one of a CPU, MPU, GPU, and TPU. The memory includes ROM and RAM. The computer system may also be configured by a non-von Neumann type computer such as an ASIC or FPGA. The display control unit 43 is an example of a determination unit.

The display control unit 43 is configured to control the HUD 42 to display on the HUD 42 map information M of the surroundings of the vehicle 1, position information of the vehicle 1, position information of at least one other vehicle around the vehicle 1, and driving assistance instructions X that assist in driving the vehicle 1. The driving assistance instructions X include a stop instruction X1 or a warning instruction X2. The HUD 42 may display an identifier along with the position information of at least one other vehicle. If there are multiple other vehicles around the vehicle 1, the HUD 42 may display an identifier for each position information along with the position information of each other vehicle.

In this embodiment, the vehicle control unit 3 and the display control unit 43 are provided as separate components, but the vehicle control unit 3 and the display control unit 43 may also be configured as an integrated unit. The display control unit 43 and the vehicle control unit 3 may be configured as a single electronic control unit. In this case, the vehicle control unit 3 serves as a determination unit and determines the driving mode of at least one other vehicle around the vehicle 1. The display control unit 43 may also be configured as two electronic control units: an electronic control unit configured to control the headlamps 20, and an electronic control unit configured to control the operation of the HUD 42. The control board 425 that controls the operation of the HUD 42 may also be configured as part of the display control unit 43.

The display control unit 43 is configured to control the sensor 5 to designate at least one other vehicle around the vehicle 1 based on instructions from the occupant of the vehicle 1. The sensor 5 is, for example, a sound sensor that detects the voice of the occupant. When the occupant of the vehicle 1 speaks the identifiers of at least one other vehicle, the sensor 5 is configured to receive instructions corresponding to the identifiers. The sensor 5 may include at least one of an acceleration sensor, a speed sensor, and a gyro sensor. The sensor 5 is configured to detect the driving state of the vehicle 1 and output driving state information to the vehicle control unit 3. The sensor 5 may further include a seating sensor that detects whether the driver is sitting in the driver's seat, a face direction sensor that detects the direction of the driver's face, an external weather sensor that detects external weather conditions, a human presence sensor that detects whether there is a person inside the vehicle, etc. The sensor 5 is an example of a designation unit.

The camera 6 is a camera that includes an imaging element such as a CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). The camera 6 includes one or more external cameras 6A and an internal camera 6B. The external camera 6A is configured to acquire image data showing the vehicle's surroundings and transmit the image data to the vehicle control unit 3. The vehicle control unit 3 acquires surrounding environment information based on the transmitted image data. Here, the surrounding environment information may include information about objects (pedestrians, other vehicles, signs, etc.) outside the vehicle 1. For example, the surrounding environment information may include information about the attributes of objects outside the vehicle 1 and information about the distance and position of the objects relative to the vehicle 1. The external camera 6A may be configured as a monocular camera or a stereo camera. The imaging camera 30 may also be substituted for the external camera 6A.

The internal camera 6B is disposed inside the vehicle 1 and is configured to acquire image data showing the occupant. The display control unit 43 is configured to control the internal camera 6B to designate at least one other vehicle around the vehicle 1 based on instructions from the occupant of the vehicle 1. The internal camera 6B is a tracking camera that detects and tracks the occupant's viewpoint P. Here, the occupant's viewpoint P may be either the viewpoint of the occupant's left eye or the viewpoint of the occupant's right eye. Alternatively, the viewpoint P may be defined as the midpoint of a line segment connecting the viewpoints of the left eye and the right eye. The display control unit 43 may identify the position of the occupant's viewpoint P based on the image data acquired by the internal camera 6B. The position of the occupant's viewpoint P may be updated at a predetermined interval based on the image data, or may be determined only once when the vehicle 1 is started. The internal camera 6B is an example of a designation unit.

The radar 7 includes at least one of a millimeter wave radar, a microwave radar, and a laser radar (e.g., a LiDAR unit). For example, the LiDAR unit is configured to detect the surrounding environment of the vehicle 1. In particular, the LiDAR unit is configured to acquire 3D mapping data (point cloud data) indicating the surrounding environment of the vehicle 1 and then transmit the 3D mapping data to the vehicle control unit 3. The vehicle control unit 3 identifies surrounding environment information based on the transmitted 3D mapping data.

The HMI 8 is composed of an input unit that accepts input operations from the driver, and an output unit that outputs driving information, etc. to the driver. The input unit includes a steering wheel, accelerator pedal, brake pedal, a driving mode selector switch that switches the vehicle's driving mode, etc. The output unit is a display (excluding the HUD 42) that displays various driving information. The GPS 9 is configured to acquire location information of the vehicle 1 via communication with a satellite, and output the acquired location information to the vehicle control unit 3. The GPS 9 is an example of a first communication unit. A satellite is an example of an external device.

The wireless communication unit 10 is configured to receive information about at least one other vehicle around vehicle 1 (e.g., location information and driving information about the other vehicle, etc.) from the other vehicle, and to transmit information about vehicle 1 (e.g., location information and driving information about vehicle 1, etc.) to the other vehicle (vehicle-to-vehicle communication). The wireless communication unit 10 may also acquire location information for multiple other vehicles around vehicle 1. The wireless communication unit 10 is configured to receive map information M and infrastructure information from infrastructure facilities such as traffic lights and marker lights, and to transmit driving information about vehicle 1 to the infrastructure facilities (road-to-vehicle communication). The wireless communication unit 10 is also configured to receive information about pedestrians from portable electronic devices (smartphones, tablets, wearable devices, etc.) carried by pedestrians, and to transmit driving information about vehicle 1 to the portable electronic device (pedestrian-to-vehicle communication).

The wireless communication unit 10 may communicate directly with other vehicles, infrastructure equipment, or portable electronic devices, or may communicate via an access point. Furthermore, the wireless communication unit 10 may communicate with other vehicles, infrastructure equipment, or portable electronic devices via a communication network (not shown). The communication network includes at least one of the Internet, a local area network (LAN), a wide area network (WAN), and a radio access network (RAN). Examples of wireless communication standards include Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), LPWA, DSRC (registered trademark), or Li-Fi. Furthermore, the vehicle 1 may communicate with other vehicles, infrastructure equipment, or portable electronic devices using a fifth-generation mobile communication system (5G). The wireless communication unit 10 is an example of a second communication unit and a third communication unit.

The storage device 11 is an external storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage device 11 may store two-dimensional or three-dimensional map information M and/or a vehicle control program. For example, the three-dimensional map information may be composed of 3D mapping data (point cloud data). The storage device 11 is configured to output the map information M and the vehicle control program to the vehicle control device 3 in response to a request from the vehicle control device 3. The map information M and the vehicle control program may be updated via the wireless communication unit 10 and a communication network.

When vehicle 1 is traveling in autonomous driving mode, vehicle control unit 3 automatically generates at least one of a steering control signal, an accelerator control signal, and a brake control signal based on driving condition information, surrounding environment information, vehicle 1 position information, map information M, etc. Steering actuator 12 is configured to receive a steering control signal from vehicle control unit 3 and control steering device 13 based on the received steering control signal. Brake actuator 14 is configured to receive a brake control signal from vehicle control unit 3 and control brake device 15 based on the received brake control signal. Accelerator actuator 16 is configured to receive an accelerator control signal from vehicle control unit 3 and control accelerator device 17 based on the received accelerator control signal. In this way, vehicle control unit 3 automatically controls the traveling of vehicle 1 based on driving condition information, surrounding environment information, vehicle 1 position information, map information M, etc. In other words, in autonomous driving mode, the traveling of vehicle 1 is automatically controlled by vehicle system 2.

On the other hand, when the vehicle 1 is driven in manual driving mode, the vehicle control unit 3 generates a steering control signal, an accelerator control signal, and a brake control signal in accordance with the driver's manual operation of the accelerator pedal, brake pedal, and steering wheel. In this way, in manual driving mode, the steering control signal, accelerator control signal, and brake control signal are generated by the driver's manual operation, and therefore the driving of the vehicle 1 is controlled by the driver.

Next, the driving modes of vehicle 1 will be explained. Note that at least one other vehicle surrounding vehicle 1 also has the following driving modes. Explanations of the driving modes of other vehicles will be omitted as they overlap with the explanation of the driving modes of vehicle 1.

The driving modes consist only of an autonomous driving mode and a manual driving mode. In the manual driving mode, the vehicle system 2 does not automatically perform driving control, and the driver drives the vehicle 1 without driving assistance from the vehicle system 2. The manual driving mode is an example of a second driving mode. The autonomous driving modes include a first autonomous driving mode and a second autonomous driving mode, which is an autonomous driving level lower than the autonomous driving level of the first autonomous driving mode. Specifically, the autonomous driving modes include five levels. The autonomous driving modes include a fully autonomous driving mode at level 5, advanced driving assistance modes at levels 4 and 3, and driving assistance modes at levels 2 and 1. The autonomous driving levels are set in the order of level 5 being the highest, followed by level 4, level 3, and level 2, with level 1 being the lowest.

In fully autonomous driving mode (Level 5), the vehicle system 2 automatically performs all driving control, including steering control, braking control, and accelerator control, and the driver is not in a state where he or she can drive the vehicle 1. If fully autonomous driving mode (Level 5) is the first autonomous driving mode, the advanced driving assistance mode and driving assistance mode (Level 4 to Level 1) other than fully autonomous driving mode are the second autonomous driving modes.

In the Level 4 advanced driving assistance mode, when vehicle 1 is under specific conditions, vehicle system 2 automatically performs all driving control, including steering control, braking control, and accelerator control, and the driver is in a state where he or she can drive vehicle 1 but does not drive vehicle 1. For example, at Level 4, when vehicle 1 is driving within a specific area, vehicle system 2 automatically performs all driving control, and when vehicle 1 is driving outside the specific area, the driver may drive vehicle 1. If Level 4 advanced driving assistance mode is the first autonomous driving mode, the autonomous driving modes from Level 3 to Level 1 are the second autonomous driving modes.

In the Level 3 advanced driving assistance mode, when vehicle 1 is under certain conditions, vehicle system 2 automatically performs all driving control, and the driver may drive vehicle 1 in response to requests from vehicle system 2. For example, at Level 3, when vehicle 1 is traveling on a highway, vehicle system 2 basically automatically performs driving control, but vehicle system 2 may send requests to the driver at predetermined times, and the driver may drive vehicle 1 in response to those requests.

In level 2 driving assistance mode, vehicle system 2 automatically performs some driving control, including steering control, braking control, and accelerator control, under certain conditions, while the driver drives vehicle 1 with the driving assistance of vehicle system 2. For example, at level 2, when vehicle 1 is traveling on a highway and there is a slow vehicle ahead, vehicle system 2 may perform driving control so that the vehicle automatically overtakes the vehicle ahead. In level 1 driving assistance mode, vehicle system 2 only performs some driving control, and the driver basically drives vehicle 1. For example, at level 1, vehicle system 2 performs driving control so that vehicle 1 automatically stops to prevent collision with objects, drives vehicle 1 while maintaining a certain distance from the vehicle ahead, and drives vehicle 1 so that vehicle 1 does not deviate from its lane.

When at least one other vehicle around vehicle 1 is in Level 5 fully autonomous driving mode or Level 4 advanced driving assistance mode, vehicle system 2 performs most of the driving control of the other vehicle, making it easier to predict the driving of the other vehicle. On the other hand, when the other vehicle is in Level 3 advanced driving assistance mode, Level 2 driving assistance mode, or Level 1 driving assistance mode, the driver of the other vehicle is more likely to be driving the other vehicle than when vehicle system 2 controls the driving, making it more difficult to predict the driving of the other vehicle. For this reason, in this example, autonomous driving modes from Level 3 to Level 1 are set as second autonomous driving modes.

The display control unit 43 is configured to determine whether another vehicle that has communicated with the wireless communication unit 10 is in an autonomous driving mode. Furthermore, the vehicle control unit 3 or the display control unit 43 is configured to determine whether the other vehicle is in a first autonomous driving mode or a second autonomous driving mode via communication with the wireless communication unit 10. The determination process of the vehicle control unit 3 or the display control unit 43 will be described later.

Next, the details of the HUD 42 will be described. Figure 2 is a schematic diagram of the HUD 42 according to this embodiment. As shown in Figure 2, the HUD 42 includes a HUD main body 420. The HUD main body 420 has a housing 422 and an exit window 423. The exit window 423 is a transparent plate that transmits visible light. Inside the housing 422, the HUD main body 420 has an image generation unit (PGU: Picture Generation Unit) 424, a control board 425, a concave mirror 426, a drive mechanism 427, and a plane mirror 428.

The image generation unit 424 has a light source, optical components, and a display device. The light source is, for example, a laser light source or an LED light source. The laser light source is, for example, an RGB laser light source configured to emit red laser light, green laser light, and blue laser light, respectively. The optical components include a prism, a lens, a diffuser, a magnifying glass, etc. The display device is, for example, a liquid crystal display, a DMD (Digital Mirror Device), etc. The drawing method of the image generation unit 424 may be a raster scan method, a DLP method, or an LCOS method. If the DLP method or LCOS method is adopted, the light source of the HUD 42 may be an LED light source. Note that if the liquid crystal display method is adopted, the light source of the HUD 42 may be a white LED light source.

The control board 425 is configured to control the operation of the image generation unit 424 and the drive mechanism 427. The control board 425 is equipped with a processor such as a CPU (Central Processing Unit) and memory, and the processor executes a computer program read from the memory to control the operation of the image generation unit 424. The control board 425 is configured to generate a control signal for controlling the operation of the image generation unit 424 based on image data transmitted from the display control unit 43, and then transmit the generated control signal to the image generation unit 424. The control board 425 may also control the drive mechanism 427 to change the orientation of the concave mirror 426.

The concave mirror 426 is disposed on the optical path of the light emitted from the image generation unit 424 and reflected by the plane mirror 428. Specifically, the concave mirror 426 is disposed in front of the image generation unit 424 and the plane mirror 428 within the HUD main body 420. The concave mirror 426 is configured to reflect the light emitted by the image generation unit 424 toward the windshield 18 (e.g., the front window of the vehicle 1) via the exit window 423. The concave mirror 426 has a concavely curved reflective surface that forms a virtual image, and reflects the image of the light emitted from the image generation unit 424 and focused at a predetermined magnification.

Light emitted from the exit window 423 of the HUD main body 420 is irradiated onto the windshield 18. A portion of the light irradiated from the HUD main body 420 onto the windshield 18 is reflected toward the occupant's viewpoint P. As a result, the occupant perceives the light (predetermined image) emitted from the HUD main body 420 as a virtual image formed at a predetermined distance in front of the windshield 18. In this way, the image displayed by the HUD 42 is superimposed on the real space in front of the vehicle 1 through the windshield 18, allowing the occupant to visually perceive a virtual image object I formed by the predetermined image as floating above the road outside the vehicle.

When forming a 2D image (planar image) as the virtual image object I, a predetermined image is projected so as to become a virtual image at a single, arbitrarily determined distance. When forming a 3D image (stereoscopic image) as the virtual image object I, multiple predetermined images, which may be identical or different, are projected so as to become virtual images at different distances. The distance of the virtual image object I (the distance from the occupant's viewpoint P to the virtual image) can be adjusted by adjusting the distance from the image generation unit 424 to the occupant's viewpoint P (for example, by adjusting the distance between the image generation unit 424 and the concave mirror 426). The HUD main body 420 does not need to have a plane mirror 428. In this case, the light emitted from the image generation unit 424 enters the concave mirror 426 without being reflected by the plane mirror 428.

Next, we will explain the processing flow of the display system 4 according to this embodiment. Figure 3 is a flowchart showing the processing flow of the display system 4.

As shown in FIG. 3, first, the GPS 9 acquires the position information of the vehicle 1 through communication with a satellite, and the wireless communication unit 10 acquires the position information of at least one other vehicle around the vehicle 1 (step 1). If there are multiple other vehicles around the vehicle 1, the wireless communication unit 10 acquires the position information of each of the multiple other vehicles. The GPS 9 or the wireless communication unit 10 may update the position information of the vehicle 1 or the position information of the other vehicles at predetermined intervals. The acquired position information of the vehicle 1 and the acquired position information of the other vehicles are transmitted to the display control unit 43 via the vehicle control unit 3. At this time, if map information M is stored in the storage device 11, the map information M is also transmitted to the display control unit 43 via the vehicle control unit 3. The map information M may be acquired through road-to-vehicle communication between infrastructure equipment and the wireless communication unit 10 and transmitted to the display control unit 43.

When the display control unit 43 receives the location information of the other vehicle, it assigns an identifier to the location information of the other vehicle (step 2). If there are multiple other vehicles around vehicle 1, an identifier is assigned to the location information of each other vehicle. The identifier may be assigned to the location information of the other vehicle by the vehicle control unit 3 before it is sent to the display control unit 43.

The display control unit 43 displays map information M of the surroundings of vehicle 1 on the HUD 42, and displays the acquired location information of vehicle 1, the acquired location information of other vehicles, and the assigned identifiers on the map information M (step 3).

Next, the display control unit 43 determines whether the sensor 5 or the internal camera 6B has received an instruction corresponding to the identifier (step 4). In other words, the display control unit 43 determines whether another vehicle corresponding to the identifier has been designated. When an occupant of vehicle 1 sends an instruction corresponding to one of the identifiers displayed on the HUD 42 to the sensor 5 or the internal camera 6B, the other vehicle is designated. Note that in this example, the occupant is the driver of vehicle 1, but the occupant is not limited to the driver. The occupant may also be a passenger sitting in the passenger seat or the rear seat.

If no other vehicle is specified (NO in step 4), the display system 4 returns to the process of acquiring the position information of vehicle 1 and the position information of other vehicles (step 1). If another vehicle is specified (YES in step 4), the specification is sent to the display control unit 43. Based on the transmitted specification, the display control unit 43 determines whether the wireless communication unit 10 can communicate with one of the specified other vehicles (step 5).

If communication with the specified other vehicle is not possible (NO in step 5), the driving mode of the other vehicle is not determined, making it difficult for the occupants of vehicle 1 to predict the driving of the other vehicle. In this example, to alert the occupants of vehicle 1, if communication with the specified other vehicle is not possible, the display control unit 43 displays a warning indication X2 on the HUD 42 (step 12). If communication with the specified other vehicle is possible (YES in step 5), the display control unit 43 determines whether the driving mode of the other vehicle communicated with by the wireless communication unit 10 is manual driving mode or automatic driving mode (step 6).

If the display control unit 43 determines that the driving mode of the other vehicle communicated with by the wireless communication unit 10 is manual driving mode (YES in step 6), the driver of the other vehicle drives the other vehicle without driving assistance from the vehicle system 2, making it difficult for the occupants of vehicle 1 to predict the driving of the other vehicle. In this case, the display control unit 43 displays a driving assistance instruction X on the HUD 42. To determine whether to display a stop instruction X1 or a warning instruction X2 among the driving assistance instructions X, the display control unit 43 determines whether the area around vehicle 1 is a public road or a highway based on the map information M and the position information of vehicle 1 (step 10). If the display control unit 43 determines that the driving mode of the other vehicle communicated with by the wireless communication unit 10 is autonomous driving mode (NO in step 6), the display control unit 43 determines whether the driving mode of the other vehicle is fully autonomous driving mode (step 7).

If the display control unit 43 determines that the driving mode of the other vehicle is fully autonomous driving mode (level 5) (YES in step 7), the vehicle system 2 of the other vehicle controls the driving of the other vehicle, making it easier for the occupants of vehicle 1 to predict the driving of the other vehicle. In this case, there is not much need to alert the occupants to the other vehicle, so the display control unit 43 does not display the driving assistance instruction X on the HUD 42 (step 13). If the display control unit 43 determines that the driving mode of the other vehicle is not fully autonomous driving mode (NO in step 7), the display control unit 43 determines whether the driving mode of the other vehicle is advanced driving assistance mode (step 8).

If the display control unit 43 determines that the driving mode of the other vehicle is not the advanced driving assistance mode (NO in step 8), the other vehicle is in level 2 driving assistance mode or level 1 driving assistance mode. In this case, the driver of the other vehicle often drives the other vehicle manually rather than through driving control by the vehicle system 2, making it difficult for the occupants of vehicle 1 to predict the driving of the other vehicle. In this case, the display control unit 43 determines whether the area around vehicle 1 is a public road or a highway (step 10) in order to display a driving assistance instruction X on the HUD 42 to alert the occupants to the other vehicle. If the display control unit 43 determines that the driving mode of the other vehicle is the advanced driving assistance mode (YES in step 8), the driving mode of the other vehicle is the advanced driving assistance mode of level 4 or level 3. Next, the display control unit 43 determines whether the other vehicle is under specific conditions (step 9).

If the display control unit 43 determines that the other vehicle is operating under specific conditions (YES in step 9), the other vehicle's driving mode is a level 4 advanced driving assistance mode and is operating under specific conditions. In this case, the vehicle system 2 of the other vehicle performs driving control, making it easy for the occupants of vehicle 1 to predict the other vehicle's driving. Therefore, there is not much need to alert the occupants to the other vehicle, and the display control unit 43 does not display the driving assistance instruction X on the HUD 42 (step 13). If the display control unit 43 determines that the other vehicle is not operating under specific conditions (NO in step 9), the other vehicle's driving mode is a level 3 advanced driving assistance mode or a level 4 advanced driving assistance mode that is not operating under specific conditions. Because the driver of the other vehicle often drives the other vehicle manually and it is difficult for the occupants of vehicle 1 to predict the other vehicle's driving, in this case the display control unit 43 displays the driving assistance instruction X on the HUD 42. Specifically, the display control unit 43 determines whether the area around vehicle 1 is an ordinary road or a highway (step 10). Between step 8 and step 9, the display control unit 43 may determine whether the other vehicle is in a level 3 advanced driving assistance mode or a level 4 advanced driving assistance mode.

If the display control unit 43 determines that the area around vehicle 1 is a public road (public road in step 10), the display control unit 43 displays a driving assistance instruction X, a stop sign X1, on the HUD 42 to alert the occupants of vehicle 1 to other vehicles (step 11). On the other hand, if the display control unit 43 determines that the area around vehicle 1 is a highway (highway in step 10), the display control unit 43 displays a driving assistance instruction X, a warning sign X2, on the HUD 42 (step 12). After performing any of steps 11, 12, and 13, the display system 4 returns to the process of acquiring the position information of vehicle 1 and the position information of other vehicles (step 1). In this example, the display system 4 always displays map information M of the area around vehicle 1, the position information of vehicle 1, and, if there is at least one other vehicle around vehicle 1, its position information and its identifier on the HUD 42. The display system 4 also displays a driving assistance instruction X on the HUD 42 depending on the driving mode of the other vehicle.

Figure 4 shows an example of the display of map information M, vehicle 1's position information, and other vehicle's position information and identifiers when vehicle 1 is traveling on a two-lane road. As shown in Figure 4, map information M of the area around vehicle 1 is displayed, and vehicle 1's position information is displayed on the map information M. Furthermore, the other vehicles around vehicle 1 include an oncoming vehicle 2A, a preceding vehicle 2B, and a following vehicle 2C. Oncoming vehicle 2A is traveling in the lane opposite the lane in which vehicle 1 is traveling. The preceding vehicle 2B and following vehicle 2C are traveling in the same lane as vehicle 1. Identifier A is displayed on map information M along with the oncoming vehicle 2A's position information. Identifier B is displayed on map information M along with the position information of preceding vehicle 2B. Identifier C is displayed on map information M along with the position information of following vehicle 2C (step 3 in Figure 3).

For example, an occupant of vehicle 1 vocalizes the identifier A of an oncoming vehicle 2A among multiple other vehicles. Sensor 5 detects the occupant's voice and accepts the occupant's instruction to designate oncoming vehicle 2A (YES in step 4 of Figure 3). Alternatively, the occupant fixes viewpoint P on the position information or identifier A of oncoming vehicle 2A displayed on HUD 42 for a certain period of time, and internal camera 6B detects the occupant's viewpoint P, thereby accepting the occupant's instruction and designating oncoming vehicle 2A (YES in step 4 of Figure 3).

When the oncoming vehicle 2A is designated, the wireless communication unit 10 performs vehicle-to-vehicle communication with the oncoming vehicle 2A (YES in step 5 of FIG. 3). The display control unit 43 then determines the driving mode of the oncoming vehicle 2A (steps 6-9 of FIG. 3). If the driving mode of the oncoming vehicle 2A is level 5 fully automated mode (YES in step 7 of FIG. 3) or level 4 advanced driving assistance mode under specific conditions (YES in step 9 of FIG. 3), there is not much need to alert the occupants of vehicle 1 to other vehicles, so the display control unit 43 does not display the driving assistance instruction X on the HUD 42 (step 13). On the other hand, if the driving mode of the oncoming vehicle 2A is manual driving mode (YES in step 6 of FIG. 3), level 2 or level 1 driving assistance mode (NO in step 8 of FIG. 3), level 3 advanced driving assistance mode, or level 4 advanced driving assistance mode not under specific conditions (YES in step 9 of FIG. 3), the display control unit 43 displays the driving assistance instruction X on the HUD 42. At this time, the display control unit 43 determines whether the area around the vehicle 1 is an ordinary road or an expressway (step 10 in FIG. 3).

When vehicle 1 is traveling on a public road, the display control unit 43 displays a stop sign X1 on the HUD 42 (step 11 in Figure 3). This alerts the occupants of vehicle 1 to the oncoming vehicle 2A, thereby reducing the number of collisions between vehicle 1 and other vehicles on public roads.

Figure 5 shows an example of a virtual image object I when the area around vehicle 1 is a public road as display example 1 of HUD 42. As shown in Figure 5, map information M, position information of vehicle 1, and position information of other vehicles are displayed in the lower right corner of the display area of HUD 42. Furthermore, as a driving assistance instruction X, a stop sign X1 is displayed in the upper right corner of the display area of HUD 42. Note that Figure 5 is an example of a display, and the size, shape, and display location of the map information M and stop sign X1 are not limited.

When vehicle 1 is traveling on a highway, the display control unit 43 displays a warning indication X2 on the HUD 42 (step 12 in Figure 3). If vehicle 1 stops temporarily on the highway, there is a possibility that vehicle 1 will collide with the following vehicle 2C. Therefore, when vehicle 1 is traveling on a highway, it is preferable that the warning indication X2 be displayed on the HUD 42 rather than the stop indication X1.

Figure 6 shows another example of a virtual image object I when the vehicle 1 is surrounded by a highway, as another display example 2 of the HUD 42. As shown in Figure 6, map information M, position information of the vehicle 1, and position information of other vehicles are displayed in the lower right corner of the display area of the HUD 42. Furthermore, as a driving assistance instruction X, a warning instruction X2 is displayed in the upper right corner of the display area of the HUD 42. Note that Figure 6 is only an example of a display, and the size, shape, and display location of the map information M and warning instruction X2 are not limited.

As described above, the display system 4 in this example displays a driving assistance instruction X on the HUD 42 when the display control unit 43 determines that the specified other vehicle is not in autonomous driving mode. In other words, when a other vehicle to watch is specified and the other vehicle is not in autonomous driving mode and there is a possibility that the other vehicle may affect the vehicle, the driving assistance instruction X is displayed. This allows the occupants of vehicle 1 to focus their attention on the other vehicle. Furthermore, since the driving assistance instruction X is displayed when the display control unit 43 determines that the other vehicle is not in autonomous driving mode, there is no need for the driving assistance instruction X to be displayed on the HUD 42 all the time. This reduces the display load on the display system 4.

The display system 4 in this example includes a GPS 9 that acquires position information of the vehicle 1 and a wireless communication unit 10 that acquires position information of the oncoming vehicle 2A. The position information of the vehicle 1 and the position information of the oncoming vehicle 2A are displayed on the HUD 42. This allows the occupants of the vehicle 1 to easily grasp the positional relationship between the vehicle 1 and the oncoming vehicle 2A, making it easier to decide whether to indicate the identifier of the oncoming vehicle 2A. The position information of the oncoming vehicle 2A may also be acquired by the external camera 6A. However, generally, other vehicles that may affect the vehicle 1 are not necessarily present within the detection range of the external camera 6A. When the wireless communication unit 10 acquires the position information of the oncoming vehicle 2A through vehicle-to-vehicle communication as in this example, the position information of the oncoming vehicle 2A can be acquired regardless of the performance or detection range of the external camera 6B.

In this example, the display system 4 is equipped with a wireless communication unit 10 that communicates with a designated oncoming vehicle 2A, and the display control unit 43 determines whether the oncoming vehicle 2A is in autonomous driving mode through communication from the wireless communication unit 10. This makes it possible to accurately obtain the driving mode of the oncoming vehicle 2A through vehicle-to-vehicle communication with the oncoming vehicle 2A. Furthermore, the wireless communication unit 10 only communicates with the designated oncoming vehicle 2A, and does not need to communicate with the undesignated leading vehicle 2B and following vehicle 2C. This reduces the communication load on the display system 4.

Generally, when acquiring and displaying other vehicle information for all other vehicles, it is difficult for the occupants of vehicle 1 to distinguish between the information for the oncoming vehicle 2A, which should be the one they pay the most attention to, and the information for the other vehicles, such as the leading vehicle 2B and the following vehicle 2C. However, the display system 4 of this example uses the sensor 5 or internal camera 6B to specify one of the multiple other vehicles, and communicates only with that one other vehicle. This reduces the communication load on the display system 4 and allows the occupants of vehicle 1 to focus on the vehicle that should be the most important to watch.

In the display system 4 of this example, the position information of the oncoming vehicle 2A acquired by the wireless communication unit 10 is displayed with an identifier, making it easy for the occupant to identify the oncoming vehicle 2A. Furthermore, even if there are multiple other vehicles around the vehicle 1, the position information is displayed with multiple identifiers, making it easy to identify one vehicle from among the multiple other vehicles.

The sensor 5 or internal camera 6B is configured to accept instructions corresponding to the identifier, allowing the display system 4 to accurately recognize other vehicles corresponding to the identifier. Furthermore, even when there are multiple other vehicles around the vehicle, the display system 4 accurately recognizes other vehicles corresponding to the identifier, improving the recognition accuracy of the system.

If the driving mode of the designated other vehicle is manual driving mode, it is relatively difficult to predict the movements of that other vehicle, increasing the possibility that that other vehicle will affect vehicle 1. According to the display system 4 of this example, the driving assistance instruction X is displayed on the HUD 42 when the display control unit 43 determines that the other vehicle is in manual driving mode, so there is no need to always display the driving assistance instruction X. This reduces the display load of the display system 4.

In this example, in the display system 4, when the display control unit 43 determines that the designated oncoming vehicle 2A is in the second mode, the driving assistance instruction X is displayed on the HUD 42. In other words, when an oncoming vehicle 2A that should be watched is designated and the oncoming vehicle 2A has a low autonomous driving level and there is a possibility that the oncoming vehicle 2A will affect the vehicle 1, the driving assistance instruction X is displayed. This allows the occupants of the vehicle 1 to focus their attention on the oncoming vehicle 2A.

Generally, when the designated other vehicle is in fully autonomous driving mode, it is relatively easy to predict the movements of that other vehicle, and it is not necessarily necessary to display driving assistance instruction X. On the other hand, when the designated other vehicle is in a driving mode other than fully autonomous driving mode, it is relatively difficult to predict the movements of that other vehicle, and it is necessary to display driving assistance instruction X. According to the display system 4 of this example, the driving assistance instruction X is displayed on the HUD 42 when the display control unit 43 determines that the oncoming vehicle 2A is in a driving mode other than fully autonomous driving mode, so it is not necessary to always display driving assistance instruction X. Therefore, the display load of the display system 4 can be reduced.

According to the present disclosure, sensor 5 detects the voice of the occupant and internal camera 6B detects the line of sight of the occupant of vehicle 1, so that the occupant can specify an oncoming vehicle 2A via sensor 5 or internal camera 6B without interrupting the driving operation of vehicle 1.

The HUD 42 displays a stop sign X1 or a warning sign X2, thereby reducing the communication load and display load of the display system 4 and more accurately urging the occupants of the vehicle 1 to pay attention to the oncoming vehicle 2A.

In this example, the display control unit 43 determines the driving mode of the oncoming vehicle 2A and whether the area around the vehicle 1 is an ordinary road or an expressway, but the vehicle control unit 3 may also determine the driving mode of the oncoming vehicle 2A and whether the area around the vehicle 1 is an ordinary road or an expressway. The vehicle control unit 3 may control the GPS 9, wireless communication unit 10, sensor 5, and internal camera 6B, or the display control unit 43 may control these via the vehicle control unit 3. Furthermore, the GPS 9 acquires the position information of the vehicle 1, and the wireless communication unit 10 acquires the position information and driving mode of the oncoming vehicle 2A, but this is not limited to this. The wireless communication unit 10 may also acquire the position information and driving mode from infrastructure equipment as road-to-vehicle communication. In this case, the first communication unit, second communication unit, and third communication unit are the same device.

In this example, the display system 4 specifies the oncoming vehicle 2A, but the other vehicles that can be specified are not limited to the oncoming vehicle 2A. The display system 4 may also specify the leading vehicle 2B or the following vehicle 2C. Of the three other vehicles, two, the oncoming vehicle 2A and the leading vehicle 2B, may be specified simultaneously. In this case, vehicle-to-vehicle communication is performed with only the two specified oncoming vehicle 2A and the leading vehicle 2B, rather than with all three other vehicles, thereby reducing the communication load on the display system 4.

While Figure 4 shows a display example when vehicle 1 is traveling on a two-lane road, the present disclosure is not limited to this case. Figure 7 shows a display example of map information M, vehicle 1's location information, and the location information and identifiers of other vehicles 2B and 2C when vehicle 1 changes lanes. As shown in Figure 7, vehicle 1 is traveling in one lane, and multiple other vehicles around vehicle 1 are a leading vehicle 2B and a following vehicle 2C traveling in the other lane. When vehicle 1 is in the other lane and changes lanes between leading vehicle 2B and following vehicle 2C, the occupant of vehicle 1 may want to check the driving mode of leading vehicle 2B or following vehicle 2C. Even in the case of a lane change such as that shown in Figure 7, display system 4 of this example can receive an instruction from the occupant of vehicle 1 using identifier B of leading vehicle 2B or identifier C of following vehicle 2C, and determine the driving mode of the vehicle corresponding to the instruction by performing vehicle-to-vehicle communication with the vehicle corresponding to the instruction.

While Figure 7 shows a display example when vehicle 1 changes lanes, the present disclosure is not limited to this case. Figure 8 shows a display example of map information M, location information of vehicle 1, location information of other vehicle 2D, and identifier D when another vehicle 2D changes lanes toward the lane in which vehicle 1 is traveling. As shown in Figure 8, vehicle 1 is traveling in one lane, and another vehicle 2D, as one of the other vehicles surrounding vehicle 1, is traveling in the other lane. When other vehicle 2D changes lanes toward the lane in which vehicle 1 is traveling, the occupant of vehicle 1 may want to check the driving mode of other vehicle 2D. Even in the case of a lane change such as that shown in Figure 8, the display system 4 of this example can receive an instruction from the occupant of vehicle 1 using the identifier D of other vehicle 2D and determine the driving mode of other vehicle 2D by performing vehicle-to-vehicle communication with other vehicle 2D.

Figure 9 shows an example of the display of map information M, vehicle 1's position information, vehicle 2E's position information, and identifier E when vehicle 1 and vehicle 2E are traveling toward merging point J. As shown in Figure 9, vehicle 1 is traveling in one lane, and vehicle 2E, one of the surrounding vehicles, is traveling in the other lane. When vehicle 2E is traveling toward merging point J, the occupant of vehicle 1 may want to check the driving mode of vehicle 2E. Even in the case of merging point J as shown in Figure 9, display system 4 of this example can determine the driving mode of vehicle 2E by receiving an instruction from vehicle 1's occupant regarding vehicle 2E's identifier E and performing vehicle-to-vehicle communication with vehicle 2E. Note that display system 4 does not need to always display map information M, vehicle 1's position information, and vehicle 2E's position information and identifier E. For example, map information M may not be displayed normally, and may be displayed only when vehicle 1 passes a point 300 m before merging point J. The distance from the merging point J may be set to be relatively short when the vehicle 1 is traveling on an ordinary road, and may be set to be relatively long when the vehicle 1 is traveling on an expressway. The display system 4 does not normally display the map information M, etc., but may display the map information M, etc. based on a display instruction from the occupant of the vehicle 1.

Figure 10 shows an example display of map information M, vehicle 1's location information, and the location information and identifiers F and G of other vehicles 2F and 2G when vehicle 1 is traveling toward a first intersection IS1 and a second intersection IS2. As shown in Figure 10, there is a first intersection IS1 and a second intersection IS2 in the direction of vehicle 1's travel. One of the other vehicles surrounding vehicle 1 is vehicle 2F traveling toward the first intersection IS1. However, vehicle 2F is traveling away from the stop line L1 at the first intersection IS1. Meanwhile, one of the other vehicles surrounding vehicle 1 is vehicle 2G traveling toward the second intersection IS2. Vehicle 2G is traveling near the stop line L2 at the second intersection IS2. When vehicle 1 passes through the first intersection IS1 and the second intersection IS2, the occupants of vehicle 1 may want to check the driving mode of vehicle 2F or vehicle 2G. In particular, they may want to check the driving mode of vehicle 2G approaching the second intersection IS2 before checking the driving mode of vehicle 2F traveling away from the first intersection IS1. Even in the case of a first intersection IS1 and a second intersection IS2 as shown in Figure 10, the display system 4 of this example can determine the driving mode of the other vehicle 2G by receiving an instruction of the identifier G of the other vehicle 2G from the occupant of the vehicle 1 and performing vehicle-to-vehicle communication with the other vehicle 2G. Note that in Figure 10, the occupant of the vehicle 1 may simultaneously specify both the other vehicle 2F and the other vehicle 2G. Furthermore, the display system 4 may not normally display map information M, etc., but may display map information M, etc., when the vehicle 1 passes a point before the intersection IS1, for example, 300 m away. Furthermore, although the first intersection IS1 and the second intersection IS2 are shown as multiple intersections in this example, the number of intersections may be one or three or more.

The above describes an embodiment of the present disclosure, but it goes without saying that the technical scope of the present disclosure should not be interpreted as being limited by the description of this embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications of the embodiment are possible within the scope of the invention described in the claims. The technical scope of the present disclosure should be determined based on the scope of the invention described in the claims and their equivalents.

This application claims priority to Japanese Application No. 2020-190156, filed November 16, 2020, and incorporates by reference all of the contents of that Japanese application.

Claims (11)

A vehicle display system provided in a vehicle,
a display unit that displays position information of at least one other vehicle on map information around the vehicle;
a designation unit that designates the other vehicle based on an instruction from an occupant of the vehicle;
a determination unit that determines whether the other vehicle is in an autonomous driving mode,
When the determination unit determines that the other vehicle is not in the autonomous driving mode, the display unit displays a driving assistance instruction to assist driving of the vehicle;
When the determination unit determines that the other vehicle is in a fully autonomous driving mode or when the determination unit determines that the other vehicle is in an advanced driving assistance mode and that the vehicle is traveling within a specific area , the display unit does not display the driving assistance instructions. a first communication unit that acquires location information of the vehicle through communication with an external device;
a second communication unit that acquires position information of the other vehicle,
The vehicular display system according to claim 1 , wherein the display unit displays the acquired position information of the vehicle and the acquired position information of the other vehicle. a third communication unit that communicates with the designated other vehicle,
The vehicular display system according to claim 1 , wherein the determination unit determines whether the other vehicle is in the autonomous driving mode via communication with the third communication unit. the second communication unit acquires position information of each of a plurality of other vehicles around the vehicle,
The vehicular display system according to claim 2 , wherein the designation unit designates one of the plurality of other vehicles. The vehicle display system described in any one of claims 1 to 4, wherein the display unit displays the position information of the at least one other vehicle by assigning an identifier to the position information. The vehicle display system described in claim 5, wherein the designation unit is configured to accept instructions corresponding to the identifier displayed on the display unit. The vehicle display system of any one of claims 1 to 6, wherein the display unit displays the driving assistance instruction when the determination unit determines that the other vehicle is in a manual driving mode in which the driver of the other vehicle is driving the other vehicle without driving assistance from a vehicle system. The autonomous driving modes include a first autonomous driving mode and a second autonomous driving mode having an autonomous driving level lower than the autonomous driving level of the first autonomous driving mode,
8. The vehicle display system according to claim 1, wherein the display unit displays the driving assistance instruction when the determination unit determines that the other vehicle is in the second autonomous driving mode and that the vehicle is not traveling within the specific area . The vehicular display system of claim 8, wherein the first autonomous driving mode is a fully autonomous driving mode in which a vehicle system controls all vehicle driving, and the second autonomous driving mode includes an autonomous driving mode other than the fully autonomous driving mode. The vehicle display system described in any one of claims 1 to 9, wherein the designation unit is a sensor that detects the voice or line of sight of the occupant. The vehicle display system described in any one of claims 1 to 10, wherein the driving assistance instruction includes a stop instruction or a warning instruction.