JP7779356B2 - Display control device, display control method and program - Google Patents

Description

The present invention relates to a display control device, a display control method, and a program.

Patent Document 1 discloses a perimeter monitoring device having a distance measurement detector that detects the distance from a vehicle to an object. The perimeter monitoring device of Patent Document 1 has a camera that captures an image of the object using an imaging unit provided in the vehicle. The perimeter monitoring device displays the color of the object and a color corresponding to the height of the object in a superimposed manner.

Patent Document 2 discloses a vehicle periphery visual recognition device that includes an obstacle detection means for detecting the distance between the vehicle and an obstacle and an imaging unit for capturing an image of the scenery outside the vehicle. When an obstacle is detected, the device of Patent Document 2 displays a graphic on the captured image that allows the user to recognize the direction of the obstacle. This device changes the display color or size of the graphic depending on the distance to the obstacle.

JP 2016-97843 A Japanese Patent Application Laid-Open No. 2006-311222

In such a device, it is desirable to appropriately warn the driver.
When multiple obstacles are present, it is preferable to draw attention to the more dangerous object. However, when multiple objects are present around the vehicle, changing the display color according to distance or height may result in a cluttered display. For example, if objects with a low level of danger are emphasized, it may not be possible to effectively draw attention to objects with a high level of danger.

In view of the above-mentioned problems, the present invention aims to provide a display control device, a display control method, and a program that can effectively alert a user by appropriately setting a risk level.

The display control device of this embodiment comprises an image data acquisition unit that acquires image data of an image taken of the outside of the vehicle, an object detection unit that detects objects included in the image, a ranging data acquisition unit that acquires ranging data regarding the distance from the vehicle to the object, a risk setting unit that sets a risk level for the object according to its size when the distance is below a threshold, an emphasis processing unit that performs emphasis processing on the image data according to the risk level, and an output unit that outputs an emphasis image after the emphasis processing, wherein the threshold distance is less than the total length of the vehicle when the depth of the parking space is approximately 10% longer than the total length of the vehicle.

The display control method of this embodiment includes the steps of acquiring image data of an image taken outside the vehicle, detecting an object contained in the image, acquiring ranging data regarding the distance from the vehicle to the object, setting a danger level for the object according to its size when the distance is equal to or less than a threshold, performing highlighting processing on the image data according to the danger level, and outputting a highlight image after the highlighting processing, wherein the threshold distance is less than the total length of the vehicle when the depth of the parking space is approximately 10% greater than the total length of the vehicle.

The program according to this embodiment includes the steps of: acquiring image data of an image taken of the outside of the vehicle; detecting an object included in the image; acquiring distance measurement data relating to the distance from the vehicle to the object; setting a threshold distance less than the overall length of the vehicle when the depth of the parking space is approximately 10% longer than the overall length of the vehicle; and setting a danger level for the object according to the size of the object when the distance is equal to or less than the threshold; and performing highlighting processing on the image data according to the danger level.
and outputting the highlighted image that has undergone the highlighting process.

According to the present invention, it is possible to provide a display control device, a display control method, and a program that can effectively alert a user by appropriately setting a risk level.

1 is a diagram illustrating a vehicle according to an embodiment. 1 is a diagram illustrating a display control device according to an embodiment; This is a color bar that indicates the display color according to the degree of danger. 3 is a flowchart illustrating a display control method according to the first embodiment. FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. FIG. 2 is a schematic diagram for explaining display control according to the first embodiment. 10 is a flowchart illustrating a display control method according to a second embodiment. FIG. 10 is a schematic diagram for explaining display control according to the second embodiment. FIG. 10 is a schematic diagram for explaining another display control method.

The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential means for solving the problems. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. Note that in each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

First Embodiment
1 is a diagram illustrating a vehicle 1 according to a first embodiment. The vehicle 1 includes a sensor unit 2 and a display control device 100. In the following description, in order to distinguish between the vehicle 1 and its surrounding vehicles, the vehicle 1 may be referred to as the subject vehicle and the surrounding vehicles as surrounding vehicles.

The sensor unit 2 has at least one imaging device and captures an image of the outside of the vehicle. The sensor unit 2 outputs image data captured by the imaging device to the display control device 100. The sensor unit 2 has a distance measurement sensor that detects obstacles. The sensor unit 2 outputs distance measurement data measured by the distance measurement sensor to the display control device 100.

The display control device 100 can be installed at any position in the vehicle 1.
The display control device 100 may be connected to a CAN (Controller Area Network). The display control device 100 performs image processing to detect objects (obstacles) in the images captured by the sensor unit 2. The display control device 100 performs highlighting to draw the user's attention. For example,
The display control device 100 highlights objects included in the image in a color such as red, which makes it easier for a user such as a driver to recognize the presence of the object and avoid contact with it. The display control device 100 will be described later.

Hereinafter, the term "video" also means "video data representing a video" as an object of information processing. Similarly, hereinafter, the term "image" also means "image data representing an image" as an object of information processing. Hereinafter, images and videos may be moving images or successive still images.

2 is a block diagram showing the configuration of a display control device 100 according to the first embodiment and a display control system 10 having the display control device 100. The display control system 10 has a sensor unit 2, a display unit 30, an ECU (Electronic Control Unit) 40, and the display control device 100. The display control device 100 has the sensor unit 2, the display unit 30, the ECU 40,
are communicatively connected to each of the above.

The display unit 30 is, for example, a display. The display unit 30 displays an image under the control of the display control device 100. The display unit 30 may include a speaker. In this case, the display unit 30 may output sound from the speaker. Furthermore, when the display control device 100 is mounted on the vehicle 1, the display unit 30 may be provided inside the vehicle 1 in a position where it can be seen by the driver of the vehicle while driving. In this case, the display unit 30 may be realized by the interface unit 108 described later. The display unit 30 may be a monitor of a car navigation system. Furthermore, the display unit 30 does not need to be an in-vehicle device permanently installed in the vehicle 1. The display unit 30 may be,
For example, the display control unit 100 may be configured by a display of an information terminal such as a smartphone or tablet terminal owned by the user. Furthermore, part of the display control process may be performed by a smartphone or the like having a display unit.

The sensor unit 2 includes a front camera 21F and a rear camera 21R. When the vehicle 1 is moving forward, the front camera 21F captures a front image of the vehicle 1. When the vehicle 1 is moving backward, the rear camera 21R captures a rear image of the vehicle 1. The front camera 21F and the rear camera 21R generate image data at, for example, 30 frames per second (30 fps) and supply the generated image data to the display control device 100 every 1/30th of a second. The image data may be, for example,
The image may be generated using a format such as H.264 or H.265. Alternatively, the sensor unit 2 may be a 360° camera that captures an image of the entire periphery of the vehicle 1.

The sensor unit 2 has a distance measurement sensor 22 for detecting the distance to an object in the vicinity of the vehicle 1. The distance measurement sensor 22 detects the distance from the vehicle to the object. The distance measurement sensor 22 is, for example, a laser imaging detection and ranging (LIDAR). Alternatively, the distance measurement sensor 22 may be an infrared camera, a stereo camera, a millimeter-wave radar, or the like. Furthermore, the distance measurement sensor 22 may be configured by combining these sensors. Furthermore, the distance measurement sensor 22 is not limited to a single physical device. For example, the distance measurement sensor 22 may be configured by LIDARs arranged at multiple locations on the vehicle 1.

The distance measurement sensor 22 transmits distance measurement data relating to the distance from the vehicle to the object to the display control device 100.
Furthermore, the distance measurement sensor 22 may detect the direction of an obstacle as well as the distance to the obstacle. In other words, the distance measurement data may be associated with the distance to the object and the direction of the object. Here, the direction of the object is a direction relative to the host vehicle. Furthermore, the forward/backward/left/right directions of the object may be indicated by azimuth angles, and the up/down directions may be indicated by elevation/depression angles.

The ECU 40 is included in the vehicle 1 and is a part of the configuration that controls the vehicle 1. The display control device 100 and the ECU 40 are connected to each other so as to be able to communicate with each other via an in-vehicle network such as a CAN.

The display control device 100 includes a control unit 102, a storage unit 104, a communication unit 106, and an interface unit 108 (IF: Interface). The display control device 100 also includes a driving information acquisition unit 110, an image data acquisition unit 121, an object detection unit 122, and a distance measurement data acquisition unit 125.
The driving information acquisition unit 110 includes a risk level setting unit 127, an emphasis display processing unit 128, and an output unit 129. The driving information acquisition unit 110 includes a traveling direction acquisition unit 111 and a speed information acquisition unit 113.

The control unit 102 is a processor such as a CPU (Central Processing Unit).
The control unit 102 functions as a calculation device that performs control processing, calculation processing, etc. The storage unit 104 is a storage device such as a memory or a hard disk.
For example, it is a ROM (Read Only Memory) or a RAM (Random Access Memory).
The storage unit 104 has a function for storing a control program, an arithmetic program, etc., which are executed by the control unit 102 or as a function of the control unit 102.
has a function for temporarily storing processing data and the like.

The communication unit 106 performs processing necessary for communicating with the sensor unit 2 and the display unit 30. The communication unit 106 may also perform processing necessary for communicating with a CAN (not shown). The communication unit 106 may include a communication port. The interface unit 108 is, for example, a user interface (UI). The interface unit 108 has an input device such as a keyboard, a touch panel, or a mouse, and an output device such as a speaker.

Therefore, the display control device 100 has a function as a computer. In addition, the display control device 100 includes a traveling information acquisition unit 110, an image data acquisition unit 121, an object detection unit 131, and a navigation system, and the display control device 100 functions as a computer by the processor of the control unit 102 executing a program stored in the storage unit 104.
22, distance measurement data acquisition unit 125, risk level setting unit 127, highlighting processing unit 128, output unit 1
29. Furthermore, each component of the display control device 100 is not limited to being realized by software programs, but may be realized by any combination of hardware and software. Furthermore, each component of the display control device 100 may be realized by, for example, an FPGA (field-programmable gate array), a microcomputer, or an SOC (System on Chip).
Alternatively, the present invention may be realized using a user-programmable integrated circuit, such as a programmable system on a chip. In this case, the integrated circuit may be used to realize a program configured from the above-described components. This also applies to the other embodiments described below.

The driving information acquisition unit 110 acquires driving information related to the driving of the vehicle 1. The driving information acquisition unit 110 acquires various information from the ECU 40. The driving information acquisition unit 110 includes a traveling direction acquisition unit 111 and a speed information acquisition unit 113.

The traveling direction acquisition unit 111 acquires information indicating the traveling direction of the vehicle 1. For example, the traveling direction acquisition unit 111 detects whether the vehicle 1 is moving forward or backward based on the shift position of the vehicle 1. When the shift position of the vehicle 1 is in the reverse (R) position, the traveling direction acquisition unit 111 detects that the vehicle 1 is moving backward. Alternatively, when the shift lever of the vehicle 1 is in a position such as drive (D), the traveling direction acquisition unit 111 detects that the vehicle 1 is moving forward.

Furthermore, the traveling direction acquisition unit 111 may acquire steering angle information as the traveling direction. The traveling direction acquisition unit 111 acquires a signal from a CAN or the like to acquire steering angle information indicating the steering angle of the wheels of the vehicle 1. Note that the steering angle information includes information indicating the steering direction, such as right or left, in addition to information indicating the steering angle.

The speed information acquisition unit 113 acquires speed information indicating the speed of the vehicle 1. For example, the speed information acquisition unit 113 detects the vehicle speed based on a vehicle speed pulse from the ECU 40. The speed information acquisition unit 113 acquires speed information indicating a vehicle speed of, for example, 40 kilometers per hour. Alternatively, the speed information may be information in stages such as low speed, medium speed, high speed, etc.

The image data acquisition unit 121 acquires image data captured by the front camera 21F and the rear camera 21R. The image data acquisition unit 121 may switch the image to be acquired depending on the traveling direction acquired by the traveling direction acquisition unit 111. For example, when the vehicle 1 is moving forward, the image data acquisition unit 121 acquires a front image from the front camera 21F. When the vehicle 1 is moving backward, the image data acquisition unit 121 acquires a rear image from the rear camera 21R. Of course, the image data acquisition unit 121 may acquire both a front image and a rear image. In this case, the display control device 100 simply switches the image to be processed depending on the traveling direction.

The object detection unit 122 detects an object included in an image. The object detection unit 122 recognizes an object in the image by performing image analysis on the image data. For example, the object detection unit 122
can perform object recognition by comparing images with pre-registered patterns.
Furthermore, when an image contains multiple objects, the object detection unit 122 detects each of the objects. Furthermore, the object detection unit 122 may identify the position coordinates and size of each object in the image.

Here, the objects include nearby vehicles, walls, wheel stops, overpasses, bicycles, etc. The objects may also include people such as pedestrians. Note that the object detection process in the object detection unit 122 can use known techniques, and therefore detailed description thereof will be omitted.

The distance measurement data acquisition unit 125 acquires distance measurement data relating to the distance to an object. Furthermore, if the image contains images of multiple objects, the distance measurement data acquisition unit 125 obtains the distance to each object. In other words, the distance from the vehicle to each object is calculated based on the distance measurement data from the distance measurement sensor 22.

The risk level setting unit 127 sets a risk level for each object. If an image contains images of multiple objects, the risk level setting unit 127 sets a risk level for each object. In other words, a value indicating a risk level is assigned to each object.

Specifically, the risk level setting unit 127 sets the risk level according to the possibility that the vehicle will come into contact with the object and the impact of the contact. The risk level setting unit 127 sets the risk level according to the distance to the object. The risk level setting unit 127 expresses the risk level value, for example, on a scale of 1 to 10, with 10 being the highest risk level and 0 being the lowest risk level. The risk level value increases as the object is closer and decreases as the object is farther away. Furthermore, the risk level setting unit 127 may set the risk level to 0 for an object that is extremely unlikely to come into contact with the object, or may set the risk level to 0 for an object that is significantly off the path of the vehicle 1, for example.

Alternatively, the risk level setting unit 127 sets the risk level according to the size of the object. The risk level setting unit 127 sets a higher risk level for larger objects and a lower risk level for smaller objects. For example, the risk level setting unit 127 can calculate the size of an object according to the size of the object image in the image and the distance to the object. The processing by the risk level setting unit 127 will be described later. Here, the size of an object can be one or both of the size in the horizontal direction (left-right direction) and the size in the vertical direction (up-down direction).

The highlighting processing unit 128 performs highlighting processing to highlight objects in the image according to the level of danger. For example, the highlighting processing unit 128 superimposes a highlighting color such as red on objects in the image that are highly dangerous. The highlighting processing unit 128 superimposes a non-highlighting color such as blue on objects in the image that are low in danger. The highlighting processing unit 128 does not perform highlighting processing on objects that are not dangerous, that is, objects that are unlikely to come into contact with the vehicle. The highlighting processing unit 128 superimposes a color that is more emphasized the higher the level of danger on objects in the image in order to display a warning to the user.

The risk level can be represented by a color bar with a color distribution as shown in FIG. 3. The higher the risk level, the redder the color, i.e., the longer the wavelength, and the lower the risk level, the bluer the color, i.e., the shorter the wavelength. The highlighting processing unit 128 superimposes a color corresponding to the risk level on an object in the image. For example, red is superimposed on an object with a high risk level in the image, and blue is superimposed on an object with a low risk level. Green is superimposed on an object with an intermediate risk level in the image. The highlighting processing unit 128 then generates a superimposed image in which a color corresponding to the risk level is superimposed on the object in the image. The output unit 129 outputs the image data so that the superimposed image is displayed on the display unit 30 as a highlighted image.

By changing the overlay color according to the degree of danger in this way, it is possible to appropriately call the user's attention. In other words, the higher the degree of danger, the more emphasized the display, so the user can easily recognize high-risk objects. By overlaying an emphasis color on an object in an image, a warning display can be performed. The user can drive while paying more attention to high-risk objects than to low-risk objects. It is possible to appropriately call the user's attention to objects.
This contributes to safer driving.

The highlighting is not limited to the color distribution shown in FIG. 3 . For example, the danger level may be indicated by shading. For example, the same color may be applied to the image of the object, and only the shading may be changed depending on the danger level. The highlighting processing unit 128 may perform the highlighting processing by changing the color tone, shading, brightness, transparency, etc. of the object image. The highlighting processing unit 128 may also perform the highlighting processing by applying an outline to the object. In this case, the thicker the outline is, the higher the danger level of the object. In this way, the highlighting processing unit 128 performs the highlighting processing such that the higher the danger level, the more emphasized the object is.

An example of the risk level setting process and highlighting process will be described with reference to Figs. 4 to 7. Fig. 4 is a flowchart showing the display control process according to this embodiment. Figs. 5 to 7 are diagrams explaining the risk level when reversing and the highlighting process. Figs. 5 to 7 show an example of the vehicle 1 reversing into a parking space 300. Figs. 5 to 7 show, from top to bottom, a superimposed image,
The color scheme, camera image, and top view are shown according to the risk level. As the vehicle 1 moves backward at a low speed, the state changes in the order of Fig. 5, Fig. 6, and Fig. 7.

A rear wall 301 is disposed at the rear of the parking space 300, and side walls 302 are disposed on the left and right. In other words, the parking space 300 is surrounded on three sides by the rear wall 301 and the side walls 302. The vehicle then backs up into the parking space 300, which is partitioned by the rear wall 301 and the side walls 302. Wheel stops 304 are disposed on the road surface of the parking space 300. Rear camera 2
1R captures an image of the parking space 300. The distance measurement sensor 22 captures the image of the rear wall 301 and the side wall 30
The distance to the object is detected as the distance to the object.

First, the image data acquisition unit 121 acquires image data of an image captured by the rear camera 21R (S401), and the object detection unit 122 detects an object included in the image (S402).
For example, the object detection unit 122 detects the rear wall 301, the side wall 302, and the wheel stopper 304 included in the image. The distance measurement data measured by the distance measurement sensor 22 is acquired (S40
3). As a result, the distances from the vehicle 1 to the rear wall 301, the side wall 302, and the wheel stopper 304 are calculated. The risk level setting unit 127 detects the size of the object based on the distance measurement data and image data (S404). As a result, the distance and size of each object are detected.

Next, the risk setting unit 127 determines whether the distance from the vehicle to the object is equal to or less than a threshold distance (S405). For example, the risk setting unit 127 compares the distance to the closest object with the threshold distance. When multiple objects are included in the image, the risk setting unit 127 determines whether the distance to the closest object is equal to or less than the threshold distance. Here, the distance to the rear wall 301 is compared with the threshold distance.

If the distance to the object is not equal to or less than the threshold distance (NO in S405), the risk level setting unit 127
The danger level setting unit 127 sets the danger level according to the distance (S406). That is, the closer the distance to the object, the higher the danger level of the object set by the danger level setting unit 127.

If the distance to the object is equal to or less than the threshold distance (YES in S405), the risk setting unit 127
The risk level setting unit 127 sets the risk level according to the size of the object (S407). That is, the larger the object, the higher the risk level set by the risk level setting unit 127.

Then, the highlighting processing unit 128 performs highlighting processing according to the degree of danger (S408).
That is, the highlighting processing unit 128 generates a superimposed image by superimposing red on objects in the image that are considered to be highly dangerous. The output unit 129 outputs the superimposed image that has been subjected to the highlighting processing to the display unit 30 (S409). The processing is repeated while the vehicle 1 is moving backward. As a result, the display unit 128 generates a superimposed image by superimposing red on the objects in the image that are considered to be highly dangerous.
The superimposed image shown in is displayed.

For example, in Figures 5 and 6, the distance to the rear wall 301 is greater than the threshold distance. Therefore, as in S406, the risk level setting unit 127 sets the risk level according to the distance to the object. As shown in Figure 5, when the distance to the rear wall 301 is great, the risk level is low. Then, as the vehicle 1 moves backward, it approaches the rear wall 301 and the side wall 302. Therefore, the risk level gradually increases. In the state shown in Figure 6,
5 to 6, the vehicle 1 is closer to the rear wall 301 and the side wall 302 than in the state shown in FIG. 5, and therefore the risk to the rear wall 301 and the side wall 302 is higher. Therefore, as the vehicle 1 moves backward from the position shown in FIG. 5 to the position shown in FIG. 6, the color of the superimposed image changes.

On the other hand, in FIG. 7, the distance to the rear wall 301 is equal to or less than the threshold distance.
7, the risk level setting unit 127 sets the risk level according to the size of the object. Since the rear wall 301 and the side wall 302 are larger than the predetermined size, the highest risk level is set. In the superimposed image of FIG. 7, the color red, which indicates the highest risk level, is superimposed on the rear wall 301 and the side wall 302.

In this way, when the distance to the rear wall 301 is equal to or less than the threshold distance, the risk level setting unit 127 sets the risk level according to the size of the object. This allows the user to easily recognize large objects that are highly dangerous. For example, if the risk level changes only according to the distance, the same risk level is set regardless of the size of the object. Therefore, it is difficult to draw attention to large objects that are far away from the vehicle 1. In contrast, according to this embodiment, when an object is closer than the threshold distance, the risk level setting unit 127 sets the risk level according to the size of the object. If the vehicle 1 comes into contact with a large object, the impact is large. In this embodiment, the risk level of large objects can be set high, so the vehicle 1
It can alert you to large objects that are far away.

In this way, when the vehicle 1 approaches an object within a threshold distance, the risk level setting unit 127 sets the risk level according to the size of the object. Therefore, the user can appropriately recognize large objects that are highly dangerous. The user (the driver) can decelerate and park appropriately. This can effectively assist driving. Furthermore, when the vehicle 1 is away from the object by more than the threshold distance, the risk level setting unit 127 sets the risk level according to the distance to the object. Therefore, the risk level changes according to the approach to the object, allowing the user to recognize the distance to the object. In this way, the risk level can be appropriately set according to the situation, thereby effectively alerting the user.

In step S407, if the size of an object closer than the threshold distance is smaller than a predetermined size, the risk level setting unit 127 may set the risk level according to the distance, as in step S405. That is, as the distance becomes shorter, the risk level setting unit 127 increases the risk level of small-sized objects. In other words, if the size of an object closer than the threshold distance is smaller than a predetermined size, the risk level may be set to a medium level. Furthermore, the size of an object may be divided into three or more levels, and the risk level may be set in multiple levels.

Furthermore, the risk level setting unit 127 may exclude objects smaller than a predetermined size from being compared with the threshold distance in S405. In other words, the risk level setting unit 127 compares the distance to the object closest to the vehicle 1, among objects equal to or larger than a certain size, with the threshold distance.
Specifically, the wheel stopper 304 is an object smaller than the predetermined size, so step S4
It is not subject to comparison with 04.

Furthermore, for objects that are unlikely or impossible to come into contact with the vehicle 1, the risk level may be set to 0. For example, the risk level setting unit 127 calculates height information indicating the height of the object as the size of the object. If the height of the object is less than the minimum ground clearance of the vehicle 1,
There is little possibility of contact with the vehicle 1. For example, the height of the wheel stopper 304 is lower than the minimum ground clearance of the vehicle 1, so the risk is low. Therefore, for the wheel stopper 304,
It may be excluded from the comparison process at 404 .

The threshold distance may be determined according to the size of the vehicle 1. For example, a 4-ton truck has a long overall length of approximately 8 m. Therefore, the threshold distance from the object to the vehicle is set to approximately 4 m. Furthermore, for example, a compact or standard passenger car has a short overall length of approximately 4.70 m or less. Therefore, the threshold distance from the object to the vehicle can be set to approximately 2 m. Alternatively, if the depth of the parking space 300 is approximately 10% longer than the overall length of the vehicle, the threshold distance may be set to less than the overall length of the vehicle. The threshold distance may also be determined according to the width of the vehicle 1.

<Second Embodiment>
In the second embodiment, the risk setting unit 127 performs risk setting processing in accordance with the travel information. More specifically, the setting processing performed by the risk setting unit 127 is switched depending on the traveling direction. For example, when the vehicle 1 is moving forward, the risk setting unit 127 performs a first setting processing.
When the vehicle 1 is reversing, the risk level setting unit 127 performs a second setting process that is different from the first setting process. Note that the configurations of the vehicle 1 and the display control system 10 are the same as those shown in Figures 1 and 2, and therefore a description thereof will be omitted.

8 is a flowchart showing the display control process according to the present embodiment. First, the driving information acquisition unit 110 acquires driving information from the ECU 40 (S801). That is, the traveling direction acquisition unit 111 acquires the traveling direction. Furthermore, the speed information acquisition unit 113 may acquire speed information.

The image data acquisition unit 121 acquires image data of the image captured by the rear camera 21R (S802). The object detection unit 122 detects an object included in the image (S803). The distance measurement data measured by the distance measurement sensor 22 is acquired (S804). The risk level setting unit 127 detects the size of the object based on the distance measurement data and image data (S805). Step S8
The processing of steps S802 to S805 is basically the same as the processing of steps S401 to S404 in embodiment 1, and therefore description thereof will be omitted. However, step S802 differs from step S401 in that image data acquisition unit 121 acquires image data of a front image captured by front camera 21F and a rear image captured by rear camera 21R.

Next, it is determined whether the vehicle 1 is moving forward (S806).
806, the risk level setting unit 127 sets the risk level by the first setting process (S
807) The first setting process will be described later.

If the vehicle 1 is not moving forward (NO in S806), the risk level setting unit 127 sets the risk level by a second setting process (S808). Note that the second setting process is different from the first setting process. If the vehicle 1 is moving backward, the risk level setting unit 127 sets the risk level by a second setting process that is different from the first setting process. For the second setting process, for example, the method shown in the first embodiment can be used. That is, in the second setting process, the risk level setting unit 1
27 sets the danger level according to the size and distance of the object.

Then, the highlighting processing unit 128 performs highlighting processing according to the degree of risk (S809).
That is, the highlighting processing unit 128 generates a superimposed image by superimposing red on objects in the image that are highly dangerous. The output unit 129 outputs the superimposed image that has been subjected to the highlighting processing to the display unit 30 (S810).

An example of the first setting process will be described below with reference to Fig. 9. Fig. 9 shows a camera image of a road 310 ahead of the vehicle and a superimposed image in which the camera image has been subjected to highlighting processing. A viaduct 311 is provided ahead of the vehicle. In other words, the vehicle 1 is traveling on the road 310 that passes under the viaduct 311.

Here, it is assumed that the vehicle height of the vehicle 1 is such that the vehicle cannot pass under the viaduct 311. For example, it is assumed that the viaduct 311 is 1.9 m above the ground and the vehicle height is higher than 1.9 m.

The object detection unit 122 detects an overpass 311 ahead in the traveling direction as an object. The risk setting unit 127 detects the height of the overpass 311 from the ground. In other words, it detects the clearance height from the road to the overpass 311. The risk setting unit 127 calculates the height of the vehicle 1 and the height of the overpass 311.
The risk level setting unit 127 then sets the risk level based on the comparison result.

If the vehicle 1 is at a height where it cannot pass under the viaduct 311, the risk level setting unit 1
27 sets a high danger level for the viaduct 311. The highlighting processing unit 128 performs highlighting according to the danger level. The highlighting processing unit 128 superimposes a red highlight color, which corresponds to a high danger level, on the viaduct 311 in the image as a warning display 312. Because there is a high possibility that the vehicle 1 will come into contact with the viaduct 311, the display unit 30 can display the warning display 312 to the user. In the drawing, the warning display 312 is superimposed only on the viaduct 311 above in the traveling direction, but it may also be superimposed on the entire surface of the viaduct 311.

Generally, the vehicle speed when moving forward is faster than the vehicle speed when reversing. For example, when reversing, the vehicle 1 moves at a relatively slow speed, for example, when parking in a garage. When reversing to park, the vehicle speed is slow, so the user is likely to notice the height restriction. In contrast, when moving forward, the vehicle speed is fast, so the user may overlook the height restriction. Therefore, when moving forward, the risk level setting unit 127 detects the height of an object and compares the vehicle height with the height of the object. By switching the risk level setting process depending on the direction of travel in this way, the risk level can be set appropriately. This makes it possible to more appropriately alert the user, who is the driver.

In the above description, an example of passing through the road 310 under the viaduct 311 has been described, but the processing of this embodiment can also be used for other than the viaduct 311. For example,
Parking lots, such as indoor parking lots and mechanical parking lots, may have height restrictions. When entering such parking lots with height restrictions, the processing of this embodiment can be applied. The risk setting unit 127 sets a risk level for the object depending on the clearance height limit of the viaduct 311 and the height inside the building. The risk setting unit 127 compares the height of the object with the height of the vehicle 1 to determine whether the vehicle 1 can pass under the object. The risk setting unit 127 sets a risk level depending on the determination result.

A margin may be provided for the comparison result between the vehicle height of the vehicle 1 and the clearance height of the object. For example, in the above description, a warning is displayed when the vehicle height of the vehicle 1 is higher than the clearance height limit of the viaduct 311. However, a warning may be displayed if the vehicle height of the vehicle 1 is approximately the same as the clearance height limit of the viaduct 311. The risk level setting unit 127 may increase the risk level when the difference between the vehicle height of the vehicle 1 and the ground clearance of the object is equal to or less than a predetermined value. Specifically, it is possible to provide a height margin taking into account road surface steps and the like.

In the first setting process, the risk level may be set according to the distance to the object. For example, weighting may be performed so that the closer the distance, the higher the risk level. Furthermore, if the distance to the object is farther than a threshold distance, the risk level may be set according to the distance as in the first embodiment. Then, the height may be compared when the distance to the object becomes equal to or less than the threshold distance. In this way, the user can be more appropriately warned.
For example, when the distance is long, the accuracy of height detection is low.

The risk level setting unit 127 may also set the risk level based on speed information. For example,
The risk level setting unit 127 may adjust the timing of displaying a warning according to the speed information and the distance to the object. When the vehicle 1 is traveling at high speed, the time from object detection to arrival at the viaduct 311 is short. Therefore, it is preferable to display a warning to the user early. The faster the speed, the earlier the timing of displaying the highlight color. It is possible to display a warning at an appropriate timing.

The risk setting unit 127 may measure at least one of the height of the object from the road surface and the position of the object in the left-right direction. The risk setting unit 127 determines whether there is a possibility that the object will collide with the vehicle 1 based on the measurement result. The risk setting unit 127 sets a risk level for the object according to the determination result. This process will be described with reference to FIG. 10. Here,
An example of parking forward or backward into the parking space 320 will be described.

Parking space 320 is provided with a wheel stopper 324.
A nearby vehicle 321 is parked to the side of parking space 20.
A nearby vehicle 322 is parked. The distances to the nearby vehicles 321 and 322 to vehicle 1 are closer than the threshold distance. Their size in the left-right direction (horizontal direction) is approximately the same as that of vehicle 1. Furthermore, a pedestrian 325 is located at the far end of the parking space 320. The distance to the pedestrian 325 is farther than the threshold distance.

The risk level setting unit 127 measures the left and right positions of the surrounding vehicles 321 and 322 based on the distance measurement data.
The risk level setting unit 127 determines whether there is a possibility that the nearby vehicles 321 and 322 will collide with the nearby vehicles 321 and 322. The risk level setting unit 127 sets the risk level of the nearby vehicles 321 and 322 according to the determination result.

For example, the nearby vehicle 321 is located right next to the parking space 320, so the risk level setting unit 1
27 determines that there is a possibility of a collision. Because nearby vehicle 322 is located ahead of parking space 320 in the direction of travel, risk level setting unit 127 determines that there is a possibility of a collision. Therefore, risk level setting unit 127 sets a high risk level for nearby vehicles 321 and 322. Note that "ahead in the direction of travel" means behind vehicle 1 when vehicle 1 is reversing, and means ahead of vehicle 1 when vehicle 1 is moving forward.

In the superimposed image, warning signs 331 and 332 are superimposed on the surrounding vehicles 321 and 322, respectively.
21, 322 are superimposed.

In the left-right direction (horizontal direction), the pedestrian 325 is located far away from the front of the vehicle 1 in the traveling direction. The risk level setting unit 127 measures the left-right position of the pedestrian 325 based on the distance measurement data. The risk level setting unit 127 determines whether there is a possibility that the vehicle 1 will collide with the pedestrian 325 based on the measurement result of the left-right position. The risk level setting unit 127 sets a risk level for the pedestrian 325 in accordance with the determination result. The risk level setting unit 127 determines that the possibility of colliding with the pedestrian 325 is extremely low. A low risk level is set for the pedestrian 325.
The risk level setting unit 127 sets the risk level of the pedestrian 325 to 0, for example. Therefore, no warning is displayed for the pedestrian 325 in the superimposed image.

In this way, the risk level setting unit 12 determines the risk level according to the left-right position and size (width) of the object.
The risk level setting unit 127 may set the risk level. Furthermore, it may determine whether there is a possibility of a collision with the vehicle 1 based on the steering angle, the lateral width, and the left-right (lateral) position of the object. If the possibility of contact with the vehicle 1 is low, the risk level setting unit 127 may set the risk level to low. Furthermore, if there is no possibility of contact, the risk level setting unit 127 may set the risk level to 0. Note that when determining whether there will be a collision with the vehicle 1 based on the left-right position, steering angle information of the vehicle 1 may be taken into consideration. That is, the risk level setting unit 127 may determine whether there is an object ahead in the traveling direction based on the steering angle, for example, ahead to the left or right.

The risk level setting unit 127 measures the height of the wheel stopper 324 based on the distance measurement data. The risk level setting unit 127 determines whether or not the object will collide with the wheel stopper 324 based on the height measurement result. The risk level setting unit 127 sets a risk level for the object in accordance with the determination result.
24 is lower than the minimum ground clearance of the vehicle 1. Therefore, the risk level setting unit 127 determines that there is no possibility of a collision with the wheel stopper 324, and sets the risk level to 0. Therefore, no warning is displayed on the wheel stopper 324 in the superimposed image.

In this way, the danger level setting unit 127 can set the danger level more appropriately. Therefore, it becomes possible to effectively alert the user. Furthermore, the display control device 100 may output a voice message or a warning sound for alerting the user from a speaker based on the danger level. In other words, the output unit 129 may generate voice data according to the danger level.

Furthermore, the program for executing the above-described display control method can be stored in various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read-Only Memory), and the like.
Only Memory), CD-R, CD-R/W, semiconductor memory (e.g., mask ROM, PR
ROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM
(random access memory). The program may also be provided to the computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable medium can provide the program to the computer via a wired communication path such as an electrical wire or optical fiber, or via a wireless communication path.

REFERENCE SIGNS LIST 1 vehicle 2 sensor unit 21F front camera 21R rear camera 22 distance measurement sensor 30 display unit 40 ECU
100 Display control device 110 Traveling information acquisition unit 111 Traveling direction acquisition unit 113 Speed information acquisition unit 121 Image data acquisition unit 122 Object detection unit 125 Distance measurement data acquisition unit 127 Risk level setting unit 128 Highlighting processing unit 129 Output unit 301 Rear wall 302 Side wall 304 Wheel stopper 310 Road 311 Viaduct 312 Warning display 320 Parking space 321, 322 Nearby vehicles 324 Wheel stopper 325 Pedestrians 331, 332 Warning display

Claims (5)

an image data acquisition unit that acquires image data of an image captured outside the vehicle;
an object detection unit that detects an object included in the image;
a distance measurement data acquisition unit that acquires distance measurement data relating to the distance from the host vehicle to the object;
a danger level setting unit that sets a danger level for the object in accordance with a size of the object when the distance is equal to or less than a threshold;
a highlighting processing unit that performs highlighting processing on image data according to the risk level;
an output unit that outputs a highlighted image that has been subjected to the highlighting process;
Equipped with
The threshold distance is set to a value when the depth of the parking space is approximately 10% longer than the overall length of the vehicle when the vehicle is parked in the parking space.
The length is less than the total length of the vehicle.
Display control device. The display control device according to claim 1 , wherein the risk level setting unit sets a risk level for the object in accordance with the distance to the object when the distance is greater than a threshold value. The risk level setting unit
measuring at least one of the height of the object from the road surface and the position of the object in the left-right direction;
determining whether the object will collide with the host vehicle based on the measurement results;
The danger level of the object is set according to the judgment result.
The display control device according to claim 1 or 2. acquiring image data of an image captured outside the vehicle;
detecting an object contained in the image;
acquiring distance measurement data relating to a distance from the host vehicle to the object;
When the distance is equal to or less than a threshold, setting a risk level for the object according to a size of the object;
performing a highlighting process on the image data according to the risk level;
outputting a highlighted image that has undergone the highlighting process;
Equipped with
The threshold distance is set to a value when the depth of the parking space is approximately 10% longer than the overall length of the vehicle when the vehicle is parked in the parking space.
The length is less than the total length of the vehicle.
Display control method. acquiring image data of an image captured outside the vehicle;
detecting an object contained in the image;
acquiring distance measurement data relating to a distance from the host vehicle to the object;
a step of setting a threshold distance to be less than the overall length of the vehicle when the depth of the parking space is approximately 10% longer than the overall length of the vehicle when the vehicle is parked in the parking space, and setting a risk level for the object according to the size of the object when the distance is equal to or less than the threshold distance;
a step of performing a highlighting process on the image data according to the risk level;
outputting a highlighted image that has undergone the highlighting process;
A program that causes a computer to execute the following.