JP6950775B2 - Vehicle display devices, vehicle display methods and programs - Google Patents

Description

The present invention relates to a vehicle display technology, and more particularly to a vehicle display device for displaying an image, a vehicle display method, and a program.

A viewpoint conversion process is performed to convert a photographed image taken by a camera installed in the vehicle into a bird's-eye view image. In such a viewpoint conversion process, it is assumed that the subject reflected in the captured image is projected on the road surface, and the projected subject is converted into an image viewed from a virtual viewpoint, so that a three-dimensional object is the subject. In some cases, the subject may be distorted and distorted. In order to suppress this distortion, the bird's-eye view image is modified by the contour of the three-dimensional object specified based on the result of transmitting and receiving the exploration wave (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-156632

Further, in the bird's-eye view image, that is, the viewpoint conversion for generating the bird's-eye view image, it is generally impossible to accurately represent the distance and the positional relationship between the objects existing around the vehicle and having a difference in the vertical direction. Therefore, a feeling of strangeness may occur, or the distance to the object and the positional relationship may not be accurately grasped. If the distance to the object and the positional relationship cannot be accurately grasped, there is a risk of contact with the object.

The present invention has been made in view of such a situation, and an object of the present invention is to be able to take a sufficient response while grasping the situation around the vehicle using a bird's-eye view image, and when an object such as an obstacle exists. , To provide a technique capable of reducing the risk of contact with an object.

In order to solve the above problems, the vehicle display device of an embodiment of the present invention has an image acquisition unit that acquires an image of the periphery of the vehicle, and an image acquired by the image acquisition unit from above the vehicle. by converting the viewpoint as seen, a generation unit that generates an overhead image, the overhead image generated by the generation unit, and place an image of a vehicle, an area that contains the image showing the vehicle first and region, a region that contains the object is an obstacle present around the vehicle as the first area and distinguishably colored second region, and an image processing unit for superimposing the first region in the image processing unit The image processing unit includes a display processing unit that displays a bird's-eye view image in which the image and the second region are superimposed on the display unit . If the contact with the object is an obstacle, if the object is an obstacle is not in contact with the ground so as to become the vehicle side of the object is an obstacle, superimposed on an overhead image.

Another aspect of the present invention is a vehicle display method. This method includes a step of acquiring an image of the surroundings of the vehicle, a step of generating a bird's-eye view image by converting the viewpoint of the acquired image as seen from above the vehicle, and a step of generating a bird's-eye view. the image, place the image of a vehicle, an area that contains the images of a vehicle as a first area, identifying a first region area containing the object is an obstacle present around the vehicle capable colored in a second region, comprising an image processing step of superimposing, the step of displaying the overhead image by superimposing a first region and a second region which is the image processing to the display unit, the image processing step, the the boundaries of the first region and the second region, if the object is an obstacle in contact with the ground so as to be in contact with the object is an obstacle, if the object is an obstacle is not in contact with the ground obstacle It is superimposed on the bird's-eye view image so that it is closer to the vehicle than a certain object.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, recording media, computer programs and the like are also effective as aspects of the present invention.

According to the present invention, it is possible to take a sufficient response while grasping the situation around the vehicle using a bird's-eye view image, and reduce the risk of contact with an object when an object such as an obstacle exists. Can be done.

It is a perspective view which shows the imaging range formed around the vehicle which concerns on embodiment of this invention. 2 (a)-(b) are diagrams showing an outline of processing in the comparison target of the examples of the present invention. It is a figure which shows the structure of the display device for a vehicle which concerns on embodiment of this invention. It is a figure which shows the bird's-eye view image generated in the generation part of FIG. It is a figure which shows the outline of the process in the distance acquisition part of FIG. It is a figure which shows the outline of another process in the distance acquisition part of FIG. It is a figure which shows the bird's-eye view image generated in the image processing part of FIG. It is a flowchart which shows the display procedure example by the display device for a vehicle of FIG.

Before the present invention is specifically described, the premise will be described first. An embodiment of the present invention relates to a vehicle display device that generates a bird's-eye view image by viewpoint conversion with respect to an image captured by a plurality of imaging units installed in a vehicle and displays the generated bird's-eye view image. In the viewpoint conversion, the captured image is mapped to a plane on the ground, so that each object included in the captured image is displayed as if they are lined up on the plane on the ground. Therefore, due to the viewpoint conversion, the distance or positional relationship between the icon indicating the own vehicle shown in the bird's-eye view image and the obstacle may differ from the actual distance or positional relationship.

For example, the distance to the obstacle in the bird's-eye view image is displayed so as to be farther than the actual distance to the obstacle. Specifically, even if the object is a three-dimensional object, it is displayed as if it is stuck on the ground, so that the distance to the three-dimensional object is the closest to the lower end of the three-dimensional object. When the three-dimensional object is in contact with the ground, the distance to the ground contact surface is the actual distance to the three-dimensional object, but when the three-dimensional object is not in contact with the ground, the distance to the three-dimensional object appears to be farther than the actual distance. Under such circumstances, when the driver decides that he / she will not come into contact with objects existing around the vehicle, such as obstacles, when he / she enters the garage while looking at the bird's-eye view image displayed on the monitor, he / she steers. Even so, it can come into contact with objects and make proper avoidance difficult.

In order to deal with this, the vehicle display device according to the present embodiment executes the following processing. The vehicle display device measures the distance to the object included in the captured image, and generates a bird's-eye view image capable of identifying the area including the object based on the measured distance. Further, the vehicle display device displays the generated bird's-eye view image. It provides the driver with an indication of the actual distance as the area containing the object is identifiable, and warns that there is an error due to viewpoint conversion.

Hereinafter, examples of the present invention will be described with reference to the drawings. The specific numerical values and the like shown in the examples are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and the drawings, elements having substantially the same function and configuration are designated by the same reference numerals to omit duplicate description, and elements not directly related to the present invention are not shown. ..

FIG. 1 is a perspective view showing an imaging range formed around the vehicle 100 according to the embodiment of the present invention. A front imaging unit 12 is installed in a front portion of the vehicle 100, for example, a bumper, a bonnet, or the like. The front imaging unit 12 forms a front imaging region 20 so as to face forward from the front imaging unit 12, and images an image in the front imaging region 20. The left side imaging unit 14 is installed on the left side portion of the vehicle, for example, the lower part of the left side door mirror. The left-side imaging unit 14 forms a left-side imaging region 22 so as to go to the left from the left-side imaging unit 14, and images an image in the left-side imaging region 22.

A rear imaging unit 16 is installed in a rear portion of the vehicle, for example, a bumper, a trunk, or the like. The rear imaging unit 16 forms a rear imaging region 24 so as to go backward from the rear imaging unit 16, and images an image in the rear imaging region 24. A right-side imaging unit 18 (not shown) is installed on the right side of the vehicle so as to be symmetrical with the left-side imaging unit 14. The right-side imaging unit 18 forms a right-side imaging region 26 so as to go to the right from the right-side imaging unit 18, and images an image in the right-side imaging region 26. The front image pickup unit 12, the left side image pickup unit 14, the rear image pickup unit 16, and the right side image pickup unit 18 are used as the image pickup unit 10, and the periphery of the vehicle 100 is imaged by the image captured by the image pickup unit 10.

2 (a)-(b) show the outline of the process in the comparative object of the Example of this invention. As a comparison target, as shown in FIG. 1, it is assumed that an image is captured by an imaging unit 10 installed in the vehicle 100. In particular, in the comparison target, the process for generating a bird's-eye view image by converting the viewpoint of the image corresponds to the process that has been performed so far. FIG. 2A shows a case where the vehicle 100 of FIG. 1 is viewed from the left side. As described above, the rear imaging unit 16 is installed in the rear portion of the vehicle 100. In addition, another vehicle 110 is parked or stopped behind the vehicle 100.

Further, a frame line 60 is drawn on the ground, and the frame line 60 exists below the front end portion of the other vehicle 110. Here, due to the tires of the other vehicle 110, there is a gap between the front end portion of the other vehicle 110 and the drawing. Therefore, when the vehicle 100 and the other vehicle 110 are looked down from above, the frame line 60 is shielded by the other vehicle 110. On the other hand, since the rear imaging unit 16 images the other vehicle 110 and the frame line 60 diagonally from a position not above the vehicle 100, the image captured by the rear imaging unit 16 shows the front end of the other vehicle 110. The part and the frame line 60 are included.

FIG. 2B shows a bird's-eye view image 78 generated by performing viewpoint conversion on the image captured by the imaging unit 10. As described above, in the process for generating the bird's-eye view image 78, the processes that have been performed so far are executed. In the central portion of the bird's-eye view image 78, the own vehicle icon 80, which is an image showing the vehicle 100, is arranged. The front image 70 is arranged in front of the own vehicle icon 80, the left side image 72 is arranged on the left side of the own vehicle icon 80, the rear image 74 is arranged behind the vehicle 100, and the right side of the vehicle 100 is arranged. The right side image 76 is arranged in. In particular, the rear image 74 is generated based on the image obtained by capturing the imaging range 62 in FIG. 2A. Therefore, in the rear image 74, the other vehicle 110 is arranged behind the vehicle 100, and the frame line 60 is arranged between the vehicle 100 and the other vehicle 110. That is, the frame line 60, which should be shielded by the other vehicle 110 when viewed from above, is no longer shielded by the other vehicle 110, and the bird's-eye view image 78 differs from the actual situation. Therefore, the driver may mistakenly think that the other vehicle 110 is behind the frame line 60 on the ground.

FIG. 3 shows the configuration of the vehicle display device 30 according to the embodiment of the present invention. The vehicle display device 30 is connected to the front image pickup unit 12, the left side image pickup unit 14, the rear image pickup unit 16, and the right side image pickup unit 18 constituting the image pickup unit 10, and is also connected to the display unit 50. The vehicle display device 30 includes an image acquisition unit 32, a generation unit 34, a distance acquisition unit 36, a selection unit 38, an image processing unit 40, and a display control unit 42.

The front image pickup unit 12, the left side image pickup unit 14, the rear image pickup unit 16, and the right side image pickup unit 18 capture an image as described above. The image is a moving image, but may be a still image. The front image pickup unit 12, the left side image pickup unit 14, the rear image pickup unit 16, and the right side image pickup unit 18 output an image to the image acquisition unit 32. The image acquisition unit 32 acquires images from each of the front image pickup unit 12, the left side image pickup unit 14, the rear image pickup unit 16, and the right side image pickup unit 18. That is, the image acquisition unit 32 acquires an image of the periphery of the vehicle 100. The image acquisition unit 32 outputs the acquired image to the generation unit 34 and the distance acquisition unit 36.

The generation unit 34 inputs an image from the image acquisition unit 32. The generation unit 34 generates a bird's-eye view image by changing the viewpoint of the image as seen from above the vehicle 100. A known technique may be used for the conversion. For example, each pixel of the image is projected on a three-dimensional curved surface in a virtual three-dimensional space, and a three-dimensional object is formed according to a virtual viewpoint from above the vehicle 100. Cut out the required area of the curved surface. The cut out area corresponds to the image from which the viewpoint has been changed.

FIG. 4 shows a bird's-eye view image 98 generated by the generation unit 34. The own vehicle icon 80 is arranged in the central portion of the bird's-eye view image 98 in FIG. The own vehicle icon 80 is an image of the vehicle 100 viewed from above, and has the size of the vehicle 100 in the bird's-eye view image 98. The front image 90 is arranged in front of the own vehicle icon 80, the left side image 92 is arranged on the left side of the own vehicle icon 80, and the rear image 94 is arranged behind the own vehicle icon 80. The right side image 96 is arranged on the right side of the 80. Further, in the bird's-eye view image 98, the first object 140 and the second object 142 are shown as examples of the objects existing around the vehicle 100. Here, when the first object 140 and the second object 142 are three-dimensional objects, it is unknown whether or not they are in contact with the ground. Return to FIG. The generation unit 34 outputs the bird's-eye view image 98 to the image processing unit 40.

The distance acquisition unit 36 acquires the distance from the object captured in the image acquired by the image acquisition unit 32. The distance acquisition unit 36 also acquires height information of the object captured in the image acquired by the image acquisition unit 32. For example, the following two methods are shown as methods for measuring the distance from the image to the object and the height. In the first, when the vehicle 100 moves, the images before and after the movement are compared and measured, and in the second, the images from the plurality of imaging units 10 are compared and measured. The distance acquisition unit 36 may use or use another method such as a millimeter wave radar or a laser radar, but the device can be simplified by using the image acquired by the image acquisition unit 32. First, the first will be described with reference to FIG.

FIG. 5 shows an outline of processing in the distance acquisition unit 36. As shown in the figure, the x-axis, y-axis, and z-axis, which are Cartesian coordinate systems, are defined. The x-axis indicates the horizontal direction on the ground, the y-axis indicates the vertical direction in the shape of the ground, and the z-axis indicates the direction perpendicular to the ground. The vehicle 100 and the rear imaging unit 16 are as described above. Further, the vehicle 100 is present at the position indicated by the vehicle 100 in the initial state. The vehicle 200 after retreating corresponds to the vehicle 100 that has moved to the rear, that is, to the negative side of the z-axis. The rear imaging unit 16 in the vehicle 200 after retreating is the rear imaging unit 216 after retreating. The obstacle 120 is a three-dimensional object existing behind the vehicle 100, and the figure 122 is a plan view on the ground, for example, the above-mentioned frame line 60. It is assumed that the figure 122 overlaps the obstacle 120 in the image captured by the rear imaging unit 16.

Here, assuming that the height from the ground to the rear imaging unit 16 is "a" with the point perpendicular to the ground from the position of the rear imaging unit 16 as the origin, the upper surfaces of the obstacle 120 and the figure 122 are as follows. Derived to. Since the obstacle 120 and the figure 122 appear to be the same from the rear imaging unit 16, they are shown as the first straight line 250 from the rear imaging unit 16. The first straight line 250 is shown as follows when the angle from the z-axis is “θ1” and the angle from the x-axis is “φ1”. (X / (sinθ1 × cosφ1)) = (y / (sinθ1 × sinφ1)) = ((z + a) / (cosθ1)) ... Equation (1)

Further, in the second straight line 252 from the rear image pickup unit 216 to the obstacle 120 after retreating, the amount of movement of the vehicle is "b", the angle from the z-axis is "θ2", and the angle from the x-axis is "φ2". If, it is shown as follows.
((X−b) / (sinθ2 × cosφ2)) = (y / (sinθ2 × sinφ2)) = ((z + a) / (cosθ2)) ... Equation (2)
Further, the third straight line 254 from the rear image pickup unit 216 to the figure 122 after retreating is shown as follows when the angle from the z-axis is "θ3" and the angle from the x-axis is "φ3".
((X−b) / (sinθ3 * cosφ3)) = (y / (sinθ3 * sinφ3)) = ((z + a) / (cosθ3)) ... Equation (3)

The distance acquisition unit 36 derives the position of the obstacle 120, which is the intersection of the equations (1) and (2), by solving the simultaneous equations of the equations (1) and (2). Further, the distance acquisition unit 36 derives the position of the figure 122, which is the intersection of the equations (1) and (3), by solving the simultaneous equations of the equations (1) and (3). Further, the distance acquisition unit 36 derives the distance from the rear end of the vehicle 100 to the obstacle 120 and the figure 122 by subtracting the movement amount “b” from the value of the x-axis of the solution of the simultaneous equations. The distance acquisition unit 36 derives the distance from the side surface of the vehicle to the obstacle 120 and the figure 122 by subtracting half the width of the vehicle from the value of the y-axis of the solution of the simultaneous equations. The distance acquisition unit 36 sets the value of the z-axis of the solution of the simultaneous equations as the height of the obstacle 120 and the figure 122.

Here, the case where the rear image pickup unit 16 is used has been described, but the position of the obstacle can also be measured by the front image pickup unit 12, the left side image pickup unit 14, and the right side image pickup unit 18 in the same manner. .. Since the measurement using the movement of the vehicle 100 is premised on the movement of the imaging unit 10, it is impossible to measure when the vehicle 100 is stationary. However, since measurement can be performed with only one imaging unit, it is possible to measure the entire circumference of the vehicle 100 by using each of the other imaging units.

Next, the second will be described with reference to FIG. FIG. 6 shows an outline of another process in the distance acquisition unit 36. In FIG. 6, a Cartesian coordinate system similar to that in FIG. 5 is defined. The vehicle 100, the rear image pickup unit 16, the right side image pickup unit 18, the obstacle 120, and the figure 122 are as described above. Here, assuming that the height from the ground to the rear imaging unit 16 is "a" with the point perpendicular to the ground from the position of the rear imaging unit 16 as the origin, the upper surfaces of the obstacle 120 and the figure 122 are as follows. Derived to.

Since the obstacle 120 and the figure 122 appear to be the same from the right-side imaging unit 18, they are shown as the first straight line 260 from the right-side imaging unit 18. In the first straight line 260, the distance from the y-axis of the right-side imaging unit 18 is "b", the distance from the x-axis is "c", the distance from the ground is "d", and the angle from the z-axis is set. When "θ1" is set and the angle from the x-axis is "φ1", it is shown as follows.
((X + c) / (sinθ1 * cosφ1)) = ((y + b) / (sinθ1 * sinφ1)) = ((z + d) / (cosθ1)) ... Equation (4)

On the other hand, since the obstacle 120 also looks different from the figure 122 from the rear imaging unit 16, the second straight line 262 from the rear imaging unit 16 to the obstacle 120 has an angle of “θ2” from the z-axis. When the angle from the x-axis is "φ2", it is shown as follows.
(X / (sinθ2 * cosφ2)) = (y / (sinθ2 * sinφ2)) = ((z + a) / (cosθ2)) ... Equation (5)
Further, the third straight line 264 from the rear imaging unit 16 to the figure 122 is shown as follows when the angle from the z-axis is “θ3” and the angle from the x-axis is “φ3”.
(X / (sinθ3 * cosφ3)) = (y / (sinθ3 * sinφ3)) = ((z + a) / (cosθ3)) ... Equation (6)

The distance acquisition unit 36 derives the position of the obstacle 120, which is the intersection of the equations (4) and (5), by solving the simultaneous equations of the equations (4) and (5). Further, the distance acquisition unit 36 derives the position of the figure 122, which is the intersection of the equations (4) and (6), by solving the simultaneous equations of the equations (4) and (6). Further, the distance acquisition unit 36 sets the x-axis value of the solution of the simultaneous equations as the distance from the vehicle 100 to the obstacle 120 and the figure 122, and sets the value of the z-axis of the solution of the simultaneous equations of the obstacle 120 and the figure 122. The height. Further, the distance acquisition unit 36 derives the distance from the side surface of the vehicle to the obstacle 120 and the figure 122 by subtracting half the width of the vehicle from the value of the y-axis of the solution of the simultaneous equations.

Here, the case where the rear image pickup unit 16 and the right side image pickup unit 18 are used has been described, but in the same manner, the positions of obstacles can be determined by the left side image pickup unit 14 or the right side image pickup unit 18 and the front image pickup unit 12 as well. It is possible to measure. In the second method, the measurement can be performed even when the vehicle 100 is stationary. Return to FIG. The distance acquisition unit 36 outputs information on the distance and height to the object derived by the first method or the second method to the selection unit 38. When a plurality of objects are included in the image, the distance acquisition unit 36 outputs information on the distances and heights to the selection unit 38.

The selection unit 38 inputs the distance and height information regarding the object one level above from the distance acquisition unit 36. The selection unit 38 selects at least one of the distances of the objects acquired by the distance acquisition unit 36. Specifically, the selection unit 38 selects (1) the distance to an object whose acquired height information is equal to or higher than a preset height, based on one or more input information. Here, the "preset height" is set to, for example, 5 cm. As the preset height, a height that is difficult for the vehicle 100 to get over and a height of an object that should be avoided as an obstacle are set. Further, as another setting example of the "preset height", for example, 20 cm is set. This height is set so that the vehicle body of the vehicle 100 comes into contact with the vehicle body. Further, the selection unit 38 selects (2) a distance less than a preset distance based on one or more input information. Here, the "preset distance" is set, for example, 50 cm. The preset distance is set, for example, the distance at which the presence of an object must be grasped during a parking operation or a slow-moving operation. The selection unit 38 may execute the selection of either (1) or (2), or may select a distance that satisfies both (1) and (2). When a plurality of objects exist in the same direction, the distance to the object closest to the vehicle 100 is selected. Such a choice corresponds to ignoring an object that has a low risk of collision with the vehicle 100, or ignoring an object that is sufficiently distant from the vehicle 100. The selection unit 38 outputs information on at least one selected distance to the image processing unit 40.

The image processing unit 40 inputs the bird's-eye view image 98 from the generation unit 34, and inputs information on at least one distance from the selection unit 38. Further, the image processing unit 40 also acquires information on the selected gear from the ECU (Electronic Control Unit) via the CAN (Control Area Network). When the reverse gear is not selected, the image processing unit 40 outputs the bird's-eye view image 98 from the generation unit 34 to the display control unit 42. On the other hand, when the reverse gear is selected, the image processing unit 40 executes the next process. Execution of the next process by selecting the reverse gear is an example, and may be a case where it is determined that the parking operation is performed based on the current position information and the traveling speed information, for example. It may also be started by the operation of the driver.

The image processing unit 40 has a first area 130 in which the own vehicle icon 80 of the vehicle 100 is displayed and a selection unit 38 with respect to the bird's-eye view image 98 generated by the generation unit 34 based on at least one distance. Generates a bird's-eye view image that can be distinguished from the second region 132 that includes objects separated by at least one distance selected in. Here, this process will be specifically described with reference to FIG. 7.

FIG. 7 shows a bird's-eye view image 98 generated by the image processing unit 40, and is shown in the same manner as in FIG. Here, the image processing unit 40 inputs information on the distance to the first object 140 and information on the distance to the second object 142 as an example. By identifying the first object 140 on the bird's-eye view image 98, the image processing unit 40 recognizes that the first object 140 exists behind the vehicle 100. Further, the image processing unit 40 generates a first boundary line 134a parallel to the rear portion of the own vehicle icon 80, which is the vehicle 100, and separated by a distance to the first object 140, and superimposes the first boundary line 134a on the bird's-eye view image 98. The first boundary line 134a is a boundary for distinguishing the first region 130 including the own vehicle icon 80 and the first and second regions 132a including the first object 140, and is, for example, a straight line. Shown. Here, the first region 130 and the first and second regions 132a both have a rectangular shape. Since the first object 140 is an obstacle that is not in contact with the ground, the first boundary line 134a is superimposed on the vehicle 100 side of the first object 140. Further, the first and second regions 132a are colored in order to easily distinguish them from the first region 130.

The image processing unit 40 executes the same processing on the second object 142, and the second object 142 is parallel to the right side of the own vehicle icon 80, which is the vehicle 100, and is separated by the distance to the second object 142. The boundary line 134b is generated and superimposed on the bird's-eye view image 98. The second boundary line 134b is a boundary for distinguishing the first region 130 including the own vehicle icon 80 and the second second region 132b including the second object 142. Since the second object 142 is an obstacle in contact with the ground, the second boundary line 134b is in contact with the image of the second object 142. Further, the second and second regions 132b are also colored in order to easily distinguish them from the first region 130.

Here, when a plurality of objects exist in the same direction with respect to the vehicle 100, the image processing unit 40 may execute the above-described processing on the closest object. Further, the first and second regions 132a and the second and second regions 132b may not be colored, and the first boundary line 134a and the second boundary line 134b may only be superimposed. Further, the first boundary line 134a and the second boundary line 134b may not be superimposed, and only the first-second region 132a and the second second region 132b may be colored. When coloring the second region 132, the second region 132 is colored so that the object projected on the bird's-eye view image 98, the condition of the road surface, and the like can be discriminated and can be distinguished from the first region 130. Specifically, the display is a light gray transparent display. As long as the first area and the second area can be identified on the display screen of the display unit 50, the display form in which they can be identified is arbitrary. Since such processing in the image processing unit 40 is performed every time the bird's-eye view image 98 is updated in the generation unit 34 or every short fixed period, even if the vehicle 100 moves, the second region 132 and the boundary line 134 are performed. Is followed by it. The image processing unit 40 outputs the bird's-eye view image 98 or the bird's-eye view image 98 on which the boundary line 134 is superimposed (hereinafter, also referred to as “bird's-eye view image 98”) to the display control unit 42.

The display control unit 42 inputs the bird's-eye view image 98 from the image processing unit 40. The display control unit 42 causes the display unit 50 to display the bird's-eye view image 98 by executing a process of displaying the bird's-eye view image 98. The display unit 50 is, for example, a display panel. The bird's-eye view image 98 as shown in FIG. 7 is displayed on the display unit 50.

This configuration can be realized by the CPU, memory, or other LSI of any computer in terms of hardware, and by programs loaded in memory in terms of software, but here it is realized by cooperation between them. It depicts a functional block to be done. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by hardware only, software only, or a combination thereof.

The operation of the vehicle display device 30 with the above configuration will be described. FIG. 8 is a flowchart showing an example of a display procedure by the vehicle display device 30. When the image processing unit 40 acquires the information that the reverse gear is selected (Y in S10), the display control unit 42 causes the display unit 50 to display the bird's-eye view image 98 generated by the generation unit 34 (S12). When the image processing unit 40 has not acquired the information that the reverse gear has been selected (N in S10), the processing is terminated. The distance acquisition unit 36 acquires the distance to at least one object (S14). A distance that is greater than or equal to the preset height (Y in S16) and less than or equal to the preset distance (Y in S18) is selected by the selection unit 38 (S20). The image processing unit 40 superimposes the boundary line 134 on the bird's-eye view image 98 based on the distance (S22). If the distance is not greater than or equal to the preset height (N in S16), or if the distance is not less than the preset distance (N in S18), steps 20 and 22 are skipped. The display control unit 42 causes the display unit 50 to display the bird's-eye view image 98 (S24). If the image processing unit 40 does not acquire the reverse gear release information (N in S26), the process returns to step 14. On the other hand, if the image processing unit 40 acquires the reverse gear release information (Y in S26), the processing is completed.

According to this embodiment, the boundary line for distinguishing the first region including the vehicle and the second region containing the object is superimposed on the bird's-eye view image based on the distance to the object. Therefore, it is possible to provide the driver with an estimate of the distance to the object. In addition, since the driver is provided with a guideline for the distance to the object, the driver can take a sufficient response while grasping the situation around the vehicle using the bird's-eye view image. In addition, since the driver is provided with a guideline for the distance to the object, the risk of contact with the object can be reduced when an object such as an obstacle is present. Further, since the guideline of the distance to the actual object is provided regardless of the object that can be seen only from the captured image, it is possible to suppress the occurrence of an accident due to a mistake in the display. Moreover, since the boundary line is a straight line, the processing can be simplified. Moreover, since the height of the object is limited, it is possible to eliminate an object that does not become an obstacle such as a bollard. Moreover, since the distance to the object is limited, the object away from the vehicle can be excluded.

The present invention has been described above based on the examples. This embodiment is an example, and it will be understood by those skilled in the art that various modifications are possible for each of these components and combinations of each processing process, and that such modifications are also within the scope of the present invention. ..

In this embodiment, the image processing unit 40 superimposes the straight line boundary line 134 on the bird's-eye view image 98. However, the present invention is not limited to this, and for example, when the boundary line 134 is superimposed on the curved surface included in the bird's-eye view image 98, the boundary line 134 may be a curve along the curved surface. In this case, the shape of the second region 132 is not a rectangular shape but a curved surface. According to this modification, the second region 132 and the boundary line 134 suitable for the bird's-eye view image 98 can be displayed.

10 Imaging unit, 12 Front imaging unit, 14 Left side imaging unit, 16 Rear imaging unit, 18 Right side imaging unit, 30 Vehicle display device, 32 Image acquisition unit, 34 Generation unit, 36 Distance acquisition unit, 38 Selection unit, 40 image processing unit, 42 display control unit, 50 display unit, 100 vehicles.

Claims (3)

An image acquisition unit that acquires an image of the surroundings of the vehicle,
A generation unit that generates a bird's-eye view image by changing the viewpoint of the image acquired by the image acquisition unit as seen from above the vehicle.
The overhead image generated by said generation unit, an image showing the vehicle is arranged, the region that contains the image that shows the vehicle as the first area, includes the object is an obstacle present around the vehicle is an area which in the second regions distinguishable colored with said first region, and an image processing unit to be superimposed,
A display processing unit that displays a bird's-eye view image in which the first region and the second region are superimposed on the image processing unit.
With
The image processing unit demarcates the boundary between the first region and the second region.
When the object which is an obstacle is in contact with the ground, it is brought into contact with the object which is an obstacle, and when the object which is an obstacle is not in contact with the ground, the vehicle is more than the object which is an obstacle. Overlay on the bird's-eye view image so that it is on the side,
A vehicle display device characterized by this. Steps to acquire an image of the surroundings of the vehicle,
A step of generating a bird's-eye view image by changing the viewpoint of the acquired image as seen from above the vehicle.
An image showing the vehicle is arranged in the generated bird's-eye view image , the area including the image showing the vehicle is set as the first area, and the area including an object which is an obstacle existing around the vehicle is included. the as the second regions distinguishable colored with said first region, and an image processing step of superimposing,
A step of displaying a bird's-eye view image in which the image-processed first region and the second region are superimposed on the display unit, and
With
In the image processing step, the boundary between the first region and the second region is set.
When the object which is an obstacle is in contact with the ground, it is brought into contact with the object which is an obstacle, and when the object which is an obstacle is not in contact with the ground, the vehicle is more than the object which is an obstacle. Overlay on the bird's-eye view image so that it is on the side,
A display method for vehicles characterized by the fact that. Steps to acquire an image of the surroundings of the vehicle,
A step of generating a bird's-eye view image by changing the viewpoint of the acquired image as seen from above the vehicle.
An image showing the vehicle is arranged in the generated bird's-eye view image, the area including the image showing the vehicle is set as the first area, and the area including an object which is an obstacle existing around the vehicle is included. the as the second regions distinguishable colored with said first region, and an image processing step of superimposing,
A step of displaying a bird's-eye view image in which the image-processed first region and the second region are superimposed on the display unit, and
With
In the image processing step, the boundary between the first region and the second region is set.
When the object which is an obstacle is in contact with the ground, it is brought into contact with the object which is an obstacle, and when the object which is an obstacle is not in contact with the ground, the vehicle is more than the object which is an obstacle. Overlay on the bird's-eye view image so that it is on the side,
A display program for vehicles that lets a computer do things.

Images