JP6644264B2 - Overhead video generation device, overhead video generation system, overhead video generation method and program - Google Patents

Description

  The present invention relates to an overhead image generation apparatus, an overhead image generation system, an overhead image generation method, and a program.

  A display panel of an in-vehicle device such as a navigation device that displays a bird's-eye view image of a vehicle is often horizontally long. For this reason, the vertically long overhead image is not displayed on the entire surface of the display panel. Therefore, a technology is known in which, when an obstacle close to the vehicle is detected, a bird's-eye view image of the vehicle and a video image of the direction of the obstacle are displayed side by side (for example, see Patent Document 1).

JP 2011-251681 A

  In this technique, when an obstacle is detected, the driver moves his / her line of sight to check the bird's-eye view image and the image of the obstacle detection direction.

  The present invention has been made in view of the above, and an object of the present invention is to suppress the movement of the driver's line of sight and to appropriately check obstacles around the vehicle.

  In order to solve the above-described problems and achieve the object, an overhead image generation device according to the present invention includes an image acquisition unit that acquires peripheral images from a plurality of cameras that photograph the front, rear, left, and right of a vehicle, and a periphery of the vehicle. Obstacle information acquisition unit that acquires the obstacle information of the obstacle detected in the above, and performs a viewpoint conversion process on the peripheral image acquired by the image acquisition unit to generate an overhead view image as viewed from above the vehicle, and generates In the center of the bird's-eye view video, a video generation unit that generates a bird's-eye video that combines peripheral images from the camera corresponding to the direction in which the obstacle information acquisition unit detected the obstacle, and the video generation unit generated And a display control unit for displaying the bird's-eye view video on the display unit.

  The overhead view image generation system according to the present invention includes the above overhead view image generation device, the camera, and an obstacle detection unit that detects an obstacle around the vehicle and supplies obstacle information to the obstacle information acquisition unit. , At least one of the display units.

  The overhead view image generation method according to the present invention includes an image acquisition step of acquiring peripheral images from a plurality of cameras that photograph the front, rear, left, and right of the vehicle, and an obstacle that acquires obstacle information of an obstacle detected around the vehicle. Information obtaining step, generating a bird's-eye view image as viewed from above the vehicle by performing a viewpoint conversion process on the peripheral image obtained in the image obtaining step, and, at the center of the generated bird's-eye view image, the obstacle information obtaining step A video generating step of generating a bird's-eye video synthesized from the peripheral video from the camera corresponding to the direction in which the obstacle was detected in the step, and a display control step of displaying the bird's-eye video generated in the video generating step on a display unit. .

  A program according to the present invention includes an image acquisition step of acquiring peripheral images from a plurality of cameras that photograph the front, rear, left and right of a vehicle, and an obstacle information acquisition step of acquiring obstacle information of an obstacle detected around the vehicle. And performing a viewpoint conversion process on the peripheral image acquired in the image acquiring step to generate a bird's-eye view image as viewed from above the vehicle, and, at the center of the generated bird's-eye view image, an obstacle in the obstacle information acquiring step. A video generation step of generating a bird's-eye video synthesized from the peripheral video from the camera corresponding to the direction in which the video was detected, and a display control step of displaying a bird's-eye video generated in the video generation step on a display unit as a bird's-eye video generation device Run on a working computer.

  ADVANTAGE OF THE INVENTION According to this invention, there exists an effect that movement of a driver | operator's line of sight can be suppressed and an obstacle around a vehicle can be appropriately confirmed.

FIG. 1 is a block diagram illustrating a configuration example of the overhead video generation system according to the first embodiment. FIG. 2 is a diagram illustrating a display panel that displays an overhead image generated by the overhead image generation system according to the first embodiment. FIG. 3 is a diagram illustrating a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the first embodiment. FIG. 4 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the first embodiment. FIG. 5 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 4. FIG. 6 is a flowchart illustrating a flow of processing in the overhead image generation device of the overhead image generation system according to the first embodiment. FIG. 7 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the second embodiment. FIG. 8 is a diagram illustrating an example of a horizontally-long overhead view video generated from the overhead view video of FIG. 7. FIG. 9 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the third embodiment. FIG. 10 is a diagram illustrating an example of a horizontally-long overhead-view video generated from the overhead-view video of FIG. 9. FIG. 11 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the fourth embodiment. FIG. 12 is a diagram illustrating an example of a horizontally-long overhead view video generated from the overhead view video of FIG. 11. FIG. 13 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the fifth embodiment. FIG. 14 is a diagram illustrating an example of a horizontally-long overhead-view video generated from the overhead-view video of FIG. 13. FIG. 15 is a diagram illustrating another example of a method of generating a horizontally long overhead image in the overhead image generation system. FIG. 16 is a diagram illustrating an example of a horizontally-long overhead-view video generated from the overhead-view video of FIG. 15.

  Hereinafter, an embodiment of an overhead video generation device 40, an overhead video generation system 1, an overhead video generation method, and a program according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited by the following embodiments.

[First embodiment]
FIG. 1 is a block diagram illustrating a configuration example of the overhead video generation system according to the first embodiment. The overhead view image generation system 1 is mounted on a vehicle. The overhead view image generation system 1 may be a device that is portable and can be used in a vehicle, in addition to the device mounted on the vehicle.

  The overhead view video generation system 1 will be described with reference to FIG. The overhead view image generation system 1 includes a front camera 11, a rear camera 12, a left camera 13, a right camera 14, a front sensor (obstacle detection unit) 21, and a rear sensor (obstacle detection unit) 22. , A left side sensor (obstacle detection unit) 23, a right side sensor (obstacle detection unit) 24, a display panel 31, and a bird's-eye view image generation device 40.

  The front camera 11 is arranged in front of the vehicle, and photographs an area around the front of the vehicle. The front camera 11 outputs the captured image to the image acquisition unit 42 of the overhead image generation device 40.

  The rear camera 12 is arranged behind the vehicle, and captures an image around the rear of the vehicle. The rear camera 12 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.

  The left camera 13 is disposed on the left side of the vehicle, and captures an image around the left side of the vehicle. The left camera 13 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.

  The right side camera 14 is arranged on the right side of the vehicle, and photographs the periphery around the right side of the vehicle. The right camera 14 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.

  The front camera 11, the rear camera 12, the left camera 13 and the right camera 14 capture images in all directions of the vehicle.

  The front sensor 21 is disposed in front of the vehicle and detects an obstacle in front of the vehicle. The front sensor 21 is, for example, an infrared sensor or an ultrasonic sensor. The front sensor 21 detects an object having a height from the ground, which may come into contact with the vehicle. The front sensor 21 detects, for example, an obstacle at a distance of about 5 m from the vehicle. The front sensor 21 detects, for example, an obstacle in a range of about 30 ° around the center of the sensor when viewed in the vertical direction. The detection range of the front sensor 21 may partially overlap the detection range of the left sensor 23 and the right sensor 24. The front sensor 21 may be configured by a combination of a plurality of sensors. The front sensor 21 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.

  The rear sensor 22 is disposed behind the vehicle and detects an obstacle behind the vehicle. The rear sensor 22 is, for example, an infrared sensor or an ultrasonic sensor. The rear sensor 22 detects an object having a height from the ground, which may come into contact with the vehicle. The rear sensor 22 detects, for example, an obstacle at a distance of about 5 m from the vehicle. The rear sensor 22 detects, for example, an obstacle in a range of about 30 ° around the center of the sensor when viewed in the vertical direction. The detection range of the rear sensor 22 may overlap a part of the detection range of the left sensor 23 and the right sensor 24. The rear sensor 22 may be configured by a combination of a plurality of sensors. The rear sensor 22 outputs the obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view image generation device 40.

  The left sensor 23 is disposed on the left side of the vehicle, and detects an obstacle on the left side of the vehicle. The left sensor 23 is, for example, an infrared sensor or an ultrasonic sensor. The left sensor 23 detects an object having a height from the ground, which may come into contact with the vehicle. The left sensor 23 detects, for example, an obstacle at a distance of about 5 m from the vehicle. The left sensor 23 detects, for example, an obstacle in a range of about 30 ° around the center of the sensor when viewed in the vertical direction. The detection range of the left sensor 23 may partially overlap the detection range of the front sensor 21 and the rear sensor 22. The left sensor 23 may be configured by a combination of a plurality of sensors. The left sensor 23 outputs the obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view image generation device 40.

  The right side sensor 24 is disposed on the right side of the vehicle, and detects an obstacle on the right side of the vehicle. The right sensor 24 is, for example, an infrared sensor or an ultrasonic sensor. The right sensor 24 detects an object having a height from the ground, which may come into contact with the vehicle. The right sensor 24 detects, for example, an obstacle at a distance of about 5 m from the vehicle. The right sensor 24 detects, for example, an obstacle in a range of about 30 ° around the center of the sensor when viewed in the vertical direction. The detection range of the right sensor 24 may partially overlap the detection range of the front sensor 21 and the rear sensor 22. The right sensor 24 may be configured by a combination of a plurality of sensors. The right sensor 24 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.

  The display panel 31 is a display including a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display, for example. The display panel 31 displays the overhead video 100 and the overhead video 200 based on the video signal output from the overhead video generation device 40 of the overhead video generation system 1. The display panel 31 may be dedicated to the overhead view image generation system 1 or may be used jointly with another system including a navigation system, for example. The display panel 31 is arranged at a position that is easily visible from the driver.

  The display panel 31 has a horizontally long rectangular shape. On the display panel 31, a vertically long overhead image 100 as shown in FIG. 2 or a horizontally long overhead image 200 as shown in FIG. 3 is displayed. The horizontally-long overhead-view video 200 is a horizontally-long rectangular overhead-view video that matches the aspect ratio of the display surface of the display panel 31.

  The overhead view video generation device 40 includes a control unit 41 and a storage unit 49.

  The control unit 41 is, for example, an arithmetic processing device including a CPU (Central Processing Unit). The control unit 41 loads a program stored in the storage unit 49 into a memory and executes an instruction included in the program. The control unit 41 includes an image acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, an image generation unit 45, and a display control unit 48.

  The image acquisition unit 42 acquires a peripheral image of the periphery of the vehicle. More specifically, the image acquisition unit 42 acquires images output by the front camera 11, the rear camera 12, the left camera 13, and the right camera 14. The video acquisition unit 42 outputs the acquired video to the video generation unit 45.

  The obstacle information acquisition unit 43 acquires obstacle information of obstacles detected around the vehicle. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the front sensor 21, the rear sensor 22, the left sensor 23, and the right sensor 24. In the present embodiment, the obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle. The obstacle information acquisition unit 43 outputs the acquired obstacle information to the video generation unit 45.

  The vehicle information acquisition unit 44 acquires vehicle information serving as a trigger for displaying a bird's-eye view image, such as vehicle gear operation information, from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the video generation unit 45.

  The video generator 45 generates the overhead video 100 and the overhead video 200. The video generation unit 45 includes an overhead video generation unit 46 and a video synthesis unit 47.

  The bird's-eye view image generation unit 46 generates a bird's-eye view image 100 that has been subjected to a viewpoint conversion process from the peripheral image acquired by the image acquisition unit 42 so as to look down on the vehicle from above. More specifically, the overhead image generation unit 46 generates the overhead image 100 based on the peripheral images captured by the front camera 11, the rear camera 12, the left camera 13, and the right camera 14. The method of generating the overhead view video 100 may be any known method, and is not limited. The overhead image generation unit 46 outputs the generated overhead image 100 to the display control unit 48.

  The overhead view video 100 will be described with reference to FIG. FIG. 2 is a diagram illustrating a display panel that displays an overhead image generated by the overhead image generation system according to the first embodiment. The bird's-eye view video 100 has a vertically long rectangular shape. The overhead image 100 includes a front image 101, a rear image 102, a left image 103, and a right image 104. A range surrounded by the front image 101, the rear image 102, the left image 103, and the right image 104 is referred to as a central portion of the overhead image 100. The central part of the overhead view image 100 is the position of the vehicle in the overhead view image 100. The front image 101, the rear image 102, the left image 103, the right image 104, and the center of the overhead image 100 may be separated by a frame-shaped boundary 110.

  In FIG. 2, an oblique dashed line indicating a boundary between the front image 101, the rear image 102, the left image 103, and the right image 104 is illustrated for explanation, but is actually displayed on the display panel 31. The broken line may or may not be displayed in the overhead view video 100. The same applies to other figures.

  The overhead view image generation unit 46 does not change the display direction of the generated overhead view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 when the direction in which the obstacle is detected is ahead or behind the vehicle. Then, a bird's-eye view image 200 having a horizontally long shape is generated.

  Unchanging the display direction refers to maintaining a state in which the vertical direction of the overhead image 100 and the vertical direction of the overhead image 200 match the front-back direction of the vehicle. In other words, that the display direction is not changed, in the landscape image 200 having a horizontally long shape, the front image 201 is arranged on the upper side, the rear image 202 is arranged on the lower side, the left image 203 is arranged on the left side, and the right image 203 is arranged on the left side. Means that the state where the right side image 204 is arranged is maintained. By keeping the display direction unchanged, the vertical direction of the bird's-eye view video 200 and the front-back direction of the vehicle match, so that the horizontally long bird's-eye view video 200 can be kept recognizable.

  The bird's-eye view video 200 will be described with reference to FIG. FIG. 3 is a diagram illustrating a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the first embodiment. The overhead image 200 includes a front image 201, a rear image 202, a left image 203, and a right image 204. A range surrounded by the front video 201, the rear video 202, the left video 203, and the right video 204 is referred to as a central portion of the overhead video 200. The central part of the overhead view video 200 is the position of the vehicle in the overhead view video 200. The front image 201, the rear image 202, the left image 203, the right image 204, and the center of the overhead image 200 may be separated by a frame-shaped boundary 210.

  With reference to FIGS. 4 and 5, a description will be given of a method in which the bird's-eye view image generation unit 46 generates the horizontally-long bird's-eye view video 200. FIG. 4 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the first embodiment. FIG. 5 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 4.

  In FIG. 4, solid lines indicating the cut-out range A1, the cut-out range A2, the cut-out range A3, and the cut-out range A4 are shown for explanation, but an overhead image actually displayed on the display panel 31. 100 does not display the solid line. The same applies to other figures.

  In the present embodiment, the bird's-eye view image generation unit 46 generates a horizontally-long bird's-eye view image 200 by enlarging a part of the peripheral image from the front camera 11 or the rear camera 12 while fixing the aspect ratio. More specifically, the bird's-eye view image generation unit 46 enlarges a part of the front image 101 and a part of the rear image 102 with the fixed aspect ratio from the generated portrait image 100 having a vertically long shape, and forms a horizontally long bird's-eye view. An image 200 is generated.

  The bird's-eye view image generation unit 46 generates a bird's-eye view video 200 having a horizontally long shape from the clipping range A1, the clipping range A2, the clipping range A3, and the clipping range A4 of the bird's-eye-view video 100. More specifically, the bird's-eye view image generation unit 46 fixes and enlarges the aspect ratio of the cutout range A1 and the cutout range A2 of the bird's-eye view image 100 to obtain a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. The generated overhead image 200 is generated.

  The cut-out range A1 is a rectangular range including a range in front of the vehicle that requires confirmation by the driver. The cutout range A1 includes a part of the front image 101 of the overhead image 100, a part of the left image 103, and a part of the right image 104. The cutout range A1 includes a boundary between the front image 101 and the left image 103 and a boundary between the front image 101 and the right image 104. In the present embodiment, the cut-out range A1 includes the front end of the vehicle up to about several tens of centimeters forward. In the present embodiment, the cutout range A1 includes the left side of the vehicle from the left end by about several tens cm. In the present embodiment, the cut-out range A1 includes a portion from the right end of the vehicle to the right by about several tens cm.

  The cut-out range A2 is a rectangular range including a range behind the vehicle that requires confirmation by the driver. The cutout range A2 includes a part of the rear image 102 of the bird's-eye view image 100, a part of the left image 103, and a part of the right image 104. The cutout range A2 includes a boundary between the rear image 102 and the left image 103 and a boundary between the rear image 102 and the right image 104. In the present embodiment, the cutout range A2 includes a range from the rear end of the vehicle to the rear by about several tens cm. In the present embodiment, the cutout range A2 includes the left side of the vehicle from the left end to about several tens cm to the left. In the present embodiment, the cutout range A2 includes the right side of the vehicle from the right end by about several tens cm. In the present embodiment, the extraction range A2 has the same shape and the same area as the extraction range A1.

  The cutout range A3 is a rectangular range located between the cutout range A1 and the cutout range A2 of the overhead view video 100. In the present embodiment, the cut-out range A3 includes the left side of the vehicle from the left end by about several tens cm. The left end of the extraction range A3 is located on the same straight line as the left end of the extraction range A1 and the left end of the extraction range A2.

  The cutout range A4 is a rectangular range located between the cutout range A1 and the cutout range A2 of the overhead view video 100. In the present embodiment, the cutout range A4 includes the right side of the vehicle from the right end by about several tens cm. The right end of the extraction range A4 is located on the same straight line as the right end of the extraction range A1 and the right end of the extraction range A2. In the present embodiment, the cutout range A4 has the same shape and the same area as the cutout range A3.

  The bird's-eye view image 200 having a horizontally long shape generated from such a bird's-eye view image 100 includes a video 211 of the clipping range A1, a video 212 of the clipping range A2, a video 213 of the clipping range A3, and a clipping range. A4 image 214.

  The image 211 is an image obtained by enlarging the image of the cut-out range A1 of the bird's-eye view image 100 until the width becomes equal to the width of the display panel 31 with a fixed aspect ratio. In FIG. 5, the image of the tree displayed in the image 211 is an image obtained by enlarging the image of the tree displayed in the cut-out range A1 of the overhead image 100 shown in FIG. 4 with a fixed aspect ratio.

  The image 212 is an image obtained by enlarging the image of the cut-out range A2 of the overhead view image 100 until the width becomes the same as the width of the display panel 31 with a fixed aspect ratio. 5, the image of the tree displayed in the image 212 is an image obtained by enlarging the image of the tree displayed in the cutout range A2 of the overhead view image 100 shown in FIG. 4 while fixing the aspect ratio.

  The image 213 is an image obtained by changing the size and shape of the image in the cut-out range A3 of the overhead view image 100 according to the display panel 31 so that the boundary between the image 211 and the image 212 is smoothly continuous. For example, the video 213 may be a video obtained by compressing the video in the cut-out range A3 of the overhead view video 100 in the vertical direction and expanding it in the horizontal direction.

  The image 214 is an image obtained by changing the size and shape of the image in the cut-out range A4 of the overhead view image 100 according to the display panel 31 such that the boundary between the image 211 and the image 212 is smoothly continuous. For example, the video 214 may be a video obtained by compressing the video in the cutout range A4 of the overhead view video 100 in the vertical direction and expanding it in the horizontal direction.

  The overhead image 200 includes a front image 201, a rear image 202, a left image 203, and a right image 204. The front image 201 is a part of the image 211. The rear image 202 is a part of the image 212. The left image 203 includes a lower left portion of the image 211, an upper left portion of the image 212, and an image 213. The right image 204 includes a lower right portion of the image 211, an upper right portion of the image 212, and an image 214.

  The video synthesizing unit 47 includes the front camera 11 or the rear camera 12 at the center of the overhead video 100 and the overhead video 200 generated by the overhead video generation unit 46, corresponding to the direction in which the obstacle information acquisition unit 43 has detected the obstacle. Alternatively, an overhead image 100 and an overhead image 200 are generated by combining peripheral images from the left camera 13 or the right camera 14. Peripheral images from the front camera 11, the rear camera 12, the left camera 13, or the right camera 14 to be combined are images before viewpoint conversion. The video composition unit 47 outputs the generated overhead video 100 and overhead video 200 to the display control unit 48.

  In the present embodiment, when the direction in which the obstacle information acquisition unit 43 detects the obstacle is the left direction or the right direction of the vehicle, the image synthesis unit 47 displays the obstacle detection direction in the center of the overhead view image 100. , A bird's-eye view image 100 is generated by synthesizing a vertically long peripheral image from the left camera 13 or the right camera 14 corresponding to. The peripheral video synthesized at the center of the bird's-eye-view video 100 is a peripheral video on which the viewpoint conversion processing has not been performed. The video synthesizing unit 47 may display the boundary line 110 of the bird's-eye view video 100 by thickening or changing the color in the direction in which the obstacle is detected.

  For example, when the direction in which the obstacle is detected is the left direction, the image synthesis unit 47 synthesizes the vertically long peripheral image 103A from the left camera 13 with the center of the overhead image 100 as shown in FIG. I do. The rear image 102 and the peripheral image 103A of the bird's-eye image 100 include an obstacle image 300 of a rear obstacle. The left boundary 110A of the overhead view video 100 is a thick line.

  In the present embodiment, when the direction in which the obstacle information acquisition unit 43 has detected the obstacle is ahead or behind the vehicle, the image synthesis unit 47 detects an obstacle in the center of the landscape-shaped overhead view video 200. Overhead image 200 is generated by combining horizontally long peripheral images from the front camera 11 or the rear camera 12 corresponding to the set direction. The peripheral video combined with the central part of the bird's-eye view video 200 is a peripheral video on which the viewpoint conversion processing has not been performed. The video synthesizing unit 47 may display the boundary 210 of the overhead view video 200 by thickening or changing the color in the direction in which the obstacle is detected.

  For example, when the direction in which the obstacle is detected is backward, the video combining unit 47 combines the horizontally long peripheral video 202A from the rear camera 12 with the center of the overhead video 200 as shown in FIG. The rear image 202 and the peripheral image 202A of the overhead image 200 include an obstacle image 300 of a rear obstacle. The lower boundary 210A of the overhead view video 200 is a thick line.

  The display control unit 48 causes the display panel 31 to display the overhead image 100 and the overhead image 200 generated by the image generation unit 45.

  The storage unit 49 stores data required for various types of processing in the overhead view image generation device 40 and various types of processing results. The storage unit 49 is a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

  Next, the flow of processing in the overhead image generation device 40 of the overhead image generation system 1 will be described with reference to FIG. FIG. 6 is a flowchart illustrating a flow of processing in the overhead image generation device of the overhead image generation system according to the first embodiment.

  The control unit 41 determines whether or not to start displaying the overhead image display (step S11). In the present embodiment, the control unit 41 determines whether or not to start the bird's-eye view video display based on the presence or absence of the backward trigger. The reverse trigger means, for example, that the shift position has been set to “reverse”. Alternatively, the backward trigger means that the traveling direction of the vehicle is behind the vehicle. When there is no retreat trigger, the control unit 41 determines that the overhead view image display is not started (No in step S11), and executes the process of step S11 again. When there is a backward trigger, the control unit 41 determines that the overhead view image display is to be started (Yes in step S11), and proceeds to step S12.

  The control unit 41 generates and displays the overhead image 100 (step S12). More specifically, the control unit 41 causes the overhead image generation unit 46 to generate an overhead image 100 that has been subjected to viewpoint conversion processing from the peripheral image acquired by the image acquisition unit 42 so as to look down on the vehicle from above. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.

  The control unit 41 determines whether an obstacle is detected (Step S13). More specifically, the control unit 41 determines whether the obstacle information acquisition unit 43 has acquired the obstacle information. When the control unit 41 determines that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step S13), the process proceeds to step S14. When the control unit 41 determines that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step S13), the process proceeds to step S17.

  In the present embodiment, based on the obstacle information acquired by the obstacle information acquisition unit 43, the control unit 41 determines whether an obstacle located in the traveling direction of the vehicle has been detected. To be located in the traveling direction of the vehicle means to be located in a direction in which the distance to the vehicle decreases as the vehicle travels, and includes a range in which obstacles interfere with the vehicle as the vehicle travels. Specifically, when the vehicle is traveling straight in the traveling direction as the vehicle travels, the control unit 41 determines whether an obstacle is detected in a range centered on the traveling direction of the vehicle, and the vehicle is steered. When the vehicle is traveling while traveling, it is determined whether an obstacle is detected in a range including a range in which the obstacle interferes with the vehicle due to the traveling direction and steering of the vehicle.

  The control unit 41 determines whether the obstacle detection direction is the left direction or the right direction (step S14). In the present embodiment, based on the obstacle information acquired by the obstacle information acquisition unit 43, the control unit 41 determines whether the detection direction of the obstacle with the shortest distance to the vehicle is the left direction or the right direction of the vehicle. Is determined. More specifically, the control unit 41 determines whether the obstacle information of the obstacle with the shortest distance to the vehicle acquired by the obstacle information acquisition unit 43 is the obstacle information from the left sensor 23 or the right sensor 24. Determine whether or not.

  When the control unit 41 determines that the obstacle information of the obstacle with the shortest distance to the vehicle is the obstacle information from the left sensor 23 or the right sensor 24 (Yes in step S14), the control unit 41 proceeds to step S15. move on. When the control unit 41 determines that the obstacle information of the obstacle with the shortest distance to the vehicle is not the obstacle information from the left sensor 23 or the right sensor 24 (No in step S14), the process proceeds to step S16. move on.

  The control unit 41 generates the overhead image 100 in which the peripheral image in the obstacle detection direction is displayed at the center of the overhead image 100 (step S15). More specifically, the control unit 41 controls the image synthesizing unit 47 so that the center of the overhead image 100 generated by the overhead image generation unit 46 corresponds to the direction in which the obstacle with the smallest distance to the vehicle is detected. A bird's-eye view image 100 is generated by combining a vertically long peripheral image from the camera 13 or the right camera 14. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.

  The control unit 41 generates the overhead image 200 in which the peripheral image in the obstacle detection direction is displayed at the center of the horizontally long overhead image 200 (step S16). More specifically, the control unit 41 causes the bird's-eye view image generation unit 46 to generate a bird's-eye view image 200 in a horizontally long shape without changing the display direction of the bird's-eye view image 100 generated in step S12. Then, the control unit 41 controls the image synthesizing unit 47 to control the center of the landscape-shaped overhead image 200 from the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected. A bird's-eye view image 200 is generated by synthesizing a horizontally long peripheral image. Then, the control unit 41 causes the display control unit 48 to display the generated landscape-shaped overhead view video 200 on the display panel 31.

  The control unit 41 determines whether or not the overhead view video display is continued (step S17). More specifically, the control unit 41 determines whether or not to continue the bird's-eye view video display based on the presence or absence of the backward trigger. When there is a backward trigger, the control unit 41 determines that the overhead view image display is to be continued (Yes in step S17), and returns to step S13 to continue the processing. When there is no retreat trigger, in other words, when the retreat trigger is released, the control unit 41 determines that the overhead view image display is not continued (No in step S17), and ends the process.

  In this way, when an obstacle is detected in the leftward or rightward direction of the vehicle, the overhead view image generation system 1 places, at the center of the overhead view image 100, the left camera 13 or the left camera 13 corresponding to the direction in which the obstacle is detected. A bird's-eye view image 100 is generated by combining a vertically long peripheral image from the right camera 14.

  When an obstacle is detected in the forward or rearward direction of the vehicle, the overhead view image generation system 1 places a front camera 11 or a rear camera corresponding to the direction in which the obstacle is detected, in the center of the horizontally-oriented overhead view image 200. An overhead view image 200 is generated by combining the horizontally long peripheral images from FIG.

  As described above, in the present embodiment, the overhead view video 100 and the overhead view video 200 are generated by combining the peripheral video corresponding to the direction in which the obstacle is detected with the central portion of the overhead view video 100 and the overhead view video 200. Thus, in the present embodiment, the surrounding image in the direction in which the obstacle is detected can be confirmed together with the overhead image 100 and the overhead image 200 without moving the line of sight. As described above, in the present embodiment, it is possible to suppress the movement of the driver's line of sight and appropriately check obstacles around the vehicle.

  In the present embodiment, when an obstacle is detected in the leftward or rightward direction of the vehicle, a vertically long image from the left camera 13 or the right camera 14 corresponding to the direction in which the obstacle is detected is provided at the center of the overhead view image 100. A bird's-eye view image 100 in which a peripheral image having a shape is synthesized is generated. In the present embodiment, when an obstacle is detected in the front direction or the rear direction of the vehicle, a central portion of the landscape-shaped overhead view image 200 is displayed from the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle is detected. The overhead image 200 is generated by synthesizing the peripheral images of. As described above, in the present embodiment, it is possible to maintain a state in which the vertical direction of the overhead image 100 and the vertical direction of the overhead image 200 match the longitudinal direction of the vehicle regardless of the obstacle detection direction. Therefore, according to the present embodiment, when the driver checks the overhead view video 100 or the overhead view video 200, the driver can easily check the periphery of the vehicle.

  In the present embodiment, when an obstacle is detected in the forward or rearward direction of the vehicle, a bird's-eye image 200 in which a peripheral image corresponding to the direction in which the obstacle is detected is synthesized at the center of the bird's-eye image 200 having a landscape shape. Is generated and displayed on the display panel 31. In the present embodiment, the bird's-eye view image 200 having a horizontally long shape can be displayed on the entire display surface of the display panel 31. Thus, in the present embodiment, when an obstacle is detected in the forward or rearward direction of the vehicle, the peripheral image can be displayed large in the center of the overhead image 200.

  In the present embodiment, the image 211 of the overhead image 200 has the same aspect ratio as the image of the cutout range A1 of the overhead image 100. In other words, the front image 201 of the overhead image 200 has the same aspect ratio as the image of the cutout range A1 of the overhead image 100. The video 212 of the overhead video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead video 100. In other words, the rear image 202 of the overhead image 200 has the same aspect ratio as the image of the cutout range A2 of the overhead image 100. As described above, in the present embodiment, the front image 201 and the rear image 202 of the bird's-eye view image 200 do not change in aspect ratio and are not deformed. Therefore, according to the present embodiment, the driver can check the front image 201 and the rear image 202 of the bird's-eye view image 200 without any discomfort.

[Second embodiment]
The overhead view image generation system 1 according to the present embodiment will be described with reference to FIGS. FIG. 7 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the second embodiment. FIG. 8 is a diagram illustrating an example of a horizontally-long overhead view video generated from the overhead view video of FIG. 7. The overhead view image generation system 1 has the same basic configuration as the overhead view image generation system 1 of the first embodiment. In the following description, the same components as those of the overhead view video generation system 1 will be assigned the same reference numerals or corresponding reference numerals, and detailed description thereof will be omitted. The overhead view video generation system 1 of the present embodiment is different from the overhead view video generation system 1 of the first embodiment in a method of generating the horizontally-long overhead view video 200 by the overhead view video generation unit 46.

  When the direction in which the obstacle is detected is ahead or behind the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view image generation unit 46 generates the peripheral image from the front camera 11 or the rear camera 12. Are compressed in the vertical direction to generate a bird's-eye view video 200 having a horizontally long shape. More specifically, the bird's-eye view image generation unit 46 compresses the front image 101 and the rear image 102 in the vertical direction from the generated vertically-shaped bird's-eye image 100 to generate a bird's-eye view image 200 in a horizontally long shape.

  The clipping range A1 illustrated in FIG. 7 has the same vertical width as the vertical width of the front image 101 of the overhead view image 100. The cutout range A1 includes a boundary between the front image 101 and the left image 103 and a boundary between the front image 101 and the right image 104. The cutout range A1 includes the left side of the vehicle about several tens of centimeters to the left. The cut-out range A1 includes the right side of the vehicle from the right end by about several tens cm.

  The cutout range A2 has the same vertical width as the vertical width of the rearward video 102 of the overhead view video 100. The cutout range A2 includes a boundary between the rear image 102 and the left image 103 and a boundary between the rear image 102 and the right image 104. The cut-out range A2 includes the left side of the vehicle about several tens of centimeters to the left. The cutout range A2 includes the right side of the vehicle from the right end by about several tens cm. The cutout range A2 has the same shape and the same area as the cutout range A1.

  The cutout range A3 and the cutout range A4 are the same as in the first embodiment.

  The bird's-eye view image generation unit 46 compresses the cut-out range A1 and the cut-out range A2 of the bird's-eye view image 100 in the vertical direction, and generates a bird's-eye view image 200 in a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. .

  A bird's-eye video 200 having a horizontally long shape and generated from such a bird's-eye video 100 will be described.

  The image 211 is an image obtained by compressing the image in the cutout range A1 of the overhead image 100 in the vertical direction. 8, the image of the tree displayed in the image 211 is an image obtained by compressing the image of the tree displayed in the cutout range A1 of the overhead view image 100 shown in FIG. 7 in the vertical direction. The image of the tree displayed on the image 211 includes the entire image of the tree displayed on the front image 101 of the overhead image 100.

  The video 212 is a video obtained by compressing the video in the cutout range A2 of the overhead view video 100 in the vertical direction. 8, the image of the tree displayed in the image 212 is an image obtained by compressing the image of the tree displayed in the cutout range A2 of the overhead view image 100 shown in FIG. 7 in the vertical direction. The image of the tree displayed on the image 212 includes the entire image of the tree displayed on the rear image 102 of the overhead image 100.

  The images 213 and 214 are the same as in the first embodiment.

  As described above, according to the present embodiment, the image 211 of the overhead image 200 is an image obtained by compressing the image of the cutout range A1 of the overhead image 100 in the vertical direction. The image 211 of the overhead image 200 includes the entire vertical image of the front image 101. In the present embodiment, the image 212 of the overhead image 200 is an image obtained by compressing the image of the cutout range A2 of the overhead image 100 in the vertical direction. The image 212 of the overhead image 200 includes the entire vertical image of the rear image 102. For this reason, according to the present embodiment, the driver can check the bird's-eye video 200 in the same range as the vertical direction of the front video 101 of the bird's-eye video 100 and the vertical direction of the rear video 102.

[Third embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. FIG. 9 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the third embodiment. FIG. 10 is a diagram illustrating an example of a horizontally-long overhead-view video generated from the overhead-view video of FIG. 9. The overhead view video generation system 1 of the present embodiment is different from the overhead view video generation system 1 of the first embodiment in a method of generating the horizontally-long overhead view video 200 by the overhead view video generation unit 46.

  When the direction in which the obstacle is detected is ahead or behind the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view image generation unit 46 outputs the image from the left camera 13 or the right camera 14. The vertically long peripheral image is compressed in the vertical direction to generate a horizontally long overhead image 200. More specifically, the bird's-eye view video generation unit 46 generates the bird's-eye view video 200 by compressing the clipping range A3 and the clipping range A4 of the generated bird's-eye video 100 in the vertical direction.

  The clipping range A3 illustrated in FIG. 9 has the same horizontal width as the horizontal width of the left image 103 of the overhead view image 100. The cutout range A3 includes a boundary between the left image 103 and the front image 101 and a boundary between the left image 103 and the rear image 102. The cutout range A3 includes the front end of the vehicle up to about several meters ahead. The cutout range A3 includes the rear end of the vehicle from the rear end by about several meters.

  The cutout range A4 has the same horizontal width as the horizontal width of the right-side image 104 of the overhead view image 100. The cutout range A4 includes a boundary between the right image 104 and the front image 101 and a boundary between the right image 104 and the rear image 102. The cutout range A4 includes the front end of the vehicle up to about several meters ahead. The cut-out range A4 includes the rear end of the vehicle from the rear end by about several meters. The cutout range A4 has the same shape and the same area as the cutout range A3.

  The cutout range A1 is a rectangular range located between the cutout range A3 and the cutout range A4. The cutout range A1 includes the front end of the vehicle up to about several meters ahead. The front end of the cutout range A1 is located on the same straight line as the front end of the cutout range A3 and the front end of the cutout range A4.

  The cutout range A2 is a rectangular range located between the cutout range A3 and the cutout range A4. The cut-out range A2 includes a portion from the front end of the vehicle to about several meters ahead. The front end of the cutout range A2 is located on the same straight line as the front end of the cutout range A3 and the front end of the cutout range A4.

  The bird's-eye view image generation unit 46 compresses the cut-out range A1 and the cut-out range A2 of the generated bird's-eye view image 100 in the vertical direction, and forms a horizontally long bird's-eye view image 200 in accordance with the aspect ratio of the display surface of the display panel 31. Generate

  A bird's-eye video 200 having a horizontally long shape and generated from such a bird's-eye video 100 will be described.

  The video 213 is a video obtained by compressing the video in the cutout range A3 of the overhead view video 100 in the vertical direction. In FIG. 10, the image of the tree displayed in the image 213 is an image obtained by vertically compressing the image of the tree displayed in the cutout range A3 of the overhead image 100. The image of the tree displayed on the image 213 includes the entire image of the tree displayed on the left image 103 of the overhead image 100.

  The video 214 is a video obtained by compressing the video in the cutout range A4 of the overhead view video 100 in the vertical direction. In FIG. 10, the image of the tree displayed in the image 214 is an image obtained by compressing the image of the tree displayed in the cutout range A4 of the overhead image 100 in the vertical direction. The image of the tree displayed in the image 214 includes the entire image of the tree displayed in the right image 104 of the overhead image 100.

  The image 211 is an image obtained by changing the size and shape of the image in the cut-out range A1 of the overhead view image 100 according to the display panel 31 so that the boundary between the image 213 and the image 214 is smoothly continuous. For example, the image 211 may be an image obtained by enlarging the image in the cutout range A1 of the overhead image 100 in the horizontal direction.

  The image 212 is an image obtained by changing the size and shape of the image in the cut-out range A2 of the bird's-eye view image 100 according to the display panel 31 so that the boundary between the image 213 and the image 214 is smoothly continuous. For example, the video 212 may be a video obtained by enlarging the video of the cut-out range A2 of the overhead view video 100 in the horizontal direction.

  As described above, according to the present embodiment, the image 213 of the overhead image 200 is an image obtained by compressing the image of the clipping range A3 of the overhead image 100 in the vertical direction. The image 213 of the overhead image 200 includes the whole of the left image 103 in the horizontal direction. In the present embodiment, the video 214 of the overhead video 200 is a video obtained by compressing the video of the cutout range A4 of the overhead video 100 in the vertical direction. The image 214 of the overhead image 200 includes the whole of the right image 104 in the horizontal direction. For this reason, according to the present embodiment, the driver can check the bird's-eye video 200 in the same range as the horizontal direction of the left video 103 and the horizontal direction of the right video 104 of the bird's-eye video 100.

[Fourth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. FIG. 11 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the fourth embodiment. FIG. 12 is a diagram illustrating an example of a horizontally-long overhead view video generated from the overhead view video of FIG. 11. The overhead view image generation system 1 according to the present embodiment differs from the overhead view image generation system 1 according to the third embodiment in a method of generating the horizontally-oriented overhead view image 200 by the overhead view image generation unit 46.

  When the direction in which the obstacle is detected is ahead or behind the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view image generation unit 46 outputs the image from the left camera 13 or the right camera 14. The upper part and the lower part are combined except for the central part of the vertically long peripheral image to generate the horizontally long overhead view image 200.

  The cutout range A3 shown in FIG. 11 includes an upper part A31, a central part A32, and a lower part A33. The upper part A31 includes a boundary between the left image 103 and the front image 101. The lower portion A33 includes a boundary between the left image 103 and the rear image 102.

  The cutout range A4 includes an upper part A41, a central part A42, and a lower part A43. The upper part A41 includes a boundary between the right image 104 and the front image 101. The lower part A43 includes a boundary between the right image 104 and the rear image 102. The cutout range A4 has the same shape and the same area as the cutout range A3.

  The cutout range A1 and the cutout range A2 are the same as in the third embodiment.

  The bird's-eye view image generation unit 46 combines the upper part A31 and the lower part A33 of the clipping range A3 of the generated bird's-eye view image 100, combines the upper part A41 and the lower part A43 of the clipping range A4, and displays the display surface of the display panel 31. , An overhead view image 200 having a horizontally long shape according to the aspect ratio of the image is generated.

  A bird's-eye video 200 having a horizontally long shape and generated from such a bird's-eye video 100 will be described.

  The image 213 is an image obtained by combining the upper part A31 and the lower part A33 except for the central part A32 of the cutout range A3 of the overhead view image 100. 12, the image 213 is an image that does not include the image of the tree displayed in the central portion A32 of the cutout range A3 of the overhead image 100 shown in FIG. In the image 213, a line may be displayed at the boundary between the upper part A31 and the lower part A33 to indicate that the central part A32 of the cutout range A3 of the overhead view image 100 is omitted.

  The image 214 is an image obtained by combining the upper part A41 and the lower part A43 except for the central part A42 of the cutout range A4 of the overhead view image 100. 12, the image 214 is an image that does not include the image of the tree displayed in the central portion A42 of the cutout range A4 of the overhead image 100 shown in FIG. In the image 214, a line may be displayed at the boundary between the upper part A41 and the lower part A43 to indicate that the central part A42 of the cutout range A4 of the overhead view image 100 is omitted.

  The video 211 and the video 212 are the same as in the third embodiment.

  As described above, according to the present embodiment, the image 213 of the overhead image 200 combines the upper part A31 and the lower part A33 except for the central part A32 of the cutout range A3 of the overhead image 100. Accordingly, the image 213 of the overhead image 200 can display a large area in front of and behind the left image 103 of the overhead image 100. The image 214 of the overhead image 200 is obtained by combining the upper part A41 and the lower part A43 except for the central part A42 of the cutout range A4 of the overhead image 100. Accordingly, the image 214 of the overhead image 200 can display a large area in front of and behind the right image 104 of the overhead image 100.

[Fifth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. FIG. 13 is a diagram illustrating a method of generating a horizontally long overhead image in the overhead image generation system according to the fifth embodiment. FIG. 14 is a diagram illustrating an example of a landscape overhead image generated from the overhead image of FIG. 13. The overhead view image generation system 1 according to the present embodiment differs from the overhead view image generation system 1 according to the third embodiment in a method of generating the horizontally-oriented overhead view image 200 by the overhead view image generation unit 46.

  When the direction in which the obstacle is detected is ahead or behind the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view image generation unit 46 outputs the image from the left camera 13 or the right camera 14. The vertically long peripheral image is deviated in the direction in which the obstacle is detected, and a horizontally long overhead image 200 is generated. In the present embodiment, the direction in which an obstacle is detected will be described as backward.

  The cut-out range A3 shown in FIG. 13 is a range deviated rearward in the left-side video 103 of the overhead view video 100, which is the obstacle detection direction. The cutout range A3 includes the lower part from the center in the vertical direction of the overhead view video 100.

  The cutout range A4 is a range deviated rearward in the right side image 104 of the overhead view image 100, which is the obstacle detection direction. The cutout range A4 includes the lower part from the center in the vertical direction. The cutout range A4 has the same shape and the same area as the cutout range A3.

  The cutout range A5 is a rectangular range located between the cutout range A3 and the cutout range A4 of the overhead view video 100. The upper end of the extraction range A5 is located on the same straight line as the upper end of the extraction range A1 and the upper end of the extraction range A2. The lower end of the extraction range A5 is located on the same straight line as the lower end of the extraction range A1 and the lower end of the extraction range A2.

  The bird's-eye view image generation unit 46 converts the bird's-eye view image 200 into a horizontally long shape matching the aspect ratio of the display surface of the display panel 31 from the clipping range A3, the clipping range A4, and the clipping range A5 of the generated bird's-eye video 100. Generate.

  A bird's-eye video 200 having a horizontally long shape and generated from such a bird's-eye video 100 will be described.

  The image 213 is an image of the cutout range A3 of the overhead image 100. In FIG. 14, the image of the tree displayed in the image 213 includes the entire image of the tree displayed in the cutout range A3 of the overhead image 100 shown in FIG.

  The image 214 is an image of the cut-out range A4 of the overhead image 100. 14, the image of the tree displayed in the image 214 includes the entire image of the tree displayed in the cutout range A4 of the overhead image 100 shown in FIG.

  The image 215 is an image obtained by changing the size and the shape of the image in the cut-out range A5 of the bird's-eye view image 100 according to the display panel 31 so that the boundary between the image 213 and the image 214 is smoothly continuous. For example, the image 215 may be an image obtained by enlarging the image in the cutout range A5 of the overhead image 100 in the horizontal direction.

  As described above, in the present embodiment, when the obstacle detection direction is forward or rearward, the left image 103 and the right image 104 of the bird's-eye view image 100 are biased in the obstacle detection direction to have a horizontally long shape. The overhead view video 200 is generated. Thereby, the overhead view video 200 can display the direction in which the obstacle is detected in the overhead view video 100 in a large size.

[Sixth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described. The bird's-eye view image generation system 1 of the present embodiment differs from the bird's-eye view image generation system 1 of the fifth embodiment in a method of generating a horizontally-long bird's-eye view image 200 by the bird's-eye view image generation unit 46.

  When the direction in which the obstacle is detected is ahead or behind the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view image generation unit 46 outputs the image from the left camera 13 or the right camera 14. The vertically long peripheral image is deviated in the traveling direction of the vehicle to generate a horizontally long overhead image 200.

  As described above, according to the present embodiment, when the obstacle detection direction is forward or backward, the left image 103 and the right image 104 of the overhead image 100 are weighted in the traveling direction of the vehicle to form a horizontally long overhead image. An image 200 is generated. Thereby, the overhead view video 200 can display the traveling direction of the vehicle in the overhead view video 100 largely.

  Now, the overhead view image generation system 1 according to the present invention has been described above, but may be implemented in various different forms other than the above-described embodiment.

  The illustrated components of the overhead view video generation system 1 are functionally conceptual, and need not necessarily be physically configured as illustrated. In other words, the specific form of each device is not limited to the one shown in the figure, and all or a part of each device may be functionally or physically dispersed or integrated in arbitrary units according to the processing load and usage status of each device. You may.

  The configuration of the bird's-eye view video generation system 1 is realized by, for example, a program loaded into a memory as software. In the above-described embodiment, the description has been given as the functional block realized by the cooperation of the hardware or the software. That is, these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof.

  The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the configurations described above can be appropriately combined. Further, various omissions, substitutions, or changes in the configuration can be made without departing from the spirit of the present invention.

  In the second embodiment, the bird's-eye view image generation unit 46 may compress the central part of the peripheral image from the front camera 11 or the rear camera 12 in the vertical direction, and generate the bird's-eye view image 200 in a horizontally long shape. This will be described in more detail with reference to FIGS. FIG. 15 is a diagram illustrating another example of a method of generating a horizontally long overhead image in the overhead image generation system. FIG. 16 is a diagram illustrating an example of a horizontally-long overhead-view video generated from the overhead-view video of FIG. 15. The bird's-eye view image generation unit 46 vertically compresses the clipping range A6 at the center of the left image 103 of the bird's-eye image 100 and the clipping range A7 at the center of the right image 104, and displays the display panel 31. A bird's-eye view video 200 having a horizontally long shape that matches the aspect ratio of the surface is generated.

  When there are a plurality of obstacles, the control unit 41 may combine the peripheral video from the camera corresponding to the direction in which the obstacle with the higher priority is detected. For example, the priority may be such that the one with the smallest distance to the obstacle is the highest priority. Alternatively, for example, the priority may be such that the obstacle is a moving object having the highest priority. Alternatively, for example, the priority may be such that an obstacle is a moving object and is approaching the vehicle as the highest priority. Alternatively, for example, when there is an obstacle in both the front-rear direction and the left-right direction when the traveling direction of the vehicle is not determined, the obstacle in the front-rear direction may be given a higher priority.

  In the present embodiment, the control unit 41 combines the peripheral image corresponding to the direction in which the obstacle with the smallest distance to the vehicle is detected at the center of the overhead image 100 and the overhead image 200. The peripheral video may be selectable by the user. For example, on the display surface of the display panel 31, a peripheral image of the overhead image 100 and the overhead image 200 of the position touched by the user may be combined.

  The control unit 41 may reduce, for example, the color information of the left image 203 and the right image 204 of the overhead view image 200 when the obstacle detection direction is forward or backward. Alternatively, when the obstacle detection direction is forward or backward, the control unit 41 may display, for example, the left image 203 and the right image 204 of the overhead image 200 in a single color. Alternatively, when the obstacle detection direction is forward or backward, the control unit 41 may reduce, for example, the color information of the left image 203 and the right image 204 of the overhead image 200. Alternatively, when the detection direction of the obstacle is forward or backward, the control unit 41 may reduce, for example, the luminance of the left image 203 and the right image 204 of the overhead image 200. By doing so, the driver can be made aware that the information amount of the left image 203 and the right image 204 of the overhead image 200 has been reduced.

  When the direction in which the obstacle is detected is ahead or behind the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view image generation unit 46 outputs the images from the left camera 13 and the right camera 14 from the left camera 13 and the right camera 14, respectively. Peripheral images may be hidden, and the landscape images from the front camera 11 and the rear camera 12 may be used to generate a landscape-shaped overhead view image 200. In this case, the driver can visually check the left and right sides of the vehicle.

  The control unit 41 may determine whether or not to start the overhead image display based on, for example, whether or not the operation of the operation unit for starting the overhead image display is detected.

  In the present embodiment, four sensors of the front sensor 21, the rear sensor 22, the left sensor 23, and the right sensor 24 are provided as sensors serving as obstacle detection units, but the number of sensors is not limited. If it is desired to detect the direction of the obstacle in more detail, the number of sensors may be increased.

1 overhead view image generation system 11 front camera 12 rear camera 13 left camera 14 right camera 21 front sensor (obstacle detection unit)
22 Rear sensor (obstacle detector)
23 Left sensor (obstacle detector)
24 Right side sensor (obstacle detector)
31 display panel 40 overhead image generation device 41 control unit 42 image acquisition unit 43 obstacle information acquisition unit 44 vehicle information acquisition unit 45 image generation unit 46 overhead image generation unit 47 image synthesis unit 48 display control unit 49 storage unit 100 overhead image 200 Bird's eye view

Claims (11)

An image acquisition unit that acquires peripheral images from a plurality of cameras that photograph the front, rear, left, and right of the vehicle,
An obstacle information acquisition unit that acquires obstacle information of an obstacle detected around the vehicle;
The image acquisition unit performs a viewpoint conversion process on the peripheral image acquired to generate an overhead image as viewed from above the vehicle, and the obstacle information acquisition unit detects an obstacle in the center of the generated overhead image. An image generating unit that generates an overhead image that combines peripheral images from the camera corresponding to the direction,
A display control unit that causes the display unit to display the overhead image generated by the image generation unit,
Have a,
If the direction in which the obstacle information acquisition unit has detected an obstacle is forward or backward of the vehicle, the image generation unit generates an overhead bird's-eye image in a horizontally long shape without changing the display direction, and the generated bird's-eye image In the center of the obstacle information acquisition unit to generate a bird's-eye view image synthesized from a horizontally long peripheral image from the camera corresponding to the direction in which the obstacle was detected,
The display control unit causes the overhead view image generated by the image generation unit to be displayed on a display unit including a horizontally long display surface,
Overhead image generating apparatus, characterized in that. An image acquisition unit that acquires peripheral images from a plurality of cameras that photograph the front, rear, left, and right of the vehicle,
An obstacle information acquisition unit that acquires obstacle information of an obstacle detected around the vehicle,
The image acquisition unit performs a viewpoint conversion process on the acquired peripheral image to generate an overhead image as viewed from above the vehicle, and the obstacle information acquisition unit detects an obstacle in the center of the generated overhead image. A video generation unit that generates a bird's-eye video that combines the peripheral video from the camera corresponding to the direction
A display control unit that causes the display unit to display the overhead image generated by the image generation unit,
Has,
When the direction in which the obstacle information acquisition unit has detected an obstacle is the left direction or the right direction of the vehicle, the image information generation unit may include an obstacle at the center of the generated bird's-eye view image. Generating a bird's-eye view image that combines a vertically long peripheral image from the camera corresponding to the detected direction,
Overhead image generating apparatus, characterized in that. When the obstacle information acquisition unit detects an obstacle in front or behind the vehicle, the image generation unit may include a peripheral image from the camera corresponding to a front direction and a rear direction of the vehicle. The section is fixed and the aspect ratio is fixed, and the bird's-eye view image in the horizontally long shape is generated. Generate a bird's-eye view image that combines landscape-shaped peripheral images,
The overhead view image generation device according to claim 1 . If the direction in which the obstacle information acquisition unit has detected an obstacle is forward or rearward of the vehicle, the image generation unit vertically displays peripheral images from the cameras corresponding to the front and rear directions of the vehicle. Compressed in the direction to generate a landscape-shaped overhead view video, and in the center of the generated overhead view video, a landscape-shaped peripheral video from the camera corresponding to the direction in which the obstacle information acquisition unit detected an obstacle. Generate a synthesized bird's-eye view video,
The overhead view image generation device according to claim 1 . When the direction in which the obstacle information acquisition unit has detected an obstacle is forward or rearward of the vehicle, the image generation unit may include a vertically long periphery from the camera corresponding to a left direction and a right direction of the vehicle. The image is compressed in the vertical direction to generate a landscape-shaped overhead view video, and in the center of the generated overhead view video, a landscape-shaped landscape view from the camera corresponding to the direction in which the obstacle information acquisition unit has detected an obstacle Generate a bird's-eye view image that combines peripheral images,
The overhead view image generation device according to claim 1 . When the direction in which the obstacle information acquisition unit has detected an obstacle is forward or rearward of the vehicle, the image generation unit may include a vertically long periphery from the camera corresponding to a left direction and a right direction of the vehicle. The upper and lower parts are combined except for the central part of the video to generate a horizontally long overhead image, and the central part of the generated overhead image corresponds to the direction in which the obstacle information acquisition unit has detected an obstacle. Generate a bird's-eye view image that combines landscape-shaped peripheral images from the camera,
The overhead view image generation device according to claim 1 . When the direction in which the obstacle information acquisition unit has detected an obstacle is forward or rearward of the vehicle, the image generation unit may include a vertically long periphery from the camera corresponding to a left direction and a right direction of the vehicle. The image is deviated in the direction in which the obstacle information acquisition unit has detected an obstacle, and a bird's-eye view image having a horizontally long shape is generated.In the center of the generated bird's-eye view image, the obstacle information acquisition unit has detected an obstacle. Generating a bird's-eye view image that combines a horizontally long peripheral image from the camera corresponding to the direction,
The overhead view image generation device according to claim 1 . When the direction in which the obstacle information acquisition unit has detected an obstacle is forward or rearward of the vehicle, the image generation unit may include a vertically long periphery from the camera corresponding to a left direction and a right direction of the vehicle. The image is deviated in the traveling direction of the vehicle, and a bird's-eye view image having a horizontally-long shape is generated. Generate a bird's-eye view image that combines landscape-shaped peripheral images,
The overhead view image generation device according to claim 1 . When the direction in which the obstacle information acquisition unit detects an obstacle is forward or backward of the vehicle, the image generation unit may be configured to control a center of a peripheral image from the camera corresponding to a front direction and a rear direction of the vehicle. Compress the section in the vertical direction, generate a landscape-shaped overhead view video, in the center of the generated overhead view video, the oblong information from the camera corresponding to the direction in which the obstacle information acquisition unit has detected an obstacle Generate a bird's-eye view image that combines peripheral images,
The overhead view image generation device according to claim 1 . An image acquisition step of acquiring peripheral images from a plurality of cameras that photograph the front, rear, left and right of the vehicle,
Obstacle information acquisition step of acquiring obstacle information of an obstacle detected around the vehicle,
A viewpoint conversion process is performed on the peripheral image acquired in the image acquisition step to generate an overhead image as viewed from above the vehicle, and an obstacle is detected in the center part of the generated overhead image in the obstacle information acquisition step. Image generation step of generating an overhead image that combines peripheral images from the camera corresponding to the direction,
A display control step of displaying the overhead image generated in the image generation step on a display unit,
Only including,
In the image generation step, when the direction in which the obstacle was detected in the obstacle information acquisition step is forward or backward of the vehicle, a bird's-eye view image in a horizontally long shape is generated without changing the display direction, and the generated bird's-eye view image is generated. In the center of the obstacle information obtaining step, an overhead view image that combines a horizontally long peripheral image from the camera corresponding to the direction in which the obstacle was detected in the obstacle information obtaining step is generated,
In the display control step, the overhead image generated in the image generation step is displayed on a display unit having a horizontally long display surface,
Overhead image generating method, characterized in that. An image acquisition step of acquiring peripheral images from a plurality of cameras that photograph the front, rear, left and right of the vehicle,
Obstacle information acquisition step of acquiring obstacle information of an obstacle detected around the vehicle,
A viewpoint conversion process is performed on the peripheral image acquired in the image acquiring step to generate an overhead image as viewed from above the vehicle, and an obstacle is detected in the center part of the generated overhead image in the obstacle information acquiring step. Image generation step of generating an overhead image that combines peripheral images from the camera corresponding to the direction,
A display control step of displaying the overhead image generated in the image generation step on a display unit,
Including
In the image generation step, when the direction in which the obstacle is detected in the obstacle information acquisition step is a left direction or a right direction of the vehicle, the obstacle information acquisition step includes an obstacle at a central portion of the generated overhead view image. Generating a bird's-eye view image that combines a vertically long peripheral image from the camera corresponding to the direction in which
Overhead image generating method, characterized in that.

Images