JP6866765B2 - Bird's-eye view image generation device, bird's-eye view image generation system, bird's-eye view image generation method and program - Google Patents

Description

The present invention relates to a bird's-eye view image generation device, a bird's-eye view image generation system, a bird's-eye view image generation method, and a program.

Conventionally, a technique has been known in which a plurality of cameras installed around a vehicle are used to photograph the surroundings of the vehicle, and a bird's-eye view image synthesized by performing viewpoint conversion processing on the plurality of captured images is displayed on a monitor, for example, when the vehicle is reversing. (See, for example, Patent Document 1). In this technology, an icon indicating a vehicle is generally displayed in the center of the display screen, and the positional relationship with the surroundings with respect to the vehicle can be clearly recognized in the bird's-eye view image. Further, in general, the bird's-eye view image is displayed so that the front direction of the vehicle coincides with the upward direction in the display screen.

Japanese Unexamined Patent Publication No. 2014-129093

The bird's-eye view image as described in Patent Document 1 is displayed on the monitor in various situations such as when the vehicle is moving backward and when the vehicle is traveling on a narrow road. Further, in recent years, the size and definition of information display monitors in vehicles have been increasing, and various information may be displayed on one monitor at the same time. For example, a map may be displayed by a navigation system and a bird's-eye view image may be displayed at the same time. As described above, in the bird's-eye view image, the front direction of the vehicle is generally set to the upward direction in the display screen, whereas in the map by the navigation system, the north direction is set to the upward direction in the display screen. It is displayed in various directions such as north-up. If the display orientation of the map and the display orientation of the bird's-eye view image are different in this way, the user may not be able to quickly recognize the information displayed on the monitor.

The present invention has been made in view of the above, and an object of the present invention is to enable a user to quickly recognize information on a bird's-eye view image and a map of a vehicle displayed on a display unit of the vehicle.

In order to solve the above-mentioned problems and achieve the object, the bird's-eye view image generation device according to the present invention has an image data acquisition unit that acquires peripheral image data obtained by photographing the periphery of the vehicle, and a map displayed on the display unit. A bird's-eye view image is generated by performing viewpoint conversion processing and composition processing on the map display information acquisition unit that acquires display orientation information and the peripheral image data acquired by the image data acquisition unit, and displays the bird's-eye view image on the display unit. A control unit is provided, and the control unit has a display orientation that matches the display orientation of the map with the display orientation of the bird's-eye view image based on the display orientation information of the map acquired by the map display information acquisition unit. It is characterized by executing an adjustment process.

In order to solve the above-mentioned problems and achieve the object, the bird's-eye view image generation system according to the present invention includes the above-mentioned bird's-eye view image generation device, a plurality of cameras for photographing the periphery of the vehicle, and at least one of the display units. , It is characterized in that.

In order to solve the above-mentioned problems and achieve the object, the bird's-eye view image generation method according to the present invention includes an image data acquisition step of acquiring peripheral image data obtained by photographing the surroundings of a vehicle, and a map displayed on a display unit. A bird's-eye view image is generated by performing viewpoint conversion processing and composition processing on the map display information acquisition step for acquiring display orientation information and the peripheral image data acquired in the image data acquisition step, and displaying the bird's-eye view image on the display unit. Including the control step, in the control step, the display direction of the map and the display direction of the bird's-eye view image are matched based on the display direction information of the map acquired in the map display information acquisition step. It is characterized by executing the orientation adjustment process.

In order to solve the above-mentioned problems and achieve the object, the program according to the present invention includes a video data acquisition step of acquiring peripheral video data obtained by photographing the surroundings of a vehicle, and display orientation information of a map displayed on a display unit. A map display information acquisition step for acquiring the above, and a control step for generating a bird's-eye view image by performing viewpoint conversion processing and composition processing on the peripheral image data acquired in the image data acquisition step and displaying the bird's-eye view image on the display unit. In the control step, the display orientation adjustment process for matching the display orientation of the map with the display orientation of the bird's-eye view image based on the display orientation information of the map acquired in the map display information acquisition step. Have the computer perform each of the above steps, which comprises executing.

The bird's-eye view image generation device, the bird's-eye view image generation system, the bird's-eye view image generation method and the program according to the present invention can quickly recognize the information of the vehicle's bird's-eye view image and the map displayed on the display unit of the vehicle. It has the effect of being able to do it.

FIG. 1 is a block diagram showing a configuration example of a bird's-eye view video generation system according to the first embodiment. FIG. 2 is a schematic view illustrating an example of the position of a virtual viewpoint in the bird's-eye view video generation system according to the first embodiment. FIG. 3 is an explanatory diagram showing an example of a bird's-eye view image and a map generated by the bird's-eye view image generation system according to the first embodiment and displayed on a display panel. FIG. 4 is an explanatory diagram showing an example of a bird's-eye view image and a map when the display orientation adjustment process is performed in the bird's-eye view image generation system according to the first embodiment. FIG. 5 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the first embodiment. FIG. 6 is an explanatory diagram showing a bird's-eye view image and a map generated in the bird's-eye view image generation system according to the second embodiment and displayed on the display panel. FIG. 7 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the second embodiment. FIG. 8 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the third embodiment. FIG. 9 is an explanatory diagram showing a bird's-eye view image and a map generated by the bird's-eye view image generation system according to the fourth embodiment and displayed on the display panel. FIG. 10 is an explanatory diagram showing an example of the position of the virtual viewpoint in the bird's-eye view image generation system according to the fourth embodiment. FIG. 11 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the fourth embodiment. FIG. 12 is an explanatory diagram showing another example of the bird's-eye view image and the map generated by the bird's-eye view image generation system according to the fourth embodiment and displayed on the display panel. FIG. 13 is an explanatory diagram showing a bird's-eye view image and a map generated by the bird's-eye view image generation system according to the fifth embodiment and displayed on the display panel. FIG. 14 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the fifth embodiment.

Hereinafter, with reference to the accompanying drawings, the bird's-eye view image generation device 40, the bird's-eye view image generation system 1, the bird's-eye view image generation method, and the embodiment of the program according to the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following embodiments.

In the following explanation, the front-rear direction is a direction parallel to the traveling direction when the vehicle goes straight, the side from the driver's seat to the windshield is "front" in the front-rear direction, and the side from the windshield to the driver's seat is front-rear. Let it be "after" the direction. The front-back direction is the X-axis direction. The left-right direction is a direction that is horizontally orthogonal to the front-back direction. The left hand side is "left" and the right hand side is "right" toward the windshield. The left-right direction is the Y-axis direction. The vertical direction is a direction orthogonal to the front-back direction and the left-right direction. The vertical direction is the Z-axis direction. Therefore, the front-back direction, the left-right direction, and the vertical direction are orthogonal in three dimensions. In the following description, front / rear, left / right, and up / down are front / rear, left / right, and up / down with the bird's-eye view image generation system 1 mounted on the vehicle.

[First Embodiment]
FIG. 1 is a block diagram showing a configuration example of a bird's-eye view video generation system according to the first embodiment. The bird's-eye view image generation system 1 is mounted on the vehicle V. The bird's-eye view image generation system 1 may be a portable device that can be used in the vehicle V, in addition to the one mounted on the vehicle V.

The bird's-eye view image generation system 1 will be described with reference to FIG. The bird's-eye view image generation system 1 includes a front camera 11, a rear camera 12, a left side camera 13, a right side camera 14, a map information storage unit 21, a current position information detection unit 22, a setting operation unit 23, and the like. It has a display panel (display unit) 30, a bird's-eye view image generation device 40, and a navigation processing device 50. The bird's-eye view image generation system 1 generates a bird's-eye view image 100 (see FIG. 3) of the vehicle V.

The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 will be described with reference to FIG. FIG. 2 is a schematic view illustrating an example of the position of a virtual viewpoint in the bird's-eye view video generation system according to the first embodiment. The front camera 11 is arranged in front of the vehicle V and photographs the surroundings centered on the front of the vehicle V. The front camera 11 captures, for example, a shooting range A1 of about 180 °. The front camera 11 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The rear camera 12 is arranged behind the vehicle V and photographs the surroundings centered on the rear of the vehicle V. The rear camera 12 captures, for example, a shooting range A2 of about 180 °. The rear camera 12 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The left-side camera 13 is arranged on the left side of the vehicle V, and photographs the surroundings centered on the left side of the vehicle V. The left-side camera 13 captures, for example, a shooting range A3 of about 180 °. The left-side camera 13 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The right-side camera 14 (see FIG. 1) is arranged on the right side of the vehicle V and photographs the periphery centered on the right side of the vehicle V. The right-side camera 14 captures, for example, a shooting range (not shown) of about 180 °. The right-side camera 14 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 capture all directions of the vehicle V.

Returning to FIG. 1, the map information storage unit 21 stores the map information. The map information is, for example, a road map including an intersection. The map information storage unit 21 outputs the stored map information to the guide map generation unit (map generation unit) 53 of the navigation processing device 50. The map information storage unit 21 may be a storage device such as an external server that acquires map information via a communication function (not shown).

The current position information detection unit 22 detects the current position of the vehicle V. The current position information detection unit 22 includes, for example, a GPS (Global Positioning System) receiver, and detects the current position by calculating the current position based on a signal from the GPS. The current position information detection unit 22 outputs the detected current position information of the vehicle V to the current position acquisition unit 51 of the navigation processing device 50.

The setting operation unit 23 is an interface on which the user can input the setting operation of the navigation processing device 50. The setting operation unit 23 is, for example, a remote controller or the like. The setting operation unit 23 may also be used as, for example, a display panel 30 which is a touch panel. The setting operation unit 23 may be an interface that has a voice recognition function and performs voice input. The setting operation unit 23 outputs the setting operation information input by the user to the guidance setting unit 52 of the navigation processing device 50. The setting operation information input from the user is, for example, the content of the display orientation setting operation of the map 200 (see FIG. 3) displayed on the display panel 30, the content of the scale setting operation of the map 200, and the setting of the destination. The contents of the operation are included.

The display panel 30 is, for example, a display including a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminence) display. The display panel 30 is, for example, a touch panel type display. The display panel 30 is arranged at a position that is easily visible to the user.

The display panel 30 displays the bird's-eye view image 100 based on the image signal output from the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1 (see FIG. 3). Further, in the present embodiment, the display panel 30 displays the map 200 based on the signal output from the navigation processing device 50 in addition to the bird's-eye view image 100 (see FIG. 3). The map 200 includes a route guidance display. The display panel 30 may be composed of a plurality of display panels including a display panel for displaying the bird's-eye view image 100 and a display panel for displaying the map 200. In this case, the display panel 30 may be a display of a mobile communication device such as a smartphone. Further, the display panel 30 may be a movable display that can be rotated, or may be a display that can rotate and display the display information on the screen according to its own rotation.

In the present embodiment, the display panel 30 is a horizontally long rectangular panel. The display panel 30 may be a vertically long rectangular panel or may have a shape other than the rectangular shape. In the present embodiment, the display panel 30 is divided into a plurality of display ranges. The display panel 30 has a display range for displaying the bird's-eye view image 100 and a display range for displaying the map 200 arranged on the side of the display range of the bird's-eye view image 100. In the present embodiment, the display range for displaying the bird's-eye view image 100 is a rectangular range having an aspect ratio of 1: 1. That is, the control unit 41 causes the display panel 30 to display the bird's-eye view image 100 having an aspect ratio of 1: 1. The display range for displaying the bird's-eye view image 100 is not limited to a rectangular shape, and may be a circular shape or a rhombus shape having an aspect ratio of 1: 1 and may not have an aspect ratio of 1: 1. .. The display panel 30 may include a display range for displaying other information such as an audio screen.

The bird's-eye view image generation device 40 includes a control unit 41 and a storage unit 49.

The control unit 41 is, for example, an arithmetic processing unit composed of a CPU (Central Processing Unit) or the like. The control unit 41 loads the program stored in the storage unit 49 into the memory and executes the instruction included in the program. The control unit 41 includes a video data acquisition unit 42, a map display information acquisition unit 43, a vehicle information acquisition unit 44, a bird's-eye view image generation unit 45, and a display control unit 46.

The video data acquisition unit 42 acquires peripheral video data obtained by photographing the periphery of the vehicle V. More specifically, the video data acquisition unit 42 acquires peripheral video data output by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video data acquisition unit 42 outputs the acquired peripheral video data to the bird's-eye view video generation unit 45.

The map display information acquisition unit 43 acquires the display orientation information of the map 200 from the guide map generation unit 53 of the navigation processing device 50. The map display information acquisition unit 43 outputs the acquired map display information to the display control unit 46.

Here, the "display direction of the map 200" means the direction set in the upper part of the screen on the display panel 30. More specifically, the "display orientation" may be heading up or other than heading up. "In the case of heading up" means a case where the front side of the vehicle V is set to the upper part in the screen. The "case other than heading up" includes cases such as north up, destination up, and user operation. "North up" means that the north direction is set to the upper part of the screen. “Destination up” means a case where the destination set in the map 200 is set in the upper part of the screen. "When following the user operation" means a case where the direction arbitrarily set by the user is set at the upper part of the screen.

The vehicle information acquisition unit 44 acquires vehicle information that is a trigger for displaying the bird's-eye view image 100, such as gear operation information of the vehicle V, from CAN (Control Area Network), various sensors that sense the state of the vehicle V, and the like. .. In the present embodiment, the vehicle information includes information indicating the traveling direction of the vehicle V. The vehicle information acquisition unit 44 is a display trigger of the bird's-eye view image 100 that, for example, the vehicle V is traveling on a narrow road with a width less than a predetermined width from the guide map generation unit 53 of the navigation processing device 50 that generates the map 200. Acquire vehicle information. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the bird's-eye view image generation unit 45.

The bird's-eye view image generation unit 45 generates a bird's-eye view image 100 with a virtual viewpoint P above the vehicle V by performing a viewpoint conversion process and synthesizing the peripheral image data acquired by the image data acquisition unit 42.

The virtual viewpoint P will be described with reference to FIG. The virtual viewpoint P is located above the center of the vehicle V. The virtual viewpoint P is a viewpoint that looks down from directly above the center of the vehicle V. The center of the vehicle V is the center of the vehicle V in the vehicle width direction and the center in the front-rear direction. The position directly above the vehicle V is a position on the vertical line of the reference plane of the vehicle V. The reference plane is a plane horizontal to the road surface when the vehicle V is located on a horizontal and flat road surface. Let the position of the virtual viewpoint P be (x, y, z).

The generated bird's-eye view image 100 will be described with reference to FIG. FIG. 3 is an explanatory diagram showing an example of a bird's-eye view image and a map generated by the bird's-eye view image generation system according to the first embodiment and displayed on a display panel. The bird's-eye view image 100 displays the peripheral image of the display range A (see FIG. 2) by synthesizing the image obtained by performing the viewpoint conversion process as the virtual viewpoint P. In the present embodiment, the bird's-eye view image generation unit 45 basically generates a bird's-eye view image 100 in which the display direction is headed up, that is, the front side of the vehicle V is set upward in the screen. The bird's-eye view image generation unit 45 may generate the bird's-eye view image 100 in which the display direction is set to a direction other than the heading-up, that is, the front side of the vehicle V is set to a direction other than the upper side in the screen, depending on various conditions. The bird's-eye view image 100 includes a front image 101, a rear image 102, a left side image 103, and a right side image 104. The display area of the front image 101 and the display area of the rear image 102 are the same area. The display area of the left side image 103 and the display area of the right side image 104 are the same area. In the central portion of the bird's-eye view image 100, the own vehicle icon 110 indicating the vehicle V is displayed. The own vehicle icon 110 shows a form in which the vehicle V is looked down from directly above.

In FIG. 3, a composite boundary B1 between the front image 101 and the left side image 103, a composite boundary B2 between the front image 101 and the right side image 104, and a composite boundary B3 between the rear image 102 and the left side image 103 are shown. A diagonal broken line showing the composite boundary B4 between the rear image 102 and the right side image 104 is shown for explanation, but the broken line is displayed on the bird's-eye view image 100 actually displayed on the display panel 30. Does not have to be displayed. The same applies to other figures.

When the bird's-eye view image generation unit 45 starts the bird's-eye view image display, the viewpoint conversion process is performed from the virtual viewpoint P located directly above the vehicle V so that the vehicle V can take a bird's-eye view of the peripheral image acquired by the image data acquisition unit 42. To generate a composite bird's-eye view image. “When starting the bird's-eye view image display” means, for example, when the vehicle V travels on a narrow road with a width less than a predetermined width, or when the vehicle V moves backward, an operation for starting the bird's-eye view image display from the user to the operation unit. For example, when is detected. The "operation unit" may be, for example, a touch panel type display panel 30. The "operation unit" may also be used as the setting operation unit 23.

The bird's-eye view image generation unit 45 includes a viewpoint conversion processing unit 451, a cutting processing unit 452, and a composition processing unit 453.

The viewpoint conversion processing unit 451 performs viewpoint conversion processing on the peripheral video data acquired by the video data acquisition unit 42 so as to look down on the vehicle V from the virtual viewpoint P above. More specifically, the viewpoint conversion processing unit 451 generates an image that has undergone viewpoint conversion processing based on peripheral image data captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method of the viewpoint conversion process may be any known method and is not limited. The viewpoint conversion processing unit 451 outputs the peripheral video data that has undergone the viewpoint conversion processing to the cutout processing unit 452.

The cutout processing unit 452 performs cutout processing for cutting out an image in a predetermined range from the peripheral image data that has undergone the viewpoint conversion process. The cutout processing unit 452 cuts out the front cutout range from the peripheral image data from the front camera 11 that has undergone the viewpoint conversion process. The cutout processing unit 452 cuts out the rear cutout range from the peripheral image data from the rear camera 12 that has undergone the viewpoint conversion process. The cutout processing unit 452 cuts out the left side cutout range from the peripheral image data from the left side camera 13 that has undergone the viewpoint conversion process. The cutout processing unit 452 cuts out the right side cutout range from the peripheral image data from the right side camera 14 that has undergone the viewpoint conversion process. The cutout processing unit 452 outputs the video data of the cutout processed video to the synthesis processing unit 453.

The front cutout range is a range in which the front image 101 is cut out from the peripheral image data that has undergone the viewpoint conversion process. The front cutout range is the range from the front end of the vehicle V to the front. The rear cutout range is a range in which the rear image 102 is cut out from the peripheral image data that has undergone the viewpoint conversion process. The rear cutting range is the range from the rear end of the vehicle V to the rear. The left side cutout range is a range in which the left side image 103 is cut out from the peripheral image data that has undergone the viewpoint conversion process. The left cutout range is the range from the left side to the left side of the vehicle V. The right side cutout range is a range in which the right side image 104 is cut out from the peripheral image data that has undergone the viewpoint conversion process. The right cutout range is the range from the right side to the right side of the vehicle V.

The compositing processing unit 453 synthesizes a plurality of images cut out by the cutting processing unit 452 to generate a bird's-eye view image 100. The compositing processing unit 453 outputs the generated bird's-eye view image 100 to the display control unit 46.

When synthesizing the image obtained by the viewpoint conversion process, the composition processing unit 453 synthesizes the own vehicle icon 110 that imitates the form of the own vehicle as seen from the virtual viewpoint in the central portion of the combined bird's-eye view image. The own vehicle icon when the virtual viewpoint position is the virtual viewpoint P when looking down from directly above the vehicle V has a form as if the own vehicle is viewed from directly above as shown in the own vehicle icon 110 shown in FIG. .. It is preferable that the own vehicle icon has a display form so that a blind spot does not occur due to the own vehicle icon. For example, the own vehicle icon may be semi-transparent. For example, the own vehicle icon may have a frame shape indicating the outer shape.

Returning to FIG. 1, the display control unit 46 displays the bird's-eye view image 100 on the display panel 30. In the present embodiment, the display control unit 46 causes the display panel 30 to display the map 200 generated by the navigation processing device 50. That is, in the present embodiment, the bird's-eye view image generation device 40 and the navigation processing device 50 also serve as the display control unit 46. In addition to the display control unit 46, the navigation processing device 50 may have a display control unit for displaying the map 200 on the display panel 30.

When the display orientation of the map 200 is other than the heading-up, the display control unit 46 executes a display orientation adjustment process for matching the display orientation of the map 200 with the display orientation of the bird's-eye view image 100. More specifically, as shown in FIG. 4, the display control unit 46 has a front direction (direction along the two-dot chain arrow in the figure) of the vehicle V shown as the own vehicle icon 210 in the screen on the map 200. The front direction (direction along the alternate long and short dash arrow in the figure) of the vehicle V shown as the own vehicle icon 110 in the bird's-eye view image 100 is matched. FIG. 4 is an explanatory diagram showing an example of a bird's-eye view image and a map when the display orientation adjustment process is performed in the bird's-eye view image generation system according to the first embodiment. In the example shown in FIG. 4, the display orientation of the map 200 is set to north-up. As shown in the figure, the display direction of the map 200 is north-up, and the front direction of the vehicle V shown as the own vehicle icon 210 is inclined with respect to the vertical direction in the screen. In this case, by executing the display orientation adjustment process, the display control unit 46 also moves the front direction of the vehicle V in the bird's-eye view image 100 at the same angle as the front direction of the vehicle V indicated by the own vehicle icon 210, up and down in the screen. The bird's-eye view image 100 is rotated so as to be tilted with respect to the direction. The display control unit 46 executes the same display orientation adjustment process even when the display orientation of the map 200 is up to the destination and the display orientation is according to the user operation. The display control unit 46 outputs the bird's-eye view image 100 whose display direction matches that of the map 200 to the display panel 30 by the display direction adjustment process.

The display control unit 46 does not execute the display orientation adjustment process when the display orientation of the map 200 is heading up. When the display direction of the map 200 is heading-up, the display control unit 46 displays the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 in the same display direction (in this embodiment, the heading-up display direction) on the display panel 30. Output.

The storage unit 49 stores data required for various processes in the bird's-eye view image generation device 40 and various processing results. The storage unit 49 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

The navigation processing device 50 includes a current position acquisition unit 51, a guidance setting unit 52, and a guidance map generation unit (map generation unit) 53. The navigation processing device 50 has a storage unit (not shown) that stores data required for various processing and various processing results. A storage unit (not shown) may also be used as a storage unit 49.

The current position acquisition unit 51 acquires the current position information of the vehicle V from the current position information detection unit 22. The current position acquisition unit 51 outputs the acquired current position information of the vehicle V to the guide map generation unit 53.

The guidance setting unit 52 acquires the setting operation information of the navigation processing device 50 from the setting operation unit 23. The guidance setting unit 52 sets the display orientation of the map 200, the scale of the map 200, the destination, etc. based on the acquired setting operation information, and outputs the set setting information of the map 200 to the guidance map generation unit 53. ..

The guide map generation unit 53 acquires map information from the map information storage unit 21. The guide map generation unit 53 acquires the current position information of the vehicle V from the current position acquisition unit 51. The guide map generation unit 53 acquires the setting information of the map 200 from the guide setting unit 52. The guide map generation unit 53 generates the map 200 based on the acquired map information, the current position information of the vehicle V, and the setting information. More specifically, the guide map generation unit 53 generates a map 200 in which the own vehicle icon 210 indicating the current position of the vehicle V is superimposed on the map information and drawn. When the setting information of the map 200 includes the destination set by the user, the guide map generation unit 53 generates a guidance route from the current position of the vehicle V to the destination included in the setting information, and generates the generated guidance route. It is further superimposed on the map 200 and drawn. The guide map generation unit 53 outputs the generated map 200 to the display control unit 46. The guide map generation unit 53 outputs the display orientation information of the generated map 200 to the map display information acquisition unit 43 of the bird's-eye view image generation device 40. Based on the generated map 200, the guide map generation unit 53 generates the bird's-eye view image generation device 40 for displaying vehicle information that triggers the display of the bird's-eye view image 100, such as the fact that the vehicle V is traveling on a narrow road with a width less than a predetermined width. It is output to the vehicle information acquisition unit 44 of.

Next, the flow of processing in the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1 will be described with reference to FIG. FIG. 5 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the first embodiment. The flowchart shown in FIG. 5 is executed by the control unit 41.

The control unit 41 determines whether or not to start the bird's-eye view video display (step S11). In the present embodiment, the control unit 41 determines whether or not to start the bird's-eye view image display based on the presence or absence of detection of the display trigger, the backward trigger, and the operation of starting the bird's-eye view image display on the operation unit. As described above, the "display trigger" is vehicle information such that the vehicle V is traveling on a narrow road having a width less than a predetermined width, for example. The "backward trigger" means, for example, that the shift position is set to "reverse". Alternatively, the "reverse trigger" means that the traveling direction of the vehicle V is behind the vehicle V. When the control unit 41 determines that the bird's-eye view image display is not started (No in step S11), the control unit 41 executes the process of step S11 again. When the control unit 41 determines that the bird's-eye view image display is to be started (Yes in step S11), the control unit 41 proceeds to step S12.

The control unit 41 acquires the display orientation of the map 200 (step S12). More specifically, the control unit 41 acquires the display orientation information of the map 200 from the guide map generation unit 53 by the map display information acquisition unit 43, and proceeds to step S13.

The control unit 41 determines whether or not the display orientation of the map 200 is heading up (step S13). More specifically, the control unit 41 determines whether the display orientation of the map 200 acquired by the map display information acquisition unit 43 is heading-up or other than heading-up. When the control unit 41 determines that the display direction of the map 200 is heading up (Yes in step S13), the control unit 41 proceeds to step S14. When the control unit 41 determines that the display orientation of the map 200 is not heading up (No in step S13), the control unit 41 proceeds to step S15.

The control unit 41 generates and displays the heading-up bird's-eye view image 100 (step S14). More specifically, the control unit 41 performs a viewpoint conversion process in the bird's-eye view image generation unit 45 so as to look down on the vehicle V from above at the virtual viewpoint P from the peripheral image data acquired by the image data acquisition unit 42, and the display direction is heading. The bird's-eye view image 100 is generated and output to the display control unit 46. As shown in FIG. 3, the control unit 41 outputs the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 by the display control unit 46 to the display panel 30 in the same display direction and displays it. The control unit 41 proceeds to step S16.

The control unit 41 generates and displays a bird's-eye view image 100 that matches the display orientation of the map 200 (step S15). That is, the control unit 41 executes a display orientation adjustment process for matching the display orientation of the map 200 with the display orientation of the bird's-eye view image 100 based on the display orientation information of the map 200 acquired by the map display information acquisition unit 43. More specifically, the control unit 41 performs a viewpoint conversion process in the bird's-eye view image generation unit 45 so as to look down on the vehicle V from above at the virtual viewpoint P from the peripheral image data acquired by the image data acquisition unit 42, and the display direction is heading. The bird's-eye view image 100 is generated and output to the display control unit 46. The control unit 41 rotates the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 by the display control unit 46, and the front direction of the vehicle V shown as the own vehicle icon 210 on the screen on the map 200 and the bird's-eye view image 100. The vehicle V is aligned with the front direction, output to the display panel 30, and displayed. The control unit 41 proceeds to step S16.

The control unit 41 determines whether or not to end the bird's-eye view video display (step S16). More specifically, the control unit 41 determines whether or not to end the bird's-eye view image display based on the presence or absence of detection of the display end trigger, the backward end trigger, and the operation of the bird's-eye view image display end for the operation unit. The "display end trigger" means that, for example, vehicle information that serves as a display trigger, such as the fact that the vehicle V is traveling on a narrow road with a width less than a predetermined width, is no longer output to the vehicle information acquisition unit 44. The "backward end trigger" means, for example, that the shift position has changed from "reverse" to another position. When the control unit 41 determines that the bird's-eye view image display is not finished (No in step S16), the control unit 41 returns to step S12 and continues the process. When the control unit 41 determines that the bird's-eye view video display is finished (Yes in step S16), the control unit 41 ends the process.

As described above, in the present embodiment, the display orientation adjustment process for matching the display orientation of the map 200 with the display orientation of the bird's-eye view image 100 is executed, and the bird's-eye view image 100 whose display orientation matches the display orientation of the map 200 is obtained. It is displayed on the display panel 30 (step S15).

According to this configuration, the front direction of the vehicle V in the map 200 and the front direction of the vehicle V in the bird's-eye view image 100 can be matched with each other. As a result, the user can visually associate the information displayed on the map 200 with the information displayed on the bird's-eye view image 100, including information on roads and buildings around the vehicle V. .. Therefore, in the present embodiment, the user can quickly recognize the information of the bird's-eye view image 100 and the map 200 of the vehicle V displayed on the display panel 30 of the vehicle V.

Further, when the display orientation of the map 200 acquired by the map display information acquisition unit 43 is other than heading up (No in step S13), the control unit 41 executes the display orientation adjustment process (step S15), and the map display information. When the display orientation of the map 200 acquired by the acquisition unit 43 is heading up (Yes in step S13), the display orientation adjustment process is not executed (step S14).

According to this configuration, when the display direction of the map 200 is, for example, north-up, and the front direction of the vehicle V in the map 200 can change from moment to moment, the front direction of the vehicle V in the bird's-eye view image 100 is also the map 200. Can be taken in the same direction as. Therefore, the information of the bird's-eye view image 100 and the map 200 of the vehicle V displayed on the display panel 30 of the vehicle V can be more reliably recognized by the user. Further, when the display direction of the map 200 is heading up and the front direction of the vehicle V on the map 200 always takes a constant direction, the bird's-eye view image 100 is adjusted to the situation by not executing the display direction adjustment process. It can be displayed in the display orientation (heading up in the present embodiment). It should be noted that displaying the bird's-eye view image 100 in a display orientation suitable for the situation is not limited to heading-up, and the bird's-eye view image 100 is generated and displayed by a viewpoint change process using a virtual viewpoint set according to various conditions. Including making it.

Further, the control unit 41 displays the bird's-eye view image 100 having an aspect ratio of 1: 1 on the display panel 30.

According to this configuration, when the bird's-eye view image 100 is rotated in order to match the display direction of the bird's-eye view image 100 with the display direction of the map 200, the front direction of the vehicle V is facing any direction in the screen. Even so, the information around the vehicle V can be sufficiently displayed on the bird's-eye view image 100.

Further, the control unit 41 acquires the map 200 by the display control unit 46, and displays both the map 200 and the bird's-eye view image 100 on the same display panel 30.

According to this configuration, the display orientations of the map 200 and the bird's-eye view image 100 displayed close to each other on the same display panel 30 can be matched by the display orientation adjustment process, so that the user can match the display directions with the map 200 and the bird's-eye view image 100. Can be associated and made easier to recognize.

[Second Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 6 is an explanatory diagram showing a bird's-eye view image and a map generated in the bird's-eye view image generation system according to the second embodiment and displayed on the display panel. The bird's-eye view image generation system 1 according to the second embodiment is different from the bird's-eye view image generation system 1 according to the first embodiment in the contents of processing in the map display information acquisition unit 43 and the display control unit 46.

The map display information acquisition unit 43 further acquires the scale information of the map 200 from the guide map generation unit 53 in addition to the display orientation information of the map 200. The map display information acquisition unit 43 outputs the scale information of the acquired map 200 to the display control unit 46.

When the display orientation of the map 200 is other than the heading-up and the scale of the map 200 is equal to or greater than a predetermined value, the display control unit 46 executes the same display orientation adjustment process as in the first embodiment. The display control unit 46 outputs a bird's-eye view image 100 whose display direction matches that of the map 200 by the display direction adjustment process to the display panel 30 (see, for example, FIG. 4). The display control unit 46 does not execute the display orientation adjustment process when the display orientation of the map 200 is other than the heading-up and the scale of the map 200 is less than a predetermined value. In this case, as shown in FIG. 6, the display control unit 46 outputs the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 to the display panel 30 in the same display direction (heading-up display direction in the present embodiment). .. The predetermined value is preferably a value 1/1000. The predetermined value is more preferably 1/500 of the value.

When the display orientation of the map 200 is heading up, the display control unit 46 does not execute the display orientation adjustment process as in the first embodiment. When the display direction of the map 200 is heading-up, the display control unit 46 displays the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 in the same display direction (in this embodiment, the heading-up display direction) on the display panel 30. Output (see, for example, FIG. 3).

FIG. 7 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the second embodiment. The flowchart shown in FIG. 7 is executed by the control unit 41.

The processes of steps S21 to S23 are the same as the processes of steps S11 to S13 in the first embodiment. The processes of steps S24 and S26 are the same as the processes of steps S14 and S15 in the first embodiment. The process of step S27 is the same process as the process of step S16 in the first embodiment.

When the control unit 41 determines that the display orientation of the map 200 is not heading-up (No in step S23), the control unit 41 proceeds to step S25. The control unit 41 determines whether or not the scale of the map 200 is equal to or greater than a predetermined value (step S25). More specifically, the control unit 41 acquires the scale information of the map 200 from the guide map generation unit 53 in the map display information acquisition unit 43. The control unit 41 determines whether or not the scale of the map 200 acquired from the guide map generation unit 53 by the map display information acquisition unit 43 is equal to or greater than a predetermined value. As described above, the predetermined value is preferably a value of 1/1000. The predetermined value is more preferably 1/500 of the value. When the control unit 41 determines that the scale of the map 200 is less than a predetermined value (No in step S25), the control unit 41 proceeds to step S24. When the control unit 41 determines that the scale of the map 200 is equal to or greater than a predetermined value (Yes in step S25), the control unit 41 proceeds to step S26.

As described above, in the present embodiment, the map display information acquisition unit 43 further acquires the scale of the map 200, and the control unit 41 when the scale acquired by the map display information acquisition unit 43 is equal to or greater than a predetermined value. (Yes in step S25), the display orientation adjustment process is executed (step S26), and when the scale acquired by the map display information acquisition unit 43 is less than a predetermined value (No in step S25), the display orientation adjustment process is not executed. (Step S24).

According to this configuration, when the scale of the map 200 is equal to or larger than a predetermined value and the user can easily recognize the map 200 and the bird's-eye view image 100 in association with each other, including information such as roads and buildings around the vehicle V. The bird's-eye view image 100 can be displayed on the display panel 30 by matching the display directions of the map 200 and the bird's-eye view image 100. Further, when the scale of the map 200 is less than a predetermined value and it is difficult for the user to associate the map 200 with the bird's-eye view image 100 and recognize it, the bird's-eye view image 100 is adjusted to the situation by not executing the display orientation adjustment process. It can be displayed in the display orientation (heading up in this embodiment).

[Third Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described. The bird's-eye view image generation system 1 according to the third embodiment is different from the bird's-eye view image generation system 1 according to the second embodiment in the processing in the display control unit 46.

When the display orientation of the map 200 is other than the heading-up and the scale of the map 200 is equal to or greater than the above-mentioned predetermined value, the display control unit 46 displays the map 200 at the scale acquired from the map display information acquisition unit 43. Output to. At this time, the display control unit 46 executes the same display orientation adjustment processing as in the first embodiment and the second embodiment, and outputs a bird's-eye view image 100 whose display orientation matches that of the map 200 to the display panel 30 ( For example, see FIG. 4). When the display orientation of the map 200 is other than the heading-up and the scale of the map 200 is less than the above-mentioned predetermined value (see, for example, FIG. 6), the display control unit 46 displays the map 200 at a scale of the above-mentioned predetermined value or more. Output to panel 30 (see, for example, FIG. 4). At this time, the display control unit 46 executes the same display orientation adjustment processing as in the first embodiment and the second embodiment, and outputs a bird's-eye view image 100 whose display orientation matches that of the map 200 to the display panel 30 ( For example, see FIG. 4).

When the display orientation of the map 200 is heading up, the display control unit 46 does not execute the display orientation adjustment process as in the first embodiment and the second embodiment. When the display direction of the map 200 is heading-up, the display control unit 46 displays the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 in the same display direction (in this embodiment, the heading-up display direction) on the display panel 30. Output (see, for example, FIG. 3).

FIG. 8 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the third embodiment. The flowchart shown in FIG. 8 is executed by the control unit 41.

The process from step S31 to step S33 is the same process as the process from step S21 to step S23 in the second embodiment. Further, the processes of steps S34, S35 and S37 are the same as the processes of steps S24, S25 and S26 in the second embodiment. Further, the process of step S38 is the same process as the process of step S27 in the second embodiment.

When the control unit 41 determines that the scale of the map 200 is less than the above-mentioned predetermined value (No in step S35), the control unit 41 proceeds to step S36. When the control unit 41 determines that the scale of the map 200 is equal to or greater than the predetermined value (Yes in step S35), the control unit 41 proceeds to step S37.

The control unit 41 changes the scale of the map 200 (step S36). More specifically, the control unit 41 changes the scale of the map 200 to a value equal to or greater than the predetermined value in the display control unit 46. As a result, the map 200 having a scale equal to or larger than a predetermined value is output from the display control unit 46 to the display panel 30 and displayed on the display panel 30. The control unit 41 proceeds to step S37.

As described above, in the present embodiment, the map display information acquisition unit 43 further acquires the scale information of the map 200, and the control unit 41 acquires the map 200 and the scale acquired by the map display information acquisition unit 43. Is equal to or greater than a predetermined value (Yes in step S35), the display orientation adjustment process is executed (step S37), and the map 200 is displayed on the display panel 30 at the same scale and acquired by the map display information acquisition unit 43. When the scale is less than the predetermined value (No in step S35), the display orientation adjustment process is executed (step S37), and the scale is changed to the predetermined value or more and the map 200 is displayed on the display panel 30 (step S36). ..

According to this configuration, when the display orientation adjustment process is performed, the map 200 having a scale of the predetermined value or more and the bird's-eye view image 100 have a scale of the predetermined value or more regardless of whether the scale of the map 200 is equal to or more than the predetermined value or less than the predetermined value. Can be displayed on the display panel 30 in the same display direction. As a result, when the display directions of the map 200 and the bird's-eye view image 100 are matched, it is possible for the user to always associate the map 200 and the bird's-eye view image 100 so that they can be easily recognized.

Further, as a modification of the third embodiment, when it is determined that the scale acquired by the map display information acquisition unit 43 is less than a predetermined value (No in step S35), as shown in FIG. 13 of the fifth embodiment described later. An enlarged map 300 having a scale equal to or larger than a predetermined value may be superimposed on the map 200. In this case, the control unit 41 displays the enlarged map 300 in which the scale of the map is changed in step S36 of FIG.

[Fourth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 9 is an explanatory diagram showing a bird's-eye view image and a map generated by the bird's-eye view image generation system according to the fourth embodiment and displayed on the display panel. The bird's-eye view image generation system 1 according to the fourth embodiment is different from the bird's-eye view image generation system 1 according to the second embodiment in the processing of the bird's-eye view image generation unit 45 and the display control unit 46.

The bird's-eye view image generation unit 45 acquires the display orientation information and the scale information of the map 200 from the map display information acquisition unit 43. The bird's-eye view image generation unit 45 displays the map 200 displayed on the display panel 30 as shown in FIG. 9 when the display direction of the map 200 is other than the heading-up and the scale of the map 200 is equal to or larger than the above predetermined value. A virtual viewpoint PF (see FIG. 10) that is located below the vehicle V in the screen and is set so that the front direction of the vehicle V in the map 200 and the front direction of the vehicle V in the bird's-eye view image 100 coincide with each other. ), The viewpoint conversion process for viewing the vehicle V is performed to generate the bird's-eye view image 100. As a result, the bird's-eye view image generation unit 45 executes a display direction adjustment process for matching the display direction of the map 200 with the display direction of the bird's-eye view image 100.

The bird's-eye view image generation unit 45 has a display orientation when the display orientation of the map 200 is heading up, and when the display orientation of the map 200 is other than the heading up and the scale of the map 200 is less than the above predetermined value. Do not execute the adjustment process. In this case, the bird's-eye view image generation unit 45 generates the bird's-eye view image 100 by performing the viewpoint conversion process so as to look down on the vehicle V from the virtual viewpoint P above (see, for example, FIG. 3).

The virtual viewpoint PF will be described with reference to FIGS. 9 and 10. FIG. 10 is an explanatory diagram showing an example of the position of the virtual viewpoint in the bird's-eye view image generation system according to the fourth embodiment. When the display direction of the map 200 is other than the heading-up and the scale of the map 200 is equal to or larger than a predetermined value, the viewpoint conversion processing unit 451 is a virtual viewpoint located directly above the vehicle V as shown in FIG. A viewpoint conversion process is performed so that the virtual viewpoint PF is located at least in either the vehicle width direction or the front-rear direction from P to generate an image. The position of the virtual viewpoint PF is represented by (xF, yF, zF). In the example shown in FIG. 9, the display direction of the map 200 is north-up, and the front direction of the vehicle V shown as the own vehicle icon 210 (the direction along the two-dot chain arrow in the figure) is the vertical direction in the screen. On the other hand, it is inclined. In the virtual viewpoint PF, the front direction of the vehicle V shown as the own vehicle icon 110 in the bird's-eye view image 100 (the direction along the one-point chain arrow in the figure) is the same as the front direction of the vehicle V shown as the own vehicle icon 210. The position is set so that the vehicle V is viewed so as to be tilted with respect to the vertical direction in the screen by an angle. Further, in the example shown in FIG. 9, the virtual viewpoint PF located on the lower side of the vehicle V in the screen of the map 200 is set at a position diagonally rearward of the vehicle V as shown in FIG. As shown in FIG. 9, the position of the virtual viewpoint PF is preferably set to a position where the vehicle V shown as the own vehicle icon 110 is viewed diagonally.

The display control unit 46 outputs the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 to the display panel 30 in the same display orientation.

FIG. 11 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the fourth embodiment. The flowchart shown in FIG. 11 is executed by the control unit 41.

The process from step S41 to step S43 is the same process as the process from step S21 to step S23 in the second embodiment. Further, the processes of steps S44 and S45 are the same as the processes of steps S24 and S25 in the second embodiment. Further, the process of step S47 is the same process as the process of step S27 in the second embodiment.

When the control unit 41 determines that the scale of the map 200 is equal to or greater than a predetermined value (Yes in step S45), the control unit 41 proceeds to step S46. The control unit 41 generates and displays a bird's-eye view image 100 in which the virtual viewpoint position is changed according to the display direction of the map 200 (step S46). More specifically, the control unit 41 is a bird's-eye view image generation unit 45, located below the vehicle V in the screen of the map 200 displayed on the display panel 30, and is located in the front direction of the vehicle V on the map 200. The viewpoint conversion process for viewing the vehicle V from a virtual viewpoint set so as to coincide with the front direction of the vehicle V in the bird's-eye view image 100 is performed to generate the bird's-eye view image 100 shown in FIG. The control unit 41 causes the display control unit 46 to output the bird's-eye view image 100 shown in FIG. 9 generated by the bird's-eye view image generation unit 45. The control unit 41 causes the display control unit 46 to output the bird's-eye view image 100 generated by the bird's-eye view image generation unit 45 to the display panel 30 in the same display orientation.

As described above, in the present embodiment, the control unit 41 is located below the vehicle V (own vehicle icon 210) in the screen of the map 200 displayed on the display panel 30 as the display direction adjustment process. In addition, the bird's-eye view image 100 is obtained by performing viewpoint conversion processing in which the vehicle V is viewed from the virtual viewpoint PF set so that the front direction of the vehicle V on the map 200 and the front direction of the vehicle V in the bird's-eye view image 100 coincide with each other. Generate (step S46).

According to this configuration, it is not limited to the bird's-eye view image 100 (see FIG. 3 and the like) in which the vehicle V is viewed from directly above, and as shown in FIG. 9, the bird's-eye view image 100 in which the vehicle V is viewed from diagonally behind, for example. Can be displayed on the display panel 30 by matching the display directions of the map 200 and the bird's-eye view image 100 while using the above. As a result, as shown in FIG. 9, the peripheral information of the vehicle V can be displayed in more detail in the bird's-eye view image 100.

The guide map generation unit 53 may generate the map 200 in three dimensions, and the control unit 41, together with the bird's-eye view image 100 shown in FIG. 9, seems to be viewed from the same virtual viewpoint as the bird's-eye view image 100. The three-dimensional map 200 may be displayed on the display panel 30. As a result, the user can more easily associate and recognize the map 200 and the bird's-eye view image 100.

FIG. 12 is an explanatory diagram showing another example of the bird's-eye view image and the map generated by the bird's-eye view image generation system according to the fourth embodiment and displayed on the display panel. As shown in FIG. 12, the control unit 41 generates a bird's-eye view image 100 generated by performing a viewpoint conversion process using a virtual viewpoint as if the vehicle V is viewed from behind in step S46, and displays the bird's-eye view image 100 on the display panel 30. It may be displayed.

[Fifth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 13 is an explanatory diagram showing a bird's-eye view image and a map generated by the bird's-eye view image generation system according to the fifth embodiment and displayed on the display panel. In the bird's-eye view video generation system 1 according to the fifth embodiment, the processing of the setting operation unit 23, the guidance setting unit 52, the guide map generation unit 53, the map display information acquisition unit 43, and the display control unit 46 is the bird's-eye view according to the first embodiment. It is different from the video generation system 1.

The setting operation unit 23 outputs to the guidance setting unit 52 that the setting operation for drawing the enlarged map 300 on the map 200 has been performed as the setting operation information input from the user. The enlarged map 300 is a map in which the periphery of the vehicle V is enlarged and displayed at a scale less than the scale of the map 200.

When the guide setting unit 52 acquires that the setting operation for drawing the enlarged map 300 on the map 200 has been performed, the guide setting unit 52 outputs the setting for generating and drawing the enlarged map 300 to the guide map generation unit 53.

When the guide map generation unit 53 acquires the setting for generating and drawing the enlarged map 300, the guide map generation unit 53 generates the enlarged map 300. The guide map generation unit 53 may automatically generate the enlarged map 300, for example, when arriving in the vicinity of the destination or when traveling on a narrow road having a width less than a predetermined width. In the present embodiment, the enlarged map 300 is generated in the same display orientation as the map 200. The display orientation of the enlarged map 300 may be set by the user via the setting operation unit 23. The guide map generation unit 53 draws the generated enlarged map 300 on the map 200, and outputs it to the display control unit 46. The guide map generation unit 53 outputs the display direction of the enlarged map 300 and the display direction of the map 200 to the map display information acquisition unit 43.

The map display information acquisition unit 43 outputs the acquired display direction of the enlarged map 300 and the display direction of the map 200 to the display control unit 46.

When the display control unit 46 acquires the enlarged map 300 and the map 200 from the guide map generation unit 53, the display control unit 46 superimposes the enlarged map 300 on the map 200 and displays it on the display panel 30.

When the display orientation of the enlarged map 300 is other than the heading-up, the display control unit 46 executes a process of matching the display orientation of the enlarged map 300 with the display orientation of the bird's-eye view image 100. More specifically, as shown in FIG. 13, the display control unit 46 indicates the front direction of the vehicle V (direction along the two-dot chain arrow in the figure) indicated as the own vehicle icon 310 on the screen of the enlarged map 300. , The front direction of the vehicle V shown as the own vehicle icon 110 in the bird's-eye view image 100 (the direction along the alternate long and short dash arrow in the figure) is matched. In the example shown in FIG. 13, the display orientation of the map 200 and the enlarged map 300 is set to north-up. As shown in the figure, the display direction of the enlarged map 300 is north-up, and the front direction of the vehicle V shown as the own vehicle icon 310 is inclined with respect to the vertical direction in the screen. In this case, the display control unit 46 tilts the front direction of the vehicle V in the bird's-eye view image 100 with respect to the vertical direction in the screen at the same angle as the front direction of the vehicle V shown as the own vehicle icon 310. , Rotate the bird's-eye view image 100. The display control unit 46 outputs a bird's-eye view image 100 whose display direction matches that of the enlarged map 300 to the display panel 30.

When the display direction of the magnified map 300 is heading up, the display control unit 46 does not execute the process of matching the display direction of the magnified map 300 with the display direction of the bird's-eye view image 100. In the present embodiment, when the display direction of the enlarged map 300 is heading-up, the display control unit 46 displays the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 as it is (in this embodiment, the heading-up display direction). ) Is output to the display panel 30.

Next, the processing flow of the bird's-eye view video generation system 1 will be described with reference to FIG. FIG. 14 is a flowchart showing a processing flow in the bird's-eye view video generation system according to the fifth embodiment. The flowchart shown in FIG. 14 is executed by the control unit 41.

The control unit 41 determines whether or not to start the enlarged map display (step S51). In the present embodiment, the control unit 41 determines that the enlarged map display is started when the enlarged map 300 is output to the display control unit 46, and enlarges when the enlarged map 300 is not output to the display control unit 46. It is determined that the map display is not started. When the control unit 41 determines that the enlarged map display is not started (No in step S51), the control unit 41 executes the process of step S51 again. When the control unit 41 determines that the enlarged map display is to be started (Yes in step S51), the control unit 41 proceeds to step S52.

The control unit 41 acquires the display orientation of the enlarged map 300 (step S52). More specifically, the control unit 41 acquires the display orientation information of the enlarged map 300 from the guide map generation unit 53 in the map display information acquisition unit 43, and proceeds to step S53.

The control unit 41 determines whether or not the display direction of the enlarged map 300 is heading up (step S53). More specifically, the control unit 41 determines whether the display direction of the enlarged map 300 acquired by the map display information acquisition unit 43 is heading-up or other than heading-up. When the control unit 41 determines that the display direction of the enlarged map 300 is heading-up (Yes in step S53), the control unit 41 proceeds to step S54. When the control unit 41 determines that the display direction of the enlarged map 300 is not heading up (No in step S53), the control unit 41 proceeds to step S55.

The control unit 41 generates and displays the heading-up bird's-eye view image 100 (step S54). More specifically, the control unit 41 performs a viewpoint conversion process in the bird's-eye view image generation unit 45 so as to look down on the vehicle V from above at the virtual viewpoint P from the peripheral image data acquired by the image data acquisition unit 42, and the display direction is heading. The bird's-eye view image 100 is generated and output to the display control unit 46. As shown in FIG. 3, the control unit 41 outputs the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 by the display control unit 46 to the display panel 30 in the same display direction and displays it. The control unit 41 proceeds to step S56.

The control unit 41 generates and displays a bird's-eye view image 100 that matches the display direction of the enlarged map 300 (step S55). That is, the control unit 41 executes a process of matching the display orientation of the enlarged map 300 with the display orientation of the bird's-eye view image 100 based on the display orientation information of the enlarged map 300 acquired by the map display information acquisition unit 43. More specifically, the control unit 41 performs a viewpoint conversion process in the bird's-eye view image generation unit 45 so as to look down on the vehicle V from above at the virtual viewpoint P from the peripheral image data acquired by the image data acquisition unit 42, and obtains the bird's-eye view image 100. It is generated and output to the display control unit 46. The control unit 41 rotates the bird's-eye view image 100 acquired from the bird's-eye view image generation unit 45 by the display control unit 46, and the front direction of the vehicle V shown as the own vehicle icon 310 on the screen of the enlarged map 300 and the bird's-eye view image 100. The front direction of the vehicle V shown as the own vehicle icon 110 is matched with the display panel 30. The control unit 41 proceeds to step S56.

The control unit 41 determines whether or not to end the enlarged map display (step S56). More specifically, the control unit 41 determines that the enlarged map display is terminated when the enlarged map 300 is no longer output to the display control unit 46, and the enlarged map 300 is enlarged while the enlarged map 300 is output to the display control unit 46. It is determined that the map display is not finished. When the control unit 41 determines that the enlarged map display is not finished (No in step S56), the control unit 41 returns to step S52 and continues the process. When the control unit 41 determines that the enlarged map display is finished (Yes in step S56), the control unit 41 proceeds to step S57.

The control unit 41 ends the bird's-eye view video display (step S57). More specifically, the control unit 41 ends the generation process of the bird's-eye view image 100 in the bird's-eye view image generation unit 45 and the display process of the bird's-eye view image 100 in the display control unit 46, and ends the process.

As described above, in the present embodiment, the map display information acquisition unit 43 further acquires the display orientation information of the enlarged map 300, which is drawn on the map 200 and is a part of the map 200 enlarged, and is a control unit. 41 matches the display orientation of the enlarged map 300 with the display orientation of the bird's-eye view image 100 based on the display orientation information of the enlarged map 300 acquired by the map display information acquisition unit 43.

According to this configuration, when the enlarged map 300 that enlarges and displays the periphery of the vehicle V is displayed, the display orientations of the enlarged map 300 and the bird's-eye view image 100 are matched, and the bird's-eye view image 100 is displayed on the display panel 30. Can be made to. As a result, when the user can easily recognize the enlarged map 300 and the bird's-eye view image 100 in association with each other, including information such as roads and buildings around the vehicle V, the user can promptly display the information displayed on the display panel 30. It can be recognizable.

Although the bird's-eye view image generation system 1 according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated bird's-eye view image generation system 1 is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of each device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the bird's-eye view video generation system 1 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the configurations described above can be combined as appropriate. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

In the first embodiment, the processes of steps S13 and S14 may be omitted. That is, the control unit 41 may execute the display orientation adjustment process regardless of whether the display orientation of the map 200 acquired by the map display information acquisition unit 43 is heading up or other than heading up.

In the second embodiment, the process of step S23 may be omitted. That is, the control unit 41 adjusts the display orientation when the scale acquired by the map display information acquisition unit 43 is equal to or greater than a predetermined value regardless of whether the display orientation of the map 200 is heading up or other than heading up. When the process is executed and the scale acquired by the map display information acquisition unit 43 is less than a predetermined value, the display orientation adjustment process may not be executed.

In the third embodiment, the processes of steps S33 and S34 may be omitted. That is, the control unit 41 adjusts the display orientation when the scale acquired by the map display information acquisition unit 43 is equal to or greater than a predetermined value regardless of whether the display orientation of the map 200 is heading up or other than heading up. While executing the process, the map 200 is displayed on the display panel 30 at the same scale, and when the scale of the map 200 is less than the predetermined value, the display orientation adjustment process is executed and the scale is changed to the predetermined value or more. The map 200 may be displayed on the display panel 30.

In the fourth embodiment, the process of step S43 or step S45 may be omitted. That is, the control unit 41 determines whether the display orientation of the map 200 is heading up or other than heading up, or whether the scale of the map 200 is greater than or equal to a predetermined value or less than a predetermined value. Instead, the display orientation adjustment process may be executed by generating a bird's-eye view image 100 of the vehicle V viewed from the virtual viewpoint PF as shown in FIG.

In the fifth embodiment, the processes of steps S53 and S54 may be omitted. That is, the control unit 41 has a display orientation of the enlarged map 300 and the bird's-eye view image 100 regardless of whether the display orientation of the enlarged map 300 acquired by the map display information acquisition unit 43 is heading up or other than heading up. May match.

In the fifth embodiment, the control unit 41 performs the vehicle V in the screen of the enlarged map 300 displayed on the display panel 30 as the display direction adjustment process in step S55 as in the process of step S46 of the fourth embodiment. A viewpoint conversion process in which the vehicle V is viewed from a virtual viewpoint that is located on the lower side of the map and is set so that the front direction of the vehicle V in the enlarged map 300 and the front direction of the vehicle V in the bird's-eye view image 100 coincide with each other. This may be performed to generate a bird's-eye view image 100 and display it on the display panel 30.

In the fifth embodiment, the control unit 41 may execute the process of step S11 before the process of step S51. Further, the control unit 41 may execute the flowchart shown in any of FIGS. 5, 7, 8 and 11 in parallel with the flowchart shown in FIG. In this case, when the control unit 41 determines that the enlarged map display is to be started while executing the flowchart of any one of FIGS. 5, 7, 8 and 11 (Yes in step S51), FIG. 5. When the execution of the flowchart shown in any of FIGS. 7, 8 and 11 is temporarily completed, the processes after step S52 shown in FIG. 14 are executed, and the execution of the flowchart shown in FIG. 14 is completed, FIG. 5. Execution of the flowchart shown in any of FIGS. 7, 8 and 11 may be started.

In the first to third embodiments and the fifth embodiment, the control unit 41 generates the bird's-eye view image 100 oriented to the heading-up display in the bird's-eye view image generation unit 45, and the display control unit 46 generates the bird's-eye view image 100. Is rotated according to the display direction of the map 200, and the display direction adjustment process for matching the display direction of the bird's-eye view image 100 with the display direction of the map 200 is executed. As a display orientation adjustment process, the control unit 41 generates a bird's-eye view image 100 that matches the display orientation of the map 200 or the enlarged map 300 in advance in the bird's-eye view image generation unit 45, and the display control unit 46 generates the bird's-eye view image generation unit 45. The bird's-eye view image 100 may be displayed on the display panel 30 in the same display orientation.

Further, in the first to third embodiments, the control unit 41 may rotate the map 200 to match the display orientation of the map 200 with the display orientation of the bird's-eye view image 100 as the display orientation adjustment process. .. For example, in the process of step S15, step S26, or step S37, the control unit 41 displays the bird's-eye view image 100 on the display panel 30 in the same display orientation (heading up in this embodiment), and heads up the map 200. May be displayed on the display panel 30 as the display direction of. Similarly, in the fifth embodiment, the control unit 41 may rotate the enlarged map 300 to match the display direction of the enlarged map 300 with the display direction of the bird's-eye view image 100. For example, in the process of step S55, the control unit 41 displays the bird's-eye view image 100 on the display panel 30 in the same display direction (heading up in the present embodiment), and displays the enlarged map 300 as the heading up display direction. It may be displayed on the panel 30.

In the first to fifth embodiments, the control unit 41 constantly displays the map 200 on the display panel 30 while executing the flowcharts of FIGS. 5, 7, 8, 11 and 14. It does not have to be. For example, the control unit 41 may temporarily stop the display of the map 200 when an operation to display the bird's-eye view image 100 for the operation unit from the user on the full screen of the display panel 30 is detected.

In the first to fifth embodiments, the guide map generation unit 53 generates a map 200 in which the own vehicle icon 210 indicating the current position of the vehicle V is superimposed on the map information to generate a map 200. The map 200 generated by the part 53 is not limited to this. The guide map generation unit 53 may generate a map 200 that always displays the front side of the vehicle V without drawing the own vehicle icon 210.

In the first to fifth embodiments, the map generated by the guide map generation unit 53 has been described as an example of a plane map, but in any of the embodiments, various maps such as a three-dimensional map and a bird's-eye view map have been described. The display form is applicable.

1 Bird's-eye view image generation system 11 Front camera 12 Rear camera 13 Left-side camera 14 Right-side camera 21 Map information storage unit 22 Current position information detection unit 23 Setting operation unit 30 Display panel 40 Bird's-eye view image generator 41 Control unit 42 Video data acquisition unit 43 Map display information acquisition unit 44 Vehicle information acquisition unit 45 Bird's-eye view image generation unit 451 Viewpoint conversion processing unit 452 Cutting processing unit 453 Synthesis processing unit 46 Display control unit 49 Storage unit 50 Navigation processing device 51 Current position acquisition unit 52 Guidance setting unit 53 Guide map generator P, PF Virtual viewpoint V Vehicle

Claims (11)

A video data acquisition unit that acquires peripheral video data of the surroundings of the vehicle,
A map display information acquisition unit that acquires scale information of a map displayed on the display screen of the display unit and display orientation information indicating a direction set above the display screen.
A control unit that generates a bird's-eye view image by performing viewpoint conversion processing and composition processing on the peripheral image data acquired by the image data acquisition unit and displays it on the display screen of the display unit.
With
When the scale of the map acquired by the map display information acquisition unit is equal to or greater than a predetermined value , the control unit moves forward of the own vehicle icon displayed on the map based on the display orientation information of the map. A bird's-eye view image generation device characterized by executing a display direction adjustment process for matching the display direction of the vehicle with the display direction of the own vehicle icon displayed in the bird's-eye view image in the front direction. The control unit acquires the map, and when the scale acquired by the map display information acquisition unit is equal to or greater than the predetermined value, the control unit executes the display orientation adjustment process and displays the map at the same scale. When the scale obtained by the map display information acquisition unit is less than the predetermined value, the display orientation adjustment process is executed, and the map is displayed by changing the scale to the predetermined value or more. The bird's-eye view image generation device according to claim 1 , wherein the image is displayed on a unit. The control unit has a heading-up in which the display direction of the map acquired by the map display information acquisition unit is set to the front direction of the own vehicle icon displayed on the map above the display screen. The bird's-eye view image generation device according to claim 1 or 2 , wherein the display orientation adjustment process is executed when other than the above. The control unit is located behind the own vehicle icon displayed on the map displayed on the display unit as the display orientation adjustment process, and has the front direction of the vehicle on the map and the bird's-eye view image. 1. Any one of claims 1 to 3 , wherein the bird's-eye view image is generated by performing a viewpoint conversion process of viewing the vehicle from a virtual viewpoint set so as to match the front direction of the vehicle. The bird's-eye view image generator described in the section. The map display information acquisition unit further acquires display orientation information indicating a direction set above the display screen of an enlarged map that is drawn on the map and is a part of the map enlarged.
Wherein, based on the display direction information of the enlarged map obtained by the map display information acquisition unit, is displayed on the overhead image a front direction of the display direction of the vehicle icon displayed on the enlarged map The bird's-eye view image generation device according to any one of claims 1 to 4 , wherein the display direction of the own vehicle icon in the front direction is matched with the display direction. The bird's-eye view image generation device according to any one of claims 1 to 5 , wherein the control unit displays the bird's-eye view image having an aspect ratio of 1: 1 on the display unit. The bird's-eye view image generation device according to any one of claims 1 to 6 , wherein the control unit acquires the map and displays both the map and the bird's-eye view image on the same display unit. .. The bird's-eye view image generator according to any one of claims 1 to 7.
A plurality of cameras for photographing the periphery of the vehicle, and at least one of the display units.
A bird's-eye view video generation system characterized by being equipped with. The current position acquisition unit that acquires the current position of the vehicle and
A map generation unit that acquires map information and generates the map displayed on the display unit based on the current position of the vehicle acquired by the current position acquisition unit and the map information.
The bird's-eye view image generation system according to claim 8 , further comprising a navigation processing device including the above. The video data acquisition step to acquire the peripheral video data of the surroundings of the vehicle,
A map display information acquisition step for acquiring display orientation information indicating a direction set above the display screen of the map displayed on the display screen of the display unit, and
A control step of generating a bird's-eye view image by performing viewpoint conversion processing and compositing processing on the peripheral image data acquired in the image data acquisition step and displaying it on the display screen of the display unit.
Including
In the control step, when the scale of the map acquired in the map display information acquisition step is equal to or greater than a predetermined value , in front of the own vehicle icon displayed on the map based on the display orientation information of the map. A method for generating a bird's-eye view image, which comprises executing a display direction adjustment process for matching the display direction of the direction with the display direction of the own vehicle icon displayed in the bird's-eye view image in the front direction. The video data acquisition step to acquire the peripheral video data of the surroundings of the vehicle,
A map display information acquisition step for acquiring display orientation information indicating a direction set above the display screen of the map displayed on the display screen of the display unit, and
A control step of generating a bird's-eye view image by performing viewpoint conversion processing and compositing processing on the peripheral image data acquired in the image data acquisition step and displaying it on the display screen of the display unit.
Including
In the control step, when the scale of the map acquired in the map display information acquisition step is equal to or greater than a predetermined value , in front of the own vehicle icon displayed on the map based on the display orientation information of the map. and executes the display orientation adjustment processing to match the front direction of the display direction of the vehicle icon displayed on the overhead image and the direction of the display direction, a program for executing the respective steps in a computer ..

Images