JP7772220B2 - Parking assistance method and parking assistance device - Google Patents

Description

The present invention relates to a parking assistance method and a parking assistance device.

The parking assistance device described in Patent Document 1 below draws an extension line of the target parking space and the predicted trajectory of the vehicle on the overhead image when the target parking space is located outside the overhead image generated from images taken around the vehicle.

Japanese Patent Application Laid-Open No. 2008-80959

However, if the vehicle is far from the target parking position and the display position of the target parking frame is set outside the range of the overhead image, the relative position of the target parking frame with respect to the vehicle cannot be correctly grasped.
An object of the present invention is to provide an image that makes it easy to grasp the relative position of a target parking frame with respect to one's own vehicle even when the own vehicle is far from the target parking position.

One aspect of the present invention provides a parking assistance method that sets a target parking position for a vehicle, generates an overhead image of the surroundings of the vehicle from captured images of the surroundings of the vehicle, and displays the overhead image and a guide figure representing an area of a predetermined size surrounding the target parking position in a predetermined display area of a display screen visible to an occupant of the vehicle. In this parking assistance method, when the overhead image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen; and when the overhead image is displayed at the first display position on the display screen, if a predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen different from the first display position.

According to the present invention, an image can be provided that makes it easy to grasp the relative position of the target parking frame with respect to the vehicle, even if the vehicle is far from the target parking position.
The objects and advantages of the invention will be realized and attained by means of the elements and combinations set forth in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

FIG. 1 is a diagram illustrating an example of a schematic configuration of a parking assistance device. 1A is an explanatory diagram of an example of a process for registering a target parking position, and FIG. 1B is an explanatory diagram of an example of a process when parking assistance is performed. FIG. 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1 . 4A to 4C are explanatory diagrams illustrating an example of a parking assistance method according to the first embodiment. 10A to 10D are schematic diagrams of examples of overlapping target parking frames. 3 is a flowchart illustrating an example of a parking assistance method according to the first embodiment. 10A and 10B are diagrams illustrating an example of a parking assistance method according to a second embodiment. 10 is a flowchart illustrating an example of a parking assistance method according to a second embodiment.

(First embodiment)
(composition)
Referring to FIG. 1 , the host vehicle 1 is equipped with a parking assistance device 10 that assists in parking the host vehicle 1 at a target parking position. The parking assistance device 10 assists the host vehicle 1 in traveling along a target traveling trajectory from the current position of the host vehicle 1 to the target parking position. For example, the host vehicle 1 may be automatically driven to travel along the target traveling trajectory of the host vehicle 1 to the target parking position (i.e., control of all or part of the steering angle, driving force, and braking force of the host vehicle to automatically perform all or part of traveling along the target traveling trajectory of the host vehicle 1). Parking of the host vehicle 1 may be assisted by displaying the target parking position, the target traveling trajectory, and the current position of the host vehicle 1 on a display device visible to an occupant (e.g., the driver) of the host vehicle 1.

The positioning device 11 measures the current position and attitude of the vehicle 1 (e.g., the longitudinal direction of the vehicle body). The positioning device 11 is equipped with a Global Navigation System (GNSS) receiver, such as a Global Positioning System (GPS) receiver. The human-machine interface (HMI) 13 is an interface device that exchanges information between the parking assistance device 10 and the occupant, and is equipped with a display device with a display screen visible to the occupant, a speaker, a buzzer, and controls. The shift switch (shift SW) 14 is a switch that allows the occupant of the vehicle 1 or the parking assistance device 10 to change the shift position of the vehicle 1.

The external sensor 15 detects objects within a predetermined distance range from the host vehicle 1. The external sensor 15 detects the surrounding environment of the host vehicle 1, such as the relative position of the host vehicle 1 and objects present around the host vehicle 1, the distance between the host vehicle 1 and the objects, and the direction in which the objects are located. The external sensor 15 may include, for example, a camera that captures the surrounding environment of the host vehicle 1. In the following description, the camera possessed by the external sensor 15 will be referred to simply as "camera." The external sensor 15 may also include a distance measuring device such as a laser range finder, radar, or LiDAR. The vehicle sensor 16 detects various information (vehicle information) about the host vehicle 1. The vehicle sensor 16 may include, for example, a vehicle speed sensor, a wheel speed sensor, a three-axis acceleration sensor that detects the acceleration of the host vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle, a steering angle sensor that detects the steering angle of the steered wheels, a gyro sensor, and a yaw rate sensor.

The controller 17 is an electronic control unit that performs parking assistance control. The controller 17 includes a processor 20 and peripheral components such as a memory device 21. The processor 20 may be, for example, a CPU or MPU. The memory device 21 may include a semiconductor memory device, a magnetic memory device, an optical memory device, etc. The functions of the controller 17 are realized, for example, by the processor 20 executing a computer program stored in the memory device 21. The steering actuator 19a controls the steering direction and steering amount of the steering mechanism of the vehicle 1 in response to a control signal from the controller 17. The accelerator actuator 19b controls the accelerator opening of the drive device, such as the engine or drive motor, in response to a control signal from the controller 17. The brake actuator 19c activates the braking device in response to a control signal from the controller 17.

Next, parking assistance control by the parking assistance device 10 will be explained. When using parking assistance by the parking assistance device 10, a target parking position where the vehicle 1 should be parked is registered in the parking assistance device 10. Specifically, landmarks existing around the target parking position are extracted and stored in advance in the memory device 21. In the following explanation, landmarks around the target parking position stored in the memory device 21 will be referred to as "learned landmarks." Figure 2(a) is an explanatory diagram of an example of the process for registering a target parking position, with circles representing learned landmarks. When registering a target parking position 30 in the parking assistance device 10, the occupant performs an operation to instruct the registration of the target parking position 30 (hereinafter sometimes referred to as the "registration operation"). The registration operation may be, for example, the operation of a "parking position registration switch" provided in the HMI 13.

For example, when the vehicle 1 is located near the target parking position 30 (for example, when the occupant manually parks the vehicle 1 at the target parking position 30), the parking assistance device 10 detects the surroundings of the vehicle 1 using the external sensor 15 and stores the detected targets as learned targets. For example, targets may be detected from a surrounding image obtained by capturing an image of the surroundings of the vehicle 1 with a camera. In this case, targets are detected as edge points or points with characteristic shapes (feature points) where the brightness of adjacent pixels changes by more than a predetermined amount, such as the edges and corners of targets such as road markings, road boundaries, and obstacles in the surrounding image. Targets around the vehicle 1 may also be detected using a ranging device.

The parking assistance device 10 stores learned target data related to the learned target in the storage device 21. For example, the learned target data includes data representing the feature amounts of the learned target (hereinafter referred to as "feature amount data"), data on the relative positional relationship between the learned target and the target parking position (hereinafter referred to as "relative position data"), and coordinate data of the target parking position 30 (hereinafter referred to as "target parking position coordinate data").
As the relative position data, for example, the relative position of the learned target with respect to the target parking position 30 may be stored. The parking assistance device 10 can acquire the position of the learned target detected when the host vehicle 1 is located at the target parking position 30 as the relative position of the learned target with respect to the target parking position 30. For example, the occupant may operate the HMI 13 to input that the host vehicle 1 is located at the target parking position 30. Alternatively, the coordinates of the learned target and the target parking position 30 in a coordinate system with a fixed point as the reference point (hereinafter referred to as a "map coordinate system") may be stored.
As the target parking position coordinate data, coordinate data of the target parking position 30 in the map coordinate system and angle data indicating the fore-and-aft direction 32 of the parking space where the vehicle is to be parked at the target parking position 30 are stored. As the target parking position coordinate data, the own position of the vehicle 1 in the map coordinate system and the fore-and-aft direction of the vehicle body when the vehicle 1 is located at the target parking position 30 may be stored.

Figure 2(b) is an explanatory diagram of an example of processing when parking assistance is performed. The parking assistance device 10 starts parking assistance for the vehicle 1 when the vehicle 1 is positioned near a registered target parking position 30 and an occupant performs an operation (hereinafter sometimes referred to as an "activation operation") to instruct activation of parking assistance control for the vehicle 1 toward the target parking position 30. The activation operation may be, for example, operation of a "parking assistance activation switch" provided in the HMI 13, or a shift operation to switch between forward and reverse movement of the vehicle 1 (for example, an operation to switch from drive range (D range) to reverse range (R range) or from R range to D range).

The parking assistance device 10 extracts landmarks around the host vehicle 1 from a surrounding image obtained by capturing images of the surroundings of the host vehicle 1 with a camera. In the following description, the landmarks around the host vehicle 1 extracted during parking assistance are referred to as "surrounding landmarks." In Figure 2(b), triangular plots represent surrounding landmarks. The parking assistance device 10 matches the learned landmarks stored in the memory device 21 with the surrounding landmarks and associates identical feature points. The parking assistance device 10 then calculates the relative position of the host vehicle 1 with respect to the target parking position 30 based on the relative positional relationship between the surrounding landmarks detected during parking assistance and the host vehicle 1, and the relative positional relationship between the learned landmarks associated with the surrounding landmarks and the target parking position 30. For example, the parking assistance device 10 calculates the position of the target parking position 30 in a coordinate system based on the current position of the host vehicle 1 (hereinafter referred to as the "vehicle coordinate system"). In addition, when the coordinates of the learned targets and the target parking position 30 on the map coordinate system are stored in the storage device 21, the coordinates of the target parking position 30 on the map coordinate system may be converted into coordinates on the vehicle coordinate system based on the positions of the surrounding targets detected when parking assistance is performed and the positions of the learned targets in the map coordinate system. The own position of the vehicle 1 on the map coordinate system may be obtained based on the positions of the surrounding targets detected when parking assistance is performed and the positions of the learned targets in the map coordinate system, and the relative position of the vehicle 1 with respect to the target parking position 30 may be calculated from the difference between the coordinates of the vehicle 1 and the coordinates of the target parking position 30 in the map coordinate system.

The parking assistance device 10 calculates a target driving trajectory 34 from the current position 33 of the vehicle 1 to the target parking position 30 based on the relative position of the vehicle 1 with respect to the target parking position 30. The parking assistance device 10 performs parking assistance control of the vehicle 1 based on the calculated target driving trajectory 34.
Furthermore, the parking assistance device 10 generates an overhead image (around view monitor image) of the surroundings of the vehicle 1 from the captured image generated by the camera, and displays a target parking frame representing a parking space for parking the vehicle at the target parking position 30 on the overhead image on the display device of the HMI 13. The target parking frame is an example of a "guide figure representing an area of a predetermined size surrounding the target parking position" as set forth in the claims.

The functional configuration of the controller 17 will be described in more detail below. FIG. 3 is a block diagram showing an example of the functional configuration of the controller 17.
The HMI control unit 40 detects an operation by the occupant to register a target parking position 30 and an operation to activate parking assist control for the registered target parking position 30. When the registration operation is detected, the HMI control unit 40 outputs a map generation command to the map generation unit 45 to store learned target data in the storage device 21. When the activation operation is detected, the HMI control unit 40 outputs a control start command to the parking assist control unit 41 to start parking assist control of the host vehicle 1 to the target parking position 30. The image conversion unit 42 converts the captured image from the camera into a bird's-eye view image seen from a virtual viewpoint directly above the host vehicle 1. The image conversion unit 42 converts the captured image into a bird's-eye view image at predetermined intervals (for example, every time the host vehicle 1 travels a predetermined distance (for example, 50 cm) or a predetermined time (for example, 1 second)), and generates a surrounding image that is an image of the area around the host vehicle 1 by accumulating the converted bird's-eye view images along the travel route of the host vehicle 1.

The self-position calculation unit 43 calculates the self-position and attitude (e.g., the longitudinal direction of the vehicle body), which are the current position of the vehicle 1 on the map coordinate system, using odometry (e.g., dead reckoning) based on vehicle information output from the vehicle sensor 16. The self-position calculation unit 43 corrects the calculation results of the self-position and attitude based on the detection results of the self-position and attitude by the positioning device 11. The target detection unit 44 detects targets from the surrounding image output from the image conversion unit 42. The target detection unit 44 may detect the positions of feature points of the targets and their image feature quantities as targets. Methods such as SIFT, SURF, ORB, BRIAK, KAZE, and AKAZE can be used to detect the positions of feature points and calculate image feature quantities. The target detection unit 44 outputs the detected positions of feature points and image feature quantities as target data to the map generation unit 45 and the matching unit 47. In addition, the self-position acquired from the self-position calculation unit 43 in synchronization with the detection of the target object is output to the map generation unit 45 and the matching unit 47 .

When the map generation unit 45 receives a map generation command from the HMI control unit 40 (i.e., when the target parking position 30 is registered), the map generation unit 45 generates learned target data and stores it in the storage device 21 as map data 46. For example, the map generation unit 45 receives, from the target detection unit 44, the target data and the vehicle's own position on the map coordinate system synchronized with the target data.
The map generation unit 45 acquires coordinate data of the target parking position 30 in the map coordinate system and angle data in the fore-and-aft direction 32 of the parking space where the vehicle is to be parked at the target parking position 30. For example, when the vehicle 1 is located at the target parking position 30, the map generation unit 45 may acquire the own position calculated by the own position calculation unit 43 as the coordinate data of the target parking position 30, and may acquire the angle of the vehicle body in the fore-and-aft direction 32 as the angle data in the fore-and-aft direction 32.
The map generation unit 45 generates relative position data based on the positions of the feature points included in the target object data, the position information of the host vehicle 1 synchronized therewith, and the position data of the target parking position 30. The map generation unit 45 also acquires feature amount data from the target object data. The position data of the target parking position 30 and the angle data of the parking space in the fore-aft direction 32 are used as target parking position coordinate data. The learned target object data including the relative position data, feature amount data, and target parking position coordinate data is stored in the storage device 21 as map data 46.

When the parking assist control unit 41 receives a control start command from the HMI control unit 40 (i.e., when the parking assist control is started), it outputs a parking position calculation command to the matching unit 47. The matching unit 47 receives the target data output from the target detection unit 44 as target data of surrounding targets, and also receives the own position of the vehicle 1 in the map coordinate system in synchronization therewith.
The matching unit 47 reads out the learned object data stored in the storage device 21 as map data 46, and determines whether the host vehicle 1 is located near the registered target parking position 30 based on the target parking position coordinate data. If the host vehicle 1 is located near the registered target parking position 30, the matching unit 47 matches the learned objects stored in the map data 46 with the surrounding objects to associate objects with each other that have the same characteristic points. The matching unit 47 calculates the current relative position of the host vehicle 1 with respect to the target parking position 30 based on the relative positional relationship between the surrounding objects and the host vehicle 1 and the relative positional relationship between the learned objects associated with the surrounding objects and the target parking position 30. For example, the surrounding objects are represented as ( _xi , _yi ), and the learned objects associated with the surrounding objects ( _xi , _yi ) are represented as ( _xmi , _ymi ) (i = 1 to N). The matching unit 47 calculates the affine transformation matrix M _affine1 using the following equation based on the least squares method.

The matching unit 47 converts the position (targetx _m , targety _m ) of the target parking position 30 on the map coordinate system stored in the map data 46 into a position (targetx, targety) in the vehicle coordinate system based on the following equation.

Furthermore, the matching unit 47 converts the angle targetyaw _m indicating the front-rear direction 32 of the parking space of the target parking position 30 on the map coordinate system into an angle targetyaw indicating a direction on the vehicle coordinate system based on the following equation.

When the target trajectory generation unit 48 receives a driving trajectory calculation command from the parking assist control unit 41, it calculates a target driving trajectory from the current position of the host vehicle 1 on the vehicle coordinate system to the target parking position 30, and a target vehicle speed profile. When the steering control unit 49 receives a steering control command from the parking assist control unit 41, it controls the steering actuator 19a so that the host vehicle 1 drives along the target driving trajectory. When the vehicle speed control unit 50 receives a vehicle speed control command from the parking assist control unit 41, it controls the accelerator actuator 19b and the brake actuator 19c so that the vehicle speed of the host vehicle 1 changes in accordance with the target vehicle speed profile. When the host vehicle 1 reaches the target parking position 30 and the parking assist control is completed, the parking assist control unit 41 activates the parking brake 18 and switches the shift position to the parking range (P range).

While parking assist control is being performed, the HMI control unit 40 receives an overhead image of the surroundings of the vehicle 1 from the image conversion unit 42 and displays it on the display device of the HMI 13. The HMI control unit 40 also receives information on the coordinates (targetx, targety) of the target parking position 30 and the angle targetyaw in the fore-and-aft direction 32 from the abutment unit 47, and displays a target parking frame representing a parking space where the vehicle will be parked at the target parking position 30 by superimposing it on the overhead image.
4A is a schematic diagram of an example of a display area in which an overhead image is displayed on the display device of the HMI 13. Reference numeral 60 denotes a display screen of the display device of the HMI 13. An overhead image 62 is displayed within a predetermined display area 61 on the display screen 60. For convenience, the following description will exemplify a case in which the reference point Pr in the lower left corner of the display area 61 is set as the coordinate origin (0,0), and the rightward and upward directions are set as the positive x-axis and positive y-axis directions, respectively. However, the setting of the coordinate system is not limited to this example and may be changed as appropriate.

Display area 61 is a pre-defined area within display screen 60. The number of pixels in display area 61 in the x-axis and y-axis directions is Wavm and Havm, respectively. For example, the number of pixels in display area 61 in the x-axis and y-axis directions may be set equal to the number of pixels in the x-axis and y-axis directions of overhead image 62 output from image conversion unit 42 to HMI control unit 40, respectively. In FIG. 4A , overhead image 62 is drawn slightly smaller than display area 61 to make the boundary of overhead image 62 easier to see, but overhead image 62 may also be displayed filling display area 61 (i.e., the number of pixels in the x-axis and y-axis directions of overhead image 62 displayed in display area 61 may be Wavm and Havm, respectively).
The overhead image 62 is an image in which an icon 63 representing the vehicle 1 is superimposed at the center of the overhead image, which represents the range of the longitudinal length DL and the vehicle width length DT centered on the current position of the vehicle 1. The longitudinal length DL and the vehicle width length DT are appropriately set so that the occupant can grasp the sense of distance from the features (road markings such as white lines and road boundaries such as shoulders) around the vehicle 1 shown in the overhead image 62.
In the following description, the display position for displaying the overhead image 62 in the display area 61 is designated as the position where the lower left corner of the overhead image 62 is displayed (position Pb1 in the example of FIG. 4A).

The HMI control unit 40 calculates the position where the target parking space 64 is superimposed and displayed on the overhead image 62. Specifically, the target parking position Pt in the coordinate system on the display screen 60 is calculated based on the coordinates (targetx, targety) of the target parking position 30 received from the matching unit 47 and the scale at which the overhead image 62 is displayed on the display screen 60. In addition, the HMI control unit 40 sets the display position and direction of the target parking space 64 in the coordinate system on the display screen 60 based on the angle targetyaw and the target parking position Pt received from the matching unit 47. For example, the target parking position Pt is set to the center position of the target parking space 64, and the direction indicated by the angle targetyaw is set to the front-rear direction of the target parking space 64.
The size (front-rear length and lateral length) of the target parking space 64 displayed on the display screen 60 may be set, for example, according to the scale of the overhead-view image 62 and the size of the host vehicle 1. For example, the size of the target parking space 64 may be set to be approximately equal to the size of the area occupied by the host vehicle 1 in the overhead-view image 62. Also, the size of the target parking space 64 may be set slightly larger than the area occupied by the host vehicle 1 to provide a margin, or may be set slightly smaller than the area occupied by the host vehicle 1. Also, the size of the target parking space 64 when the overhead-view image 62 is displayed to fill the display area 61 (i.e., when the overhead-view image 62 is displayed with pixel numbers Wavm and Havm in the x-axis and y-axis directions, respectively) may be set in advance as an initial value.

Here, when the parking assist control is started in a state where the host vehicle 1 is away from the target parking position 30, the display position of the target parking frame 64 may be set outside the range of the overhead image 62 as shown in Fig. 4(a). In this case, the relative position of the target parking frame 64 with respect to the icon 63 representing the host vehicle 1 cannot be correctly grasped.
Therefore, when the bird's-eye view image 62 is displayed at the first display position Pb1 of the display screen 60, if a predetermined range or more of the target parking space 64 can be displayed within the display area 61, the HMI control unit 40 displays the bird's-eye view image 62 at the first display position Pb1 of the display screen 60. For example, when the display position for displaying the bird's-eye view image 62 is set to the first display position Pb1, it is determined whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61. In the example of FIG. 4A , when the bird's-eye view image 62 is displayed to fill the display area 61 (i.e., when the bird's-eye view image 62 has pixel counts Wavm and Havm in the x-axis and y-axis directions, respectively, and the first display position Pb1 of the bird's-eye view image 62 is set to the reference point Pr in the lower left corner of the display area 61), it is determined whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61. In the following description, the predetermined range of the target parking space 64 may be referred to as a "required display range."

On the other hand, when the overhead image 62 is displayed at the first display position Pb1 on the display screen 60, if more than a predetermined range of the target parking space 64 cannot be displayed within the display area 61, the overhead image 62 is displayed at a second display position Pb2 on the display screen 60, which is different from the first display position Pb1, as shown in FIG. 4(b). This makes it possible to display more than the necessary display range of the target parking space 64 within the display area 61. In this case, for example, the portion of the overhead image 62 that falls outside the display area 61 due to a change in display position and the portion of the target parking space 64 that does not fit within the display area 61 may be cut out, and the remaining portion may be displayed within the display area 61.

The HMI control unit 40 determines the offset direction (movement direction) and offset amount (movement distance) of the display position of the overhead image 62 relative to the reference position (i.e., first display position Pb1) so that more than the required display range of the target parking space 64 can be displayed within the display area 61, and calculates the second display position Pb2 by offsetting the first display position Pb1 in the offset direction by the offset amount. See Figure 4(c). The offset direction of the display position of the overhead image 62 is determined by the position of the target parking position Pt when the overhead image 62 is displayed to fill the display area 61. Region R1 is the region where the x coordinate is less than 0 and the y coordinate is greater than Havm, region R2 is the region where the x coordinate is between 0 and Wavm but less than Wavm and the y coordinate is greater than Havm, and region R3 is the region where the x coordinate and y coordinate are greater than Wavm and Havm, respectively. Region R4 is a region where the x coordinate is less than 0 and the y coordinate is between 0 and Havm, region R5 is a region where the x coordinate is greater than Wavm and the y coordinate is between 0 and Havm, region R6 is a region where both the x coordinate and the y coordinate are less than 0, region R7 is a region where the x coordinate is between 0 and Wavm and the y coordinate is less than 0, and region R8 is a region where the x coordinate is greater than Wavm and the y coordinate is less than 0.

For example, if the target parking position Pt is in region R1, the offset direction of the display position of the overhead image 62 may be downward and to the right (increasing the x coordinate and decreasing the y coordinate); if the target parking position Pt is in region R2, the offset direction of the display position of the overhead image 62 may be downward (decreasing the y coordinate without changing the x coordinate); and if the target parking position Pt is in region R3, the offset direction of the display position of the overhead image 62 may be downward and to the left (decreasing the x coordinate and y coordinate). If the target parking position Pt is in region R4, the offset direction of the display position of the overhead image 62 may be right (increasing the x coordinate without changing the y coordinate); and if the target parking position Pt is in region R5, the offset direction of the display position of the overhead image 62 may be left (decreasing the x coordinate without changing the y coordinate). When the target parking position Pt is in region R6, the offset direction of the display position of the overhead image 62 may be set to the upper right direction (increasing the x-coordinate and y-coordinate), when the target parking position Pt is in region R7, the offset direction of the display position of the overhead image 62 may be set to the upper right direction (increasing the y-coordinate without changing the x-coordinate), and when the target parking position Pt is in region R8, the offset direction of the display position of the overhead image 62 may be set to the upper left direction (decreasing the x-coordinate and increasing the y-coordinate).

4(b) shows an example of setting the offset direction and offset amount of the display position of the overhead image 62 so that an area including the center of the target parking frame 64 (for example, the target parking position Pt) or more can be displayed within the display area 61 as the required display range of the target parking frame 64. In this example, the x-direction offset amount ofx is set to Px, and the y-direction offset amount ofy is set to Py based on the coordinates (-Px, -Py) of the target parking position Pt when the overhead image 62 is displayed to fill the display area 61 in FIG.
The required display range of the target parking frame 64 is not limited to the above example, and may be appropriately set so that the target parking frame 64 displayed on the display screen 60 is easily understood by the occupant. For example, the required display range may be a range including a part of at least two sides 64a, 64b of the rectangular target parking frame 64 (FIG. 5(a)), a range including the entire length of at least two sides 64a, 64b of the rectangular target parking frame 64 (FIG. 5(b)), a range including the entire length of at least one short side 64a of the rectangular target parking frame 64 (FIG. 5(c)), or a range including all four sides of the rectangular shape (FIG. 5(d)).
For example, the HMI control unit 40 may calculate the coordinates of the four corners of the rectangular target parking space 64 based on the target parking position Pt and the angle targetyaw, and determine the offset direction and offset amount of the display position of the overhead image 62 so that more than the required display range can be displayed within the display area 61.

(operation)
6 is a flowchart of an example of the parking assistance method of the first embodiment. In step S1, the HMI control unit 40 calculates the target parking position Pt in the coordinate system on the display screen 60. Also, it calculates the display position and direction of the target parking space 64. In step S2, the HMI control unit 40 determines whether or not a predetermined range or more of the target parking space 64 can be displayed within the display area 61 when the overhead image 62 is displayed at the first display position Pb1. If a predetermined range or more of the target parking space 64 cannot be displayed within the display area 61 (step S2: N), the process proceeds to step S3. If a predetermined range or more of the target parking space 64 can be displayed within the display area 61 (step S2: Y), the process proceeds to step S7.
In step S3, the HMI control unit 40 calculates an offset amount (ofx, ofy) of the display position of the overhead image 62. In step S4, the HMI control unit 40 calculates the display position of the target parking frame 64 after offset by the offset amount (ofx, ofy).

In step S5, the HMI control unit 40 displays the overhead image 62 at the second display position Pb2, which is a display position offset by the offset amount (ofx, ofy). In step S6, the HMI control unit 40 displays the target parking space 64 superimposed on the display position offset by the offset amount (ofx, ofy). Then, the process ends.
In step S7, the HMI control unit 40 displays the overhead image 62 at the first display position Pb1. In step S8, the HMI control unit 40 displays the target parking space 64 superimposed on the position calculated in step S1. Thereafter, the process ends.

Second Embodiment
If the display position is offset while maintaining the size of the overhead image 62 (i.e., the scale of the overhead image 62), a part of the overhead image 62 may extend outside the display area 61 and become unable to be displayed on the display screen 60. Therefore, in the second embodiment, the overhead image 62 displayed in the display area 61 is reduced in size, as shown in FIG.
This allows the overhead image 62 to be displayed at the second display position Pb2 so as to include the entire range that can be displayed in the display area 61 when the overhead image 62 is displayed at the first display position Pb1 (i.e., the range of the longitudinal length DL and the transverse length DT centered on the current position of the vehicle 1) as shown in Figure 4 (a).

For example, when the x-direction offset amount ofx is smaller than the y-direction offset amount ofy, the HMI control unit 40 calculates the reduction ratio scale using the following formula.
scale=(Havm-ofy)/Havm
When the x-direction offset amount ofx is larger than the y-direction offset amount ofy, the HMI control unit 40 calculates the reduction ratio scale using the following formula.
scale=(Wavm-ofx)/Wavm
The HMI control unit 40 offsets the display positions of the overhead image 62 and the target parking space 64 by an x-direction offset amount ofx and a y-direction offset amount ofy, and sets an affine transformation matrix M _affine2 that reduces the overhead image 62 and the target parking space 64 by a reduction rate scale using the following equation.

The HMI control unit 40 transforms the coordinates of the overhead-view image 62 and the target parking space 64 using an affine transformation matrix M _affine2 . Specifically, if the coordinates of each pixel of the overhead-view image 62 before transformation and the coordinates of the boundary of the target parking space 64 are (Px1, Py1), the HMI control unit 40 calculates the coordinates of each pixel of the overhead-view image 62 after offset and reduction and the coordinates of the boundary of the target parking space 64 (Px2, Py2) using the following equations.

The HMI control unit 40 may also draw grid lines 65 in the background portion of the display area 61 (the area where the overhead image 62 is not displayed) as shown in Figure 7(a). For example, the HMI control unit 40 may display in the display area 61 an image obtained by adding a background image 66 with a drawn grid to the overhead image 62 as shown in Figure 7(b).

Fig. 8 is a flowchart of an example of the parking assistance method of the second embodiment. The processes of steps S10 and S11 are the same as the processes of steps S1 and S2 in Fig. 6. If the predetermined range or more of the target parking space 64 cannot be displayed within the display area 61 (step S11: N), the process proceeds to step S12. If the predetermined range or more of the target parking space 64 can be displayed within the display area 61 (step S11: Y), the process proceeds to step S17.
The process of step S12 is the same as step S3 in Fig. 6. In step S13, the HMI control unit 40 calculates the reduction rate "scale". In step S14, the HMI control unit 40 calculates the display position of the target parking space 64 when the overhead image 62 is offset by the offset amount (ofx, ofy) and reduced by the reduction rate "scale".

In step S15, the HMI control unit 40 displays the overhead image 62 offset by the offset amount (ofx, ofy) and reduced by the reduction rate scale. In step S16, the HMI control unit 40 displays the target parking space 64 offset by the offset amount (ofx, ofy) and reduced by the reduction rate scale in a superimposed manner. Thereafter, the processing ends.
The processes in steps S17 and S18 are the same as those in steps S7 and S8 in FIG.

(Modification)
In the first and second embodiments, examples have been described in which targets around the pre-stored target parking position 30 are matched with targets detected around the vehicle 1 during execution of parking assist control to detect the relative position of the target parking position 30 with respect to the current position of the vehicle 1, but the parking assist method of the present invention is not limited to such a method of determining the target parking position. For example, during execution of parking assist control, targets indicating a parking space (for example, road markings such as parking frame lines or three-dimensional objects such as wheel chocks) may be detected to detect the relative position of the target parking position 30 with respect to the current position of the vehicle 1.

(Effects of the embodiment)
(1) A parking assistance method that sets a target parking position where vehicle 1 will park, generates an overhead image of the surroundings of vehicle 1 from captured images obtained by photographing the surroundings of vehicle 1, and displays the overhead image and a guide figure representing an area of a predetermined size surrounding the target parking position in a predetermined display area of a display screen that is visible to an occupant of vehicle 1. When the overhead image is displayed at a first display position on the display screen, if more than the predetermined range of the guide figure can be displayed within the display area, the overhead image is displayed at the first display position on the display screen. When the overhead image is displayed at the first display position on the display screen, if more than the predetermined range of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen that is different from the first display position.
This allows the bird's-eye view image of the surroundings of the vehicle 1 and a predetermined range or more of the guide figure to be displayed simultaneously even if the vehicle is far from the target parking position, so that an image that makes it easy to grasp the relative position of the target parking frame with respect to the vehicle can be provided.

(2) When the overhead image is displayed at the second display position, the overhead image displayed in the display area may be reduced in size, thereby preventing a reduction in the display range of the overhead image after the display position of the overhead image is changed.
(3) By reducing the size of the overhead image displayed in the display area, an overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position may be displayed in the second display position. This allows the entire range of the overhead image that could be displayed before the display position of the overhead image to be displayed even when the display position of the overhead image is changed.

(4) A background image with a grid drawn on it may be added to the overhead image and displayed in the display area, which makes it easier to grasp the sense of distance to the guide figure displayed outside the overhead image.
(5) The predetermined range of the guide figure may be a range that includes a portion of each of at least two sides of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes the entire length of each of at least two sides of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes the entire length of at least one short side of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes all four sides of the rectangular guide figure. The predetermined range of the guide figure may also be a range that includes the center point of the guide figure. This allows the guide figure to be displayed in a way that is easy for the occupant to understand.

(6) The target parking position may be detected based on captured images of the surroundings of the vehicle 1, a target driving trajectory from the current position of the vehicle 1 to the target parking position may be calculated, and the vehicle 1 may be controlled to drive along the target driving trajectory. This may assist in parking the vehicle 1.

All examples and conditional language described herein are intended for educational purposes to aid the reader in understanding the present invention and the concepts provided by the inventors for the advancement of technology, and should be construed without limitation to the specifically described examples and conditions above, and the configuration of examples herein for illustrating the advantages and disadvantages of the present invention. While embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present invention.

1...Own vehicle, 10...Parking assistance device, 11...Positioning device, 13...Human machine interface (HMI), 14...Shift switch (shift SW), 15...External sensor, 16...Vehicle sensor, 17...Controller, 18...Parking brake, 19a...Steering actuator, 19b...Accelerator actuator, 19c...Brake actuator, 20...Processor, 21...Storage device, 40...HMI control unit, 41...Parking assistance control unit, 42...Image conversion unit, 43...Self-position calculation unit, 44...Target detection unit, 45...Map generation unit, 46...Map data, 47...Matching unit, 48...Target trajectory generation unit, 49... Steering control unit, 50 ...Vehicle speed control unit

Claims (11)

A parking assistance method comprising: setting a target parking position where a vehicle is to be parked; generating an overhead image of the surroundings of the vehicle from captured images obtained by capturing images of the surroundings of the vehicle; and displaying the overhead image and a guide figure representing an area of a predetermined size surrounding the target parking position in a predetermined display area of a display screen visible to an occupant of the vehicle;
When a bird's-eye view image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the bird's-eye view image is displayed at the first display position on the display screen;
When an overhead image is displayed at the first display position on the display screen, if the predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen that is different from the first display position;
A parking assistance method, comprising: when displaying an overhead image at the second display position, reducing the overhead image displayed in the display area. The parking assistance method according to claim 2, characterized in that the overhead image displayed in the display area is reduced, so that the overhead image including the entire range that can be displayed in the display area when the display position of the overhead image is set to the first display position is displayed in the second display position. A parking assistance method comprising: setting a target parking position where a vehicle is to be parked; generating an overhead image of the surroundings of the vehicle from captured images obtained by capturing images of the surroundings of the vehicle; and displaying the overhead image and a guide figure representing an area of a predetermined size surrounding the target parking position in a predetermined display area of a display screen visible to an occupant of the vehicle;
When a bird's-eye view image is displayed at a first display position on the display screen, if a predetermined range or more of the guide figure can be displayed within the display area, the bird's-eye view image is displayed at the first display position on the display screen;
When an overhead image is displayed at the first display position on the display screen, if the predetermined range or more of the guide figure cannot be displayed within the display area, the overhead image is displayed at a second display position on the display screen that is different from the first display position;
A parking assistance method characterized in that a background image on which a grid is drawn is added to the overhead image and displayed in the display area. A parking assistance method as described in claim 2 or 4, characterized in that the specified range of the guide figure is a range that includes a portion of each of at least two sides of the guide figure, which is rectangular in shape. A parking assistance method as described in claim 2 or 4, characterized in that the specified range of the guide figure is a range that includes the total length of each of at least two sides of the guide figure, which is rectangular in shape. A parking assistance method as described in claim 2 or 4, characterized in that the specified range of the guide figure is a range that includes the entire length of at least one short side of the guide figure, which is rectangular in shape. A parking assistance method as described in claim 2 or 4, characterized in that the specified range of the guide figure is a range that includes all four sides of the guide figure, which is rectangular in shape. A parking assistance method as described in claim 2 or 4, characterized in that the specified range of the guide figure is a range that includes the center point of the guide figure. A parking assistance method as described in any one of claims 2 to 9, characterized in that the target parking position is detected based on an image obtained by photographing the surroundings of the vehicle, a target driving trajectory from the current position of the vehicle to the target parking position is calculated, and the vehicle is controlled to drive along the target driving trajectory. A parking assistance device comprising: a camera that captures images of the surroundings of a vehicle; and a display device having a display screen that can be viewed by an occupant of the vehicle; the parking assistance device sets a target parking position where the vehicle will be parked; generates an overhead image of the surroundings of the vehicle from an image captured by the camera; and displays the overhead image and a guide figure that represents a range of a predetermined size surrounding the target parking position in a predetermined display area of the display screen,
a controller that displays an overhead image at a first display position on the display screen when a predetermined range or more of the guide figure can be displayed within the display area, displays an overhead image at the first display position on the display screen, and when a predetermined range or more of the guide figure cannot be displayed within the display area when an overhead image is displayed at the first display position on the display screen, displays an overhead image at a second display position on the display screen that is different from the first display position, and reduces the overhead image displayed in the display area when the overhead image is displayed at the second display position. A parking assistance device comprising: a camera that captures images of the surroundings of a vehicle; and a display device having a display screen that can be viewed by an occupant of the vehicle; the parking assistance device sets a target parking position where the vehicle will be parked; generates an overhead image of the surroundings of the vehicle from an image captured by the camera; and displays the overhead image and a guide figure that represents a range of a predetermined size surrounding the target parking position in a predetermined display area of the display screen,
a controller that displays an overhead image at a first display position on the display screen when a predetermined range or more of the guide figure can be displayed within the display area when the overhead image is displayed at a first display position on the display screen, and displays an overhead image at a second display position on the display screen that is different from the first display position when a predetermined range or more of the guide figure cannot be displayed within the display area when the overhead image is displayed at the first display position on the display screen, and adds a background image on which a grid is drawn to the overhead image and displays it in the display area.