JP7743726B2 - Parking assistance method and parking assistance device - Google Patents

Description

The present invention relates to a parking assistance method and a parking assistance device.

Patent Document 1 describes a parking assistance device that detects parking spaces and surrounding landmarks and parks the vehicle in a target parking position.

Japanese Patent Application Laid-Open No. 2008-174000

When calculating a target driving trajectory to a target parking position by storing target positions around the target parking position, the more target positions are stored, the more accurate the target driving trajectory can be calculated. However, storing many target positions increases the required storage capacity, which leads to increased costs.
An object of the present invention is to reduce the number of target positions to be stored in order to calculate a target driving trajectory to a target parking position.

In one aspect of the parking assistance method of the present invention, target positions are detected, which are the positions of targets present around the vehicle. When the driver manually turns the vehicle to park at the target parking position, only those of the detected target positions within a predetermined distance range from the turning position where the vehicle will turn are stored. The relative positional relationship between the stored target positions and the target parking position is stored. The relative positional relationship between the detected target positions and the vehicle and the relative positional relationship between the stored target positions and the target parking position is calculated. The target driving trajectory from the current position of the vehicle to the target parking position via the turning position is calculated based on the relative positional relationship between the current position of the vehicle and the target parking position. Parking assistance for the vehicle is provided based on the calculated target driving trajectory.

This invention reduces the number of target positions that need to be stored to calculate the target driving trajectory to the target parking position.

1 is a diagram illustrating an example of a schematic configuration of a parking assistance device according to an embodiment; 10A is a diagram illustrating an example of a process for storing a target learning position, and FIG. 10B is a diagram illustrating an example of a process when parking assistance is performed. FIG. 2 is a block diagram illustrating an example of a functional configuration of a controller in FIG. 1 . 10A and 10B are diagrams illustrating an example of a process for extracting common feature points from two frames that are images captured at different vehicle positions. 10 is a flowchart of an example of a process for storing a target learning position. 10 is a flowchart illustrating an example of processing when parking assistance is performed.

Embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, identical or similar parts will be designated by identical or similar reference numerals, and redundant explanations will be omitted. The drawings are schematic and may differ from the actual product. The embodiments shown below exemplify devices and methods that embody the technical concept of the present invention, but the technical concept of the present invention is not limited to the devices and methods exemplified in the following embodiments. The technical concept of the present invention can be modified in various ways within the technical scope described in the claims.

(composition)
Referring to Fig. 1, a vehicle 1 is equipped with a parking assistance device 10 that assists the vehicle 1 in parking at a target parking position.
In parking assistance by the parking assistance device 10, a target driving trajectory from the current position of the vehicle 1 to the target parking position is calculated, and assistance is provided to the vehicle 1 to drive along the target driving trajectory.
Parking assistance by the parking assistance device 10 includes various forms of assistance in driving the host vehicle 1 along a target driving trajectory. For example, parking of the host vehicle 1 may be assisted by performing automatic driving, in which the host vehicle 1 is controlled to drive to a target parking position along the target driving trajectory of the host vehicle 1. Note that automatic driving, in which the host vehicle 1 is controlled to drive to a target parking position along the target driving trajectory of the host vehicle 1, refers to control that automatically controls all or part of the steering angle, driving force, and braking force of the host vehicle to drive all or part of the host vehicle 1 along the target driving trajectory and assists the occupant in parking the vehicle 1. Furthermore, for example, parking of the host vehicle 1 may be assisted by displaying the target driving trajectory and the current position of the host vehicle 1 on a display device that is visible to the occupant of the host vehicle 1.

The parking assistance device 10 includes a positioning device 11, a human-machine interface 12, a shift switch 13, an external sensor 14, a vehicle sensor 15, a controller 16, a parking brake 17, a steering actuator 18a, an accelerator actuator 18b, and a brake actuator 18c. In the drawings, the human-machine interface is referred to as "HMI" and the shift switch is referred to as "shift SW."

The positioning device 11 measures the current position of the vehicle 1. The positioning device 11 may include, for example, a Global Navigation System (GNSS) receiver. The GNSS receiver is, for example, a Global Positioning System (GPS) receiver, and receives radio waves from multiple navigation satellites to measure the current position of the vehicle 1.
The human-machine interface 12 is an interface device that exchanges information between the parking assistance device 10 and the occupant. The human-machine interface 12 includes a display device (e.g., a display screen of a navigation system) that can be seen by the occupant of the vehicle 1, and a speaker and buzzer for outputting warning sounds, notification sounds, and audio information.
The human-machine interface 12 also includes an operator that accepts an operation input from the occupant to the parking assistance device 10. The operator may be a mechanical interface device such as a button, switch, lever, dial, or keyboard, or may be a button, switch, lever, dial, or keyboard displayed on a touch panel.

The shift switch 13 is a switch that allows the driver or the parking assistance device 10 to switch the shift position of the vehicle 1 .
The external sensor 14 detects objects within a predetermined distance range (e.g., a detection area of the external sensor 14) from the vehicle 1. The external sensor 14 detects the surrounding environment of the vehicle 1, such as the relative position between the vehicle 1 and objects present around the vehicle 1, the distance between the vehicle 1 and the objects, and the direction in which the objects are present.

The external sensor 14 may include, for example, a camera that captures an image of the surrounding environment of the vehicle 1. The camera may be, for example, an around-view monitor camera that captures an image of the surroundings of the vehicle 1 and generates a captured image that is converted into a bird's-eye view (around-view monitor image).
The external sensor 14 may include a distance measuring device such as a laser range finder (LRF), radar, or LiDAR (Light Detection and Ranging) laser radar.

The vehicle sensor 15 detects various information (vehicle information) obtained from the host vehicle 1. The vehicle sensor 15 may include, for example, a vehicle speed sensor that detects the traveling speed (vehicle speed) of the host vehicle 1, a wheel speed sensor that detects the rotational speed of each tire equipped on the host vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) of the host vehicle 1 in three axial directions, a steering angle sensor that detects the steering angle (including the steering angle of the steering wheel or the turning angle of the steered wheels), a gyro sensor that detects the angular velocity generated in the host vehicle 1, and a yaw rate sensor that detects the yaw rate.

The controller 16 is an electronic control unit (ECU) that performs parking assistance control for the host vehicle 1. The controller 16 includes a processor 20 and peripheral components such as a storage device 21. The processor 20 may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit).
The storage device 21 may include a semiconductor storage device, a magnetic storage device, an optical storage device, etc. The storage device 21 may include memories such as a register, a cache memory, a ROM (Read Only Memory) used as a main memory device, and a RAM (Random Access Memory).
The functions of the controller 16 described below are realized, for example, by the processor 20 executing a computer program stored in the storage device 21 .

The controller 16 may be formed of dedicated hardware for executing each of the information processes described below.
For example, the controller 16 may include a functional logic circuit configured in a general-purpose semiconductor integrated circuit, or may include a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

The steering actuator 18a controls the steering direction and steering amount of the steering mechanism of the vehicle 1 in response to a control signal from the controller 16. The accelerator actuator 18b controls the accelerator opening of the drive device, such as the engine or drive motor, in response to a control signal from the controller 16. The brake actuator 18c activates the braking device in response to a control signal from the controller 16.

Next, an example of parking assist control by the parking assist device 10 of the embodiment will be described.
2A, when the driver uses the parking assistance provided by the parking assistance device 10, the target position of a target object that exists near the target parking position 35, which is the target position for parking the vehicle 1, is stored in the storage device 21.
A landmark is a feature that serves as a landmark for identifying the current position of the vehicle 1, and may be road markings (road signs, dividing lines, and road markings) painted on the road surface around the vehicle 1, or obstacles such as road boundaries, curbs, guardrails, and utility poles.

When storing the target position in the storage device 21, for example, the driver operates a "parking position learning switch" provided as an operator of the human-machine interface 12. Then, the driver manually parks the vehicle 1 at the target parking position 35.
Now, it is assumed that the vehicle 1 is moved forward along the track 33, then turned around at the turning position 31 to move backward along the track 34, and parked at the target parking position 35.

The external sensor 14 is a sensor that detects the position of a target that exists in a detection area within a predetermined detection distance range from the external sensor 14 .
For example, the parking assistance device 10 may detect, as feature points, images of portions representing targets such as road markings and road boundaries from an image captured by a camera serving as the external sensor 14, and may regard the positions of the feature points as the target positions. The parking assistance device 10 also detects feature quantities (e.g., shading, attributes, etc.) of the detected feature points.
Furthermore, for example, feature points and their feature amounts may be detected from point cloud information obtained by a laser range finder (LRF), radar, or LiDAR (Light Detection and Ranging), which is the external sensor 14. Hereinafter, in this specification, an example of detecting feature points from a captured image will be described.

2A, a dashed line 30 indicates the camera's image capture area when the host vehicle 1 is located at the position indicated by reference numeral 31. A dashed-dotted line 32 indicates the trajectory of the image capture area that moves as the host vehicle 1 moves. The circular plots represent feature points (i.e., target positions) detected in the camera's image capture area.
The term "reversal" refers to the action of switching the vehicle 1 between forward and reverse, and may include both switching from forward to reverse and switching from reverse to forward.
While the driver manually parks the vehicle 1 at the target parking position 35, the parking assistance device 10 stores the target positions detected by the external sensor 14 (for example, feature points and their feature amounts detected from an image captured by a camera) in the storage device 21. Hereinafter, the target positions stored in the storage device 21 may be referred to as "target learning positions."

The parking assistance device 10 also stores the relative positional relationship between the target learning position and the target parking position 35 .
For example, the driver may input to the parking assistance device 10 that the current position of the vehicle 1 is the target parking position 35 by operating the human-machine interface 12. The parking assistance device 10 may determine the relative positional relationship between the target learning position and the target parking position 35 based on the target position detected by the external sensor 14 when the vehicle 1 is located at the target parking position 35.

Further, for example, an image captured by a camera, which is the external sensor 14, may be displayed on the display device of the human-machine interface 12, and the driver may specify the position of the target parking position 35 on the captured image. The parking assistance device 10 may determine the relative positional relationship between the target learning position and the target parking position 35 based on the position of the specified target parking position 35 and the position of the target object shown in the captured image.
When storing the relative positional relationship between the target learning positions and the target parking position 35 in the storage device 21, for example, the positions of each target learning position and the target parking position 35 may be stored as coordinates on a coordinate system (hereinafter referred to as a "map coordinate system") that uses a fixed point as a reference point. The relative positional relationship between these target learning positions can be obtained from the difference in coordinates between the target learning positions and the target parking position 35 in the common map coordinate system.
When storing coordinates on a map coordinate system as the target learning position, the current position on the map coordinate system measured by the positioning device 11 when the vehicle 1 is located at the target parking position 35 may be stored as the target parking position 35.
Also, instead of using the map coordinate system, the relative positions of the target parking positions 35 with respect to each target learning position may be stored.

When the target learning position and the position of the target parking position 35 are stored in the storage device 21, parking assistance by the parking assistance device 10 becomes available.
2B, the parking assistance device 10 performs parking assistance for the host vehicle 1 when the host vehicle 1 is located near the target parking position 35.
For example, when the driver performs a shift operation to turn the vehicle 1, for example, when the driver's shift operation switches the shift position from drive range (hereinafter referred to as "D range") to reverse range (hereinafter referred to as "R range"), or when the shift position switches from R range to D range, parking assistance may be started.
For example, when the vehicle 1 is at a position 36 near the target parking position 35 (for example, near the entrance to the target parking position 35), parking assistance may be initiated when the driver operates a "parking assistance activation switch" provided as an operator on the human-machine interface 12.
Furthermore, for example, when it is detected that the vehicle 1 has reached a position 36 near the target parking position 35, parking assistance may be started automatically regardless of the driver's operation, and the conditions for starting parking assistance may be set arbitrarily.

At this time, the external sensor 14 detects the target position, which is the position of a target that exists in the detection areas 30, 32 of the external sensor 14. The target position detected by the external sensor 14 when parking assistance is being performed is shown by a triangular plot. Note that the detection of the target position may be performed at all times regardless of the shift operation or the operation of the parking assistance activation switch, and the timing of detecting the target position is not limited.
The parking assistance device 10 matches each target position (triangle plot) detected by the external sensor 14 with the target learning position (circle plot in Figure 2(a)) stored in the memory device 21, and associates the target positions detected for the same target with each other.

The parking assistance device 10 calculates the relative positional relationship between the current position of the vehicle 1 and the target parking position 35 based on the relative positional relationship between each target position (triangular plot) detected during parking assistance and the vehicle 1, and the relative positional relationship between the target learning position (circular plot) associated with these target positions (triangular plot) and the target parking position 35.
For example, the parking assistance device 10 may calculate the position of the target parking position 35 on a coordinate system (hereinafter referred to as the "vehicle coordinate system") based on the current position of the vehicle 1. For example, if the coordinates of the target parking position 30 and the target learning position (circle plot) on the map coordinate system are stored in the storage device 21, the parking assistance device 10 may convert the target parking position 30 on the map coordinate system to the target parking position 30 on the vehicle coordinate system based on the target position (triangle plot) on the vehicle coordinate system detected when parking assistance is performed and the target learning position (circle plot) on the map coordinate system.
Alternatively, the current position of the vehicle 1 on the map coordinate system can be determined based on the target position (triangle plot) in the vehicle coordinate system detected when parking assistance is performed and the target learning position (circle plot) in the map coordinate system, and the relative positional relationship between the current position of the vehicle 1 and the target parking position 30 can be calculated from the difference between the coordinates of the vehicle 1 and the coordinates of the target parking position 30 in the map coordinate system.
The parking assistance device 10 calculates a target driving trajectory from the current position of the vehicle 1 to the target parking position 35 via the turning position 31 based on the relative positional relationship between the current position of the vehicle 1 and the target parking position 35.

At this time, for example, if the position of the vehicle 1 at the time when parking assistance is started is the turning position 31, a trajectory 37 from the turning position 31 to the target parking position 35 is calculated as the target driving trajectory.
For example, if the position of the vehicle 1 at the time parking assistance is started is position 36 near the target parking position 35, a trajectory 38 proceeding from position 36 to the turning position 31 and a trajectory 37 from the turning position 31 to the target parking position 35 are calculated as the target driving trajectories.

In other words, the "target driving trajectory to reach the target parking position via the turning position" includes both the target driving trajectory 37 that starts from the turning position 31 and reaches the target parking position 35, and the target driving trajectories 38, 37 that proceed from a position 36 near the target parking position 35 to the turning position 31, turn at the turning position 31, and reach the target parking position 35.
Then, the parking assistance device 10 performs parking assistance for the host vehicle 1 based on the calculated target driving trajectory. As parking assistance for the host vehicle 1, the host vehicle 1 may be controlled to travel along the calculated target driving trajectory, or the calculated target driving trajectory may be displayed on the display device of the human-machine interface 12.

In this way, when the target positions around the target parking position are stored and the target driving trajectory to the target parking position is calculated, the more target positions are stored, the more accurate the target driving trajectory can be calculated. However, storing many target positions increases the required storage capacity, which leads to increased costs.
Therefore, the parking assistance device 10 stores in the storage device 21 as target learning positions only target positions within a predetermined distance range from the turning position 31 where the vehicle 1 turns during manual driving.

For example, only the target positions detected when the vehicle 1 is within a predetermined travel distance range (e.g., 10 meters) along the track 33 until it reaches the turning position 31 before turning, and within a predetermined travel distance range (e.g., 10 meters) along the track 34 from the turning position 31 after turning, may be stored in the memory device 21 as target learning positions, which are target positions within a predetermined distance range from the turning position 31.
Also, for example, only the target positions detected when the vehicle 1 is within a predetermined driving distance range (e.g., 10 meters) along the track 33 to the turning position 31 before the turning, and within a driving distance range along the track 34 from the turning position 31 to the target parking position 35 after the turning, may be stored in the memory device 21 as target learning positions, which are target positions within a predetermined distance range from the turning position 31.
In addition, among the target positions detected during manual driving, target positions within a predetermined distance range (for example, within a radius of 20 meters centered on the turning position) determined in advance from the turning position 31 may be stored in the memory device 21 as target learning positions.

For example, only the target position detected when the vehicle 1 is located in the range from the turning position 31 to the target parking position 35 along the track 34 after turning may be stored in the memory device 21 as a target learning position, which is a target position within a predetermined distance range from the turning position 31.
This allows the target positions near the turning position 31 required for calculating an accurate target traveling trajectory to be stored, thereby enabling the target traveling trajectory to be calculated with high precision. Also, by limiting the target positions to be stored, an increase in the required storage capacity can be avoided, thereby mitigating increases in costs.
In particular, when parking assistance begins, the target positions (triangular plots) detected by the external sensor 14 tend to coincide with the stored target positions (circular plots) near the turning position 31, so the relative positional relationship between the current position of the vehicle 1 near the turning position 31 and the target parking position 35 can be calculated with high accuracy.

The functional configuration of the controller 16 will be described in more detail below with reference to Figure 3. The controller 16 functions as a human-machine interface control unit (hereinafter referred to as "HMI control unit") 40, a parking assistance control unit 41, an image conversion unit 42, a vehicle position calculation unit 43, a feature point detection unit 44, a map generation unit 45, a matching unit 47, a target trajectory generation unit 48, a steering control unit 49, and a vehicle speed control unit 50.
When the driver operates the parking position learning switch of the human-machine interface 12 , the HMI control unit 40 outputs a map generation command to the map generation unit 45 to store the target learning position in the storage device 21 .
The HMI control unit 40 also determines whether the driver has performed a shift operation to turn the steering wheel, and outputs the determination result to the parking assist control unit 41. When it detects that the driver has operated the parking assist activation switch of the human-machine interface 12, it outputs the detection result to the parking assist control unit 41.

The parking assist control unit 41 determines whether or not the host vehicle 1 is located near the target parking position 35. For example, it determines whether or not the distance between the host vehicle 1 and the target parking position 35 is equal to or less than a predetermined distance. The parking assist control unit 41 may determine whether or not the distance between the host vehicle 1 and the target parking position 35 is equal to or less than a predetermined distance based on the current position of the host vehicle 1 measured by the positioning device 11.
For example, the characteristics of targets near the target parking position 35 may be stored in advance, and based on whether the external sensor 14 detects a target with similar characteristics, it may be determined whether the distance between the vehicle 1 and the target parking position 35 is less than a predetermined distance.

When the host vehicle 1 is located near the target parking position 35 and the HMI control unit 40 detects operation of the parking assist activation switch, the parking assist control unit 41 starts parking assist control. Alternatively, when the host vehicle 1 is located near the target parking position 35 and a shift operation for turning is detected, the parking assist control unit 41 starts parking assist control. Note that in this embodiment, an example is shown in which parking assist control is started when the host vehicle 1 is located near the target parking position 35 and operation of the parking assist activation switch or a shift operation is detected. However, parking assist control does not necessarily have to be started on the condition that operation of the parking assist activation switch or a shift operation is detected. For example, parking assist control may be started regardless of driver operation when it is detected that the distance between the host vehicle 1 and the target parking position 35 has changed from a state greater than a predetermined distance to a predetermined distance or less (i.e., the host vehicle 1 has reached the vicinity of the target parking position 35). Furthermore, instead of operation of the parking assist activation switch or a shift operation, parking assist control may be started when, for example, the hazard lights are turned on or the vehicle is stopped.
When the parking assist control is started, the parking assist control unit 41 outputs a parking position calculation command to the matching unit 47 to calculate the position of the target parking position 35 in the vehicle coordinate system.
In addition, based on the calculated position of the target parking position 35, a driving trajectory calculation command is output to the target trajectory generation unit 48 to calculate a target driving trajectory from the current position of the vehicle 1 to the target parking position 35 and a target vehicle speed profile for the vehicle 1 to drive on the target driving trajectory.
The target trajectory generating unit 48 calculates a target driving trajectory from the current position of the host vehicle 1 to the target parking position 35 and a target vehicle speed profile, and outputs them to the parking assist control unit 41. A well-known method already employed in a commonly known automatic parking device can be applied to calculate the target driving trajectory from the current position of the host vehicle 1 to the target parking position 35. As an example, the target driving trajectory can be calculated by connecting the current position of the host vehicle 1 to the target parking position 35 via the turning position 31 with a clothoid curve. Furthermore, as an example, the target vehicle speed profile can be calculated based on a predetermined set speed, such that the host vehicle 1 accelerates from the current position of the host vehicle 1 to the set speed, decelerates before the turning position 31, stops at the turning position 31, accelerates from the turning position 31 to the set speed, decelerates before the target parking position 35, and stops at the target parking position 35. Alternatively, the set speed when calculating the speed profile may be set based on the curvature of the calculated target driving trajectory, such that the larger the curvature, the lower the speed.

The parking assist control unit 41 outputs information on the target driving trajectory calculated by the target trajectory generation unit 48 and the current position of the vehicle 1 to the HMI control unit 40. If the calculated target driving trajectory includes a turnaround like the trajectories 37 and 38 in FIG. 2B , the parking assist control unit 41 outputs information on the turnaround position 31 to the HMI control unit 40.
The HMI control unit 40 displays the target driving trajectory, the current position of the vehicle 1, and the position display of the steering position on the display device of the human-machine interface 12.

The parking assist control unit 41 also outputs a steering control command to the steering control unit 49 to perform steering control so as to cause the host vehicle 1 to travel along the calculated target travel trajectory. The parking assist control unit 41 also outputs a vehicle speed control command to the vehicle speed control unit 50 to control the vehicle speed of the host vehicle 1 in accordance with the calculated target vehicle speed profile.
The image conversion unit 42 converts the image captured by the camera of the external sensor 14 into a bird's-eye view image (around view monitor image) viewed from a virtual viewpoint directly above the vehicle 1, as shown in Figures 2(a) and 2(b). Hereinafter, the bird's-eye view image converted by the image conversion unit 42 may be referred to as a "surrounding image." Figure 2(a) is the surrounding image when a target learning position is stored in the storage device 21, and Figure 2(b) is the surrounding image when parking assist control is performed.

The image conversion unit 42 generates surrounding images at predetermined intervals. For example, the image conversion unit 42 may generate a surrounding image every time the host vehicle 1 travels a predetermined distance (e.g., 50 cm). Or, for example, the image conversion unit 42 may generate a surrounding image every time the host vehicle 1 travels for a predetermined time (e.g., 1 second). Each of the surrounding images acquired when the host vehicle 1 is in a different position or each of the surrounding images acquired at a different time may be referred to as a "frame."

The vehicle position calculation unit 43 calculates the current position of the vehicle 1 on the map coordinate system by dead reckoning based on the vehicle information output from the vehicle sensor 15 .
The feature point detection unit 44 performs predetermined preprocessing such as noise removal, brightness adjustment, and edge enhancement on the surrounding image output from the image conversion unit 42. The feature point detection unit 44 detects feature points and their feature amounts from the surrounding image that has undergone preprocessing. For example, the feature point detection unit 44 may detect feature points and feature amounts using a FAST detector or a FREAK descriptor.
The feature points detected when the target learning position is stored in the storage device 21 are indicated by circular plots in Fig. 2(a), and the feature points detected when the parking assist control is performed are indicated by triangular plots in Fig. 2(b).

Furthermore, the feature point detection unit 44 extracts common feature points common to two different frames. An example of the process of extracting common feature points from two different frames will be described with reference to FIG.
At the time when the surrounding image on the left side of Fig. 4 (referred to as "first frame") is acquired, the host vehicle 1 is at the position indicated by reference numeral 50. The host vehicle 1 then moves forward as indicated by arrow 51, and at the time when the surrounding image on the right side of Fig. 4 (referred to as "second frame") is acquired, the host vehicle 1 has moved to the position indicated by reference numeral 52.
The feature points detected in each frame are shown as circles. As the host vehicle 1 moves forward between the time when the first frame is acquired and the time when the second frame is acquired, the feature points in the second frame move backward by the amount of movement of the host vehicle 1 compared to the feature points in the first frame.

The feature point detection unit 44 matches the feature points of the first frame with the feature points of the second frame, and associates the feature points of the first frame with the feature points of the second frame. For example, feature points having the same feature amount or feature points having a similarity of feature amounts equal to or greater than a threshold may be associated with each other.
The dashed dotted line in FIG. 4 indicates the correspondence between the feature points of the first frame and the feature points of the second frame.
The feature point detection unit 44 removes feature points that could not be associated.

Here, when feature point 53a of the first frame and feature point 53b of the second frame, which are feature points of the same target, are associated with each other, feature point 53b of the second frame recedes from feature point 53a of the first frame by the same length as the forward movement of host vehicle 1. In other words, the amount of change in position of the feature point between the first frame and the second frame corresponds to the amount of movement of host vehicle 1.
On the other hand, when a feature point 54a of the first frame and a feature point 54b of the second frame, which are feature points of different targets, are associated with each other, the amount of change in the positions of these feature points 54a and 54b does not correspond to the amount of movement of the vehicle 1.

The feature point detection unit 44 extracts common feature points that are common to the two frames by removing feature points that could not be matched and, from among the matched feature points, feature points whose position change amounts in the first and second frames do not correspond to the amount of movement of the vehicle 1.
By extracting common feature points in this way, feature points that detect three-dimensional objects, raindrops, the shadow of the vehicle, and moving objects can be removed.
The feature point detection unit 44 synchronizes the feature points extracted as common feature points with the current position of the vehicle 1 received from the self-position calculation unit 43 when the surrounding image was acquired, and outputs them to the map generation unit 45 and the matching unit 47.

3 , when the target learning position is stored in the storage device 21, the driver operates the parking position learning switch and manually parks the vehicle 1 at the target parking position. At this time, the map generation unit 45 receives a map generation command from the HMI control unit 40.
When the map generation unit 45 receives a map generation command, it stores feature point information including the feature points output from the feature point detection unit 44, the current position of the vehicle 1 synchronized therewith, and the feature amounts of the feature points in the storage device 21 as target learning positions, and generates map data 46.
The position of the feature point in the map coordinate system may be calculated based on the current position of the vehicle 1 synchronized with the feature point and stored as feature point information.
Furthermore, when the driver operates the human-machine interface 12 to input to the parking assistance device 10 that the current position of the vehicle 1 is the target parking position 35, the map generation unit 45 receives the current position of the vehicle 1 on the map coordinate system from the positioning device 11 or the self-position calculation unit 43, and stores it in the map data 46 as the target parking position 35. In other words, the relative positional relationship between the target parking position 35 and the multiple characteristic points is stored as map data.

At this time, the map generating unit 45 limits the target learning positions stored in the storage device 21, thereby reducing the storage capacity required to store the target learning positions.
2A , the map generation unit 45 selects only feature point information of feature points detected when the host vehicle 1 is within a predetermined distance range traveled before and after the host vehicle 1 makes a turn at the turn-around position 31 during manual driving as target learning positions to be stored, which are target positions within a predetermined distance range from the turn-around position 31. The map generation unit 45 integrates the feature point information of feature points to be stored that are detected in multiple different frames and stores the integrated information in the storage device 21.
For example, only the feature point information of the feature points detected when the vehicle 1 is within a predetermined traveling distance range (e.g., 10 meters) along the track 33 until it reaches the turning position 31 before turning, and within a predetermined traveling distance range (e.g., 10 meters) along the track 34 from the turning position 31 after turning, may be stored as the target position within the predetermined distance range from the turning position 31.

Also, for example, only the feature point information of the feature points detected when the vehicle 1 is within a predetermined driving distance range (e.g., 10 meters) along the track 33 to the turning position 31 before the turning, and when the vehicle 1 is within a range along the track 34 from the turning position 31 to the target parking position 35 after the turning, may be stored as target positions within a predetermined distance range from the turning position 31.
For example, only the feature point information of the feature points detected when the vehicle 1 is in the range from the turning position 31 to the target parking position 35 along the track 34 after turning may be stored as the target position within a predetermined distance range from the turning position 31.
Furthermore, target positions within a predetermined distance range (for example, within a radius of 20 meters centered on the turning position) determined in advance from the turning position 31 may be stored in the storage device 21 as target learning positions.

As described above, the surrounding image in which feature points are detected is generated at predetermined intervals. For example, the surrounding image is generated every time the host vehicle 1 travels a predetermined distance (e.g., 50 cm) or every time the host vehicle 1 travels for a predetermined time (e.g., 1 second). Therefore, the feature points stored as target learning positions are detected at predetermined intervals.
Therefore, the shorter the distance traveled by the host vehicle 1 after turning, the fewer the number of characteristic points stored as target learning positions, and the lower the accuracy of calculating the position of the target parking position 35. Conversely, the longer the distance traveled by the host vehicle 1 after turning, the more the number of characteristic points stored as target learning positions and the larger the required storage capacity.
Therefore, the longer the distance traveled by the vehicle 1 after turning, the longer the interval at which feature points are stored. That is, the longer the interval at which feature points are detected to be stored as learned target positions. For example, the interval at which surrounding images are generated may be longer, or the feature points detected by the feature point detection unit 44 at a predetermined detection interval may be sampled and thinned out at intervals longer than the detection interval.

Thereafter, when the parking assist control unit 41 starts parking assist control, the matching unit 47 receives a parking position calculation command from the parking assist control unit 41 .
When the matching unit 47 receives a parking position calculation command, it matches the feature point information (circle plot in FIG. 2A) stored in the map data 46 as a target learning position with the feature point information (triangle plot in FIG. 2B) output from the feature point detection unit 44 from the captured image output by the camera of the external sensor 14 when parking assistance is performed, and associates the feature point information detected for the same target with each other.
At this time, the matching unit 47 may remove, as outliers, feature points that do not match the two-dimensional affine transformation from among the associated feature points. The outlier detection may use, for example, the RANSAC (Random Sample Consensus) algorithm. Furthermore, among the associated feature points, feature points that do not match in feature amounts such as shading or attributes may be removed.

The matching unit 47 calculates the relative positional relationship between the current position of the vehicle 1 and the target parking position 35 based on the relative positional relationship between the feature points (triangular plots) when parking assistance is performed and the vehicle 1, and the relative positional relationship between the feature points (circular plots) of the map data 46 associated with these feature points (triangular plots) and the target parking position 35.
For example, a feature point detected when parking assistance is performed is represented as (x _i , y _i ), and a feature point stored as a target learning position in the map data 46 that is associated with the feature point (x _i , y _i ) is represented as (x _mi , y _mi ) (i = 1 to N).
The matching unit 47 calculates the affine transformation matrix M _affine using the following equation based on the least squares method.

The weighted least squares method may be used to calculate the column vector (a1, a2, a3, a4) ^T as shown in the following equation.

The matching unit 47 converts the position (targetx _m , targety _m ) of the target parking position 35 on the map coordinate system stored in the map data 46 into a position (targetx, targety) in the vehicle coordinate system using the following equation.

When the target trajectory generation unit 48 receives a driving trajectory calculation command from the parking assistance control unit 41, it calculates a target driving trajectory from the current position of the vehicle 1 on the vehicle coordinate system (i.e., the coordinate origin) to the target parking position 35 (targetx, targety), and a target vehicle speed profile for the vehicle 1 to travel along the target driving trajectory.
When the steering control unit 49 receives a steering control command from the parking assist control unit 41, it controls the steering actuator 18a so that the host vehicle 1 travels along the target travel trajectory.
When the vehicle speed control unit 50 receives a vehicle speed control command from the parking assist control unit 41, it controls the accelerator actuator 18b and the brake actuator 18c so that the vehicle speed of the host vehicle 1 changes in accordance with the target vehicle speed profile.
This controls the vehicle 1 to travel along the target travel path.

The parking assist control unit 41 determines whether the vehicle 1 has reached the target parking position 35 and the parking assist control has been completed. Once the parking assist control has been completed, the parking assist control unit 41 activates the parking brake 17 and switches the shift position to the parking range (hereinafter referred to as the "P range").

(operation)
5 is a flowchart of an example of a process for storing the target learning position. When storing the target learning position in the storage device 21, the driver operates the parking position learning switch and manually parks the vehicle 1 at the target parking position.
In step S1, the image conversion unit 42 converts the image captured by the camera of the external sensor 14 into a bird's-eye view image seen from a virtual viewpoint directly above the vehicle 1, thereby acquiring a surrounding image.
In step S2, the feature point detection unit 44 performs predetermined preprocessing such as noise removal, brightness adjustment, and edge emphasis on the surrounding image output from the image conversion unit 42.
In step S3, the feature point detection unit 44 detects feature points from the surrounding image that has been subjected to preprocessing.

In step S4, the feature point detection unit 44 extracts common feature points common to two different frames.
In step S5, the map generation unit 45 selects and integrates, from the plurality of different frames, feature points to be stored as target learning positions in the storage device 21 from among the feature points extracted in step S4. At this time, the map generation unit 45 integrates, as target learning positions to be stored, only feature points detected when the host vehicle 1 is within a predetermined distance range traveled before and after the host vehicle 1 makes a turn at the turning position 31.
In step S6, the map generating unit 45 stores the integrated feature points in the storage device 21.

FIG. 6 is a flowchart of an example of processing when parking assistance is performed.
In step S10, the image conversion unit 42 acquires a surrounding image.
In step S<b>11 , the feature point detection unit 44 performs a predetermined preprocessing on the surrounding image output from the image conversion unit 42 .
In step S12, feature points are detected from the surrounding image that has been preprocessed.

In step S13, the feature point detection unit 44 extracts common feature points common to two different frames.
In step S14, the parking assist control unit 41 determines whether the distance between the vehicle 1 and the target parking position 35 is equal to or less than a predetermined distance. If the distance between the vehicle 1 and the target parking position 35 is equal to or less than the predetermined distance (step S14: Y), the process proceeds to step S15. If the distance between the vehicle 1 and the target parking position 35 is not equal to or less than the predetermined distance (step S14: N), the process returns to step S10.

In step S15, the parking assist control unit 41 determines whether a shift operation for turning has been detected. If a shift operation for turning has been detected, the process proceeds to step S16. If a shift operation for turning has not been detected, the process returns to step S10.
In step S15, it may be determined whether the parking assist activation switch has been operated by the driver. If the parking assist activation switch has been operated, the process proceeds to step S16. If the parking assist activation switch has not been operated, the process returns to step S10.

In step S16, the matching unit 47 selects and integrates feature amounts within a predetermined range from among the feature points extracted in step S13 from multiple different frames. For example, the matching unit 47 integrates only feature points detected within a predetermined distance range (e.g., 10 meters) that the host vehicle 1 travels to the turning position before turning.
In step S17, the matching unit 47 reads the characteristic points stored in the map data 46 as target learning positions.
In step S18, the matching unit 47 matches the feature points integrated in step S16 with the feature points read in step S17.
In step S19, the matching unit 47 calculates the target parking position 35 based on the matched feature points.

In step S20, the target trajectory generating unit 48 calculates a target driving trajectory from the current position of the host vehicle 1 to the target parking position 35 and a target vehicle speed profile for the host vehicle 1 to travel on the target driving trajectory.
In step S21, the steering control unit 49 controls the steering actuator 18a so that the host vehicle 1 travels along the target travel trajectory. The vehicle speed control unit 50 controls the accelerator actuator 18b and the brake actuator 18c so that the vehicle speed of the host vehicle 1 changes in accordance with the target vehicle speed profile.
In step S22, the parking assist control unit 41 determines whether the parking assist control is complete. When the parking assist control is complete, the parking assist control unit 41 activates the parking brake 17 and switches the shift position to the P range. Then, the process ends.

(Effects of the embodiment)
(1) The controller 16 detects target positions, which are the positions of targets present around the vehicle 1, and when the driver manually turns the vehicle 1 to park at the target parking position, it stores only target positions within a predetermined distance range from the turning position where the vehicle 1 turns, stores the relative positional relationship between the stored target positions and the target parking position, and calculates the relative positional relationship between the current position of the vehicle 1 and the target parking position based on the relative positional relationship between the target positions detected by the sensor and the vehicle 1 and the relative positional relationship between the stored target positions and the target parking position.
Based on the relative positional relationship between the current position of the vehicle 1 and the target parking position, a target driving trajectory from the current position of the vehicle 1 to the target parking position via a turning position is calculated, and parking assistance for the vehicle 1 is performed based on the calculated target driving trajectory.
This makes it possible to reduce the number of target positions to be stored in order to calculate the target driving trajectory to the target parking position.

(2) The target position within a predetermined distance range from the turning position of the vehicle 1 may be a target position detected when the vehicle 1 travels within a predetermined traveling distance range before and after the turning position.
This allows the storage of a target position suitable for calculating a target driving trajectory to the target parking position.
(3) The specified driving distance range may be, for example, a predetermined driving distance range from when the vehicle 1 reaches the turning position before turning, and a range from when the vehicle 1 reaches the turning position to the target parking position after turning.
This allows the target positions near the turning position required for calculating an accurate target driving trajectory to be stored, making it possible to calculate the target driving trajectory with high accuracy. Also, by limiting the target positions to be stored, it is possible to avoid an increase in the required storage capacity and mitigate increases in costs.
In particular, when parking assistance begins, the target positions (triangular plots) detected by the external sensor 14 tend to coincide with the stored target positions (circular plots) near the turning position 31, so the relative positional relationship between the current position of the vehicle 1 and the target parking position 35 can be calculated with high accuracy.

(4) The controller 16 stores the target position detected by the sensor each time the vehicle 1 travels a predetermined length of time or distance as the target position to be stored when manually parking at the target parking position, and the predetermined length may be shorter when the distance between the vehicle 1 after turning and the turning position is short than when it is long.
Even if the distance traveled by the vehicle 1 after turning is short, the number of feature points stored as target learning positions can be secured, thereby preventing a decrease in the accuracy of calculating the target parking position. Also, even if the distance traveled by the vehicle 1 after turning is long, an increase in the memory capacity for storing feature points can be prevented.

(5) The controller 16 may control the vehicle 1 to travel along the target travel trajectory, and may display the target travel trajectory and the position of the vehicle 1 on a display device that is visible to the occupant. This can assist in parking the vehicle 1.

1...Vehicle, 10...Parking assistance device, 11...Positioning device, 12...Human-machine interface, 13...Shift switch, 14...External sensor, 15...Vehicle sensor, 16...Controller, 17...Parking brake, 18a...Steering actuator, 18b...Accelerator actuator, 18c...Brake actuator, 20...Processor, 21...Storage device, 40...Human-machine interface control unit, 41...Parking assistance control unit, 42...Image conversion unit, 43...Self-position calculation unit, 44...Feature point detection unit, 45...Map generation unit, 46...Map data, 47...Matching unit, 48...Target trajectory generation unit, 49...Steering control unit, 50...Vehicle speed control unit

Claims (5)

Detecting target positions that are positions of targets present around the vehicle;
When the driver manually turns the vehicle to park at a target parking position, only the target positions detected when the vehicle travels within a predetermined driving distance range before and after a turning position at which the vehicle turns are stored , and the turning position is also stored .
storing a relative positional relationship between the stored target position and the target parking position;
Calculating a relative positional relationship between the current position of the vehicle and the target parking position based on a relative positional relationship between the detected target position and the current position of the vehicle and a relative positional relationship between the stored target position and the target parking position;
calculating a target driving trajectory from the current position of the vehicle to the target parking position via the turning position based on the relative positional relationship between the current position of the vehicle and the target parking position;
providing parking assistance for the host vehicle based on the calculated target driving trajectory;
The shorter the distance between the vehicle after the steering change and the steering position, the shorter the detection interval for detecting the target position stored when parking the vehicle at the target parking position by manual driving is set.
A parking assistance method comprising: 2. The parking assistance method according to claim 1, wherein the predetermined travel distance range is within a predetermined travel distance range from the turning position of the host vehicle before the turning, to the target parking position after the turning . 3. The parking assistance method according to claim 1, wherein the host vehicle is controlled so as to travel along the target travel path. 4. The parking assistance method according to claim 1 , wherein the target driving trajectory and the position of the vehicle are displayed on a display device that is visible to a driver. a sensor for detecting a target position, which is the position of a target present around the vehicle;
When the driver manually drives the vehicle to turn around and park it at a target parking position, only the target positions detected by the sensor that are detected when the vehicle travels within a predetermined driving distance range before and after the turning position where the vehicle turns around are stored, and the turning position is stored. A relative positional relationship between the stored target positions and the target parking position is stored. The relative positional relationship between the target positions detected by the sensor and the current position of the vehicle and the relative positional relationship between the stored target positions and the target parking position are stored. and a controller that calculates a relative positional relationship between the current position of the host vehicle and the target parking position based on the above, calculates a target driving trajectory from the current position of the host vehicle to the target parking position via the turning position based on the relative positional relationship between the current position of the host vehicle and the target parking position , performs parking assistance for the host vehicle based on the calculated target driving trajectory, and shortens a detection interval for detecting the target position stored by a sensor when parking the host vehicle at the target parking position by manual driving as the distance between the host vehicle after turning and the turning position becomes shorter .
A parking assistance device comprising: