JP7785448B2 - Parking assistance system, parking assistance device, parking assistance method, and program - Google Patents

Description

The present invention relates to a parking assistance system, a parking assistance device, a parking assistance method, and a program.

In parking assistance systems that assist the vehicle when parking in a parking area, ultrasonic sensors that measure the distance from the vehicle to an obstacle using reflected ultrasonic waves may be used as a means of detecting obstacles around the vehicle.

Patent No. 6479130

When providing parking assistance, it is necessary to estimate the shape of obstacles (such as other vehicles parked in adjacent parking areas) that exist around the target parking area. Accurately estimating the direction of the obstacle's extension (such as the direction along the side of the other vehicle) is particularly important for improving the reliability and efficiency of control. However, when using sensors with a relatively narrow detection range, such as ultrasonic sensors, it is difficult to accurately estimate the shape of areas far away from the vehicle.

Therefore, one of the objectives of the present invention is to provide a parking assistance system, parking assistance device, parking assistance method, and program that can improve the accuracy of estimating the shape of an obstacle.

One form of the present invention is a parking assistance system mounted on a vehicle, comprising: a distance measurement unit that uses ultrasonic reflection to acquire distance information indicating the distance from the vehicle to an obstacle; a detection unit that detects demarcation lines indicating a parking area; an estimation unit that, when no demarcation lines are detected, sets the extension direction of the obstacle that passes through the end of the obstacle closest to the parking area to be parallel to a predetermined vehicle width direction of the vehicle, and, when a demarcation line is detected, sets the extension direction to be parallel to the extension direction of the demarcation line; a generation unit that generates a movement path for the vehicle based on the extension direction set by the estimation unit; and, when multiple obstacles are detected based on the distance information and the demarcation line is detected, performs processing to merge the multiple obstacles into a single obstacle based on an extension direction that passes through the closest part of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the demarcation line .

With the above configuration, when a lane marking is detected by the detection unit, the extension direction of the obstacle is estimated based on the lane marking. This improves the accuracy of estimating the extension direction of the obstacle, even when detecting an obstacle using ultrasound, thereby improving the reliability and efficiency of parking assistance control.

In addition, the parking assistance system may further include a calculation unit that, when a dividing line is detected, calculates a passable frontage distance when a vehicle enters the parking area based on the set extension direction.

This allows for accurate calculation of entrance distance, improving the reliability and efficiency of parking assistance control.

The parking assistance system may further include an integration unit that, when multiple obstacles are detected based on distance information and a demarcation line is also detected, performs processing to integrate the multiple obstacles into a single obstacle based on the demarcation line.

This improves the accuracy of obstacle recognition and increases the reliability and efficiency of parking assistance control.

Another aspect of the present invention is a parking assistance device that is mounted on a vehicle and performs processing to assist in the movement of the vehicle when parking the vehicle in a parking area based on distance information indicating the distance from the vehicle to an obstacle obtained using ultrasonic reflection, and that includes an estimation unit that, when a dividing line indicating the parking area is not detected, sets the extension direction of the obstacle that passes through the end of the obstacle closest to the parking area to be parallel to a predetermined vehicle width direction of the vehicle, and when a dividing line is detected, sets the extension direction to be parallel to the extension direction of the dividing line; a generation unit that generates a movement path for the vehicle based on the extension direction set by the estimation unit ; and, when multiple obstacles are detected based on the distance information and the dividing line is detected, performs processing to merge the multiple obstacles into a single obstacle based on an extension direction that passes through the closest part of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the dividing line .

Another aspect of the present invention is a parking assistance method comprising the steps of: acquiring distance information indicating the distance from the vehicle to an obstacle using reflected ultrasound; detecting demarcation lines indicating a parking area; if no demarcation lines are detected, setting the extension direction of the obstacle that passes through the end of the obstacle closest to the parking area to be parallel to a predetermined width direction of the vehicle; if a demarcation line is detected, setting the extension direction to be parallel to the extension direction of the demarcation line; generating a movement path for the vehicle based on the set extension direction; and, if multiple obstacles are detected based on the distance information and the demarcation line is detected, performing processing to combine the multiple obstacles into a single obstacle based on an extension direction that passes through the closest portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the demarcation line.

Another aspect of the present invention is a computer that is mounted on a vehicle and performs processing to assist the movement of a vehicle when parking in a parking area based on distance information indicating the distance from the vehicle to an obstacle obtained using ultrasonic reflection, and that performs the following processing: when a dividing line indicating the parking area is not detected, setting the extension direction of the obstacle that passes through the end of the obstacle closest to the parking area to be parallel to a predetermined vehicle width direction of the vehicle; when a dividing line is detected, setting the extension direction to be parallel to the extension direction of the dividing line; generating a vehicle movement path based on the set extension direction; and when multiple obstacles are detected based on the distance information and the dividing line is detected, merging the multiple obstacles into a single obstacle based on an extension direction that passes through the closest part of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the dividing line .

FIG. 1 is a plan view showing the configuration of a vehicle equipped with a parking assistance system according to an embodiment. FIG. 2 is a block diagram showing the configuration of the parking assistance system according to the embodiment. FIG. 3 is a block diagram showing the functional configuration of the parking assistance system according to the embodiment. FIG. 4 is a diagram illustrating an example of a parking lot where the parking assistance system according to the embodiment is used. FIG. 5 is a diagram showing an extension direction estimated when a lane marking is detected in the embodiment. FIG. 6 is a diagram showing an extension direction estimated when no lane marking is detected in the embodiment. FIG. 7 is a flowchart showing the processing in the parking assistance system according to the embodiment. FIG. 8 is a diagram illustrating a method for calculating the frontage distance according to the embodiment. FIG. 9 is a flowchart showing the processing in the frontage distance calculation unit according to the embodiment. FIG. 10 is a diagram illustrating an example of a detection area that can be detected by an ultrasonic sensor when a vehicle passes through a parking area in the embodiment. FIG. 11 is a diagram illustrating an example of a detection area that can be detected by an ultrasonic sensor when a vehicle enters a parking area in the embodiment. FIG. 12 is a diagram showing an example of a state in which one obstacle is recognized as two obstacles in the embodiment. FIG. 13 is a diagram showing a process of integrating multiple recognized objects into one recognized object when a lane marking is detected in the embodiment. FIG. 14 is a flowchart illustrating processing in the obstacle integrating unit according to the embodiment.

The following describes exemplary embodiments of the present invention. The configurations of the embodiments described below, as well as the actions, results, and advantages achieved by those configurations, are merely examples. The present invention can be realized using configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of the various advantages based on the basic configuration and derivative advantages.

Figure 1 is a plan view showing the configuration of a vehicle 10 equipped with a parking assistance system according to an embodiment.

Vehicle 10 is an example of a moving body. Vehicle 10 may be, for example, an automobile powered by an internal combustion engine (internal combustion engine automobile), an automobile powered by an electric motor (electric automobile, fuel cell automobile, etc.), or an automobile powered by both (hybrid automobile). Vehicle 10 may be equipped with various transmissions, and may be equipped with various devices (systems, parts, etc.) required to drive the internal combustion engine or electric motor. The type, number, layout, etc. of the devices related to driving the wheels 13 of vehicle 10 may be configured in a variety of ways.

As shown in FIG. 1, the vehicle 10 includes a body 12, four wheels 13, one or more (four in this embodiment) imaging devices 14a, 14b, 14c, and 14d, and one or more (eight in this embodiment) ultrasonic sensors 16a, 16b, 16c, 16d, 16e, 16f, 16g, 16h, 16i, 16j, 16k, and 16l. When there is no need to distinguish between the imaging devices 14a, 14b, 14c, and 14d, they will be referred to as imaging devices 14. When there is no need to distinguish between the ultrasonic sensors 16a, 16b, 16c, 16d, 16e, 16f, 16g, 16h, 16i, 16j, 16k, and 16l, they will be referred to as ultrasonic sensors 16.

The vehicle body 12 forms the vehicle cabin where the occupants ride. The vehicle body 12 houses or holds the wheels 13, the imaging device 14, the ultrasonic sensor 16, etc.

Four wheels 13 are provided on the front, rear, left and right sides of the vehicle body 12. For example, the two front wheels 13 function as steering wheels, and the two rear wheels 13 function as drive wheels.

The imaging devices 14 are digital cameras incorporating imaging elements such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging devices 14 output imaging data in the form of video or still image data containing multiple frame images generated at a predetermined frame rate. Each imaging device 14 has a wide-angle lens or a fisheye lens and can capture an image in a horizontal range of, for example, 140° to 190°. The optical axis of the imaging device 14 is set diagonally downward. Therefore, the imaging devices 14 output imaging data capturing an image of the surroundings of the vehicle 10, including the surrounding road surface.

The imaging devices 14 are provided on the outer periphery of the vehicle body 12. For example, imaging device 14a is provided in the center in the left-right direction of the front end of the vehicle body 12 (e.g., the front grille). Imaging device 14a generates images of the surroundings in front of the vehicle 10. Imaging device 14b is provided in the center in the left-right direction of the rear end of the vehicle body 12 (e.g., around the back door switch). Imaging device 14b generates images of the surroundings in the rear of the vehicle 10. Imaging device 14c is provided in the center in the front-to-rear direction of the left end of the vehicle body 12 (e.g., the left side mirror 12a). Imaging device 14c generates images of the surroundings to the left of the vehicle 10. Imaging device 14d is provided in the center in the front-to-rear direction of the right end of the vehicle body 12 (e.g., the right side mirror 12b). Imaging device 14d generates images of the surroundings to the right of the vehicle 10.

The ultrasonic sensor 16 is, for example, an ultrasonic sensor (sonar) provided on the outer periphery of the vehicle 10, which transmits ultrasonic waves as detection waves and receives reflected waves from objects (obstacles) present around the vehicle 10. The ultrasonic sensor 16 acquires (generates) distance information indicating the distance from the vehicle 10 to obstacles present around the vehicle 10. For example, the ultrasonic sensor 16 acquires the time from transmitting the detection wave to receiving the reflected wave (TOF: Time Of Flight) as distance information for determining the presence or absence, distance, position, movement, etc. of an obstacle.

Ultrasonic sensors 16a, 16b, 16c, and 16d, also known as side sonars, are installed on the left and right sides of the vehicle 10. Ultrasonic sensors 16e and 16f, also known as corner sonars, are installed further rearward (e.g., near the corners of the vehicle 10) than ultrasonic sensors 16a, 16b, 16c, and 16d and are directed further rearward (e.g., toward the outside rearward) than ultrasonic sensors 16a, 16b, 16c, and 16d. Ultrasonic sensors 16g and 16h, also known as corner sonars, are installed further forward (e.g., near the corners of the vehicle 10) than ultrasonic sensors 16a, 16b, 16c, and 16d and are directed further forward (e.g., toward the outside frontward) than ultrasonic sensors 16a, 16b, 16c, and 16d. Ultrasonic sensors 16i and 16j, also known as rear sonars, are installed at the rear end of the vehicle 10. Ultrasonic sensors 16k and 16l are also called front sonars and are installed at the front end of the vehicle 10.

The ultrasonic sensor 16a is located at the front of the left side of the vehicle 10. The ultrasonic sensor 16a is oriented toward the left. The ultrasonic sensor 16a acquires distance information about obstacles present in the detection area on the left side of the front of the vehicle 10.

Ultrasonic sensor 16b is located at the rear of the left side of vehicle 10. Ultrasonic sensor 16b is oriented toward the left. Ultrasonic sensor 16b acquires distance information regarding obstacles present in the detection area on the left side of the rear of vehicle 10.

The ultrasonic sensor 16c is located at the front of the right side of the vehicle 10. The ultrasonic sensor 16c is oriented to the right. The ultrasonic sensor 16c acquires distance information regarding obstacles present in the detection area on the front right side of the vehicle 10.

Ultrasonic sensor 16d is located at the rear of the right side of vehicle 10. Ultrasonic sensor 16d is oriented to the right. Ultrasonic sensor 16d acquires distance information regarding obstacles present in the detection area on the right side of the rear of vehicle 10.

The ultrasonic sensor 16e is located on the left side of the rear end of the vehicle 10. The ultrasonic sensor 16e is directed toward the left rear. The ultrasonic sensor 16e acquires distance information regarding obstacles present in the detection area at the left rear of the vehicle 10.

The ultrasonic sensor 16f is located on the right side of the rear end of the vehicle 10. The ultrasonic sensor 16f is directed toward the right rear. The ultrasonic sensor 16f acquires distance information regarding obstacles present in the detection area at the right rear of the vehicle 10.

The ultrasonic sensor 16g is located on the left side of the front end of the vehicle 10. The ultrasonic sensor 16g is directed toward the left front. The ultrasonic sensor 16g acquires distance information regarding obstacles present in the detection area in the left front of the vehicle 10.

The ultrasonic sensor 16h is located on the right side of the front end of the vehicle 10. The ultrasonic sensor 16h is directed toward the right front. The ultrasonic sensor 16h acquires distance information regarding obstacles present in the detection area in the right front of the vehicle 10.

Ultrasonic sensors 16i and 16j are located at the rear end of vehicle 10, spaced apart from each other in the left-right direction between ultrasonic sensors 16e and 16f. Ultrasonic sensors 16i and 16j are directed rearward. Ultrasonic sensors 16i and 16j acquire distance information regarding obstacles present in the detection area behind vehicle 10.

Ultrasonic sensors 16k and 16l are located at the front end of vehicle 10, spaced apart from each other in the left-right direction between ultrasonic sensors 16g and 16h. Ultrasonic sensors 16k and 16l face forward. Ultrasonic sensors 16k and 16l obtain distance information regarding obstacles present in the detection area ahead of vehicle 10.

Figure 2 is a block diagram showing the configuration of the parking assistance system 20 according to the embodiment.

The parking assistance system 20 is installed in the vehicle 10 and assists the movement of the vehicle 10 when parking the vehicle 10 in a designated parking area by performing automated driving (including partial automated driving).

As shown in FIG. 2, the parking assistance system 20 includes an imaging device 14, an ultrasonic sensor 16, a braking system 22, an acceleration system 24, a steering system 26, a transmission system 28, a vehicle speed sensor 30, a monitoring device 32, a parking assistance device 34, and an in-vehicle network 36.

The braking system 22 controls the deceleration of the vehicle 10. The braking system 22 includes a braking unit 40, a braking control unit 42, and a braking unit sensor 44.

The braking unit 40 includes, for example, a brake, a brake pedal, etc., and is a device for decelerating the vehicle 10.

The braking control unit 42 is a computer such as a microcomputer having a hardware processor such as a CPU (Central Processing Unit). The braking control unit 42 controls the braking unit 40 based on instructions from the parking assistance device 34, and controls the deceleration of the vehicle 10.

The brake unit sensor 44 is, for example, a position sensor that detects the position of the brake unit 40 when the brake unit 40 is a brake pedal. The brake unit sensor 44 outputs the detected position of the brake unit 40 to the in-vehicle network 36.

The acceleration system 24 controls the acceleration of the vehicle 10. The acceleration system 24 includes an acceleration unit 46, an acceleration control unit 48, and an acceleration unit sensor 50.

The acceleration unit 46 includes, for example, an accelerator pedal, and is a device for accelerating the vehicle 10.

The acceleration control unit 48 is, for example, a computer such as a microcomputer having a hardware processor such as a CPU. The acceleration control unit 48 controls the acceleration unit 46 based on instructions from the parking assistance device 34, thereby controlling the acceleration of the vehicle 10.

The acceleration sensor 50 is, for example, a position sensor, and if the acceleration unit 46 is an accelerator pedal, it detects the position of the acceleration unit 46. The acceleration sensor 50 outputs the detected position of the acceleration unit 46 to the in-vehicle network 36.

The steering system 26 controls the direction of travel of the vehicle 10. The steering system 26 includes a steering unit 52, a steering control unit 54, and a steering unit sensor 56.

The steering unit 52 includes, for example, a steering wheel, and is a device that steers the steered wheels of the vehicle 10 to steer the direction of travel of the vehicle 10.

The steering control unit 54 is, for example, a computer such as a microcomputer having a hardware processor such as a CPU. The steering control unit 54 controls the steering unit 52 based on instructions from the parking assistance device 34, and controls the direction of travel of the vehicle 10.

The steering unit sensor 56 is an example of a third detection unit, and is, for example, an angle sensor including a Hall element, etc., that detects the steering angle, which is the rotation angle of the steering unit 52. The steering unit sensor 56 outputs the detected steering angle of the steering unit 52 to the in-vehicle network 36.

The transmission system 28 controls the gear ratio of the vehicle 10. The transmission system 28 includes a transmission unit 58, a transmission control unit 60, and a transmission unit sensor 62.

The transmission unit 58 includes, for example, a shift lever, and is a device that changes the gear ratio of the vehicle 10.

The transmission control unit 60 is, for example, a computer such as a microcomputer having a hardware processor such as a CPU. The transmission control unit 60 controls the transmission unit 58 based on instructions from the parking assistance device 34, and controls the gear ratio of the vehicle 10.

The transmission sensor 62 is, for example, a position sensor, and if the transmission 58 is a shift lever, it detects the position of the transmission 58. The transmission sensor 62 outputs the detected position of the transmission 58 to the in-vehicle network 36.

The vehicle speed sensor 30 is a sensor that has, for example, a Hall element provided near the wheels 13 of the vehicle 10 and detects the amount of rotation or the number of rotations per unit time of the wheels 13. The vehicle speed sensor 30 outputs the number of wheel speed pulses indicating the detected amount of rotation or number of rotations to the in-vehicle network 36 as a sensor value for calculating the vehicle speed. The parking assistance device 34 can calculate the speed (vehicle speed), amount of movement, etc. of the vehicle 10 based on the sensor value obtained from the vehicle speed sensor 30.

The monitor device 32 is provided on the dashboard or the like inside the vehicle 10. The monitor device 32 has a display unit 64, an audio output unit 66, and an operation input unit 68.

The display unit 64 displays an image based on the image data transmitted by the parking assistance device 34. The display unit 64 is, for example, a display device such as a liquid crystal display (LCD) or an organic electroluminescent display (OELD). The display unit 64 displays an image that accepts an operation instruction to switch between automatic driving and manual driving, for example.

The audio output unit 66 outputs audio based on the audio data transmitted by the parking assistance device 34. The audio output unit 66 is, for example, a speaker. The audio output unit 66 outputs audio related to operational instructions, such as switching between automatic driving and manual driving.

The operation input unit 68 accepts input from the occupant. The operation input unit 68 is, for example, a touch panel. The operation input unit 68 is provided on the display screen of the display unit 64. The operation input unit 68 is configured to be transparent to the image displayed by the display unit 64. This allows the occupant to view the image displayed on the display screen of the display unit 64. The operation input unit 68 accepts instructions input by the occupant touching a position corresponding to the image displayed on the display screen of the display unit 64, and transmits them to the parking assistance device 34. Note that the operation input unit 68 is not limited to a touch panel, and may be a hardware switch such as a push button.

The parking assistance device 34 is a computer that includes a microcomputer such as an ECU (Electronic Control Unit), and provides parking assistance for the vehicle 10.

The parking assistance device 34 includes a CPU 34a, a ROM (Read Only Memory) 34b, a RAM (Random Access Memory) 34c, a display control unit 34d, a sound control unit 34e, and an SSD (Solid State Drive) 34f. The CPU 34a, ROM 34b, and RAM 34c may be integrated into the same package.

The CPU 34a is an example of a hardware processor that reads programs stored in a non-volatile storage device such as the ROM 34b and executes various types of arithmetic processing and control in accordance with the programs. For example, the CPU 34a executes parking assistance through automatic driving of the vehicle 10.

The ROM 34b stores programs, parameters required for program execution, etc. The RAM 34c temporarily stores various data used in the calculations by the CPU 34a. The display control unit 34d, among the calculation processes performed by the parking assistance device 34, mainly performs image processing of images obtained by the imaging device 14 and data conversion of images to be displayed on the display unit 64. The audio control unit 34e, among the calculation processes performed by the parking assistance device 34, mainly processes audio to be output to the audio output unit 66. The SSD 34f is a rewritable non-volatile storage device that retains data even when the parking assistance device 34 is powered off.

The in-vehicle network 36 includes, for example, a Controller Area Network (CAN) or a Local Interconnect Network (LIN). The in-vehicle network 36 connects the acceleration system 24, braking system 22, steering system 26, gear shift system 28, ultrasonic sensor 16, vehicle speed sensor 30, operation input unit 68 of the monitor device 32, and parking assistance device 34 so that information can be sent and received among them.

Figure 3 is a block diagram showing the functional configuration of the parking assistance system 20 according to the embodiment.

The parking assistance system 20 includes a distance measurement unit 101, a lane marking detection unit 102 (detection unit), an extension direction estimation unit 103 (estimation unit), a travel path generation unit 104 (generation unit), and a driving control unit 105.

The distance measurement unit 101 uses reflected ultrasonic waves to obtain distance information indicating the distance from the vehicle 10 to an obstacle. The distance measurement unit 101 is configured through the cooperation of the ultrasonic sensor 16, the parking assistance device 34, a program, etc.

The lane marking detection unit 102 detects lane markings that indicate parking areas. The lane marking detection unit 102 is configured through cooperation between the imaging device 14, the parking assistance device 34, a program, etc. The lane marking detection unit 102 detects lane markings that exist around the vehicle 10 by performing image recognition processing on the captured image acquired by the imaging device 14. Note that the method for detecting lane markings is not limited to this; for example, lane markings can also be detected based on information acquired by a LIDAR (Light Detection and Ranging) sensor, millimeter-wave radar, etc. instead of the imaging device 14.

The extension direction estimation unit 103 estimates the extension direction of obstacles present around the vehicle 10, particularly obstacles present near the target parking area, based on the distance information acquired by the distance measurement unit 101 and the lane line information acquired by the lane line detection unit 102. The extension direction indicates part of the obstacle's shape characteristics and is the direction in which a relatively long portion of the obstacle extends. For example, if the obstacle is a vehicle, the extension direction is the direction along the side of the vehicle. The lane line information includes the position, shape, etc. of the lane lines detected by the lane line detection unit 102. The extension direction estimation unit 103 is configured in cooperation with the parking assistance device 34, a program, etc.

In this embodiment, when a lane marking is not detected, the extension direction estimation unit 103 estimates the extension direction based on the distance information acquired by the distance measurement unit 101 and a predetermined direction (e.g., the width direction of the vehicle 10). Furthermore, when a lane marking is detected by the lane marking detection unit 102, the extension direction estimation unit 103 estimates the extension direction based on the distance information and the lane marking.

The movement path generation unit 104 generates a movement path for the vehicle 10 based on the extension direction of the obstacle estimated by the extension direction estimation unit 103. The movement path generation unit 104 is configured through cooperation between the parking assistance device 34, a program, etc.

The travel path generation unit 104 according to this embodiment includes an entrance distance calculation unit 111 and an obstacle integration unit 112.

When a demarcation line is detected by the demarcation line detection unit 102, the frontage distance calculation unit 111 calculates the passable frontage distance when the vehicle 10 enters the parking area based on the extension direction of the obstacle estimated based on the demarcation line.

When multiple obstacles are detected based on distance information acquired by the distance measurement unit 101 and a demarcation line is detected by the demarcation line detection unit 102, the obstacle integration unit 112 performs processing to integrate the multiple obstacles into a single obstacle based on the demarcation line.

The travel route generation unit 104 generates a travel route for the vehicle 10 based on the calculation results by the frontage distance calculation unit 111 and the processing results by the obstacle integration unit 112.

The driving control unit 105 performs processing to cause the vehicle 10 to drive automatically based on the travel route generated by the travel route generation unit 104. The driving control unit 105 is configured through the cooperation of the parking assistance device 34, braking system 22, acceleration system 24, steering system 26, gear shift system 28, programs, etc.

Figure 4 is a diagram showing an example of a parking lot in which the parking assistance system 20 according to the embodiment is used.

The parking lot illustrated in Figure 4 has multiple parking areas 201 arranged diagonally (in a stepped pattern). Each parking area 201 is separated by demarcation lines 211. In this example, other vehicles 200A and 200B are parked in parking areas adjacent to the parking area 201 where vehicle 10 (host vehicle) is attempting to park, and these other vehicles 200A and 200B act as obstacles. Figure 4 shows a situation in which vehicle 10 travels at a low speed in the direction of travel Df, passes through the target parking area 201 once, and then backs into the parking lot. In this situation, the direction of travel Df of vehicle 10 and the extension direction E of the other vehicles 200A and 200B are not perpendicular to each other.

When vehicle 10 passes through parking area 201, ultrasonic sensors 16 (in this example, ultrasonic sensors 16c and 16d) acquire distance information indicating the distance from vehicle 10 to other vehicles 200A and 200B, and imaging device 14 (in this example, imaging device 14d) acquires image data (captured image) of the area around parking area 201. If demarcation lines 211 are detected (recognized) in the captured image, the extension direction E of other vehicles 200A and 200B is estimated based on the distance information and demarcation lines 211. On the other hand, if demarcation lines 211 are not detected, extension direction E is estimated based on the distance information and a predetermined direction.

Figure 5 shows the extension direction E estimated when a demarcation line 211 is detected in this embodiment.

5 shows a case in which the ultrasonic sensor 16 detects four ends P1 to P4 of the other vehicle 200A, and the demarcation line 211 is detected from the image captured by the imaging device 14. In this case, the extension direction estimation unit 103 (see FIG. 3) estimates the extension direction E of the other vehicle 200A based on the positions of the ends P1 to P4 determined from the distance information and the demarcation line information related to the demarcation line 211. Specifically, the extension direction E passing through the end of the other vehicle 200A closest to the target parking area 201 (end P1 in this example) is set to be parallel to the extension direction Es of the demarcation line 211. Note that while the above description only applies to the other vehicle 200A, the same applies to the other vehicle 200B. This estimation method allows for accurate estimation of the extension direction E of the other vehicles 200A and 200B, even in a special parking lot where multiple parking areas 201 are arranged diagonally, as shown in FIG. 4.

Figure 6 shows the extension direction E that is estimated when the division line 211 is not detected in this embodiment.

6 shows a case where the ultrasonic sensor 16 detects four ends P1-P4 of the other vehicle 200A, but the partition line 211 is not detected in the image captured by the imaging device 14. In such a case, the extension direction estimation unit 103 estimates the extension direction E of the other vehicle 200A based on the positions of the ends P1-P4 determined from the distance information and the predetermined vehicle width direction W. Specifically, the extension direction E passing through the end of the other vehicle 200A closest to the target parking area 201 (end P1 in this example) is set to be parallel to the vehicle width direction W. Note that while the above description only applies to the other vehicle 200A, the same applies to the other vehicle 200B. With this estimation method, the estimation accuracy of the extension direction of the other vehicles 200A and 200B decreases in parking lots with a special structure such as that shown in FIG. 4. However, in parking lots with a normal structure in which the traveling direction Df and the extension direction E form a right angle, the extension direction E can be estimated with substantially sufficient accuracy.

Figure 7 is a flowchart showing the processing in the parking assistance system 20 according to the embodiment.

When the distance measurement unit 101 acquires distance information (S101), the extension direction estimation unit 103 determines whether or not the lane marking 211 has been detected by the lane marking detection unit 102 (S102). If the lane marking 211 has been detected (S102: Yes), the extension direction estimation unit 103 estimates the extension direction E of the obstacle (other vehicles 200A, 200B) based on the distance information and the lane marking 211 (S103). On the other hand, if the lane marking 211 has not been detected (S102: No), the extension direction estimation unit 103 estimates the extension direction E of the obstacle based on the distance information and the vehicle width direction W (S104).

The travel path generation unit 104 generates a travel path for the vehicle 10 based on the extension direction E estimated as described above (S105), and the driving control unit 105 controls the vehicle 10 according to the generated travel path (S106).

In addition, as described above, the travel path generation unit 104 generates a travel path based on the calculation results by the frontage distance calculation unit 111 and the processing results by the obstacle integration unit 112.

Figure 8 shows a method for calculating the frontage distance De according to this embodiment.

The frontage distance De indicates the width of the area through which the vehicle 10 can enter the target parking area 201. As shown in Figure 8, when a demarcation line 211 is detected, the frontage distance De is calculated based on the extension direction E estimated based on the demarcation line 211. This allows the frontage distance De to be calculated accurately even for parking lots with special structures such as those shown in Figure 4.

Figure 9 is a flowchart showing the processing performed by the frontage distance calculation unit 111 according to the embodiment.

The frontage distance calculation unit 111 determines whether the demarcation line 211 has been detected by the demarcation line detection unit 102 (S201). If the demarcation line 211 is detected (S201: Yes), the frontage distance calculation unit 111 calculates the frontage distance De based on the extension direction E (see Figures 5 and 8) of the demarcation line 211 (S202). On the other hand, if the demarcation line 211 is not detected (S201: No), the frontage distance calculation unit 111 calculates the frontage distance De based on the extension direction E of the obstacle based on the vehicle width direction W (see Figure 6) (S203). According to the processing of step S203, the calculation accuracy of the frontage distance De is low in parking lots with a special structure such as that shown in Figure 4. However, in parking lots with a normal parallel structure (a structure in which the travel direction Df and the extension direction E are perpendicular to each other), the extension direction can be estimated with substantially sufficient accuracy.

The process of merging multiple obstacles is explained below with reference to Figures 10 to 13.

Figure 10 is a diagram showing an example of a detection area A1 that can be detected by the ultrasonic sensor 16 when the vehicle 10 passes through the parking area 201 in this embodiment. Figure 11 is a diagram showing an example of a detection area A2 that can be detected by the ultrasonic sensor 16 when the vehicle 10 enters the parking area 201 in this embodiment.

At the timing shown in Figure 10, the detection area A1 of the ultrasonic sensor 16 (in this example, ultrasonic sensors 16c and 16d) is the front end of the other vehicle 200A. At the timing shown in Figure 11, the detection area A2 of the ultrasonic sensor 16 is the side of the other vehicle 200A facing the vehicle 10. In this way, the detection areas A1 and A2 of the ultrasonic sensor 16 change as the vehicle 10 moves.

Figure 12 shows an example of a state in which one obstacle is recognized as two obstacles in an embodiment.

Figure 12 shows a recognized object R1 corresponding to the detection area A1 shown in Figure 10, and a recognized object R2 corresponding to the detection area A2 shown in Figure 11. As such, when the detection areas A1 and A2 of the ultrasonic sensor 16 change as the vehicle 10 moves, there is no continuity in the acquired distance information, and one obstacle (in this example, another vehicle 200A) may be recognized as multiple obstacles.

Figure 13 shows the process of merging multiple recognition objects R1 and R2 into a single recognition object R when a demarcation line 211 is detected in an embodiment.

When a demarcation line 211 is detected, as shown in FIG. 13, multiple recognition objects R1, R2 present within a predetermined area S are integrated into a single recognition object R based on the demarcation line 211. The predetermined area S should be set appropriately depending on the usage situation, etc., but may be an area defined based on the size of a typical vehicle, for example. In this embodiment, the multiple recognition objects R1, R2 are integrated into a single recognition object R based on the extension direction E that passes through the proximate portion Pn of the obstacle acquired by the ultrasonic sensor 16 that is closest to the vehicle 10 and is parallel to the extension direction Es of the demarcation line 211. This allows the shape of the obstacle to be accurately estimated even in parking lots with a special structure such as that shown in FIG. 4.

Figure 14 is a flowchart showing the processing performed by the obstacle integration unit 112 according to the embodiment.

The obstacle integration unit 112 determines whether multiple obstacles (recognition objects R1, R2) have been recognized within the specified area S (S301). If multiple obstacles have not been recognized (S301: No), this flow ends. If multiple obstacles have been recognized (S301: Yes), the obstacle integration unit 112 determines whether the lane marking 211 has been detected by the lane marking detection unit 102 (S302).

If a demarcation line 211 is detected (S302: Yes), the obstacle integration unit 112 integrates multiple obstacles (recognized objects R1, R2) into a single obstacle (recognized object R) based on the demarcation line 211 (S303). On the other hand, if a demarcation line 211 is not detected (S302: No), the obstacle integration unit 112 integrates multiple obstacles (recognized objects R1, R2) into a single obstacle (recognized object R) based on the vehicle width direction W (see FIG. 6) (S304). According to the processing of step S304, the accuracy of estimating the shape of an obstacle is low in parking lots with a special structure such as that shown in FIG. 4. However, in parking lots with a normal parallel structure (a structure in which the traveling direction Df and the extension direction E are perpendicular to each other), the shape of the obstacle can be estimated with substantially sufficient accuracy.

The program that causes the parking assistance device 34 to perform the various processes described above may be provided as a computer program product stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD). The program may also be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. The program may also be provided or distributed via a network such as the Internet.

According to the above embodiment, the accuracy of estimating the shape of an obstacle can be improved even when detecting an obstacle using ultrasound or in a parking lot with a special structure, thereby improving the reliability and efficiency of parking assistance control.

The above describes an embodiment of the present invention, but the above embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be embodied in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its variations are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as set forth in the claims.

10...vehicle (moving object), 12...vehicle body, 13...wheel, 14 (14a-14d)...imaging device, 16 (16a-16l)...ultrasonic sensor, 20...parking assistance system, 34...parking assistance device, 101...distance measurement unit, 102...landing line detection unit (detection unit), 103...extension direction estimation unit (estimation unit), 104...travel path generation unit (generation unit), 105...driving control unit, 111...frontage distance calculation unit (calculation unit), 112...obstacle integration unit (integration unit), 200A, 200B...other vehicles (obstacles), 201...parking area, 211...landing line, A1, A2...detection area, Df...direction of travel, E...extension direction (of obstacle), Es...extension direction (of lane line), P1-P8...edge, R, R1, R2...recognized object, S...predetermined area, W...vehicle width direction

Claims (5)

A parking assistance system mounted on a vehicle,
a distance measuring unit that acquires distance information indicating the distance from the vehicle to an obstacle by using reflected ultrasonic waves;
a detection unit that detects parking area markings;
an estimation unit that, when the lane marking is not detected, sets an extension direction of the obstacle that passes through an end of the obstacle obtained from the distance information to be parallel to a predetermined vehicle width direction of the vehicle, and, when the lane marking is detected, sets an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area obtained from the distance information to be parallel to the extension direction of the lane marking;
a generation unit that generates a travel path of the vehicle based on the extension direction set by the estimation unit;
an integration unit that, when a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, performs processing to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking;
A parking assistance system equipped with a calculation unit that calculates a passable frontage distance when the vehicle enters the parking area based on the set extension direction when the demarcation line is detected;
The parking assistance system of claim 1 further comprising: A parking assistance device that is mounted on a vehicle and performs processing to assist movement of the vehicle when parking the vehicle in a parking area, based on distance information indicating a distance from the vehicle to an obstacle, the distance information being acquired by utilizing reflection of ultrasonic waves,
an estimation unit that, when a demarcation line indicating the parking area is not detected, sets an extension direction of the obstacle that passes through an end of the obstacle obtained from the distance information to be parallel to a predetermined vehicle width direction of the vehicle, and, when the demarcation line is detected, sets an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area obtained from the distance information to be parallel to the extension direction of the demarcation line;
a generation unit that generates a travel path of the vehicle based on the extension direction set by the estimation unit;
an integration unit that, when a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, performs processing to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking;
A parking assistance device comprising: obtaining distance information indicating a distance from the vehicle to an obstacle using reflected ultrasonic waves;
detecting parking zone lines;
When the lane marking is not detected, setting an extension direction of the obstacle that passes through an end of the obstacle determined from the distance information so that the extension direction is parallel to a predetermined vehicle width direction of the vehicle;
When the demarcation line is detected, setting an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area, which is determined from the distance information , to be parallel to the extension direction of the demarcation line;
generating a travel path for the vehicle based on the set extension direction;
When a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, a process is performed to integrate the plurality of obstacles into a single obstacle, based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking;
A parking assistance method including: a computer that is mounted on a vehicle and performs processing to assist the movement of the vehicle when the vehicle is parking in a parking area, based on distance information indicating a distance from the vehicle to an obstacle obtained by using reflection of ultrasonic waves;
When a parking area marking line is not detected, a process of setting an extension direction of the obstacle passing through an end of the obstacle obtained from the distance information so that the extension direction is parallel to a predetermined vehicle width direction of the vehicle;
When the demarcation line is detected, a process of setting an extension direction of the obstacle that passes through an end of the obstacle that is closest to the parking area, which is obtained from the distance information , to be parallel to the extension direction of the demarcation line;
A process of generating a travel route of the vehicle based on the set extension direction;
When a plurality of obstacles are detected within a predetermined area based on the distance information and the lane marking is also detected, a process is performed to integrate the plurality of obstacles into a single obstacle based on an extension direction that passes through a portion of the detected obstacle that is closest to the vehicle and is parallel to the extension direction of the lane marking;
A program that executes the following.