JP7111634B2 - OBJECT DETECTION DEVICE, OBJECT DETECTION SYSTEM AND OBJECT DETECTION METHOD IN VEHICLE - Google Patents

Description

The present disclosure relates to techniques for detecting objects around the vehicle used in the vehicle.

2. Description of the Related Art Techniques for executing vehicle driving assistance using a plurality of target detection devices such as imaging devices and radars are known. In this technique, for example, an object is detected based on the detection result of an imaging device (eg, Patent Literature 1).

JP 2010-271788 A

However, when the accuracy of specifying the object using the detection result by the imaging device is low, there is a case where the non-existing object is erroneously detected as the existing object. False detection of an object that intersects the planned trajectory of the vehicle also hinders proper execution of driving assistance.

Therefore, it is required to suppress or prevent erroneous detection of an object that intersects with the planned trajectory of the vehicle when detecting the object using the imaging device.

The present disclosure can be implemented as the following aspects.

A first aspect provides an object detection device in a vehicle. A target object detection device for a vehicle according to a first aspect includes an acquisition unit that acquires feature points indicating targets around the own vehicle extracted from a captured image, and a three-dimensional object and a road using the feature points. a control unit that determines a road boundary line and determines whether or not the determination of the road boundary line is appropriate when the road boundary line intersects the planned trajectory of the vehicle.

According to the object detection device for a vehicle according to the first aspect, it is possible to suppress or prevent erroneous detection of an object that intersects the planned trajectory of the vehicle when detecting the object using the imaging device.

A second aspect provides a method for object detection in a vehicle. A method for detecting an object in a vehicle according to a second aspect obtains characteristic points indicating a target around the own vehicle extracted from a captured image, and uses the characteristic points to determine a road boundary line between a three-dimensional object and a road. and determining whether or not the determination of the road boundary line is appropriate when the road boundary line intersects the planned trajectory of the vehicle.

According to the method for detecting an object in a vehicle according to the second aspect, it is possible to suppress or prevent erroneous detection of an object that intersects the planned trajectory of the vehicle when the imaging device is used to detect the object. Note that the present disclosure can also be implemented as an object detection program for a vehicle or a computer-readable recording medium that records the program.

1 is an explanatory diagram showing an example of a vehicle equipped with an object detection device according to the first embodiment; FIG. 1 is a block diagram showing the functional configuration of an object detection device according to a first embodiment; FIG. 4 is a flowchart showing a processing flow of object detection processing executed by the object detection device according to the first embodiment; FIG. 2 is an explanatory diagram showing the own vehicle running on the roadway and the guardrail; FIG. 5 is an explanatory diagram conceptually showing feature points obtained by a camera in the state shown in FIG. 4 and road boundary lines determined using the feature points; FIG. 2 is an explanatory diagram showing the own vehicle running on the roadway and the guardrail; FIG. 7 is an explanatory diagram conceptually showing feature points obtained by a camera in the state shown in FIG. 6 and road boundary lines determined using the feature points; FIG. 6 is an explanatory diagram conceptually showing the result of dividing the road boundary line shown in FIG. 5 into a plurality of areas; FIG. 8 is an explanatory diagram conceptually showing the result of dividing the road boundary line shown in FIG. 7 into a plurality of areas; FIG. 7 is an explanatory view conceptually showing the result of fusion processing of reflection points obtained by a millimeter-wave radar with feature points obtained by a camera in the state shown in FIG. 6 ; FIG. 5 is an explanatory view conceptually showing the result of fusion processing of reflection points obtained by a millimeter-wave radar with feature points obtained by a camera in the state shown in FIG. 4 ; Conceptually, the result of fusion processing of reflection points obtained by millimeter-wave radar with feature points obtained by a camera is performed on a vehicle that is traveling in a state where the guardrail is located farther than the state shown in FIG. Explanatory diagram showing. FIG. 7 is an explanatory diagram conceptually showing feature points and planned trajectory widths obtained by a camera in the state shown in FIG. 6 ; FIG. 14 is an explanatory diagram conceptually showing the result of fusion processing of feature points obtained by a camera and reflection points obtained by a millimeter-wave radar when a three-dimensional object or a two-dimensional object exists on the planned trajectory in the state shown in FIG. 13 ; FIG. 14 is an explanatory diagram conceptually showing the result of fusion processing of feature points obtained by a camera and reflection points obtained by a millimeter-wave radar when a preceding vehicle exists on a planned trajectory in the state shown in FIG. 13 ; FIG. 9 is an explanatory diagram conceptually showing a state in which the area shown in FIG. 8 is expanded in the depth direction; The flowchart which shows the processing flow of the driving assistance process performed in other embodiment.

An object detection device, an object detection system, and an object detection method in a vehicle according to the present disclosure will be described below based on several embodiments.

First embodiment:
As shown in FIG. 1, an object detection device 100 for a vehicle according to the first embodiment is mounted on a vehicle 500 and used. The object detection apparatus 100 may include at least a control unit and an acquisition unit. A camera 221 is provided. A vehicle 500 according to the first embodiment includes a rotation angle sensor 23, a wheel speed sensor 24, a yaw rate sensor 25, and a driving assistance device 31 in order to realize driving assistance. Vehicle 500 further comprises wheels 501 , braking device 502 , braking line 503 , steering wheel 504 , windshield 510 , front bumper 520 and rear bumper 521 . Vehicle 500 includes at least one of an internal combustion engine and an electric motor as a driving force source for running the vehicle.

The radar ECU 21 is connected to a millimeter wave radar 211 that emits radio waves and detects reflected waves from a target. Generate and output a detection signal representing the target. The camera ECU 22 is connected to a monocular camera 221. The camera ECU 22 identifies the shape of the target from the captured image acquired by the camera 221, quantizes it, and generates a detection signal indicating the target with nodes, that is, feature points. Generate and output. Identification of the target shape is performed by semantic segmentation using machine learning or the like. In this embodiment, a shape line indicating the shape pattern of the identified target, that is, a pixel string is subjected to quantization processing to obtain a plurality of discrete feature points. A detection signal is a signal containing pixel data corresponding to each feature point. Each ECU is a microprocessor that includes an arithmetic unit, a storage unit, and an input/output unit. The radar ECU 21 and the millimeter wave radar 211 emit detection waves, detect reflected waves and reflection points, and generally correspond to detectors for measuring the distance between the vehicle and the target. . As this detector, in addition to the millimeter wave radar 211, a lidar (LIDAR: laser radar) or an ultrasonic detector that emits sound waves and detects the reflected waves may be used. The camera ECU 22 and the camera 221 are detectors capable of three-dimensionally recognizing the shape of a target by at least a point sequence of feature points, and correspond to imaging devices. May be used. The camera 221 may be a stereo camera or a multi-camera made up of two or more cameras.

The vehicle 500 is provided with a braking device 502 for braking the vehicle 500 and a steering wheel 504 for steering the vehicle 500 . A braking device 502 is provided for each wheel 501 . Each braking device 502 is, for example, a disc brake or a drum brake, and brakes each wheel 501 with a braking force corresponding to brake fluid pressure supplied through a braking line 503 in response to the driver's brake pedal operation, Braking of the vehicle 500 is realized. Brake line 503 includes a brake piston and a brake fluid line that generate brake fluid pressure in response to brake pedal operation. As the braking line 503, a control signal line may be used instead of the brake fluid line, and a configuration may be adopted in which an actuator provided in each braking device 502 is operated. The steering wheel 504 is connected to the front wheels 501 via a steering device 42 including a steering rod, a steering mechanism and a steering shaft 44 .

As shown in FIG. 2, the object detection apparatus 100 includes a central processing unit (CPU) 101 and memory 102 as a control section, an input/output interface 103 as an acquisition section, and a bus 104 . The CPU 101, memory 102 and input/output interface 103 are connected via a bus 104 so as to be bidirectionally communicable. The memory 102 includes a memory, such as a ROM, for nonvolatilely and read-only storing an object detection program Pr1 for detecting objects around the own vehicle and a driving assistance program Pr2 for executing driving assistance for the own vehicle; It includes a memory, such as a RAM, which can be read and written by the CPU 101 . The CPU 101 implements a function as a control unit by expanding the target object detection program Pr1 stored in the memory 102 into a readable/writable memory and executing the program. The CPU 101 may be a single CPU, a plurality of CPUs executing each program, or a multi-core type CPU capable of simultaneously executing a plurality of programs.

A radar ECU 21, a camera ECU 22, a rotation angle sensor 23, a wheel speed sensor 24, a yaw rate sensor 25, and a driving support device 31 are connected to the input/output interface 103 via control signal lines. Detection signals are input from the radar ECU 21, the camera ECU 22, the rotation angle sensor 23, the wheel speed sensor 24, and the yaw rate sensor 25, and the driving support device 31 is instructed with a control signal and a braking level corresponding to the required torque. A control signal and a control signal instructing a steering angle are output. Therefore, the input/output interface 103 functions as an acquisition unit for acquiring information on targets around the own vehicle detected by various sensors.

The millimeter wave radar 211 is a sensor that detects the distance, relative speed and angle of the target with respect to the vehicle 500 by emitting millimeter waves and receiving reflected waves reflected by the target. In this embodiment, the millimeter wave radar 211 is arranged on the front bumper 520 and the rear bumper 521 . An unprocessed detection signal output from the millimeter wave radar 211 is processed in the radar ECU 21 and input to the object detection device 100 as a detection signal consisting of detection points or detection point sequences indicating one or more representative positions of the target. be done. Alternatively, a signal indicating an unprocessed received wave may be input from the millimeter wave radar 211 to the object detection device 100 as a detection signal without providing the radar ECU 21 . When an unprocessed received wave is used as a detection signal, the object detection device 100 performs signal processing for identifying the position and distance of the target.

The camera 221 is an image pickup device having an image pickup device such as a CCD or an image pickup device array, and is a sensor that outputs external shape information or shape information of an object as image data, which is a detection result, by receiving visible light. . For image data output from the camera 221, the camera ECU 22 performs object classification processing using, for example, semantic segmentation, and combines pixels indicating the same object to identify each object. Each image region shown is extracted. In this embodiment, the camera 221 can extract roads and solid objects in the vicinity of the roads or on the roads, and detect road boundary lines between the roads and the solid objects. A quantization process is performed on the detected road boundary line, that is, the pixel string, and a frame image in which the road boundary line is represented by discrete feature points is generated as road boundary line information. In the quantization process, for example, the segmented image data is subjected to bird's eye conversion, and a plurality of radial lines obtained as a result of scanning at a predetermined angle with the center of the front of the vehicle as the origin and road boundary lines. This is done by extracting the intersection points as feature points. A feature point, that is, a pixel, included in each frame image is associated with, for example, pixel value information (R, G, B) and coordinate information as position information. Unprocessed image data captured by the camera 221 may be input to the object detection device 100 as the detection signal without separately providing the camera ECU 22 . In this case, target segmentation processing and quantization processing are performed in the object detection apparatus 100 . In this embodiment, the camera 221 is arranged at the top center of the windshield 510 . The pixel data output from the camera 221 may be monochrome pixel data. In this case, luminance values are used for segmentation.

The rotation angle sensor 23 is a torque sensor that detects the amount of twist generated in the steering rod by the steering of the steering wheel 504, that is, the steering torque, as a voltage value proportional to the amount of twist, and detects the steering angle of the steering wheel 504. . In this embodiment, the rotation angle sensor 23 is provided on a steering rod that connects the steering wheel 504 and the steering mechanism.

The wheel speed sensor 24 is a sensor that detects the rotation speed of the wheel 501 and is provided on each wheel 501 . A detection signal output from the wheel speed sensor 24 is a pulse wave indicating a voltage value proportional to the wheel speed or an interval corresponding to the wheel speed. By using the detection signal from the wheel speed sensor 24, it is possible to obtain information such as vehicle speed and travel distance of the vehicle.

Yaw rate sensor 25 is a sensor that detects the rotational angular velocity of vehicle 500 . The yaw rate sensor 25 is arranged, for example, in the center of the vehicle. The detection signal output from the yaw rate sensor 25 is a voltage value proportional to the rotational direction and angular velocity.

The driving support device 31 controls the output of the internal combustion engine ICE according to the accelerator pedal operation by the driver or independently of the accelerator pedal operation by the driver, and the braking device 502 is controlled regardless of the brake pedal operation by the driver. Alternatively, steering by the steering device 42 is realized independently of the operation of the steering wheel 504 by the driver.

An object detection process executed by the object detection device 100 according to the first embodiment will be described. The processing routine shown in FIG. 3 is repeatedly executed at predetermined time intervals, for example, from the time the vehicle control system is started to the time it is stopped, or from when the start switch is turned on until the start switch is turned off. . The object detection process shown in FIG. 3 is executed by the CPU 101 executing the object detection program Pr1. In the object detection process according to the present embodiment, when the road boundary line determined using the feature points acquired by the camera 221 intersects the planned trajectory of the own vehicle, whether or not the determination of the road boundary line is appropriate. is determined.

The CPU 101 acquires, via the input/output interface 103, feature points indicating targets around the own vehicle, which are extracted from the captured image captured by the camera 221 (step S100). When the camera ECU 22 extracts the feature points from the captured image, information on the feature points is input to the CPU 101 from the camera ECU 22 . When the feature points are extracted from the captured image by the CPU 101 , data of the captured image is input to the CPU 101 from the camera ECU 22 .

The CPU 101 determines a road boundary line indicating a boundary between a three-dimensional object and a road using the acquired feature points (step S102). A road boundary line means a boundary line between a roadway, a three-dimensional object located on the side of the roadway, and the roadway. Since the feature points are points that indicate a part of the three-dimensional object, the outer shape of the three-dimensional object is specified by connecting the feature points, and the boundary line with the roadway, which is a plane, is located at the position where the outer shape of the three-dimensional object is specified. can be assumed to exist. Three-dimensional objects include, for example, curbs, guardrail beams, and plants. The CPU 101 applies a plurality of predetermined conditions, connects the plurality of acquired feature points to form a point sequence, and determines the road boundary line. The plurality of predetermined conditions include, for example, that the pixel values of two feature points to be determined are close to each other, that the distance between the two feature points is smaller than a predetermined threshold, Two feature points are obtained continuously in time at the same position. A predetermined threshold for the distance between two feature points can be set larger as the distance between the vehicle and the three-dimensional object increases. When the camera ECU 22 extracts the feature points from the captured image, the camera ECU 22 may determine the road boundary line and provide it to the CPU 101 as road boundary line information. Road boundaries also include the sides of vehicles that make up traffic lights and traffic jams, the surfaces of fences and barriers that are placed on the road and used during road construction, and the crowds that form at the edge of the road. A boundary line formed by a three-dimensional object such as a person's chest or back and a road can be included.

The CPU 101 determines whether or not the planned trajectory of the host vehicle and the road boundary line intersect (step S104). The scheduled trajectory of the own vehicle means the trajectory along which the own vehicle is scheduled to travel, and can be estimated or determined using the steering angle of the own vehicle, for example. When the CPU 101 determines that the planned trajectory and the road boundary line do not intersect (step S104: No), this processing routine ends. This is because the purpose of the present embodiment is to determine whether or not the determination of the road boundary line is appropriate when it is determined that the road boundary line exists on the planned trajectory.

When the CPU 101 determines that the planned trajectory and the road boundary line intersect (step S104: Yes), it determines whether the determination of the road boundary line is appropriate (step S106). In FIG. 4, the host vehicle M0 is traveling toward the guardrail GR on the two-way road sectioned by the center line CL. When the own vehicle M0 is in the running state shown in FIG. 4, the road boundary line BL can be determined as shown in FIG. In FIG. 5, the feature points FP are evenly dispersed. In FIG. 6, the host vehicle M0 is traveling along the center line CL on the two-way road sectioned by the center line CL. When the host vehicle M0 is in the running state shown in FIG. 6, the feature points FP are located on both sides of the road as shown in FIG. There should be no obstacles on the planned trajectory PL, and the planned trajectory PL should not cross the road boundary lines BLl and BLr. However, when the feature points FP are located on both sides of the road as shown in FIG. 7, the road boundary line BL may be determined by joining the feature points FP on both sides. That is, the feature points FP forming the road boundary line BL may be polarized. The polarization of the feature points FP forming the road boundary line BL is such that when the distance between the own vehicle M0 and the three-dimensional object is long, the intervals between the feature points FP become sparse. This is caused by setting a large threshold value. As a result, as shown in FIG. 7, a road boundary line BL that intersects the planned trajectory PL may be determined. Since it is inappropriate to ignore the road boundary line BL once determined without verification, this embodiment determines whether the determination of the road boundary line BL is appropriate. In order to simplify the explanation, the example of polarization has been described, but the same phenomenon can occur in the case of multi-polarization of tri-polarization or higher.

In the first embodiment, the CPU 101 divides the road boundary line BL into a plurality of areas, and determines whether or not the determination of the road boundary line BL is appropriate according to the aspect of the feature points FP in each area. . In the examples of FIGS. 8 and 9, the road boundary line BL is divided into three areas A1 to A3, but it may be divided into two areas or four areas. A region can also be called a virtual block. In the first embodiment,
(1) Whether or not the feature point FP is included in the front area, which is the area that intersects the planned trajectory PL of the own vehicle M0, among the plurality of areas;
(2) whether or not each of the areas A1 to A3 includes at least one feature point FP, and whether or not the number of feature points FP included in each of the areas A1 to A3 is approximately the same, or whether or not there are more feature points FP in areas other than the front area than in the front area;
determines whether or not the determination of the road boundary line BL is appropriate.

When the determination of (1) is executed, as shown in FIG. 8, if the feature point FP is included in the front area of the own vehicle M0, that is, the area A2, the CPU 101 determines the road boundary line BL. Judged as appropriate. On the other hand, as shown in FIG. 9, when the area A2 in front of the host vehicle M0 does not include the feature point FP, the CPU 101 determines that the determination of the road boundary line BL is inappropriate. If the feature point FP is included in the front area of the host vehicle M0, there is, or is likely to be, some continuous three-dimensional object existing on the planned trajectory PL, so the road boundary line BL is determined. is judged to be correct. On the other hand, if the feature point FP is not included in the front area of the own vehicle M0, there is no three-dimensional object in the area that intersects the planned trajectory PL, and the road boundary line BL is composed of a continuous point sequence. In other words, it is composed of discontinuous point sequences, so there is no continuous three-dimensional object, and determination of the road boundary line BL is determined to be inappropriate. In this specification, the determination that the feature points FP are not included in the front area A2 may include the case where the number of feature points FP included in the front area A2 is less than the reference number. . The reference number is, for example, 1 or 2 points.

When the determination of (2) is executed, as shown in FIG. If the numbers are approximately the same, the CPU 101 determines that the determination of the road boundary line BL is appropriate. Approximately the same number means, for example, ±10% of the average number of feature points FP included in each region. When feature points FP exist in each of the areas A1 to A3, and when the number of feature points FP present in each of the areas A1 to A3 are approximately the same, the road boundary line BL is composed of a continuous point sequence, Since it indicates the presence of continuous three-dimensional objects, it is determined that the determination of the road boundary line BL is appropriate. On the other hand, as shown in FIG. 9, when the number of feature points FP included in the area varies, or when the number of feature points FP included in the front area A2 is If the number of feature points FP contained in the areas A1 and A3 is smaller than the number of feature points FP, the CPU 101 determines that the determination of the road boundary line BL is inappropriate. When the number of feature points FP included in each of the areas A1 to A3 varies, the road boundary line BL is composed of discontinuous point sequences and there are no continuous three-dimensional objects. If the number of feature points FP in is small, there is a high possibility that there is no continuous three-dimensional object in the front area A2. Therefore, it is determined that the determination of the road boundary line BL is inappropriate. A ratio of feature points FP may be used instead of the number of feature points FP.

When the CPU 101 determines that the determination of the road boundary line BL is appropriate using the result of the appropriateness determination in step S106 (step S108: Yes), the CPU 101 maintains the determination of the road boundary line BL (step S110). End the processing routine. As a result, the CPU 101 executes the driving assistance process using the road boundary line BL in the subsequent driving assistance process. When the CPU 101 determines that the determination of the road boundary line BL is not appropriate using the result of the appropriateness determination in step S106 (step S108: No), the CPU 101 cancels the determination of the road boundary line BL (step S112). exit. As a result, the CPU 101 executes the driving support processing without using the road boundary line BL in the driving support processing executed thereafter.

According to the object detection device 100 according to the first embodiment described above, the road boundary line BL between the three-dimensional object and the road determined using the feature points FP extracted from the captured image captured by the camera 221 is , it can be determined whether or not the determination of the road boundary line BL is appropriate when the vehicle intersects with the planned trajectory of the own vehicle. As a result, when detecting an object using the camera 221, it is possible to suppress or prevent erroneous detection of an object that intersects the planned trajectory of the vehicle. In addition, it is possible to suppress or prevent the execution of undesired driving support processing using an inappropriate road boundary line BL in the subsequent processing using the road boundary line BL as the object, for example, the driving support processing. That is, when the road boundary line BL is the guardrail GR, even if it is determined that the guardrail GR exists on the planned trajectory PL of the own vehicle using the captured image, whether or not the determination is appropriate. By determining , it is possible to suppress or prevent the execution of the driving support process for the guardrail GR that does not actually exist on the planned trajectory PL.

Second embodiment:
A second mode of determining whether or not the determination of the road boundary line BL is appropriate will be described below. Note that the configuration of the object detection device in each of the following embodiments including the second embodiment is the same as the configuration of the object detection device 100 in the first embodiment, so the same reference numerals are given and the description is omitted. do.

Determination as to whether or not the determination of the road boundary line BL in the second embodiment is appropriate is performed using reflection points by the millimeter wave radar 211 in addition to the feature points FP. For example, the reflection point by the millimeter wave radar 211 may be acquired at the same time as the feature point FP in step S100, or may be acquired at the time of determination in step S106.
(3) Whether or not the front area A2 includes a reflection point.

In the determination of (3), as shown in FIG. 10, if the front area of the host vehicle M0, that is, the area A2 does not include the feature point FP and the reflection point RP, the CPU 101 determines the road boundary line BL. is determined to be inappropriate. If the reflection point RP is not included in addition to the feature point FP, it is reasonable to determine that there is no three-dimensional object in the front area A2 and that there is no road boundary line BL that intersects the planned trajectory PL. is. As shown in FIG. 11, when the feature point FP and the reflection point RP are included in the area A2 in front of the host vehicle M0, the CPU 101 determines that the determination of the road boundary line BL is appropriate. Reflection points RP are included in addition to the feature points FP, and it is rational to determine that there is a three-dimensional object in the front area A2 and that there is a road boundary line BL that intersects the planned trajectory PL. Furthermore, as shown in FIG. 12, even if the front area A2 does not include the feature point FP, if the reflection point RP is included, the CPU 101 determines that the road boundary line BL is not properly determined. Determine that there is. When the own vehicle M0 is traveling as shown in FIG. 6, the reflection point RP and the feature point FP overlap or exist on the same point sequence as shown in FIG. As a result, neither the reflection point RP nor the feature point FP exists in the front area A2. On the other hand, when the own vehicle M0 travels as shown in FIG. 4 and the distance between the guardrail GR and the own vehicle M0 is sufficiently long, the variation between the feature points FP, that is, the distance between the feature points is large. , the feature point FP may not be included in the front area A2. In FIG. 12, the number and pattern of feature points FP included in the region are similar to those shown in FIG. 10, but the variation in feature points FP is large, and the dispersion of point sequences is also large. In this case, even if the front area A2 does not include the feature point FP, if the reflection point RP is included, the aspect shown in FIG. 4 may be included. Therefore, even if the feature point FP is not included in the front area A2, the CPU 101 determines that the specification of the road boundary line BL is appropriate if the front area A2 includes the reflection point RP. judge. In this specification, the determination that the reflection point RP is not included in the front area A2 may include the case where the number of reflection points RP included in the front area A2 is less than the reference number. . The reference number is, for example, 1 or 2 points.

According to the object detection device 100 according to the second embodiment, by using the reflection points obtained by the millimeter wave radar 211, it is possible to determine whether the road boundary line BL determined using the feature points FP is appropriate. can be determined more accurately. In particular, when the feature point FP is not included in the front area A2, the existence of the three-dimensional object on the planned trajectory PL can be repeatedly confirmed by different measurement means.

Third embodiment:
A third mode of determining whether or not the determination of the road boundary line BL is appropriate will be described below. Determination of whether or not the determination of the road boundary line BL in the third embodiment is appropriate is based on the road boundary line BL determined using the feature point FP without using the areas A1 to A3, and the schedule of the own vehicle M0. It is performed using the track width and reflection points by the millimeter wave radar 211 . For example, the reflection point by the millimeter wave radar 211 may be acquired at the same time as the feature point FP in step S100, or may be acquired at the time of determination in step S106.
(4) Whether or not the reflection point RP is included within the planned track width of the own vehicle M0.

In the determination of (4), as shown in FIG. 13, when the reflection point RP is not included within the planned track width MD of the host vehicle M0, the CPU 101 determines that the determination of the road boundary line BL is inappropriate. The planned track width MD is the width of the vehicle M0, which is the width D±α [m], and may be set for each vehicle. May be used. The adjustment margin α is set according to the desired margin, and is, for example, 0.3 to 0.6 m. The CPU 101 aligns the center of the vehicle M0 in the width direction with the center of the planned track width MD, and determines whether or not the reflection point RP is included in a strip-shaped area extending in the traveling direction of the vehicle M0. If the reflection point RP does not exist within the planned track width of the host vehicle M0, there is no three-dimensional object on the planned track PL. It should be noted that the numerical values of the vehicle width D and the adjustment allowance α are merely examples for facilitating understanding, and are not limited to these.

According to the object detection device 100 according to the third embodiment, the road boundary line BL is determined by using the planned track width MD and the reflection point RP without dividing the road boundary line BL into a plurality of areas. is appropriate or not. That is, when there is a reflection point RP indicating a three-dimensional object on the course of the own vehicle M0, it is determined that the determination of the road boundary line BL is appropriate, and the reflection point RP indicating the three-dimensional object is determined on the course of the own vehicle M0. If it does not exist, it is determined that the determination of the road boundary line BL is appropriate. Since it is a condition that the road boundary line BL intersects with the planned trajectory PL, the road boundary line BL corresponding to the guardrail GR is determined without the reflection point RP on the planned trajectory PL. Under normal driving conditions, the determined road boundary line BL is maintained.

Fourth embodiment:
A fourth mode of determining whether or not the determination of the road boundary line BL is appropriate will be described below. The fourth embodiment is an embodiment based on the third embodiment, and whether or not the determination of the road boundary line BL in the fourth embodiment is appropriate is determined without using the areas A1 to A3. , the feature point FP, the planned track width of the own vehicle M0, and the reflection points by the millimeter wave radar 211. FIG. For example, the reflection point by the millimeter wave radar 211 may be acquired at the same time as the feature point FP in step S100, or may be acquired at the time of determination in step S106.
(5) Whether the feature point FP exists near the reflection point RP when the reflection point RP is included in the planned track width MD of the own vehicle M0.
(6) When the reflection point RP is included in the planned track width MD of the host vehicle M0, whether or not the reflection point RP advances on the planned track, or whether the feature point FP is the preceding vehicle on the planned track. whether to indicate

In the determination of (5), as shown in FIG. 14, when the reflection point RP exists within the planned track width MD of the own vehicle M0, the object Ta1 corresponding to the reflection point RP is located on the planned track PL of the own vehicle M0. It is determined whether the object is a plate-like object that reflects a reflected wave of high intensity, such as a manhole or grating, or a three-dimensional object. For example, a manhole is plate-shaped, that is, a plane body, but the intensity of the reflected wave is strong. be. Therefore, the CPU 101 also determines whether or not there is a feature point FP in the vicinity of the reflection point RP, for example, in a radius of 1 m or 1 m square around the reflection point RP. When the feature point FP does not exist in the vicinity of the reflection point RP, the CPU 101 determines that the road boundary line BL is not determined properly because the reflection point RP corresponds to a manhole or the like rather than a three-dimensional object. On the other hand, if the feature point FP exists in the vicinity of the reflection point RP, it is determined that the reflection point RP is highly likely to be a three-dimensional object, and the determination of the road boundary line BL is determined to be appropriate.

In the determination of (6), as shown in FIG. 15, if the reflection point RP exists within the planned track width MD of the own vehicle M0, it is determined whether or not the object corresponding to the reflection point RP is the preceding vehicle M1. be judged. In the determination of (5), even if it is confirmed that the feature point FP exists in the vicinity of the reflection point RP and that the reflection point RP corresponds to a three-dimensional object, the three-dimensional object advances in the traveling direction of the own vehicle M0. It may be vehicle M1. If the three-dimensional object is the preceding vehicle M1, determination of the road boundary line BL including the preceding vehicle M1 is not appropriate. Specifically, when the reflection point RP has a moving speed at which the vehicle M0 travels along the planned trajectory PL, and when the feature point FP is within the planned trajectory width MD of the own vehicle M0, the CPU 101 determines whether the preceding person is on the planned trajectory. If at least one of the above cases is established, it is determined that the reflection point RP corresponds to the preceding vehicle M1, and determination of the road boundary line BL is determined to be inappropriate. The information on the reflection point RP includes the speed information on the reflection point RP, and the determination of the preceding vehicle M1 using the feature point FP is realized by the above-described semantic segmentation or pattern matching. obtain. Furthermore, the speed of the preceding vehicle M1 can be calculated from the amount of movement of the feature point FP between imaging frames.

According to the object detection device 100 according to the fourth embodiment, the road boundary line BL can be determined appropriately using the planned track width MD and the reflection point RP without dividing the road boundary line BL into a plurality of areas. It is possible to improve the determination accuracy when determining whether or not. That is, it is determined whether or not the object of the reflection point RP is the moving preceding vehicle M1, and if it is the preceding vehicle M1, it is determined that the determination of the road boundary line BL is not appropriate. Erroneous determination of BL can be suppressed or prevented.

Fifth embodiment:
As shown in FIG. 16, in the first and second embodiments, the length of the area in the depth direction is variable according to the distance from the own vehicle M0, that is, the camera 221, to the three-dimensional object, that is, the feature point FP. It can be long. Specifically, the longer the distance from the camera 221 to the feature point FP, the longer the length of the region in the depth direction. As a result, it is possible to complement the characteristics of the camera 221 in which the positional error in the depth direction increases as the distance increases, and to improve the accuracy of determining the adequacy of determining the road boundary line BL using the area.

Other embodiments:
(1) Application examples using the identification results output in the first to fifth embodiments will be described. The CPU 101 executes the driving assistance program Pr2, so that the driving assistance processing is executed using the identification result of the target object. The flowchart shown in FIG. 17 is repeatedly executed in the same manner as the flowchart shown in FIG.

The CPU 101 obtains a target object identification result (step S200). The identification result includes, for example, the road boundary line BL determined in each of the above embodiments. The CPU 101 determines whether or not the identification result can identify the target object (step S202). After executing the support process (step S204), the process routine ends. The driving assistance process is realized by CPU 101 outputting at least one of a braking assistance control signal, a steering assistance control signal, and an output control signal to driving assistance device 31 . When the target object cannot be identified, for example, when the road boundary line BL is canceled (step S202: No), the CPU 101 does not execute the driving support process (step S206) and ends this process routine. .

According to the object detection device 100 according to the first to fourth embodiments, it is determined whether or not the determination of the road boundary line BL that intersects the planned trajectory PL is appropriate. It is possible to suppress or avoid the execution of driving assistance for non-existing road boundary lines BL, and it is possible to improve the processing accuracy of driving assistance.

(2) In the above embodiment, if the determination of the road boundary line BL is not proper, the determination of the road boundary line BL, that is, the recording of the road boundary line BL is cancelled. On the other hand, while maintaining the record of the road boundary line BL, for example, a non-use flag may be used so that the road boundary line BL is not used in the driving support process. In this case, the same effect as when the road boundary line BL is canceled can be obtained, and the determined road boundary line BL can be used in other processing such as fusion processing. Alternatively, if the determination of the road boundary line BL is not appropriate, the driving support process may not be executed.

(3) In each of the above-described embodiments, the CPU 101 executes the object detection program Pr1 to implement a control unit that determines whether or not the determination of the road boundary line BL is appropriate in terms of software. , may be implemented in hardware by pre-programmed integrated circuits or discrete circuits. That is, the control unit and its technique in each of the above embodiments can be performed by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. may be implemented. Alternatively, the controls and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring the processor with one or more dedicated hardware logic circuits. Alternatively, the controllers and techniques described in this disclosure comprise a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. may be implemented by one or more dedicated computers configured as The computer program may also be stored as computer-executable instructions on a computer-readable non-transitional tangible recording medium.

Although the present disclosure has been described above based on the embodiments and modifications, the above-described embodiments of the present invention are intended to facilitate understanding of the present disclosure, and do not limit the present disclosure. This disclosure may be modified and modified without departing from its spirit and scope of the claims, and this disclosure includes equivalents thereof. For example, the technical features in the embodiments and modifications corresponding to the technical features in each form described in the Summary of the Invention are used to solve some or all of the above problems, or In order to achieve some or all of the effects, it is possible to appropriately replace or combine them. Also, if the technical features are not described as essential in this specification, they can be deleted as appropriate.

DESCRIPTION OF SYMBOLS 10... Object detection system, 22... Camera ECU, 221... Camera, 31... Driving assistance device, 100... Object detection apparatus, 101... CPU, 102... Memory, 103... Acquisition part, 104... Bus, 500... Vehicle, Pr1 . . . Object detection program.

Claims (10)

An object detection device (100) in a vehicle (500),
an acquisition unit (103) for acquiring feature points representing targets around the own vehicle extracted from the captured image;
A road boundary line between a three-dimensional object and a road is determined using the feature points, and if the road boundary line intersects the planned trajectory of the own vehicle, whether or not the determination of the road boundary line is appropriate. and a control unit (101, Pr1) for determining. In the object detection device for a vehicle according to claim 1,
The control unit divides the road boundary line into a plurality of areas, and if the feature point is not included in a front area that intersects the planned trajectory of the own vehicle among the plurality of areas, the road boundary An object detection device in a vehicle that determines that line determination is not correct. In the object detection device for a vehicle according to claim 1,
The control unit divides the road boundary line into a plurality of areas, and determines that the number of the feature points included in the front area intersecting the planned trajectory of the vehicle among the plurality of areas is the number of the feature points other than the front area. An object detection device for a vehicle, which determines that the determination of the road boundary line is not appropriate when the number of the feature points included in the area is smaller than the number of the feature points. In the object detection device for a vehicle according to claim 2 or 3,
The acquisition unit further acquires a reflection point indicating a target around the own vehicle,
The control unit determines whether the feature points are not included in the front area, or when the number of feature points included in the front area is greater than the number of feature points included in the areas other than the front area. object detection device for a vehicle, determining that the determination of the road boundary line is appropriate when the reflection points are included in the front area even if the number of the reflection points is small. In the object detection device for a vehicle according to claim 1,
The acquisition unit further acquires a reflection point indicating a target around the own vehicle,
The object detection device for a vehicle, wherein the control unit determines that the determination of the road boundary line is not appropriate when the reflection point does not exist within the planned track width of the vehicle. In the object detection device for a vehicle according to claim 5,
The control unit determines that the road boundary line is not appropriate when the feature point does not exist near the reflection point even when the reflection point exists within the planned track width of the vehicle. An object detection device for a vehicle, which determines that In the object detection device for a vehicle according to claim 5,
Even when the reflection point exists within the width of the planned track of the vehicle, the control unit controls the control unit to control the reflection point when the reflection point advances and moves on the planned track of the vehicle, and when the feature point is on the planned track of the vehicle. and determining that the determination of the road boundary line is inappropriate when at least one of the preceding vehicles is established. An object detection system (10) mounted on a vehicle (500), comprising:
an object detection device (100) according to any one of claims 1 to 3;
an imaging device (22, 221) that acquires the captured image;
An object detection system, comprising: An object detection system (10) mounted on a vehicle (500), comprising:
an object detection device (100) according to any one of claims 4 to 7;
an imaging device (22, 221) that generates the captured image;
and a detector (21, 211) that emits a detection wave and detects the reflection point. A method for detecting an object in a vehicle (500), comprising:
Obtaining feature points indicating targets around the own vehicle extracted from the captured image,
determining a road boundary line between a three-dimensional object and a road using the feature points;
A method for detecting an object in a vehicle, comprising: determining whether or not the determination of the road boundary line is appropriate when the road boundary line intersects a planned trajectory of the own vehicle.

Images