JP4972073B2 - Environment recognition device - Google Patents

Description

  The present invention relates to an environment recognition device that recognizes an environment using a captured image obtained by an imaging device.

Extracting a region corresponding to the pedestrian's head, which is an object, from an image captured by an imaging device mounted on the vehicle, and extracting a region corresponding to the pedestrian's torso based on the head region Is known to recognize that the object is a pedestrian and inform the vehicle driver of the presence of the pedestrian (see Patent Document 1).
Japanese Patent Laid-Open No. 11-328364

  However, there are cases where not only humans such as pedestrians or bicycle occupants but also quadruped animals such as deer, sheep, dogs and horses exist around the vehicle. In this case, it is difficult to determine whether or not the object is a quadruped animal even if a method for determining whether or not the object is a human is employed. For this reason, it becomes difficult to inform the driver of the vehicle of the presence of a quadruped animal, and to drive the vehicle so that the driver avoids contact with the object.

  Therefore, an object of the present invention is to provide an apparatus capable of improving the recognition accuracy of whether or not an object existing in an environment including an imaging range by an imaging apparatus corresponds to a quadruped animal.

  An environment recognition device according to a first aspect of the present invention is an environment recognition device that recognizes an environment using a captured image obtained by an imaging device, and that extracts an image region where a target object exists from the captured image as a target region. Object extraction means, principal component extraction means for extracting a principal component vector representing the posture of the object from the object region extracted by the object extraction means, and the principal component extracted by the principal component extraction means Depends on whether the inclination angle of the vector with respect to the horizontal axis is periodically fluctuating across the reference angle, which is the inclination angle of the reference axis with respect to the horizontal axis, which is equal to or greater than the inclination angle with respect to the horizontal axis. And an object type determining means for determining whether or not the object corresponds to a quadruped animal.

  According to the environment recognition apparatus of the first invention, the principal component vector representing the posture of the object is extracted, and the inclination angle of the principal component vector with respect to the horizontal axis (the horizontal or horizontal axis of the image) is the reference angle. It is determined whether or not the object corresponds to a quadruped animal depending on whether or not it periodically changes between the two. The “reference angle” is an inclination angle of the reference axis with respect to the horizontal axis, and an axis having a posture such that the inclination angle of the “reference axis” with respect to the vertical axis is equal to or greater than the inclination angle with respect to the horizontal axis. When a walking or running quadruped animal is observed from the lateral direction, the front part of the torso is displaced more vertically than the rear part of the torso, so the tilt angle of the whole torso with respect to the horizontal direction may change periodically. is there. In view of this, by extracting a principal component vector representing the posture of the torso, whether or not the object corresponds to a quadruped animal can be recognized with high accuracy according to the above-described method. The “quadruped animal” means not all the quadruped animals but a quadruped animal that can be recognized by the technique of the present invention.

  The environment recognition apparatus according to a second aspect is characterized in that, in the environment recognition apparatus according to the first aspect, the principal component extraction means extracts the principal component vector by performing principal component analysis on the object region. And

  According to the vehicle periphery monitoring device of the second aspect of the invention, when a walking or running quadruped animal is observed from the lateral direction, the inclination angle of the entire trunk with respect to the horizontal direction may periodically change. In view of this, by extracting a principal component vector representing the posture of the torso, whether or not the object corresponds to a quadruped animal can be recognized with high accuracy according to the above-described method.

  The environment recognition apparatus according to a third aspect of the present invention is the environment recognition apparatus according to the first aspect, wherein the principal component extraction means extracts a thin line by executing a thinning process on the object region, and the principal component is targeted for the thin line. The principal component vector is extracted by performing an analysis.

  According to the environment recognition device of the third aspect of the invention, when a walking or running quadruped animal is observed from the lateral direction, the inclination angle of the entire torso with respect to the horizontal direction may periodically change. The principal component vector representing the posture of the trunk is extracted by performing the thinning process and the principal component analysis on the thin line obtained thereby, so that the object corresponds to a quadruped animal according to the above method It can be recognized with high accuracy.

  The environment recognition device according to a fourth aspect is characterized in that, in the environment recognition device according to the first aspect, the object extraction means extracts an image area where a body portion of the object is present as the object area.

  According to the environment recognition device of the fourth aspect of the invention, when a quadruped walking or running is observed from the lateral direction, the posture of the trunk may change periodically as described above. By extracting a principal component vector representing the posture of the torso as an object, whether or not the object corresponds to a quadruped animal (more precisely, the torso of a quadruped animal) according to the above method is highly accurate. Can be recognized.

  An environment recognition device according to a fifth aspect of the present invention is the environment recognition device according to any one of the first to fourth aspects, wherein the environment recognition device is mounted on a vehicle together with the imaging device.

  According to the environment recognition device of the fifth aspect of the present invention, it can be determined with high accuracy whether or not an object existing around the vehicle on which the environment recognition device is mounted together with the imaging device corresponds to a quadruped animal.

  An embodiment of the environment recognition apparatus of the present invention will be described. First, the configuration of the environment recognition device of the present embodiment will be described. The environment recognition apparatus includes an image processing unit 1 shown in FIGS. The image processing unit 1 is connected to a pair of left and right infrared cameras (corresponding to the “imaging device” of the present invention) 2R and 2L that image the front of the vehicle 10 and sensors that detect the traveling state of the vehicle 10. The yaw rate sensor 3 for detecting the yaw rate of the vehicle 10, the vehicle speed sensor 4 for detecting the traveling speed (vehicle speed) of the vehicle 10, and the brake sensor 5 for detecting the presence or absence of a brake operation of the vehicle 10 are connected. Further, the image processing unit 1 displays a captured image or visual report information captured by a speaker or voice output device 6 for outputting auditory report information such as voice and the infrared cameras 2R and 2L. The HUD or the image display device 7 arranged near the front window is connected to the front window. When the distance to the object is measured by a radar, only one infrared camera may be mounted on the vehicle 10. Instead of or in addition to the HUD, a display for displaying a traveling state such as the vehicle speed of the vehicle 10 or a display included in the in-vehicle navigation device may be employed as the image display device 7.

  The image processing unit 1 includes an ECU (electronic control unit) that includes an A / D conversion circuit, a memory such as a CPU, a RAM, and a ROM, an I / O circuit, and a bus that connects these. The image processing unit 10 (specifically, a CPU that executes arithmetic processing in accordance with a program stored in a memory or a storage device) performs “object extraction means”, “principal component extraction means”, and “object type determination” according to the present invention. Means ". Analog signals output from the infrared cameras 2R, 2L, the yaw rate sensor 3, the vehicle speed sensor 4, and the brake sensor 5 are converted into digital data by the A / D conversion circuit, and based on the data digitized by the CPU, humans (pedestrians) Or an object such as a cyclist or a two-wheeled vehicle) or a quadruped animal (such as a deer, a horse or a dog), and when the detected object satisfies a predetermined notification requirement, the object is detected through the speaker 6 or the HUD 7. The driver is notified of the presence or the possibility of contact between the vehicle 10 and the object.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion of the vehicle 10 (the front grill portion in the figure) in order to image the front of the vehicle 10. The right infrared camera 2R is disposed at a position to the right of the center of the vehicle 10 in the vehicle width direction, and the left infrared camera 2L is disposed at a position to the left of the center of the vehicle 10 in the vehicle width direction. Their positions are symmetrical with respect to the center of the vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are fixed so that their optical axes extend in the front-rear direction of the vehicle 10 in parallel with each other, and the heights of the respective optical axes from the road surface are equal to each other. The infrared cameras 2R and 2L have sensitivity in the far-infrared region, and have a characteristic that the higher the temperature of an object to be imaged, the higher the level of the output video signal (the higher the luminance of the video signal). is doing.

  Next, functions of the environment recognition apparatus having the above-described configuration will be described. Details of the image processing are disclosed in Japanese Patent Laid-Open Nos. 2001-006096 and 2007-310705, and will be briefly described. First, infrared image signals of the infrared cameras 2R and 2L are input to the image processing unit 1, the infrared image signal is A / D converted, and a gray scale image is generated based on the A / D converted infrared image signal. Then, the reference grayscale image (right image) is binarized. After that, an area where the object exists in the binarized image is extracted as the object area S (FIG. 3 / STEP002). Specifically, a pixel group constituting a high luminance area of the binarized image is converted into run length data, and a label (identifier) is attached to each of the line groups that overlap in the vertical direction of the reference image. Are extracted as the object region S. As a result, a high-luminance region constituted by a group of high-luminance pixels (pixel value = 1 pixel) as shown by hatching in FIGS. 4A and 4B is extracted as the object region S. The

  Subsequently, the position of the center of gravity of each object (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle are calculated. Further, tracking of the target object is performed, and the identity of the target object is determined for each calculation processing cycle of the image processing unit 1. Further, the outputs (vehicle speed detection value and yaw rate detection value) of the vehicle speed sensor 4 and the yaw rate sensor 5 are read. Further, in parallel with the calculation of the aspect ratio of the circumscribed rectangle and the tracking of the object during the time, the area corresponding to each object (for example, the area of the circumscribed rectangle of the object) is extracted as a search image from the reference image. Furthermore, by executing the correlation calculation, an image (corresponding image) corresponding to the search image is searched and extracted from the left image. Further, a real space distance (distance in the front-rear direction of the vehicle 10) z from the vehicle 10 to the object is calculated (FIG. 3 / STEP004). Moreover, the real space position (relative position with respect to the vehicle 10) of each object is calculated. Further, the X component of the real space position (X, Y, Z) (see FIG. 2) of the object is corrected according to the time-series data of the turning angle. Further, a relative movement vector of the object with respect to the vehicle 10 is calculated (FIG. 3 / STEP006).

  Furthermore, the level of possibility of contact between the vehicle 10 and the object is determined (FIG. 3 / STEP008). Since this determination method is also disclosed in Japanese Patent Laid-Open Nos. 2001-006096 and 2007-310705, it will be briefly described. First, it is determined whether or not the real space distance z of the object is equal to or less than the product of the relative speed Vs and the margin time T. When it is determined that the real space distance z is equal to or less than the value, it is determined whether or not the object exists in the approach determination region (first contact determination process). The approach determination area extends symmetrically in the left-right direction of the vehicle 10 in front of the vehicle 10 and has a width obtained by adding an offset value to the width of the vehicle 10. When the determination result in the first contact determination process is affirmative, it is determined that there is a high possibility that the object is in contact with the vehicle 10. On the other hand, when the determination result in the first contact determination process is negative, the object exists in the entry determination area outside the approach determination area, and the relative movement vector of the object moves toward the approach determination area. It is further determined whether or not there is (second contact determination process). When the determination result in the second contact determination process is affirmative, it is determined that there is a high possibility that the target object is in contact with the vehicle 10.

  When it is determined that the possibility of contact between the vehicle 10 and the object is high (when the determination result in the first or second contact determination process is affirmative) (FIG. 3 / STEP008... YES), the object is artificial. It is determined whether or not the object corresponds to an object whose necessity of reporting its existence is low (FIG. 3 / STEP010). For example, when an external feature that does not correspond to a pedestrian or a quadruped is detected in this object, it is determined that the object corresponds to an artificial structure. When it is determined that the possibility of contact between the vehicle 10 and the object is low (when the determination results in the first and second contact determination processes are negative) (FIG. 3 / STEP008... NO), or the object Is determined not to fall under the notification target (FIG. 3 / STEP010... NO), the processing after the extraction of the object region S is repeated without executing the notification processing described later (see FIG. 3 / STEP002, etc.). .

  When it is determined that the target object does not correspond to a notification target such as an artificial structure (FIG. 3 / STEP010... YES), it is determined whether or not the target object corresponds to a person such as a pedestrian or a bicycle occupant ( FIG. 3 / STEP012). Specifically, it is determined whether or not the object corresponds to a person based on the shape or size of the object region S on the gray scale image or the feature quantity such as the luminance distribution.

When it is determined that the object does not correspond to a human (FIG. 3 / STEP012... NO), it is determined whether or not the object corresponds to a quadruped animal such as a deer, a horse, or a dog (FIG. 3). / STEP014). The details of this determination process, which is a feature of the present invention, will be described. For example, the principal component analysis is performed on the object region S as shown in FIGS. 4A and 4B. That is, a single vector representing a plurality of vectors x ₁ , x ₂ ,... X _n representing the position of each pixel (high luminance pixel, indicated by “◯”) constituting the object area S is obtained. The first principal component (vector) y ₁ indicated by the arrow is extracted according to a known principal component analysis. Then, it is determined whether or not the object corresponds to a quadruped animal depending on whether or not the inclination angle of the principal component vector y ₁ with respect to the horizontal axis of the image is periodically changing across the reference angle. The The reference angle is an inclination angle of the reference axis with respect to the horizontal axis indicated by a broken line in FIGS. 4A and 4B, and the inclination angle of the reference axis with respect to the image vertical axis is set to be greater than or equal to the inclination angle with respect to the image horizontal axis. Yes. As shown in FIG. 4A, the first principal component vector y ₁ is on the upper side of the reference axis, and as shown in FIG. 4B, the first principal component vector y ₁ is the reference. When the state under the axis is periodically repeated, it is determined that the object corresponds to a quadruped animal. The inclination angle of the reference axis shown in FIGS. 4A and 4B with respect to the horizontal axis is 0 °, but if it is less than the inclination angle of the reference axis with respect to the vertical axis, that is, less than 45 °, It may be set arbitrarily such as 5 °, 10 ° or 15 °. In place of the object region S, the first principal component vector y is used for a thin line obtained by thinning the object region S as shown in FIGS. 4C and 4D. ₁ may be extracted.

  When it is determined that the object corresponds to a human (FIG. 3 / STEP012... YES), or when it is determined that the object corresponds to a quadruped animal (FIG. 3 / STEP014... YES), “report processing” Is executed (FIG. 3 / STEP016). Thereby, sound is output from the speaker 6. Further, a frame that highlights the target object by surrounding the target object is displayed on the HUD 7. On the other hand, when it is determined that the object does not correspond to a human or a quadruped animal (FIG. 3 / STEP012... NO, STEP014... NO), the process after the extraction of the object region S is performed without executing the notification process. Repeated (see FIG. 3 / STEP002 etc.).

According to the environment recognition apparatus 1 that exhibits the above function, the first principal component vector y ₁ representing the posture of the object is extracted, and the inclination angle of the principal component vector y ₁ with respect to the horizontal axis is a period across the reference angle. It is determined whether or not this object corresponds to a quadruped animal depending on whether or not it has changed in a moving manner (see FIG. 3 / STEP014 and FIGS. 4A to 4D). The “reference angle” is an inclination angle of the reference axis with respect to the horizontal axis, and an axis having a posture such that the inclination angle of the “reference axis” with respect to the vertical axis is equal to or greater than the inclination angle with respect to the horizontal axis. When a walking or running quadruped animal is observed from the lateral direction, the front part of the torso is displaced more vertically than the rear part of the torso, so the tilt angle of the whole torso with respect to the horizontal direction may change periodically. is there. In view of this, by extracting a principal component vector representing the posture of the torso, whether or not the object corresponds to a quadruped animal can be recognized with high accuracy according to the above-described method.

  In the above-described embodiment, the environment recognition device is mounted on the vehicle 10. However, as another embodiment, the environment recognition device is connected to the imaging device, or the device is configured integrally with the imaging device. It may be mounted on a vehicle other than the vehicle 10 (four-wheeled vehicle), or may be carried or carried by a user.

  Also, the image area where the body of the object is present (for example, a group of high-intensity areas whose width is larger than the height (height) and whose real space size belongs to the specified range) is the object area. May be extracted as According to this configuration, when a quadruped animal that is walking or running is observed from the lateral direction, the posture of the torso may change periodically as described above. By extracting the principal component vector representing the posture of the torso, it is possible to recognize with high accuracy whether or not the object corresponds to a quadruped animal (more precisely, the torso of a quadruped animal) according to the above method.

Configuration explanatory diagram of the environment recognition apparatus of the present invention Perspective view of a vehicle equipped with an environment recognition device Flow chart showing functions of environment recognition device Explanatory diagram regarding the principal component vector extraction method

Explanation of symbols

1. Image processing unit (object extracting means, object determining means), 2R, 2L, infrared camera (imaging device)

Claims (5)

An environment recognition device that recognizes an environment using a captured image obtained by an imaging device,
An object extraction means for extracting an image area where the object exists from the captured image as an object area;
Principal component extraction means for extracting a principal component vector representing the posture of the object from the object region extracted by the object extraction means;
The inclination angle of the principal component vector extracted by the principal component extraction means with respect to the horizontal axis is a reference angle that is an inclination angle of the reference axis with respect to the horizontal axis that is greater than or equal to the inclination angle with respect to the horizontal axis. And an object type determining means for determining whether or not the object corresponds to a quadruped animal according to whether or not it periodically varies. The environment recognition device according to claim 1,
The environment recognition apparatus, wherein the principal component extraction unit extracts the principal component vector by performing principal component analysis on the object region. The environment recognition device according to claim 1,
The principal component extraction means extracts a thin line by executing a thinning process on the object region, and extracts a principal component vector by executing a principal component analysis on the thin line. Environment recognition device. The environment recognition device according to claim 1,
The environment recognition apparatus, wherein the object extraction unit extracts an image area where a body portion of the object exists as the object area. In the environment recognition device according to any one of claims 1 to 4,
An environment recognition device mounted on a vehicle together with the imaging device.

Images