JP4817982B2 - Vehicle identification system and vehicle identification method - Google Patents

Description

  The present invention relates to a technology for identifying a vehicle existing in a predetermined observation space.

As a technique for identifying a vehicle existing in a predetermined observation space, a technique for identifying a vehicle reflected in a captured image by image recognition processing for the captured image is known (for example, Patent Document 1).
JP-A-8-16978

  Now, in the technology for identifying a vehicle by performing image recognition processing on the above-described captured image, in general, in order to identify the vehicle reflected in the captured image, a captured image and a vehicle provided in advance as a template are identified. Pattern matching is performed with an image pattern representing a standard form. Here, since the form of the vehicle greatly varies depending on the direction of the vehicle, usually, a plurality of image patterns representing standard forms of the vehicle facing a plurality of different directions are prepared as templates. The vehicle is identified by pattern matching between each of a plurality of image patterns prepared as templates and a captured image. That is, according to this technique, it is necessary to perform pattern matching a plurality of times for vehicle identification, which tends to increase the processing load.

  Accordingly, an object of the present invention is to more efficiently identify a vehicle existing in a predetermined observation space.

  In order to achieve the above object, the present invention provides a vehicle identification system for identifying a vehicle, a radar device, a camera, and a radar reflection intensity measured by the radar device at each time point for the same vehicle. Based on a standard pattern relationship between the vehicle orientation and radar reflection intensity set in advance, based on a transition pattern associated with a change in direction with respect to the radar device measured by the device, in the image captured by the current camera of the vehicle And an image recognition process optimized for identifying a vehicle facing the direction according to the direction estimated by the estimation unit, and an image captured by the camera. The image recognition processing means for identifying the vehicle appearing in the image is included.

  According to such a vehicle identification system, the direction of the vehicle reflected in the image photographed by the camera is estimated based on the transition pattern of the radar reflection intensity with respect to the direction change of the vehicle position, and the estimated direction is directed. When the image recognition process optimized for vehicle identification is performed to identify the other vehicle, for each direction of the vehicle, image recognition that similarly identifies the vehicle in the direction is performed to identify the vehicle Compared to the above, the vehicle can be identified efficiently.

  Here, in such a vehicle identification system, the estimation means changes the direction of the radar reflection intensity measured for the same vehicle by the radar device at each time point with respect to the radar device measured by the radar device. Based on the standard relationship between the vehicle type and orientation set in advance and the radar reflection intensity, the type of vehicle and the direction reflected in the current image captured by the camera are estimated from the transition pattern associated with In the image recognition processing means, an image recognition process optimized for identifying the vehicle of the type facing the direction is applied to the image captured by the camera according to the direction and type estimated by the estimation means. For example, the vehicle may be configured to identify the vehicle reflected in the image.

  According to such a vehicle identification system, based on the change pattern of the radar reflection intensity with respect to the direction change of the vehicle position, the type of the vehicle reflected in the image photographed by the camera is further estimated, and the estimated direction is directed. Since the other vehicle is identified by performing image recognition processing suitable for identifying the estimated type of vehicle, the identification of the type of vehicle facing the estimated direction is the same for each type of vehicle. As compared with the case where the vehicle is identified by performing the image recognition performed in the above, the vehicle can be identified more efficiently.

  Each of the above vehicle identification systems may be mounted on a vehicle and identify other vehicles existing around the vehicle.

  As described above, according to the present invention, a vehicle existing in a predetermined observation space can be identified more efficiently.

Hereinafter, an embodiment of the present invention will be described taking application to a periphery monitoring system mounted on an automobile as an example.
FIG. 1a shows the configuration of the periphery monitoring system according to the present embodiment.
As shown in the figure, the periphery monitoring system includes a radar device 1, a camera 2, a target tracking unit 3, an image recognition processing unit 4, an other vehicle direction calculation unit 5, an RCS table group 6, a tracking information holding unit 7, and a surrounding situation presentation. Part 8.
Here, assuming that the vehicle equipped with the periphery monitoring system is the own vehicle 100, the radar device 1 and the camera 2 are arranged at the rear of the own vehicle 100 as shown in FIG. Further, the radar apparatus 1 repeats the process of measuring the relative position and relative speed of an object within the scan range and the intensity of radar reflection on the object and outputting it as measurement data with the back of the host vehicle 100 as the scan range. Moreover, the camera 2 repeats the process which image | photographs the image | video of the imaging range behind the own vehicle 100. FIG.

  The target tracking unit 3 tracks each object existing behind the host vehicle 100 based on the measurement data output from the radar device 1 and generates tracking information for each tracking object that is a tracking object. The target tracking process to be managed is repeated. Here, tracking of an object is a process of serializing measurement data calculated at each time point as time-series measurement data of the object for each object for which the measurement data is calculated. In addition, the tracking information of a certain tracking object refers to time-series measurement data of the tracking object that is serialized in this way, and this tracking information is stored in the tracking information holding unit 7 by the target tracking unit 3. Is done.

In the target tracking process, for example, an object is tracked as follows.
That is, when measurement data output from the radar device 1 includes measurement data indicating a relative position that is close to the current estimated relative position of a tracking object within an allowable distance, the measurement data is The measured data of the tracking object is added to the tracking information of the tracking object stored in the tracking information holding unit 7 as the measurement data of the tracking object. Here, the current estimated relative position of the tracking object is the relative speed predicted from the relative position indicated by the most recent measurement data of the tracking object, the history of changes in the past relative position of the tracking object, and the like. Calculate based on

  Also, if there is measurement data that could not be measured as the measurement data of the object being tracked, the object for which the measurement data was measured is regarded as the new tracking object and its tracking information. Is generated on the tracking information holding unit 7, and the existing measurement data is added to the tracking information of the new tracking object. Further, in the target tracking process, tracking is terminated for the tracking object for which the corresponding measurement data has not been measured for a predetermined period or longer, and the tracking information for the tracking object is deleted from the tracking information holding unit 7. .

  Next, the other vehicle direction calculation unit 5 calculates, for each tracking target object that is tracked by the target tracking by the other vehicle direction calculation process, which will be described in detail later, a small vehicle / one box vehicle / Types of ordinary automobiles / small motorcycles / medium-sized motorcycles / large motorcycles, and the direction (front-facing, diagonally left-facing forward, right-side-facing, etc.) reflected in the image captured by the camera 2 of the vehicle being tracked It is estimated and notified to the image recognition processing unit 4 as other vehicle information together with the current relative position of the vehicle that is the object being tracked.

   Then, the image recognition processing unit 4 applies the region around the position where the relative position included in the other vehicle information notified from the other vehicle direction calculation unit 5 in the image captured by the camera 2 is reflected. Image recognition processing for recognizing the reflected object and its type is performed, and the recognition result is output to the surrounding situation presentation unit 8.

  Here, the image recognition processing unit 4 corresponds to each of the six types of vehicles, that is, a small vehicle, a one-box vehicle, a normal vehicle, a small motorcycle, a medium motorcycle, and a large motorcycle as a pattern matching template. A plurality of image pattern sets, and each image pattern set includes an image pattern provided for each of a plurality of different directions and representing a standard form of a corresponding type of vehicle facing the direction. It is.

  Then, the image recognition processing unit 4 adds the image pattern corresponding to the type and direction of the vehicle included in the other vehicle information notified from the other vehicle direction calculating unit 5 and the other vehicle information in the image captured by the camera 2. Perform pattern matching with the area around the position where the relative position included is reflected, identify the other car and its type reflected in the area, and from the position in the image where the other car is reflected, The relative position of the other vehicle is calculated. Then, the type of the identified other vehicle and the calculated relative position are output to the surrounding situation presentation unit 8 as the identification result. In addition, when pattern matching using an image pattern corresponding to the type and orientation of the vehicle included in the other vehicle information fails, another image pattern is sequentially applied to the region until the pattern matching is successful. By performing the pattern matching used, the other vehicle and its type are identified, the relative position of the other vehicle is calculated, and the other vehicle information described above is output. However, when the other vehicle information notified from the other vehicle direction calculation unit 5 indicates that the type and direction of the vehicle are unknown, each image pattern is used in accordance with a predetermined image pattern order. By performing the pattern matching until the pattern matching is successful, the other vehicle and its type are identified, the relative position of the other vehicle is calculated, and the aforementioned other vehicle information is output. In any case, if the pattern matching fails for all the image patterns, the other vehicle information is not output.

  That is, the image recognition processing unit 4 first holds a template as a template when, for example, a medium-sized motorcycle is notified as the type of the other vehicle and the front left diagonal as the direction from the other vehicle direction calculation unit 5 according to the other vehicle information. Then, pattern matching is performed between the image pattern of the medium-sized motorcycle that faces the left diagonal front and the area around the position where the relative position included in the other vehicle information in the image captured by the camera 2 is reflected. If the pattern matching is successful, the relative position of the image pattern relative to the vehicle behind the vehicle corresponding to the region in the captured image where the pattern is matched is calculated as a relative position, and the medium size automatic is calculated at the calculated relative position. An identification result indicating that the two-wheeled vehicle has been identified is output to the surrounding situation presentation unit 8. On the other hand, if pattern matching using an image pattern of a medium-sized motorcycle facing diagonally to the left fails, pattern matching using another image pattern stored as a template is sequentially performed until pattern matching succeeds. Go.

  Then, the surrounding state presentation unit 8 is based on the tracking information of each tracking target stored in the tracking information holding unit 7 and the other vehicle identification result by the image recognition processing in the image recognition processing unit 4. A process of displaying, for example, the arrangement and behavior of other vehicles behind the vehicle to the user is performed.

Hereinafter, the other vehicle direction calculation process described above performed by the other vehicle direction calculation unit 5 will be described.
First, the RCS table group 6 used in this other vehicle direction calculation process will be described.
The RCS table group 6 is a set of RCS tables provided for each type of vehicle.
In the present embodiment, RCS tables are provided for six types of vehicles: small vehicles, one-box vehicles, ordinary vehicles, small motorcycles, medium-sized motorcycles, and large motorcycles.
Each RCS table represents the RCS (Radar Cross Section; radar effective reflection area) of each direction θ of the corresponding type of vehicle. The RCS of each direction θ is obtained when the radar is irradiated from that direction. It represents the degree of radar reflection intensity.
For example, as shown in FIG. 2, the RCS 21 registered in each RCS table uses the radar reflection intensity 20 measured by irradiating the vehicle of a corresponding type from each direction with a radar from a certain distance by experiment, and the spatial frequency in the θ direction. Is smoothed and registered in the RCS table. In each graph of FIG. 2, the radial direction is the axis of radar reflection intensity and RCS.

  In FIG. 2, a is RCS21 in each direction represented by the RCS table of the small car, b is RCS21 in each direction represented by the RCS table of the one-box car, and c is RCS21 in each direction represented by the RCS table of the ordinary car. D represents the RCS 21 in each direction represented by the RCS table of the small motorcycle, e represents the RCS 21 in each direction represented by the RCS table of the medium motorcycle, and f represents the RCS 21 in each direction represented by the RCS table of the large motorcycle. ing. The value of θ is set so that θ = 0 represents the RCS on the front surface of the vehicle and θ = π / 2 represents the RCS on the rear surface of the vehicle.

Next, the other vehicle direction calculation process performed by the other vehicle direction calculation unit 5 will be described.
FIG. 3 shows a processing procedure of the other vehicle direction calculation processing.
As shown in the drawing, in this process, first, the target tracking unit 3 monitors the occurrence of update of tracking information of any tracking target held in the tracking information holding unit 7 (step 302). If the update occurs, the tracking object whose tracking information is updated is set as the calculation target object (step 304).

  First, a history of the relative position of the calculation target object and the radar reflection intensity indicated by each measurement data registered in the tracking information of the calculation target object is acquired (step 306), and the radar of the calculation target object is acquired from the acquired history. A change pattern of the radar reflection intensity with respect to the direction viewed from the apparatus 1 is obtained (step 308). Here, the change pattern of the radar reflection intensity is obtained as follows.

  That is, first, for each measurement data, the direction θ of the calculation target object viewed from the radar device 1 and the distance from the radar device 1 are calculated from the relative positions indicated by the measurement data. Then, the radar reflection intensity indicated by the measurement data is corrected according to the calculated distance to the reflection intensity F that would have been obtained if the distance to the calculation target object was a predetermined distance. Then, on the two-dimensional space having the axis of direction θ and the axis of reflection intensity F, the curve connecting (θ, F) obtained for each measurement data in the θ-axis direction is smoothed, and the radar apparatus 1 of the calculation target object The change pattern of the radar reflection intensity with respect to the direction seen from above.

  Here, when the motorcycle 400 approaches the host vehicle 100 from the rear as shown in FIG. 4a1-a2-a3 by the processing in step 308, for example, the radar device at each time point a1, a2, a3. 1 is an angle θ viewed from the radar device 1 and the camera 2 of the motorcycle 400 indicated by a relative position measured with respect to the motorcycle 400, and a radar reflection intensity measured by the radar device 1 with respect to the motorcycle 400 at each time point. From the reflection intensity F corrected in accordance with the distance to the motorcycle 400, a change pattern of the radar reflection intensity F with respect to the direction θ viewed from the radar apparatus 1 of the calculation target object as shown in FIG. . Note that the radial direction of the graph of FIG.

  Returning to FIG. 3, when the change pattern of the radar reflection intensity with respect to the direction of the object to be calculated as viewed from the radar apparatus 1 is obtained in this way, the obtained change pattern of the radar reflection intensity and the RCS table group 6 Pattern matching processing with the RCS change pattern for the direction θ represented by each RCS table is performed, and it is checked whether the obtained radar reflection intensity change pattern matches the RCS change pattern of any RCS table (step) 310). Here, in this pattern matching process, an RCS change pattern indicating the change pattern similar to the obtained radar reflection intensity change pattern with respect to an arbitrary angle range is matched with the radar reflection intensity change pattern. Calculate as change pattern.

  If no match is found (step 312), other vehicle information including the relative position indicated by the latest measurement data in the tracking information is generated to indicate that the type and orientation of the vehicle is unknown, and the image The data is output to the recognition processing unit 4 (step 320), and the process returns to step 302 to wait for the next tracking information update. On the other hand, if the obtained radar reflection intensity change pattern matches the RCS change pattern of any RCS table (step 312), the vehicle type corresponding to the matched RCS table is calculated as the calculation target object type. (Step 314).

  In addition, on the matching RCS change pattern corresponding to the point corresponding to the direction of the calculation target object and the radar reflection intensity indicated by the latest measurement data of the tracking information of the calculation target object on the obtained radar reflection intensity change pattern. From the angle of the point on the portion showing the change pattern similar to the change pattern of the radar reflection intensity obtained in step 1, the direction of the calculation target object reflected in the image taken by the camera 2 is calculated (step 316).

  Then, other vehicle information including the type calculated in step 314, the direction of the calculation target object calculated in step 318, and the relative position indicated by the latest measurement data in the tracking information of the calculation target object is generated, and image recognition processing is performed. Output to the unit 4 (step 318), and returns to step 302 to wait for the next tracking information update to occur.

The other vehicle direction calculation process performed by the other vehicle direction calculation unit 5 has been described above.
As described above, according to the present embodiment, the type of the other vehicle is estimated based on the change in the radar reflection intensity of the other vehicle, and the image recognition process for identifying the estimated type of the vehicle is performed with priority. Since the other vehicle is identified, the other vehicle can be efficiently identified.
Further, according to the present embodiment, the direction of the other vehicle reflected in the image taken by the camera 2 is estimated based on the change in the radar reflection intensity of the other vehicle, and the vehicle for identifying the vehicle facing the estimated direction is identified. Since the image recognition process is preferentially performed to identify the other vehicle, the other vehicle can be efficiently identified.

It is a block diagram which shows the structure and arrangement | positioning of the periphery monitoring system which concern on embodiment of this invention. It is a figure which shows the content of the RCS table which concerns on embodiment of this invention. It is a flowchart which shows the other vehicle direction calculation process which concerns on embodiment of this invention. It is a figure which shows the process example of the other vehicle direction calculation process which concerns on embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Radar apparatus, 2 ... Camera, 3 ... Target tracking part, 4 ... Image recognition process part, 5 ... Other vehicle direction calculation part, 6 ... RCS table group, 7 ... Tracking information holding part, 8 ... Surrounding condition presentation part .

Claims (4)

A vehicle identification system for identifying a vehicle,
A radar device;
A camera,
Based on a transition pattern of the radar reflection intensity measured for the same vehicle at each time point by the radar apparatus with respect to the direction of the radar apparatus measured by the radar apparatus, a predetermined vehicle direction and radar reflection are set. Based on a standard relationship with intensity, an estimation means for estimating the orientation of the vehicle in the image captured by the current camera;
An image recognition process that is suitable for identifying a vehicle facing in the direction according to the direction estimated by the estimation unit is performed on an image captured by the camera, and a vehicle reflected in the image is displayed. A vehicle identification system comprising image recognition processing means for identifying. The vehicle identification system according to claim 1,
The estimation means is a vehicle set in advance based on a transition pattern of the radar reflection intensity measured for the same vehicle at each time point by a change in direction of the vehicle with respect to the radar device measured by the radar device. Based on the standard relationship between the type and orientation of the vehicle and the radar reflection intensity, the type of the vehicle and the direction reflected in the current image captured by the camera are estimated,
The image recognition processing unit performs an image recognition process suitable for identifying the vehicle of the type facing the direction according to the direction and type estimated by the estimation unit with respect to an image captured by the camera. And a vehicle identification system for identifying a vehicle reflected in the image. The vehicle identification system according to claim 1 or 2,
The vehicle identification system is mounted on a vehicle and identifies other vehicles existing around the vehicle. In a vehicle identification system for identifying a vehicle, a vehicle identification method for identifying a vehicle using a radar device and a camera,
The vehicle identification system presets the radar reflection intensity measured for the same vehicle at each time point by a change pattern of the vehicle with respect to the direction of the radar device measured by the radar device. Based on the standard relationship between the vehicle orientation and the radar reflection intensity, an estimation step for estimating the orientation of the vehicle in the image captured by the current camera;
The vehicle identification system performs image recognition processing optimized for identifying a vehicle facing in the direction according to the direction estimated in the estimation step, on the image captured by the camera, and includes in the image An image recognition processing step for identifying a reflected vehicle, and a vehicle identification method.