JP7785244B2 - Object detection device - Google Patents

Description

The present disclosure relates to an object detection device that is mounted on a vehicle and detects objects.

Onboard object detection devices are devices that can quickly detect objects (people, structures, etc.) in the vehicle's environment and use the information for vehicle control, warning notifications, etc., to ensure a comfortable driving experience. These object detection devices use a variety of sensors, including radar, cameras, Lidar (Light Detection and Ranging, Laser Imaging Detection and Ranging), and ultrasonic sensors. However, with the recent proliferation of various sensors, fusion-type object detection devices that combine multiple sensors to improve performance have become widely used.

When such a fusion-type object detection device is constructed using radar and a camera, radar has excellent accuracy in detecting position and speed, while cameras have excellent accuracy in detecting objects, so it is common for radar to be used for detecting position and speed, and cameras for object recognition.

The external environment recognition device for vehicles described in Patent Document 1 determines whether an object is a pedestrian based on radar information, which is information on the position and speed of an object detected by radar, and image data of the object detected by a camera.

WO 2010/119860

However, with the technology of Patent Document 1, when an object to be detected approaches a structure with high radar reflection intensity (e.g., a utility pole, wall, guardrail, shelf pillar, delineator, traffic light, etc.), the radar reflected wave signal from the object is buried in the radar reflected wave signal from the structure, resulting in the loss of radar detection data and making object detection difficult. In this case, object detection can be maintained by replacing the radar detection data with camera detection data, but low-performance monocular cameras (hereinafter referred to as cameras) have low detection accuracy for the relative position between the object and the vehicle (hereinafter sometimes referred to as position), making it difficult to accurately detect the object. For this reason, the technology of Patent Document 1 poses the problem of being unable to accurately detect the position of an object when a camera with low object position detection accuracy is used and the object approaches a structure with high radar reflection intensity.

The present disclosure has been made in consideration of the above, and aims to provide an object detection device that can accurately detect the position of an object even when a camera with low object position detection accuracy is used, even if the object approaches a structure with high radar reflection intensity.

To solve the above-mentioned problems and achieve the objectives, the object detection device of the present disclosure includes a radar signal processor that detects the radar detection position (position) and speed of a first object based on the reflected signal of electromagnetic waves emitted by a radar toward a first object to be detected. The object detection device of the present disclosure also includes a camera image processor that detects the camera detection position (position) of a second object, the pixel coordinates of the second object, and the object type (type) of the second object based on image data obtained by a camera capturing an image of the second object. The object detection device of the present disclosure also includes a fusion processor that performs an identity determination based on the radar detection position and camera detection position to determine whether the first object detected using the radar and the second object detected using the camera are the same, and if it determines that they are the same, sends the radar detection position, speed, and object type to a vehicle control device that controls the vehicle. If it determines that they are not the same, it sends the camera detection position and object type to the vehicle control device. If the fusion processor determines that the images are identical, it sends camera update data associating pixel coordinates with radar detection positions to the camera image processor. The camera image processor has a conversion table update unit that updates a position conversion table indicating the correspondence between pixel coordinates and radar detection positions based on the camera update data, and an object position detection unit that detects camera detection positions corresponding to the pixel coordinates based on the position conversion table.

The object detection device disclosed herein has the advantage of being able to accurately detect the position of an object even when a camera with low object position detection accuracy is used, even if the object approaches a structure with high radar reflection intensity.

FIG. 1 is a diagram showing a configuration of an object detection device according to a first embodiment; 1 is a flowchart showing a processing procedure of a process executed by an object detection device according to a first embodiment; FIG. 10 is a diagram showing a configuration of an object detection device according to a second embodiment. 10 is a flowchart showing a processing procedure of a process executed by an object detection device according to a second embodiment. FIG. 1 is a diagram showing an example of the configuration of a processing circuit provided in an object detection device according to the first and second embodiments, when the processing circuit is realized by a processor and a memory; FIG. 1 is a diagram illustrating an example of a processing circuit when the processing circuit included in the object detection device according to the first and second embodiments is configured with dedicated hardware.

Below, the object detection device according to an embodiment of the present disclosure is described in detail with reference to the drawings.

Embodiment 1.
FIG. 1 is a diagram illustrating the configuration of an object detection device according to a first embodiment. The object detection device 100A is an object detection device for vehicle installation that detects objects such as moving objects (e.g., people, other vehicles) and fixed structures (e.g., walls). Hereinafter, a detection target object such as a moving object or a fixed object detected by the object detection device 100A may be simply referred to as an object. Furthermore, a vehicle equipped with the object detection device 100A may be simply referred to as a vehicle, and a vehicle detected by the object detection device 100A may be simply referred to as another vehicle.

In embodiment 1, the object to be detected is an object that is detected in order to perform vehicle control or alarm notification, etc. The object detection device 100A is connected to a vehicle control device 30 that controls the operation of the vehicle. The object detection device 100A transmits data used to control the operation of the vehicle to the vehicle control device 30. The data used to control the operation of the vehicle includes, for example, the position, speed, and type of the object.

The object detection device 100A comprises a radar 1, a camera 2, a radar signal processor 3, a camera image processor 4A, and a fusion processor 5A. At least one of the radar 1 and the camera 2 may be configured separately from the object detection device 100A. The object detected by the radar 1 is the first object, and the object detected by the camera 2 is the second object. The first object and the second object may be determined to be the same object or may be determined to be different objects.

Radar 1 emits electromagnetic waves toward an object to be detected and receives a reflected signal (radar reflection signal) from the object. Radar 1 for vehicle installation operates, for example, using the FMCW (Frequency Modulated Continuous Wave) method or the FCM (Fast Chirp Modulation) method. Radar 1 is constructed using high-frequency semiconductor components, power semiconductor components, substrates, quartz devices, chip components, antennas, etc. Radar 1 sends the received reflection signal to radar signal processor 3.

The radar signal processor 3 detects the position and speed of an object by processing the reflected signal received by the radar 1. The radar signal processor 3 uses a processor, etc., as described below.

The radar signal processor 3 has a distance detection unit 6, a speed detection unit 7, a horizontal angle detection unit 8, and a radar detection data storage unit 9. The distance detection unit 6, the speed detection unit 7, and the horizontal angle detection unit 8 each perform a Fast Fourier Transform (FFT) in the distance direction, the speed direction, and the horizontal angle direction to detect the distance, speed, and horizontal angle of an object.

That is, the distance detection unit 6 detects the distance from the vehicle to the object (hereinafter referred to as distance data) based on the reflected signal received by the radar 1. The distance detection unit 6 sends the distance data to the speed detection unit 7.

The speed detection unit 7 detects the relative speed of the object with respect to the vehicle based on the reflected signal received by the radar 1. Hereinafter, the data of the relative speed of the object with respect to the vehicle detected by the speed detection unit 7 will be referred to as speed 32. The speed detection unit 7 sends the speed 32 and distance data to the horizontal angle detection unit 8.

The horizontal angle detection unit 8 detects the horizontal angle (the angle between the vehicle's direction of travel and the direction connecting the object and the vehicle) based on the vehicle's direction of travel, based on the reflected signal received by the radar 1. Hereinafter, the horizontal angle of the object detected by the horizontal angle detection unit 8 is referred to as horizontal angle data. The horizontal angle detection unit 8 calculates the relative position of the object with respect to the vehicle based on the horizontal angle data and distance data. Hereinafter, the data of the object's relative position with respect to the vehicle calculated by the horizontal angle detection unit 8 is referred to as the radar detection position 31. The radar detection position 31 is data indicating the position of the object detected using the radar 1. The horizontal angle detection unit 8 stores the speed 32 and the radar detection position 31 in the radar detection data memory unit 9.

The speed 32 may be stored in the radar detection data memory unit 9 by the speed detection unit 7. Also, the distance detection unit 6 may send the distance data directly to the horizontal angle detection unit 8 without going through the speed detection unit 7.

The radar detection data storage unit 9 is a storage device such as a memory that stores the speed 32 and radar detection position 31. The radar detection data storage unit 9 is connected to the fusion processor 5A. The speed 32 and radar detection position 31 stored in the radar detection data storage unit 9 are read out by the subsequent fusion processor 5A. The radar signal processor 3 may also transmit the speed 32 and radar detection position 31 to the fusion processor 5A.

Camera 2 captures image data of an object by capturing the image of the object. Camera 2 is composed of components such as a lens, holder, CMOS (Complementary Metal Oxide Semiconductor) sensor, power semiconductor components, and quartz devices. Camera 2 sends the image data of the object to camera image processor 4A.

The camera image processor 4A performs signal processing on the image data acquired by the camera 2 to detect the position of the object, pixel coordinates 42, which are the coordinates of the object's pixels, and the object type, which is the type of object. The camera image processor 4A uses a processor, etc., as described below.

The camera image processor 4A has an object recognition unit 10, a detection frame assignment unit 11, a pixel coordinate assignment unit 12, an object position detection unit 13, a camera detection data memory unit 14A, a parameter memory unit 17, a position conversion table memory unit 18, and a conversion table update unit 19.

The object recognition unit 10 detects the object type based on the image data acquired by the camera 2. The object recognition unit 10 recognizes objects and detects the object type by using feature data obtained by machine learning, deep learning, etc. as a database (not shown). The object recognition unit 10 sends the object type 43, which is data on the object type, and the image data to the detection frame assignment unit 11. The object recognition unit 10 also stores the object type 43 in the camera detection data storage unit 14A.

The detection frame assigning unit 11 assigns a frame to the area of the object to be recognized on the image based on the image data and the object recognition result (object type 43) obtained by the object recognition unit 10. The detection frame assigning unit 11 sends the image with a frame indicating the object area to the pixel coordinate assigning unit 12 as frame-assigned image data.

The pixel coordinate assignment unit 12 assigns pixel coordinates 42 to pixels of an object in the image based on the frame-assigned image data. The pixel coordinate assignment unit 12 assigns pixel coordinates 42 to, for example, the center point of the bottom edge of the object's frame. As a result, the pixel coordinate assignment unit 12 assigns pixel coordinates 42 to the object in the image for the image data captured by the camera 2. The pixel coordinate assignment unit 12 sends the pixel coordinates 42, which are data of the coordinates assigned to the pixel, to the object position detection unit 13. The pixel coordinate assignment unit 12 also stores the pixel coordinates 42 in the camera detection data storage unit 14A.

The object position detection unit 13 calculates the relative position of the object with respect to the vehicle based on the pixel coordinates 42. Hereinafter, the data of the object's relative position with respect to the vehicle calculated by the object position detection unit 13 will be referred to as the camera detected position 41. The camera detected position 41 is data indicating the position of the object detected using the camera 2.

The object position detection unit 13 calculates the camera detection position 41 by converting the pixel coordinates 42 into position data indicating the object position based on the position conversion table described below stored in the position conversion table memory unit 18.

Only in the initial state immediately after the object detection device 100A is attached to the vehicle, the object position detection unit 13 calculates the camera detection position 41 by referring to the camera internal parameters 51 and camera external parameters 52 stored in the parameter storage unit 17. The object position detection unit 13 stores the camera detection position 41 in the camera detection data storage unit 14A.

The parameter storage unit 17 is a storage device such as a memory that stores camera internal parameters 51 and camera external parameters 52. The camera internal parameters 51 are parameters determined by the components included in the camera 2, such as the lens and CMOS sensor. The camera internal parameters 51 include focal length, image data size, pixel center, lens distortion coefficient, etc. The camera internal parameters 51 are adjusted for each individual camera 2 during product shipping inspection of the camera 2 and are stored in the parameter storage unit 17.

The camera extrinsic parameters 52 are parameters determined by how the camera 2 is attached to the vehicle. The camera extrinsic parameters 52 include the mounting height, mounting attitude angle, etc. after the camera 2 is attached to the vehicle. The camera extrinsic parameters 52 are adjusted for each individual object detection device 100A during product shipping inspection of the object detection device 100A and stored in the parameter storage unit 17. Note that the camera extrinsic parameters 52 may also be adjusted for each individual camera 2 when the camera 2 is attached to the vehicle and stored in the parameter storage unit 17.

The camera internal parameters 51 fluctuate due to temperature and aging. The camera external parameters 52 fluctuate due to aging changes in the attitude of the camera 2, image blurring caused by vibrations that occur while the vehicle is traveling, and other factors. Therefore, the camera internal parameters 51 and camera external parameters 52 are referenced only in the initial state immediately after the camera 2 is attached to the vehicle, and are not referenced thereafter. In other words, in the initial state, the object position detection unit 13 references the position conversion table, camera internal parameters 51, and camera external parameters 52, and thereafter, the object position detection unit 13 references the position conversion table updated in real time by the conversion table update unit 19.

The position conversion table storage unit 18 is a storage device such as a memory that stores a position conversion table that is updated in response to changes in the radar detection position 31. The position conversion table is a data table that shows the correspondence between the pixel coordinates of an object and the object's position. In other words, the position conversion table is a data table for converting the pixel coordinates of an object into the object's position.

The object position detection unit 13 in embodiment 1 calculates the camera detection position 41 corresponding to the pixel coordinates 42 using a position conversion table that is updated in accordance with changes in the radar detection position 31, thereby enabling the accurate calculation of the object position.

The conversion table update unit 19 updates the position conversion table stored in the position conversion table memory unit 18. The conversion table update unit 19 updates the position conversion table based on the camera update data (first camera update data) described below stored by the fusion processor 5A. Specifically, the conversion table update unit 19 updates the position conversion table based on the radar detection position 31 and pixel coordinates 42 included in the camera update data.

The camera detection data storage unit 14A stores the object type 43 sent from the object recognition unit 10, the pixel coordinates 42 sent from the pixel coordinate assignment unit 12, and the camera detection position 41 sent from the object position detection unit 13. The camera detection data storage unit 14A is connected to the fusion processor 5A.

The object type 43, pixel coordinates 42, and camera detection position 41 stored in the camera detection data memory unit 14A are read out by the subsequent fusion processor 5A. The camera image processor 4A may also transmit the object type 43, pixel coordinates 42, and camera detection position 41 to the fusion processor 5A.

The fusion processor 5A has an object identity determination unit 20, a detection data selection unit 21, a fusion detection data storage unit 22, and a camera update data storage unit 24A. The object identity determination unit 20 reads the speed 32 and radar detection position 31 from the radar signal processor 3. The object identity determination unit 20 also reads the object type 43, pixel coordinates 42, and camera detection position 41 from the camera image processor 4A.

The object identity determination unit 20 performs an object identity determination process based on the radar detection position 31 detected using the radar 1 and the camera detection position 41 detected using the camera 2. That is, the object identity determination unit 20 determines whether the object detected using the radar 1 and the object detected using the camera 2 are the same, based on the radar detection position 31 and the camera detection position 41.

When the object identity determination unit 20 determines through the identity determination process that the object detected using radar 1 and the object detected using camera 2 are the same, it associates the radar detection position 31 detected using radar 1 with pixel coordinates 42 in the object recognition data detected using camera 2. In this case, the object identity determination unit 20 stores camera update data, which is data associating the radar detection position 31 with the pixel coordinates 42, in the camera update data storage unit 24A.

In addition, if the object identity determination unit 20 determines that the object detected using radar 1 and the object detected using camera 2 are the same, it sends correspondence data (hereinafter sometimes referred to as first correspondence data) that associates the radar detection position 31, speed 32, and object type 43 to the detection data selection unit 21.

On the other hand, if the object identity determination unit 20 determines that the object detected using radar 1 and the object detected using camera 2 are not the same, it considers the detection data from radar 1 to have been lost and sends the data detected using camera 2 to the detection data selection unit 21. In other words, if the object detected using radar 1 and the object detected using camera 2 are not the same, the object identity determination unit 20 sends to the detection data selection unit 21 correspondence data (hereinafter sometimes referred to as second correspondence data) that associates the camera detection position 41 with the object type 43.

As a result, if the detection data selection unit 21 succeeds in making an identity determination (if it is determined to be identical), it stores first correspondence data in the fusion detection data storage unit 22, which associates the radar detection position 31 detected using radar 1, the speed 32, and the object type 43 detected using camera 2.

On the other hand, if the detection data selection unit 21 fails to determine identity (if it is determined that the objects are not identical), it stores second correspondence data in the fusion detection data storage unit 22, which associates the camera detection position 41 detected using the camera 2 with the object type 43.

The first data or second data stored in the fusion detection data storage unit 22 is data for vehicle control. Note that Figure 1 illustrates a case where the data stored in the fusion detection data storage unit 22 is a detected position 61, a speed 32, and an object type 43. If the identity determination is successful, the detected position 61 is the radar detected position 31; if the identity determination is unsuccessful, the detected position 61 is the camera detected position 41. Note that if the identity determination is unsuccessful, the speed 32 is not stored in the fusion detection data storage unit 22.

The camera update data storage unit 24A is a storage device such as a memory that stores camera update data that associates radar detection positions 31 with pixel coordinates 42. The camera update data stored in the camera update data storage unit 24A is read by the conversion table update unit 19 of the camera image processor 4A. The fusion processor 5A may also transmit the camera update data to the conversion table update unit 19.

The conversion table update unit 19 updates the position conversion table when it acquires camera update data. As a result, the conversion table update unit 19 updates the position conversion table in real time. The conversion table update unit 19 updates the position conversion table for each frame of image data captured by the camera 2, for example. In this way, the object detection device 100A updates the position conversion table in real time, thereby improving the accuracy of position detection by the camera 2.

The fusion detection data storage unit 22 is a storage device such as a memory that stores data associating the detection position 61, speed 32, and object type 43. If the identity determination is successful, the fusion detection data storage unit 22 stores first correspondence data associating the radar detection position 31, speed 32, and object type 43. If the identity determination is unsuccessful, the fusion detection data storage unit 22 stores second correspondence data associating the camera detection position 41 and object type 43.

The vehicle control device 30 reads data from the fusion detection data storage unit 22 and controls the operation of the vehicle using the read data. Specifically, if the identity determination is successful, the vehicle control device 30 controls the operation of the vehicle based on the radar detection position 31, speed 32, and object type 43. Note that if the identity determination is successful, the object detection device 100A may transmit the radar detection position 31, speed 32, and object type 43 to the vehicle control device 30.

Furthermore, if the identity determination fails, the vehicle control device 30 controls the operation of the vehicle based on the camera detection position 41 and the object type 43. Note that if the identity determination fails, the object detection device 100A may transmit the camera detection position 41 and the object type 43 to the vehicle control device 30.

Next, the processing procedure executed by the object detection device 100A will be explained. Figure 2 is a flowchart showing the processing procedure executed by the object detection device according to the first embodiment. The object detection device 100A starts frame detection (step S1). That is, the radar 1 emits electromagnetic waves toward the object to be detected and acquires a reflected signal (received data) from the object. The camera 2 also acquires image data of the object by capturing an image of the object.

When the radar 1 acquires received data, it outputs the acquired received data to the radar signal processor 3 (step S2). The radar signal processor 3 detects the position and speed of the object based on the received data. That is, the distance detection unit 6, speed detection unit 7, and horizontal angle detection unit 8 of the radar signal processor 3 detect the radar detection position 31 indicating the position of the object and the speed 32 indicating the speed of the object based on the received data (step S3). The radar signal processor 3 stores the radar detection position 31 and the speed 32 in the radar detection data memory unit 9 (step S4).

Furthermore, when the camera 2 acquires image data, it outputs the acquired image data to the camera image processor 4A (step S5). The object recognition unit 10 of the camera image processor 4A recognizes the object based on the image data (step S6). That is, the object recognition unit 10 detects the object type 43, which is the type of object, based on the image data. The object recognition unit 10 sends the object type 43 and the image data to the detection frame assignment unit 11. Furthermore, the object recognition unit 10 sends the object type 43 to the camera detection data storage unit 14A.

The detection frame assigning unit 11 assigns a frame to the area of the object on the image based on the image data and the object type 43. The detection frame assigning unit 11 sends the image with a frame indicating the area of the object assigned to the pixel coordinate assigning unit 12 as frame-assigned image data.

The pixel coordinate assignment unit 12 assigns pixel coordinates 42 to the pixels of the object in the image based on the frame-assigned image data. That is, the pixel coordinate assignment unit 12 assigns pixel coordinates 42 to the recognized object (step S7). The pixel coordinate assignment unit 12 sends the pixel coordinates 42 to the object position detection unit 13. The pixel coordinate assignment unit 12 also sends the pixel coordinates 42 to the camera detection data storage unit 14A.

The object position detection unit 13 detects the camera detection position 41, which is the relative position of the object with respect to the vehicle, by referencing the position conversion table for the pixel coordinates 42 (step S8). In this case, since the position conversion table for the previous frame is updated in real time, the camera image processor 4A can detect the position of the object with high accuracy using the camera 2. The object position detection unit 13 sends the camera detection position 41 to the camera detection data storage unit 14A.

In the initial state immediately after the object detection device 100A is installed in the vehicle, the object position detection unit 13 calculates the camera detection position 41 by referring to the camera internal parameters 51, the camera external parameters 52, and the position conversion table.

The camera image processor 4A stores the camera detection position 41, pixel coordinates 42, and object type 43 in the camera detection data memory unit 14A (step S9).

The processing of steps S2 to S4 and the processing of steps S5 to S9 are executed in parallel. The camera-detection data storage unit 14A may store the object type 43 before the processing of steps S7 and S8, provided that the processing of step S6 has been executed. Furthermore, the camera-detection data storage unit 14A may store the pixel coordinates 42 before the processing of step S8, provided that the processing of step S7 has been executed.

The object identity determination unit 20 of the fusion processor 5A reads the radar detection position 31 and speed 32 from the radar detection data storage unit 9. The object identity determination unit 20 also reads the camera detection position 41, pixel coordinates 42, and object type 43 from the camera detection data storage unit 14A.

The object identity determination unit 20 of the fusion processor 5A performs an object identity determination process based on the radar detection position 31 and the camera detection position 41 (step S10). Through this identity determination process, the object identity determination unit 20 associates the radar detection position 31 and speed 32 detected using the radar 1 with the object recognition data detected using the camera 2.

The object identity determination unit 20 determines whether the identity determination was successful (step S11). If the object identity determination unit 20 determines that the identity determination was unsuccessful, it executes the processing of step S12. On the other hand, if the object identity determination unit 20 determines that the identity determination was successful, it executes the processing of steps S13 and S17.

That is, if the object identity determination unit 20 fails to determine identity (step S11, No), it sends second correspondence data associating the camera detection position 41 with the object type 43 to the detection data selection unit 21. As a result, the detection data selection unit 21 outputs the camera detection position 41 and object type 43 detected using the camera 2 to the fusion detection data storage unit 22 (step S12).

On the other hand, if the object identity determination unit 20 is successful in determining identity (step S11, Yes), it sends first correspondence data associating the radar detection position 31, speed 32, and object type 43 to the detection data selection unit 21. As a result, the detection data selection unit 21 outputs the radar detection position 31 and speed 32 detected using radar 1, and the object type 43 detected using camera 2 to the fusion detection data storage unit 22 (step S13).

The fusion processor 5A stores the data obtained in step S12 or the data obtained in step S13 in the fusion detection data storage unit 22 (step S14). That is, if the detection data selection unit 21 fails to make an identity determination, it stores second correspondence data associating the camera detection position 41 with the object type 43 in the fusion detection data storage unit 22. On the other hand, if the detection data selection unit 21 succeeds in making an identity determination, it stores second correspondence data associating the radar detection position 31, speed 32, and object type 43 in the fusion detection data storage unit 22.

The fusion processor 5A outputs the data stored in the fusion detection data storage unit 22 to the vehicle control device 30. That is, the fusion processor 5A outputs the object detection position 61, speed 32, and object type 43 data stored in the fusion detection data storage unit 22 in the processing of step S14 to the vehicle control device 30 (step S15). Specifically, if the identity determination fails, the fusion processor 5A outputs second correspondence data associating the camera detection position 41 with the object type 43 to the vehicle control device 30. On the other hand, if the identity determination is successful, the fusion processor 5A outputs first correspondence data associating the radar detection position 31, speed 32, and object type 43 to the vehicle control device 30.

The vehicle control device 30 controls the operation of the vehicle using the data acquired from the fusion processor 5A (step S16). Specifically, if the identity determination fails, the vehicle control device 30 controls the vehicle based on the camera detection position 41 and the object type 43. If the identity determination is successful, the vehicle control device 30 controls the vehicle based on the radar detection position 31, the speed 32, and the object type 43.

Furthermore, if the object identity determination unit 20 succeeds in determining identity in the processing of step S11 (step S11, Yes), it stores camera update data that corresponds to the radar detection position 31 and pixel coordinates 42 in the camera update data memory unit 24A (step S17).

The fusion processor 5A transmits the camera update data stored in the camera update data storage unit 24A to the conversion table update unit 19 of the camera image processor 4A. As a result, the conversion table update unit 19 of the camera image processor 4A updates the position conversion table based on the camera update data (step S18).

When the processing of step S12 is executed, the processing of steps S14 to S16 is executed, and the processing of steps S13, S17, and S18 is not executed. On the other hand, when the processing of step S13 is executed, the processing of steps S14 to S18 is executed, and the processing of step S12 is not executed. Note that when the processing of step S13 is executed, the processing of steps S13 to S16 and the processing of steps S17 and S18 are executed in parallel.

When the processing of steps S16 and S18 is completed, the object detection device 100A proceeds to processing the next frame of object detection (step S19). The object detection device 100A then repeats the processing of steps S2 to S19 in Figure 2.

The object detection device 100A achieves fusion-type object detection using radar 1 and camera 2. When an object to be detected approaches a structure with high radar reflection intensity, the radar reflected wave signal from the object is buried in the radar reflected wave signal from the structure, causing the radar 1 detection data to be lost and making it difficult for the radar 1 to detect the object. The object detection device 100A maintains the detection accuracy of fusion-type object detection by replacing the radar 1 detection data with the camera 2 detection data. That is, the object detection device 100A calculates the camera detection position 41 based on the radar detection position 31, and generates vehicle control data for controlling the vehicle using the calculated camera detection position 41 instead of the radar detection position 31.

In embodiment 1, even if the accuracy of object position detection by camera 2 is low, object detection device 100A can update the position conversion table in real time based on radar detection position 31 by processing steps S17 and S18. As a result, even if radar 1 detection data is lost, object detection device 100A can detect the object position using the position conversion table that has been updated in accordance with changes in radar detection position 31. This prevents a decrease in object position detection accuracy and allows high position detection accuracy to be maintained.

For example, the camera characteristics of the camera 2 change due to temperature and aging. Furthermore, the position detection accuracy of the camera 2 decreases due to aging changes in the attitude of the camera 2 after it is attached to the vehicle, and blurring of the image due to vibrations that occur while the vehicle is moving. Even if the detection data of the radar 1 is lost in such a case, the object detection device 100A can detect the object's position using a position conversion table that is updated in accordance with changes in the radar detection position 31. This prevents a decrease in the object's position detection accuracy and maintains high position detection accuracy.

In this way, in embodiment 1, object detection device 100A performs object identity determination based on data from past frames in which radar 1 and camera 2 simultaneously detected an object (e.g., a person), and adds radar detection position 31, which is radar 1's position detection information, to the image data from camera 2. That is, for frames in which object detection device 100A determines that the object detected by radar 1 and the object detected by camera 2 are the same based on radar detection position 31 and camera detection position 41, object detection device 100A generates camera update data that associates pixel coordinates 42 with radar detection position 31. Then, based on the camera update data, object detection device 100A updates the position conversion table, which indicates the correspondence between the pixel coordinates of the object and its position, and calculates camera detection position 41 corresponding to pixel coordinate 42 based on the position conversion table. If radar 1 detection data is lost in subsequent frames, object detection device 100A provides vehicle control device 30 with camera detection position 41 calculated based on the position conversion table, which has been updated in response to changes in radar detection position 31. This enables the object detection device 100A to improve and maintain high position detection accuracy of the camera 2. Therefore, even if the detection data of the radar 1 is lost, the object detection device 100A can provide the vehicle control device 30 with a highly accurate detected position 61, and the vehicle control device 30 can control the vehicle based on the highly accurate detected position 61.

As described above, according to embodiment 1, when object detection device 100A determines that the object detected by radar 1 and the object detected by camera 2 are the same, it generates camera update data that associates pixel coordinates 42 with radar detection positions 31. Then, based on the camera update data, object detection device 100A updates the position conversion table that indicates the correspondence between pixel coordinates 42 and radar detection positions 31, and detects camera detection positions 41 that correspond to pixel coordinates 42 based on the position conversion table. Therefore, when camera 2 with low object position detection accuracy is used, object detection device 100A can accurately detect the position of an object even if the object approaches a structure with high radar reflection intensity.

Embodiment 2.
Next, a second embodiment will be described with reference to Figures 3 and 4. In the second embodiment, the object detection device updates a distance conversion table indicating the correspondence relationship between the pixel size of an object and the distance from the vehicle to the object, based on the pixel size of the object and the distance to the object detected using radar 1. The object detection device then refers to the distance conversion table to detect the distance corresponding to the pixel size, and detects the camera detection position 41 based on the detected distance.

Figure 3 is a diagram showing the configuration of an object detection device according to embodiment 2. Of the components in Figure 3, those that achieve the same functions as those in object detection device 100A according to embodiment 1 shown in Figure 1 are given the same reference numerals, and redundant explanations will be omitted.

Similar to the object detection device 100A of embodiment 1, the object detection device 100B of embodiment 2 is an object detection device mounted on a vehicle that detects objects. In embodiment 1, the object detection device 100A updated the position conversion table based on the camera update data (first camera update data) in which the radar detection position 31 and pixel coordinates 42 were associated, and detected the camera detection position 41 corresponding to the pixel coordinates by referring to the position conversion table.

In embodiment 2, object detection device 100B updates a distance conversion table indicating the correspondence between the pixel size of an object and the distance from the vehicle (camera 2) to the object, based on camera update data (second camera update data) that associates the pixel size in an image of the object with the distance from radar 1 to the object. Then, object detection device 100B detects the distance corresponding to the pixel size by referring to the distance conversion table, and detects camera detection position 41 based on the detected distance. Note that the distance to the object in embodiment 2 may be the distance from camera 2 to the object, or the distance from the vehicle to the object.

The pixel size in embodiment 2 corresponds not to the size of one pixel, but to the number of pixels in a specific area of the object. Therefore, even for the same object, the closer the object is to camera 2, the larger the pixel size will be, and the farther the object is from camera 2, the smaller the pixel size will be. In other words, the pixel size in embodiment 2 changes depending on the distance between camera 2 and the object. In other words, if the pixel size is known, the distance between camera 2 and the object can also be calculated.

Compared to object detection device 100A, object detection device 100B has a camera image processor 4B and a fusion processor 5B instead of the camera image processor 4A and the fusion processor 5A. That is, object detection device 100B has radar 1, camera 2, radar signal processor 3, camera image processor 4B, and fusion processor 5B.

Compared to camera image processor 4A, camera image processor 4B has a camera detection data memory unit 14B instead of camera detection data memory unit 14A. Also, compared to camera image processor 4A, camera image processor 4B has a distance conversion table memory unit 16 instead of position conversion table memory unit 18. Also, camera image processor 4B has a detection frame pixel size assigning unit 15. Specifically, camera image processor 4B has an object recognition unit 10, a detection frame assigning unit 11, a pixel coordinate assigning unit 12, an object position detection unit 13, a detection frame pixel size assigning unit 15, a camera detection data memory unit 14B, a parameter memory unit 17, a distance conversion table memory unit 16, and a conversion table update unit 19.

In embodiment 2, the detection frame assigning unit 11 sends the frame-assigned image data to the pixel coordinate assigning unit 12 and the detection frame pixel size assigning unit 15. The detection frame pixel size assigning unit 15 assigns a pixel size (number of pixels) to the detection frame assigned to the frame-assigned image data. Hereinafter, the pixel size data assigned to the detection frame by the detection frame pixel size assigning unit 15 will be referred to as pixel size 44.

The detection frame pixel size assigning unit 15 sends the pixel size 44 to the object position detection unit 13. The detection frame pixel size assigning unit 15 also stores the pixel size 44 in the camera detection data storage unit 14B.

In embodiment 2, the pixel coordinate assignment unit 12 sends pixel coordinates 42 to the object position detection unit 13, but they do not have to be stored in the camera detection data storage unit 14B. In embodiment 2, the object position detection unit 13 detects the orientation of the object relative to the camera 2 based on the pixel coordinates 42 sent from the pixel coordinate assignment unit 12.

The object position detection unit 13 also detects the distance from the camera 2 to the object based on the pixel size 44 sent from the detection frame pixel size assignment unit 15. At this time, the object position detection unit 13 calculates the distance from the camera 2 to the object by converting the pixel size 44 into the distance to the object based on the distance conversion table stored in the distance conversion table storage unit 16.

The distance conversion table storage unit 16 is a storage device such as a memory that stores a distance conversion table that is updated in response to changes in the radar detection position 31. The distance conversion table is a data table that shows the correspondence between the pixel size 44 of an object and the distance from the radar 1 to the object. In other words, the distance conversion table is a data table for converting the pixel size 44 of an object into the distance from the radar 1 to the object.

The object position detection unit 13 in embodiment 2 calculates the distance to an object corresponding to pixel size 44 using a distance conversion table that is updated in response to changes in radar detection position 31, allowing for accurate calculation of the distance to the object. The object position detection unit 13 detects the camera detection position 41, which is the relative position of the object with respect to the vehicle, based on the distance to the object and the direction in which the object is located. The object position detection unit 13 stores the camera detection position 41 in the camera detection data storage unit 14B.

The conversion table update unit 19 updates the distance conversion table stored in the distance conversion table memory unit 16. The conversion table update unit 19 updates the distance conversion table based on the camera update data (second camera update data) described below stored by the fusion processor 5B. Specifically, the conversion table update unit 19 updates the distance conversion table based on the pixel size 44 and distance 33 included in the camera update data. The distance 33 is the distance from the radar 1 to the object, calculated based on the radar detection position 31.

The camera detection data storage unit 14B stores the object type 43 sent from the object recognition unit 10, the pixel size 44 sent from the detection frame pixel size assignment unit 15, and the camera detection position 41 sent from the object position detection unit 13. The camera detection data storage unit 14B is connected to the fusion processor 5B.

The object type 43, pixel size 44, and camera detection position 41 stored in the camera detection data memory unit 14B are read out by the subsequent fusion processor 5B. The camera image processor 4B may also transmit the object type 43, pixel size 44, and camera detection position 41 to the fusion processor 5B.

Compared to fusion processor 5A, fusion processor 5B has a camera update data storage unit 24B instead of a camera update data storage unit 24A. That is, fusion processor 5B has an object identity determination unit 20, a detection data selection unit 21, a fusion detection data storage unit 22, and a camera update data storage unit 24B.

In the second embodiment, if the object identity determination unit 20 is successful in the identity determination, it calculates the distance 33 from the vehicle to the object and the direction of the object relative to the vehicle based on the radar detection position 31. That is, the object identity determination unit 20 calculates the distance 33 to the object using radar 1 by breaking down the radar detection position 31 into the distance 33 to the object and the direction to the object. If the object identity determination is successful, the object identity determination unit 20 associates the distance 33 with the pixel size 44 in the object recognition data detected using camera 2. In this case, the object identity determination unit 20 stores the camera update data, which is data associating the distance 33 with the pixel size 44, in the camera update data storage unit 24B.

In addition, if the object identity determination unit 20 is successful in determining identity, it sends first correspondence data that associates the radar detection position 31, speed 32, and object type 43 to the detection data selection unit 21.

As a result, if the detection data selection unit 21 successfully determines that the objects are identical, it stores first correspondence data in the fusion detection data storage unit 22, which associates the radar detection position 31 detected using radar 1, the speed 32, and the object type 43 detected using camera 2.

On the other hand, if the object identity determination unit 20 fails to determine identity, it considers that the detection data from radar 1 has been lost and sends the data detected using camera 2 to the detection data selection unit 21. In other words, if the object detected using radar 1 and the object detected using camera 2 are not the same, the object identity determination unit 20 sends second correspondence data that associates the camera detection position 41 with the object type 43 to the detection data selection unit 21.

As a result, if the detection data selection unit 21 fails to make an identity determination, it stores second correspondence data in the fusion detection data storage unit 22, which associates the camera detection position 41 detected using the camera 2 with the object type 43.

Note that Figure 3 illustrates a case where the data stored in the fusion detection data storage unit 22 is the detection position 61, speed 32, and object type 43. If the identity determination is successful, the detection position 61 is the radar detection position 31, and if the identity determination is unsuccessful, the detection position 61 is the camera detection position 41. Note that if the identity determination is unsuccessful, the speed 32 is not stored in the fusion detection data storage unit 22.

The camera update data storage unit 24B is a storage device such as a memory that stores camera update data in which distance 33 and pixel size 44 are associated. The camera update data stored in the camera update data storage unit 24B is read by the conversion table update unit 19 of the camera image processor 4B. The fusion processor 5B may also transmit the camera update data to the conversion table update unit 19.

The conversion table update unit 19 updates the distance conversion table when it acquires camera update data. As a result, the conversion table update unit 19 updates the distance conversion table in real time. The conversion table update unit 19 updates the position conversion table for each frame of image data captured by the camera 2, for example. In this way, the object detection device 100B updates the position conversion table in real time, thereby improving the accuracy of position detection by the camera 2.

Next, the processing procedure of the processing executed by the object detection device 100B will be explained. Figure 4 is a flowchart showing the processing procedure of the processing executed by the object detection device according to the second embodiment. Among the processing shown in Figure 4, the same processing as the processing executed by the object detection device 100A according to the first embodiment shown in Figure 2 is assigned the same step number, and duplicate explanations will be omitted.

Compared to object detection device 100A of embodiment 1, object detection device 100B of embodiment 2 executes step S7A in addition to step S7, and executes steps S8A and S9A instead of steps S8 and S9. Also, compared to object detection device 100A, object detection device 100B executes steps S17A and S18A instead of steps S17 and S18.

That is, when the object recognition unit 10 of the camera image processor 4B recognizes an object based on the image data (step S6), it sends the frame-assigned image data to the pixel coordinate assignment unit 12 and the detection frame pixel size assignment unit 15. As a result, the pixel coordinate assignment unit 12 assigns pixel coordinates 42 to the recognized object (step S7). The pixel coordinate assignment unit 12 then sends the pixel coordinates 42 to the object position detection unit 13.

The detection frame pixel size assigning unit 15 also assigns a pixel size of 44 to the detection frame assigned to the frame-assigned image data. That is, the detection frame pixel size assigning unit 15 assigns the pixel size of 44 to the recognized object (step S7A). The detection frame pixel size assigning unit 15 then sends the pixel size 44 to the object position detection unit 13 and the camera detection data storage unit 14B.

The object position detection unit 13 detects the orientation of the object relative to the camera 2 based on the pixel coordinates 42 sent from the pixel coordinate assignment unit 12. The object position detection unit 13 also references a distance conversion table for the pixel size 44 to detect the distance to the object (the distance from the camera 2 to the object) corresponding to the pixel size 44 of the object (step S8A). In this case, because the distance conversion table for the previous frame is updated in real time, the camera image processor 4B can detect the distance to the object with high accuracy using the camera 2.

The object position detection unit 13 detects the camera detection position 41, which is the relative position of the object with respect to the camera 2, based on the distance to the object and the orientation in which the object is located. The object position detection unit 13 sends the camera detection position 41 to the camera detection data storage unit 14B.

In the initial state immediately after the object detection device 100B is installed in the vehicle, the object position detection unit 13 calculates the camera detection position 41 by referring to the camera internal parameters 51, the camera external parameters 52, and the distance conversion table.

The camera image processor 4B stores the camera detection position 41, pixel size 44, and object type 43 in the camera detection data memory unit 14B (step S9A).

The processing of steps S2 to S4 and the processing of steps S5 to S9A are executed in parallel. The camera detection data storage unit 14B may store the object type 43 before the processing of steps S7, S7A, and S8A, as long as the processing of step S6 has been executed.

The object identity determination unit 20 of the camera image processor 4B determines whether the identity determination was successful (step S11), and if it determines that the identity determination was unsuccessful, it executes the processing of step S12. On the other hand, if it determines that the identity determination was successful, it executes the processing of steps S13 and S17A.

If the object identity determination unit 20 successfully determines identity in the processing of step S11 (step S11, Yes), it stores camera update data that corresponds to the distance 33 and pixel size 44 extracted from the radar detection position 31 in the camera update data memory unit 24B (step S17A).

The fusion processor 5B transmits the camera update data stored in the camera update data storage unit 24B to the conversion table update unit 19 of the camera image processor 4B. As a result, the conversion table update unit 19 of the camera image processor 4B updates the distance conversion table based on the camera update data (step S18A).

When the processing of step S12 is executed, the processing of steps S14 to S16 is executed, and the processing of steps S13, S17A, and S18A is not executed. On the other hand, when the processing of step S13 is executed, the processing of steps S14 to S16, S17A, and S18A is executed, and the processing of step S12 is not executed. Note that when the processing of step S13 is executed, the processing of steps S13 to S16 and the processing of steps S17A and S18A are executed in parallel.

When the processing of step S16 and step S18A is completed, the object detection device 100B proceeds to processing the next frame of object detection (step S19).The object detection device 100B then repeats the processing of steps S2 to S19 in Figure 4.

In this way, like object detection device 100A, object detection device 100B achieves fusion-type object detection using radar 1 and camera 2. And, like object detection device 100A, object detection device 100B maintains the detection accuracy of fusion-type object detection by replacing radar 1 detection data with camera 2 detection data. That is, object detection device 100B calculates camera detection position 41 based on radar detection position 31, and generates vehicle control data for controlling the vehicle using the calculated camera detection position 41 instead of radar detection position 31.

In embodiment 2, even if the accuracy of object position detection by camera 2 is low, object detection device 100B can update the distance conversion table in real time in accordance with changes in radar detection position 31 by processing steps S17A and S18A. As a result, even if radar 1 detection data is lost, object detection device 100B can detect the object position using the distance conversion table updated based on radar detection position 31, thereby preventing a decrease in object position detection accuracy and maintaining high position detection accuracy.

For example, the camera characteristics of camera 2 fluctuate with temperature and aging. Furthermore, the position detection accuracy of camera 2 decreases due to aging changes in the attitude of camera 2 after it is attached to the vehicle, and blurring of the image due to vibrations that occur while the vehicle is moving. Even if the detection data of radar 1 is lost in such a case, object detection device 100A can detect the object's position using a distance conversion table that has been updated based on radar detection position 31. This prevents a decrease in object position detection accuracy and maintains high position detection accuracy.

According to embodiment 2, as in embodiment 1, when a camera 2 with low object position detection accuracy is used, the object's position can be accurately detected even if the object approaches a structure with high radar reflection intensity.

Here, the hardware configuration of the object detection devices 100A and 100B according to embodiments 1 and 2 will be described. The object detection devices 100A and 100B are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in memory, or may be dedicated hardware.

Figure 5 is a diagram showing an example configuration of a processing circuit when the processing circuit provided in the object detection device according to embodiments 1 and 2 is realized by a processor and memory. The processing circuit 90 shown in Figure 5 includes a processor 91 and a memory 92.

When the processing circuit 90 is composed of a processor 91 and a memory 92, each function of the processing circuit 90 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a data processing program and stored in the memory 92. In the processing circuit 90, each function is realized by the processor 91 reading and executing the data processing program stored in the memory 92. That is, the processing circuit 90 includes a memory 92 for storing a data processing program that results in the processing of the object detection devices 100A, 100B. This data processing program can also be said to be a program that causes the object detection devices 100A, 100B to execute each function realized by the processing circuit 90. This data processing program may be provided by a storage medium on which the data processing program is stored, or by other means such as a communication medium.

Here, the processor 91 is, for example, a CPU (Central Processing Unit), processing device, arithmetic unit, microprocessor, microcomputer, or DSP (Digital Signal Processor). Furthermore, the memory 92 is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), or EEPROM (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc).

Figure 6 is a diagram showing an example of a processing circuit provided in the object detection device according to embodiments 1 and 2, configured with dedicated hardware. The processing circuit 93 shown in Figure 6 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. The processing circuit 93 may be partially implemented with dedicated hardware and partially implemented with software or firmware. In this way, the processing circuit 93 can realize each of the above-mentioned functions with dedicated hardware, software, firmware, or a combination of these.

In addition, in the object detection device 100A, the radar signal processor 3, the camera image processor 4A, and the fusion processor 5A may each be realized by separate processing circuits. In this case, one or more of the radar signal processor 3, the camera image processor 4A, and the fusion processor 5A are realized by the processing circuit 90 or the processing circuit 93 described above. In this case, the object detection device 100A has multiple processing circuits.

Furthermore, in the object detection device 100B, the radar signal processor 3, the camera image processor 4B, and the fusion processor 5B may each be realized by separate processing circuits. In this case, one or more of the radar signal processor 3, the camera image processor 4B, and the fusion processor 5B are realized by the processing circuit 90 or the processing circuit 93 described above. In this case, the object detection device 100B has multiple processing circuits.

The camera image processors 4A and 4B may also be realized using an MCU (microcontroller, micro control unit) or the like. The camera image processors 4A and 4B may also be realized using a GPU (graphics processing unit) or the like.

The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. Parts of the configuration may also be omitted or modified without departing from the spirit of the invention.

1 Radar, 2 Camera, 3 Radar signal processor, 4A, 4B Camera image processor, 5A, 5B Fusion processor, 6 Distance detection unit, 7 Speed detection unit, 8 Horizontal angle detection unit, 9 Radar detection data storage unit, 10 Object recognition unit, 11 Detection frame assignment unit, 12 Pixel coordinate assignment unit, 13 Object position detection unit, 14A, 14B Camera detection data storage unit, 15 Detection frame pixel size assignment unit, 16 Distance conversion table storage unit, 17 Parameter storage unit, 18 Position conversion table storage unit, 19 Conversion table update unit, 20 Object identity determination unit, 21 Detection data selection unit, 22 Fusion detection data storage unit, 24A, 24B Camera update data storage unit, 30 Vehicle control device, 31 Radar detection position, 32 Speed, 33 Distance, 41 Camera detection position, 42 Pixel coordinates, 43 Object type, 44 Pixel size, 51 camera internal parameters, 52 camera external parameters, 61 detection position, 90, 93 processing circuit, 91 processor, 92 memory, 100A, 100B object detection device.

Claims (4)

a radar signal processor that detects a radar detection position, which is the position of a first object, and a speed of the first object based on a reflected signal of an electromagnetic wave emitted by the radar toward the first object;
a camera image processor that detects a camera detection position, which is the position of a second object, pixel coordinates of the second object, and an object type, which is the type of the second object, based on image data obtained by capturing an image of the second object with a camera;
a fusion processor that determines whether the first object detected using the radar and the second object detected using the camera are the same or not based on the radar detection position and the camera detection position, and if it determines that they are the same, sends the radar detection position, the speed, and the object type to a vehicle control device that controls the vehicle, and if it determines that they are not the same, sends the camera detection position and the object type to the vehicle control device;
Equipped with
The fusion processor comprises:
If it is determined that the pixel coordinates are identical, camera update data that associates the pixel coordinates with the radar detection position is sent to the camera image processor;
The camera image processor includes:
a conversion table update unit that updates a position conversion table indicating a correspondence relationship between the pixel coordinates and the radar detection positions based on the camera update data;
an object position detection unit that detects the camera detection position corresponding to the pixel coordinates based on the position conversion table;
having
An object detection device characterized by: The camera image processor includes:
a parameter storage unit that stores camera internal parameters that are parameters determined by components included in the camera and camera external parameters that are parameters determined by how the camera is attached to the vehicle;
the object position detection unit, in an initial state immediately after the camera is attached to the vehicle, detects the camera detection position corresponding to the pixel coordinates based on the camera internal parameters, the camera external parameters, and the position conversion table;
2. The object detection device according to claim 1. a radar signal processor that detects a radar detection position, which is the position of a first object, and a speed of the first object based on a reflected signal of an electromagnetic wave emitted by the radar toward the first object;
a camera image processor that detects, based on image data obtained by a camera capturing an image of a second object, a camera detection position that is the position of the second object, a pixel size that corresponds to the number of pixels on the image of the second object, and an object type that is the type of the second object;
a fusion processor that determines whether the first object detected using the radar and the second object detected using the camera are the same or not based on the radar detection position and the camera detection position, and if it determines that they are the same, sends the radar detection position, the speed, and the object type to a vehicle control device that controls the vehicle, and if it determines that they are not the same, sends the camera detection position and the object type to the vehicle control device;
Equipped with
The fusion processor comprises:
If it is determined that the two objects are identical by the identity determination, camera update data in which the pixel size and the distance from the radar to the first object calculated from the radar detection position are associated with each other is sent to the camera image processor;
The camera image processor includes:
a conversion table update unit that updates a distance conversion table indicating a correspondence relationship between the pixel size and the distance based on the camera update data;
an object position detection unit that detects the distance corresponding to the pixel size based on the distance conversion table, detects an orientation of the second object relative to the camera based on the image data, and detects the camera detection position corresponding to the pixel size based on the distance and the orientation;
having
An object detection device characterized by: The camera image processor includes:
a parameter storage unit that stores camera internal parameters that are parameters determined by components included in the camera and camera external parameters that are parameters determined by how the camera is attached to the vehicle;
the object position detection unit, in an initial state immediately after the camera is attached to the vehicle, detects the camera detection position corresponding to the pixel size based on the camera internal parameters, the camera external parameters, and the distance conversion table;
4. The object detection device according to claim 3.