JP5215329B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image of an infrared camera attached to the vehicle.

  Conventionally, in order to make it easier for a driver to find an object to be monitored such as a pedestrian during night driving, an image of the surroundings of the vehicle is captured by an infrared camera, and the object to be monitored is detected from the captured image. There is known a vehicle periphery monitoring device that notifies the user of the above.

  An infrared imaging element of an infrared camera used in such a vehicle periphery monitoring device is configured by using an infrared absorption film that converts incident infrared rays into heat and absorbs it, and changes its electrical resistance according to the amount of heat absorbed. (For example, refer to Patent Document 1). The infrared imaging device outputs a voltage drop when a constant current is passed through the infrared absorbing film as an infrared detection signal corresponding to the amount of incident heat.

Japanese Patent No. 3921320

  As described above, the infrared absorption film used in the infrared imaging element of the infrared camera outputs a detection signal corresponding to the amount of heat absorbed, and the sensitivity of the infrared imaging element is increased by increasing the heat capacity of the infrared absorption film. Can be increased. However, it takes time depending on the thermal time constant τ determined by the heat capacity C of the infrared absorption film and the thermal conductance G of the surrounding circuit until the heat once absorbed by the infrared absorption film is released.

  Therefore, when the heat capacity of the infrared absorption film is increased, the influence of the response delay of the infrared detection signal to the change in the level of the infrared ray input to the infrared absorption film is increased. As a result, a phenomenon in which the previous captured image overlaps with the captured image of the infrared camera as an afterimage occurs, and the reliability of the captured image is reduced. Therefore, in Patent Document 1 described above, by improving the structure of the infrared imaging device, an increase in the heat capacity of the infrared absorption film is suppressed and the sensitivity of the infrared imaging device is increased.

  However, when the structure of the infrared imaging element is improved as described above, there is a disadvantage that the structure of the infrared imaging element becomes complicated and the manufacturing cost increases.

  Therefore, an object of the present invention is to provide a vehicle periphery monitoring device that prevents an afterimage from being generated in an image captured by an infrared camera without depending on an improvement in the structure of an infrared imaging device.

Department for solving problems

  The present invention has been made in order to achieve the above object, and is configured by using an infrared absorption film in which incident infrared rays are converted into heat and absorbed, and electric resistance changes according to the amount of heat absorbed. An infrared imaging device that outputs an infrared detection signal corresponding to a level of infrared rays incident on a detection region of the pixel; and an infrared camera that converts the infrared detection signal into a luminance signal and outputs the luminance signal; and the vehicle mounted on the vehicle A captured image generation unit that generates data of a captured image in each control cycle based on a luminance signal for one frame output from the infrared camera every predetermined control cycle, and an object around the vehicle using the captured image It is related with the vehicle periphery monitoring apparatus provided with the target object detection part which detects this.

Then, the luminance value by the luminance signal of for each pixel in the current control cycle output from the infrared camera, and the luminance value by the luminance signal of the pixels at the same position in the previous control cycle, of the infrared absorption film A luminance correction unit that performs a first correction that reduces an estimated value of luminance due to a residual amount of heat absorbed in the infrared absorption film in the previous control cycle, calculated using a heat capacity and a thermal time constant based on thermal conductance. The captured image generation unit generates data of the captured image based on a luminance value of each pixel subjected to the first correction.

  According to the present invention, the luminance correction unit causes the luminance value based on the luminance signal of each pixel in the current control cycle to be the luminance due to the residual amount of heat absorbed in the infrared absorption film in the previous control cycle. The first correction for reducing the estimated value is performed. Then, the captured image generation unit generates a captured image using the luminance of each pixel subjected to the first correction. Therefore, without depending on the improvement of the structure of the infrared imaging device, the captured image in the previous control cycle is determined by the remaining amount of heat from the infrared rays incident in the previous control cycle on the captured image in the current control cycle. Generation of afterimages can be suppressed.

  In addition, the luminance correction unit is configured based on the thermal time constant corresponding to the incidence of infrared rays on the infrared absorption film in the current control cycle with respect to the luminance value of each pixel subjected to the first correction. A second correction that is divided by a correction variable based on a convergence curve of the luminance signal is performed, and the captured image generation unit performs the first correction and the luminance value of each pixel on which the second correction has been performed, It is characterized by generating data.

  According to the present invention, by performing the second correction on the luminance value of each pixel subjected to the first correction, the response of the luminance data to the change in the amount of incident infrared rays in the current control cycle. The influence of delay can be reduced.

  The brightness correction unit performs the first correction when the traveling speed of the vehicle is equal to or higher than a predetermined speed, and does not perform the first correction when the traveling speed of the vehicle is lower than the predetermined speed. It is characterized by.

  According to the present invention, since the degree of change of the captured image during the control period is increased, the first correction is performed by performing the first correction at the time of high-speed traveling in which the possibility that an afterimage is displayed on the captured image increases. It is possible to more reliably prevent an afterimage from appearing in the image. In addition, since the degree of change in the captured image during the control cycle is small, the first correction is not performed during low speed travel where the possibility that an afterimage is displayed on the captured image is reduced. The calculation burden on the monitoring device can be reduced.

The block diagram of a vehicle periphery monitoring apparatus. Explanatory drawing of the process which correct | amends the luminance value of each pixel. The block diagram of an infrared image sensor. Explanatory drawing of the effect by correction | amendment of a luminance value, and this correction | amendment.

  Embodiments of the present invention will be described with reference to FIGS. Referring to FIG. 1, a vehicle periphery monitoring device according to the present embodiment includes an image processing unit 10 that processes an analog infrared detection signal (Psig, video signal) output from an infrared camera 1 provided in the vehicle, and an image. A display unit 2 that displays image data output from the processing unit 10 and a speaker 3 that outputs audio data output from the image processing unit 10 are provided.

  The image processing unit 10 is an electronic unit composed of a CPU, a memory, and the like. The image processing unit 10 converts an analog infrared detection signal (Psig) output from the infrared camera 1 into a digital luminance signal, and performs a predetermined control cycle. An image input circuit 11 that writes data (data of luminance values for m × n pixels) of an original captured image Im (k) (k indicates a time-series number of a control cycle) of the infrared camera 1 to an image memory 14 (Corresponding to the captured image generation unit of the present invention) and corrected captured image Im (k) _c data generated by correcting the original captured image Im (k) (brightness value data for m × n pixels) ) Is detected in the image memory 14 and the monitoring target object such as a pedestrian is detected from the original captured image Im (k) or the corrected captured image Im (k) _c, and the display unit 2 and the speaker 3 are detected. And an object detection unit 15 that performs the notification.

  The CPU functions as the brightness correction unit 12 and the object detection unit 15 by executing an image processing program stored in the memory.

Next, correction processing by the luminance correction unit 12 will be described with reference to FIGS. As illustrated in FIG. 2, the luminance correction unit 12 includes the original captured image Im (k) in the current control cycle (kth control cycle) composed of m × n pixels and the previous control cycle (k−). The luminance values I _k (x, y) and I _k-1 (x, y) (x = 0) of each pixel at the same position with the original image Im (k-1) in the first control cycle) , 1,..., M, y = 0, 1,. Then, the luminance correction unit 12 generates a corrected captured image Im (k) _c using the luminance value Sb ^ (x, y) obtained by this correction calculation as pixel data.

  Here, as shown in FIG. 3, the infrared imaging device of the infrared camera 1 has the infrared absorbing film 30 connected to the transistor 32 by the support leg 31 and also connected to the output signal line 34 by the support leg 33. The supply / cut-off of the heat detection current to the infrared absorption film 30 is switched by the on / off control signal of the transistor 32 output to the control line 35.

  The infrared absorption film 30 absorbs infrared In and converts it into heat, and its electrical resistance changes according to the amount of heat absorbed. Therefore, when the transistor 32 is turned on and a constant current is supplied to the infrared absorption film 30 The voltage value of the output signal line can be read out as an infrared detection signal corresponding to the level of the infrared light incident on the infrared absorption film 30.

  As described above, when the level of the infrared ray incident on the infrared absorption film 30 changes, the heat absorption amount of the infrared absorption film 30 changes, but the heat capacity C of the infrared absorption film 30 (the area and thickness of the infrared absorption film 30). And the thermal time constant τ (= C / G) determined by the thermal conductance G of the support legs 31 and 33 (which depends on the length and thickness of the support legs 31 and 33). Response delay of the infrared detection signal output from the element occurs.

  If the heat amount in the previous control period remains in the infrared absorption film 30 due to this response delay, the previous image is captured in the captured image in the current control period generated by the image input circuit 11. An afterimage of the image is generated. Therefore, the luminance correction unit 12 performs the first correction and the second correction in order to suppress the occurrence of such an afterimage.

  FIG. 4A and FIG. 4B are explanatory diagrams showing the effect of performing correction by the luminance correction unit 12. 4A shows the case where the input level of infrared rays to the infrared imaging device increases from Sa to Sb (the amount of incident heat increases). FIG. 4B shows the case where the input level of infrared rays to the infrared imaging device starts from Sa. In the case of a decrease in Sb (the amount of incident heat is reduced), the vertical axis is set to the input level and the horizontal axis is set to time.

In FIG. 4A, the luminance value I _k (x, y) of the pixel output from the image input circuit 11 increases from t ₀ when the infrared input level changes from Sa to Sb. In FIG. 4B, the pixel value I _k (x, y) output from the image input circuit 11 decreases from t ₀ when the infrared input level changes from Sa to Sb.

These luminance values I _k (x, y) are expressed as follows using the input level I _k−1 (x, y) (= Sa) in the previous control cycle and the input level Sb after t ₀ . It can be modeled by equation (1).

Where I _k (x, y): the luminance value of the pixel at the (x, y) position of the current original captured image, I _k-1 (x, y): the (x, y) position of the previous original captured image , The period of one frame, τ: the thermal time constant of the infrared imaging element of the infrared camera 1, e ^{−t / τ} : the decreasing curve of the luminance value of each pixel of the previous original captured image, (1 -e ^{-t / τ} ): Convergence curve of the luminance value by this input

Therefore, the estimated value Sb ^ of the level Sb of the current input signal can be calculated by the following formula (2) that expresses the above formula for the current input level.

Where Sb ^ (x, y): luminance value of the pixel at the (x, y) position of the corrected captured image, I _k (x, y): luminance of the pixel at the (x, y) position of the current original captured image Value, I _k-1 (x, y): luminance of the pixel at the position (x, y) of the previous original captured image, t: period of 1 frame, τ: thermal time constant of the infrared imaging element of the infrared camera 1, I _k-1 (x, y) e ^{−t / τ} : Estimated value of luminance value due to residual amount of heat absorbed by the infrared absorption film in the previous control cycle.

In the above formula (2), the luminance value I _k (x, y) of the pixel at the (x, y) position of the current original captured image in the numerator on the right side is used to calculate the The first process is to reduce the estimated luminance value by the residual amount of heat absorbed in the infrared absorption film in the previous control cycle obtained by multiplying the pixel luminance value I _k-1 (x, y) by e ^{−t / τ} . It becomes correction. Further, the process of dividing by (1−e ^{−t / τ} ) on the right side in the above equation (2) is the second correction.

As described above with reference to FIG. 2, the luminance correction unit 12 includes the original captured image Im (k) in the current control cycle (kth control cycle) and the previous control cycle (k−1th control cycle). The luminance value I _k (x, y), I _k-1 (x, y) of each pixel at the same position in the original captured image Im (k-1) is applied to the above equation (2) and Sb ^ ( x, y) is calculated. Then, the luminance correction unit 12 generates a corrected captured image Im (k) _c in which the luminance value of each pixel is Sb ^ (x, y). As a result, it is possible to obtain a corrected captured image Im (k) _c from which the residual luminance value of the previous captured image Im (k−1) that causes an afterimage is excluded from the current captured image Im (k).

  Then, the object detection unit 15 searches for an image of a monitoring object such as a pedestrian from the binary image of the corrected captured image Im (k) _c (grayscale image) or the corrected captured image Im (k) _c. Monitor objects around the vehicle. Therefore, false detection of a pedestrian due to the influence of an afterimage can be suppressed.

  Note that when the vehicle traveling speed is low, there is little change between the current captured image and the previous captured image, so that the influence of the afterimage is reduced. Further, the lower the traveling speed of the vehicle, the longer it takes for the vehicle to reach an object located at a predetermined distance from the vehicle. Therefore, in the low speed range where the traveling speed detected by the vehicle speed sensor (not shown) is equal to or lower than the predetermined speed, the correction process by the luminance correction unit 12 is prohibited, and the correction by the luminance correction unit 12 is performed only in the high speed range exceeding the predetermined speed. As a configuration for performing the processing, the calculation load of the image processing unit 10 in the low speed region may be reduced.

Further, in the present embodiment, the influence of the response delay of the luminance value with respect to the current input is reduced by performing the second correction divided by (1−e ^{−t / τ} ) by the above equation (2). However, without dividing by (1−e ^{−t / τ} ), only the operation for reducing the luminance value I _k−1 (x, y) e ^{−t / τ} by the previous input is performed. However, the effect of suppressing the occurrence of afterimages can be obtained.

Further, in the present embodiment, according to the above equation (2), the luminance I _k-1 (x, y) of each pixel of the previous original captured image is multiplied by e ^{−t / τ} , and infrared rays are transmitted in the previous control cycle. The estimated value of the luminance value based on the residual amount of heat absorbed by the absorption film was calculated, and the correspondence between the estimated value of the luminance value and the luminance I _k-1 (x, y) of each pixel of the previous original captured image A map is prepared in advance, and an estimated value of the luminance value is obtained by another configuration such as obtaining an estimated value of the luminance value by applying the luminance I _k-1 (x, y) of each pixel to the corresponding map. You may do it.

  When the monitoring target is a pedestrian or a quadruped, the correction process by the luminance correction unit 12 may be prohibited under a situation where the possibility of existence is low. Here, the situation where the possibility of existence is low is, for example, that there is no high-luminance image portion in the grayscale image or the binarized image, the luminance dispersion of the grayscale image is a predetermined value or less, and the vehicle is traveling on a highway (high speed The fact that the vehicle is traveling on a road can be recognized by a navigation device mounted on the vehicle).

  DESCRIPTION OF SYMBOLS 1 ... Infrared camera, 10 ... Image processing unit, 11 ... Image input circuit, 12 ... Brightness correction part, 14 ... Image memory, 15 ... Object detection part.

Claims (3)

Infrared detection according to the level of infrared rays that are incident on the detection area of each pixel, using infrared absorption films that change the electrical resistance according to the amount of heat absorbed by converting incident infrared rays into heat. An infrared imaging device that outputs a signal, an infrared camera that converts the infrared detection signal into a luminance signal and outputs the luminance signal;
A captured image generation unit that generates data of a captured image in each control cycle by a luminance signal for one frame output every predetermined control cycle from the infrared camera mounted on the vehicle;
In the vehicle periphery monitoring device provided with an object detection unit that detects an object around the vehicle using the captured image,
The luminance value by the luminance signal for each pixel in the current control cycle output from the infrared camera, and the luminance value by the luminance signal of the pixels at the same position in the previous control cycle, the heat capacity and the heat of the infrared absorption film A brightness correction unit that performs a first correction to reduce an estimated value of brightness due to a residual amount of heat absorbed in the infrared absorption film in a previous control cycle, calculated using a thermal time constant based on conductance;
The vehicle periphery monitoring device, wherein the captured image generation unit generates data of the captured image based on a luminance value of each pixel subjected to the first correction. The vehicle periphery monitoring device according to claim 1,
The luminance correction unit is configured to output the luminance signal based on the thermal time constant corresponding to the incidence of infrared rays on the infrared absorption film in the current control cycle with respect to the luminance value of each pixel subjected to the first correction. Perform a second correction divided by the correction variable by the convergence curve of
The vehicle periphery monitoring device, wherein the captured image generation unit generates data of the captured image based on a luminance value of each pixel subjected to the first correction and the second correction. In the vehicle periphery monitoring device according to claim 1 or claim 2,
The brightness correction unit performs the first correction when the traveling speed of the vehicle is equal to or higher than a predetermined speed, and does not perform the first correction when the traveling speed of the vehicle is lower than the predetermined speed. A vehicle periphery monitoring device.

Images