JP3925501B2 - Heat source detection device - Google Patents

Description

  The present invention relates to a heat source detection device that detects a heat source of a pedestrian or the like.

  2. Description of the Related Art An environment recognition device is known in which an infrared camera and a visible light camera are used to capture the front of a vehicle, a visible light camera image is subtracted from an infrared camera image, and an image of only a heat source is generated and displayed (for example, , See Patent Document 1).

Prior art documents related to the invention of this application include the following.
JP 2000-19259 A

  However, since the above-described conventional apparatus does not consider the influence of sunlight in the daytime, particularly in scenes with strong solar radiation, when the detection area changes from shade to sunlight, the temperature of the road surface rises so much. In spite of this, there is a problem that far-infrared rays contained in sunlight are reflected on the road surface and erroneously detected that there is a heat source on the road surface. In addition, when there is a sunlit portion and a shaded portion in the detection area, there is a problem that a heat source is erroneously detected even though there is no heat source in the sunlit portion.

Infrared imaging means for imaging an object scene with infrared rays and outputting an infrared image; Visible ray imaging means for imaging a range including an imaging range in the object scene of the infrared imaging means with visible light and outputting a visible image; When the presence of a heat source is detected by comparing the brightness of each infrared image by the infrared imaging means with the threshold, the threshold calculated based on the infrared image A threshold value changing unit that corrects using luminance, and the threshold value changing unit sets the threshold value of the sunny part higher than the threshold value of the shaded part when the imaging range includes a sunny part and a shaded part To do.

  According to the present invention, erroneous detection of a heat source can be prevented.

  FIG. 1 is a diagram showing a configuration of an embodiment. The visible light camera 1 captures an object scene with visible light and outputs a visible image. The infrared camera 2 images an object scene with infrared rays and outputs an infrared image. The visible light camera 1 and the infrared camera 2 are installed at the rear end of the vehicle and take an image of the rear of the vehicle. It is desirable that the visible light camera 1 and the infrared camera 2 are installed at the same position, and the imaging range thereof is preferably the same. However, the infrared ray is at least in the imaging range of the visible light camera 1 even if they are not the same position and the same range. The imaging range of the camera 2 needs to be included. The heat source determination device 3 is configured around a microcomputer, and processes the images captured by the visible light camera 1 and the infrared camera 2 to determine the presence or absence of a heat source in the field of view. When the heat source determination device 3 determines that there is a heat source in the field of view, the notification device 4 displays an image of the heat source and issues a warning by voice, buzzer, vibrator, or the like.

  FIG. 2 is a diagram showing a case where a shaded portion and a sunlit portion are formed behind the vehicle due to sunlight, and FIG. 3 is a diagram of the shaded portion and the sunlit portion at the rear of the vehicle as viewed from above. When sunlight is radiated from above the vehicle, a shadow of the vehicle is generated behind the vehicle, and a part of the heat source detection area by the visible light camera 1 and the infrared camera 2 becomes a shaded part. When there are a sunlit portion and a shaded portion in the heat source detection area, sunlight is irregularly reflected by the unevenness of the road surface as shown in FIG. Is incident on. The higher the intensity of sunlight, the stronger the reflected light, and it will be detected as if there is a heat source even though there is no heat source in the sun.

  When detecting a pedestrian around a vehicle by detecting a heat source emitted from a human body, basically, an image captured by the infrared camera 2 is subjected to a binarization process, and a heat source exists in a portion where the signal value is large Judge that. However, since the signal value of the entire image changes depending on the background temperature, it is necessary to change the threshold value in the binarization process, that is, the heat source determination threshold value. For example, since the heat source determination threshold value is too low, the background of the sunny part where sunlight is reflected may be mistaken as a heat source. Therefore, in this embodiment, the heat source determination threshold value in the binarization process is changed based on the maximum luminance, the minimum luminance, or the average luminance of the image captured by the visible light camera 1.

  FIG. 5 is a diagram for explaining the heat source detection operation when the heat source determination threshold value in the binarization process is not changed. On the other hand, FIG. 6 is a diagram for explaining the heat source detection operation of an embodiment when changing the heat source determination threshold value in the binarization process. When there are a shaded part and a sunny part in the captured images of the visible light camera 1 and the infrared camera 2, the visible images by the visible light camera 1 are as shown in FIGS. 5 (a) and 6 (a). The luminance in the horizontal axis direction of the visible image is as shown in FIG. 5B and part 6B. Naturally, the luminance of the sunny part of the visible image is higher than the luminance of the shaded part.

  On the other hand, when the captured images of the visible light camera 1 and the infrared camera 2 have a shaded portion and a sunny portion, the infrared images from the infrared camera 2 are as shown in FIGS. 5 (c) and 6 (c). The above-mentioned reflected light from the sunlit portion can be used to brighten or darken the light. For example, the illuminance of sunlight is 54,000 to 76462 lux in the sunny part, which is about 8 times the illuminance of the shaded part. Sunlight includes a wavelength component associated with black body radiation at a temperature of about 6000 degrees, and the intensity ratio of visible light and infrared light of 8 to 13 μm used for human body detection also changes at a substantially similar ratio. Furthermore, since the reflectance of the road surface varies depending on the road surface, it is about several percent, so that the infrared light incident on the infrared camera 2 is also reduced. Assuming that the reflectance is 5%, the infrared light incident on the infrared camera 2 changes by 8 × 0.05 = 0.4 times, that is, 40%. If the infrared luminance exceeds the binarization threshold value (heat source determination threshold value) due to this change, it is erroneously detected that there is a heat source in the sunny part where there is no heat source, as shown in FIG. 5D and 6D show the luminance in the horizontal axis direction of the infrared image, that is, the output (pixel value) for each pixel of the image sensor (not shown) of the infrared camera 2.

  Therefore, in this embodiment, as shown in FIG. 6 (d), the binarization threshold value in the sunny portion is increased by an amount corresponding to the reflected light. In this way, even if the infrared brightness of the sunlit portion increases due to the reflected light, the binarization threshold value is not exceeded, and erroneous detection of the heat source can be prevented.

  Next, as shown in FIG. 5 (e) and FIG. 6 (e), when a human body is present at the boundary between the shaded portion and the sunny portion of the captured images of the visible light camera 1 and the infrared camera 2, the human body that is a heat source The infrared luminance greatly increases at the portion, and the presence of the heat source is detected when the infrared luminance exceeds the binarization threshold as shown in FIGS. 5 (f) and 6 (f). 5 (f) and 6 (f) show the luminance of the broken line portion of the infrared image shown in FIGS. 5 (e) and 6 (e), that is, the pixel of the image sensor (not shown) of the infrared camera 2. Each output (pixel value) is shown. When the binarization threshold value of the sunny portion is not changed, the infrared luminance of the sunny portion exceeds the binarization threshold value as shown in FIG. 5 (f), and there is no heat source in the sunny portion. It is falsely detected that there is a heat source. On the other hand, in one embodiment in which the binarization threshold value of the sunny portion is changed, the infrared luminance of the sunny portion does not exceed the binarization threshold value as shown in FIG. be able to.

  FIG. 7 is a flowchart illustrating heat source determination processing according to an embodiment. The operation of the embodiment will be described with reference to this flowchart. In step 1, an infrared image is acquired by the infrared camera 2. The heat source determination threshold value (binarization threshold value) set in advance in step 2 is read from a memory (not shown), or the heat source determination threshold value (binarization threshold value) is obtained by calculation from an infrared image, In the subsequent step 3, the infrared image is binarized and the heat source is detected using the heat source determination threshold value.

Next, in step 4, a visible image is acquired by the visible light camera 1. In the subsequent step 5, the heat source determination threshold value is corrected for each image based on the luminance of the visible image.
TH ′ = TH + THC = TH + (Vmax−Vm) * W * R (1)
In equation (1), TH ′ is a corrected heat source determination threshold value, TH is a heat source determination threshold value set in advance or calculated based on an infrared image, and THC is a heat source determination value calculated based on the luminance of a visible image. This is the correction value for the threshold. Vmax is the maximum luminance of the sunny part of the visible image, Vm is the minimum luminance or the average luminance of the shaded part of the visible image, and W is the wavelength coefficient that affects the luminance of the infrared image of the visible image, that is, black body radiation. The ratio of the visible light receiving energy and the infrared light receiving energy based on R, R is the assumed road surface reflectance.

  In step 6, heat source detection processing is performed from the visible image using the corrected heat source determination threshold value TH '. Thereafter, the above process is repeated.

  Thus, according to one embodiment, the infrared camera 2 that images an object scene with infrared rays and outputs an infrared image, and the range including the imaging range in the object scene of the infrared camera 2 is imaged with visible light. And a visible camera 1 that outputs a visible image, and the presence of a heat source is detected by comparing the brightness of the infrared image by the infrared camera 2 with a threshold value, based on the brightness of the visible image by the visible camera 1. Therefore, the influence on the heat source detection such as road surface reflected light can be effectively suppressed and erroneous detection of the heat source can be prevented.

  In one embodiment, the heat source is determined based on a value obtained by multiplying the difference between the maximum luminance and the minimum luminance or the average luminance of the visible image by the ratio of the visible light reception energy and the infrared light reception energy based on black body radiation. Since the threshold value is changed, the heat source can be detected more accurately.

  The correspondence between the constituent elements of the claims and the constituent elements of the embodiment is as follows. That is, the visible camera 1 constitutes a visible light imaging means, the infrared camera 2 constitutes an infrared imaging means, and the heat source determination device 3 constitutes a heat source determination means and a threshold value changing means. In addition, as long as the characteristic function of this invention is not impaired, each component is not limited to the said structure.

  In the above-described embodiment, the heat source detection device mounted on the vehicle has been described as an example, but the present invention is not limited to the vehicle.

It is a figure which shows the structure of one embodiment. It is a figure which shows the case where the part of the shade and the part of the sun are made behind vehicles by sunlight. It is the figure which looked at the shade part and sunny part of a vehicle rear part from the top. It is a figure which shows the reflection of a road surface. It is a figure explaining the heat-source detection operation | movement when not changing a heat-source determination threshold value. It is a figure explaining the heat source detection operation in the case of changing a heat source determination threshold value. It is a flowchart which shows a heat source judgment process.

Explanation of symbols

1 Visible Light Camera 2 Infrared Camera 3 Heat Source Determination Device 4 Alarm

Claims (4)

Infrared imaging means for imaging an object scene with infrared rays and outputting an infrared image;
A visible light imaging means for imaging a range including an imaging range in the object scene of the infrared imaging means with a visible light, and outputting a visible image;
Heat source determination means for detecting the presence of a heat source by comparing the brightness of each pixel of the infrared image by the infrared imaging means with a threshold value;
Threshold value changing means for correcting the threshold value calculated based on the infrared image using the luminance of the visible image by the visible light imaging means,
The on-vehicle heat source detection device, wherein the threshold value changing means makes the threshold value of the sun part higher than the threshold value of the sun part when the imaging range has a sun part and a shade part. . In the vehicle-mounted heat source detection device according to claim 1,
The on-vehicle heat source characterized in that the threshold value changing means increases the threshold value of the sunlit portion based on a difference between the maximum brightness of the sunlit portion and the minimum brightness of the shaded portion of the visible image. Detection device. In the vehicle-mounted heat source detection device according to claim 1,
The threshold value changing means increases the threshold value of the sunny portion based on a difference between a maximum luminance of the sunny portion of the visible image and an average luminance of the entire visible image . Heat source detection device. Infrared imaging means for imaging an object scene with infrared rays and outputting an infrared image;
A visible light imaging means for imaging a range including an imaging range in the object scene of the infrared imaging means with a visible light, and outputting a visible image;
Heat source determination means for detecting the presence of a heat source by comparing the brightness of an infrared image by the infrared imaging means with a threshold value;
Threshold changing means for changing the threshold based on the luminance of the visible image by the visible light imaging means,
The threshold value changing means includes a visible light receiving energy and an infrared light based on black body radiation based on a difference between a maximum luminance of a sunlit portion of the visible image and a minimum luminance of a shaded portion or an average luminance of the entire visible image. A vehicle-mounted heat source detection device characterized in that the threshold value of the sunny portion is increased based on a value obtained by multiplying a ratio of received light energy.

Images