JP3481854B2 - In-vehicle camera - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle-mounted camera for detecting a white line on a road, capturing an image of a vehicle traveling in front, and inputting it to an image-processing device mounted on a vehicle.

[0002]

13 to 15 are block diagrams showing a configuration of a conventional vehicle-mounted camera and image processing. FIG.
In the figure, 1 is a camera that converts incident light into an electric signal, 2
Is an amplifier for amplifying the output signal of the camera 1, 3 is an A / D conversion circuit, and 4 is an image processing circuit, and shows a schematic configuration of the vehicle-mounted image pickup apparatus. As shown in FIG. 14, the camera 1 includes a lens 11, an iris 12 that adjusts incident light from the lens 11, an iris control unit 13 that controls the iris 12, and an image sensor 14 that photoelectrically converts the incident light.
A CDS 15 that performs correlated double sampling of the electrical output of the image sensor 14, and an AGC 16 that has a variable amplification function,
A signal processing unit 17 that synchronizes signals, and an AGC control unit 18 that controls the amplification factor of the AGC by the output of the signal processing unit 17.
And a pulse signal and a signal processing unit 17 for extracting the charge of each pixel of the image sensor 14 as an image signal for each field.
And a NDR circuit 20 for removing noise.
As shown in FIG. 15, the DR circuit 20 is composed of an integrating circuit 21, an adding circuit 22, and a field memory 23.

Vehicle-mounted cameras mounted on automobiles for imaging and identifying road white lines and obstacles on roads are used in low-light areas such as during night driving or in tunnels to high-light areas under direct sunlight. In order to capture an image, the iris control unit 13 controls the iris 1 according to the level of the output signal of the image sensor 14.
2 to control the amount of incident light on the image sensor 14,
Further, it is configured to control the amplification factor of the AGC 16,
Further, the noise that increases due to the increase of the incident light amount and the amplification factor is reduced by the DNR circuit 20 to prevent the deterioration of the sharpness of the image. The DNR circuit 20 shown in FIG. 15 is a general cyclic DNR circuit, and the coefficient k input to the integrating circuit 21 in the figure is adjusted within the range of 1>k> 0 to obtain a signal of k times ( 1-k) times the previous field signal is added to reduce noise having no correlation in the time axis direction.

[0004]

As described above, in the vehicle-mounted camera, in order to perform the image processing from the low illuminance range to the high illuminance range, the signal obtained by the image pickup device 14 is used as the AGC1 having the variable amplification factor.
Although the amplification factor of 6 is controlled and the iris 21 is controlled, amplification of the signal also amplifies noise at the same time. There are two types of noise caused by imaging: one that depends on the amount of incident light and one that does not depend on the amount of incident light. With respect to noise that does not depend on the amount of incident light, the noise ratio increases as the illuminance of the subject decreases, and the S / N ratio deteriorates. It reduces the sharpness.

In order to deal with such a problem, it is considered to sensitize the image pickup device, and, for example, Japanese Patent Laid-Open No. 5-56.
As disclosed in Japanese Patent Application Laid-Open No. 354 and Japanese Patent Application Laid-Open No. 6-20943, it is proposed to increase the brightness value and improve the S / N ratio by adding the charges of pixels adjacent in the horizontal direction of the image sensor. However, for example, JP-A-5-563
In Japanese Patent Laid-Open No. 54-54, since the combination of pixels to be added is changed for each field and inter-field offset sampling is performed in order to suppress the deterioration of resolution due to the addition of two pixels, the circuit scale becomes complicated and large. According to Japanese Patent Laid-Open No. 6-20943, a combination of reset pulse phase control and signal interpolation processing using vertical correlation is performed as a resolution reduction suppression measure.
Similarly, the circuit scale becomes complicated and large, and it is inaccurate for in-vehicle use. In addition, in these examples, in order to prevent the deterioration of resolution, only two pixels in the horizontal direction are added, so that the image sensor There was a limit to sensitization.

[0006] Further, the forward landscape photographed by the vehicle-mounted camera is a white line on the road or an image of a forward vehicle. The images have correlation in the time axis direction, and noise has no correlation in the time axis direction. For this reason, it has been said that it is possible to remove noise without missing a necessary image by performing a normal DNR process, but the front image suddenly changes when, for example, a sudden lane change is performed. In this case, the images have no correlation in the time axis direction, a part of the normal image representing the current situation is deleted, and the accumulated value of the past image information from the field memory 23 up to the previous time is added. However, there is a problem that various image processings become impossible.

The present invention has been made in order to solve such a problem, and it is possible to perform sensitization without affecting the performance of image processing, and when a front image suddenly changes such as a lane change. Another object of the present invention is to obtain a small and inexpensive vehicle-mounted camera equipped with a DNR circuit that can obtain a normal image.

[0008]

An on-vehicle camera according to the present invention is an image pickup device for photoelectrically converting incident light, and a signal of a predetermined pixel is sampled for each pixel of an image picked up by this image pickup device. The image processing apparatus includes an image processing apparatus that forms an image, and a luminance adding unit that adds the signals of the peripheral pixels of the sampled pixels to the signals of the sampled pixels.

Further, the pixels picked up by the image pickup device are divided in the horizontal and vertical directions, and the signals of all the pixels in the divided blocks are added by the luminance adding means. Further, the number of peripheral pixels added by the brightness adding means is determined according to the ratio between the number of pixels of the image and the resolution of the image processing device.

Further, a DNR circuit as a noise removing means is provided on the output side of the image pickup signal to which the luminance is added by the luminance adding means, and a coefficient setting means for giving a feedback coefficient to the DNR circuit is provided. . Further, the feedback coefficient of the feedback coefficient determining means is set by the brightness difference of each pixel between the frames of the image. Furthermore, the feedback coefficient of the feedback coefficient determining means is set according to the steering amount of the vehicle. Further, the feedback coefficient of the feedback coefficient determining means is set according to the steering speed of the vehicle. Further, the steering amount or steering speed signal is detected by a steering wheel angle sensor. Further, an output adjusting means for making the output signal constant is added to the brightness adding means.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1. 1 to 5
FIG. 3 is a configuration diagram and an explanatory diagram of Embodiment 1 of the present invention,
In the figure, the same reference numerals are given to the same functional parts as those of the above-mentioned conventional example. In the block diagram of FIG. 1, 11 is a condenser lens, 12 is an iris that controls the incident light from the lens 11, 13 is an iris control unit that controls the iris 12, 1
Reference numeral 4 denotes photoelectric conversion of incident light from the lens 11, for example CC
An image pickup device constituted by D and the like, 15 is an image pickup device 14
CDS that performs correlated double sampling of the electrical output of the signal, 16 is an AGC having a variable amplification factor, 24 is an addition processing unit that performs addition processing of pixel charges, 17 is a signal processing unit that synchronizes signals, and 18 is a signal processing unit 17 An AGC control unit that controls the amplification factor of the AGC 16 according to the output of the
, A timing pulse generator that gives a drive pulse signal to the signal processor 17 and the addition processor 24, and 20 is an NDR processor for removing noise.

The addition processing section 24 is for adding the luminance values of the peripheral pixels to the obtained video signal and outputting the sensitized signal. As shown in the block diagram of FIG. An A / D converter 25 for A / D conversion, a memory 26 that holds a brightness value based on the charge amount of each pixel as a digital signal, and an operation for adding the brightness values of peripheral pixels for each pixel recorded in the memory 26 It is composed of a processing unit 27, a memory 28 that holds the added signal, and a D / A converter 29 that converts the signal of the memory 28 into an analog video signal.

In the vehicle-mounted camera according to the first embodiment of the present invention configured as described above, the iris control section 13 has a C
The average value of the luminance signal obtained from the DS15 is calculated, and the iris 12 is controlled so as to approach the preset target value. The input signal of the iris control unit 13 may be the output signal of the signal processing unit 17 instead of the output signal of the CDS 15. Further, the AGC control unit 18 obtains the average value of the luminance signal obtained from the signal processing unit 17, and controls the amplification factor of the AGC so as to approach the preset target value.

In the on-vehicle image processing apparatus, real-time processing is required for the processing, and due to the limitation of the calculation processing time, even the conventional apparatus does not use all the pixels of the image pickup element of the camera, but performs thinning-out sampling of the data to obtain the image information. We are reducing. According to the sampling theorem, the resolution of the camera may be 1/2 of that on the image processing side, and the resolution of the image processing device is horizontal (hereinafter referred to as H) 256 × vertical direction (hereinafter referred to as V).
If it is 256, the resolution required by the camera is H12.
8 × V128 will suffice. For example, if the camera has an image sensor of 250,000 pixels, the resolution for both H and V is 1/4.
It is possible to reduce to the optical LP
When F is incorporated and the resolution is lowered, it is possible to freely select and add pixels of 4 pixels each for H and V to improve the brightness. Figure 3 shows the image sensor with 250,000 pixels,
It shows the pixels of the image sensor and the sampling points of the image processing in the image processing with a resolution of H256 × V256. In this case, there are three missing pixels at each sampling point, and the signals of the missing pixels are given the luminance. The sensitivity can be sensitized four times by adding all the values.

In the screen of FIG. 4, the addition of the luminance value of each pixel aij from al1 to amn is as follows depending on the number of pixels to be added. For example, when three missing pixels in the periphery are added as in (70) of FIG.
7 performs the calculation of the equation (1). Further, in order to detect the edge in the H direction, when the calculation is performed while paying attention to the pixel information in the H direction, the pixels adjacent in the H direction are added as shown in (71) of FIG. The calculation of the calculation processing device 27 in this case is as in the following expression (2). In addition, although there are addition methods from (72) to (74) in FIG. 5, an operation according to the addition method can be set in the arithmetic processing unit 27.

In this way, the arithmetic processing unit 27 of the addition processing unit 24 adds the brightness values of the peripheral pixels, the added signal is recorded in the memory 28, and the timing pulse generation unit 19 is added.
Is read out by the drive pulse of, and converted into analog by the D / A converter 29, and output as a video signal. Since the pixel to be added is the original missing pixel, according to this embodiment, the sensitization of the camera can be realized without degrading the image quality of the image of the vehicle-mounted image processing apparatus.

Embodiment 2. FIG. 6 is an explanatory diagram of the second embodiment of the present invention. In the second embodiment, a captured image is divided according to the resolution of an image processing device, and the brightness value of each pixel in the divided block is shown. Is used as the sampling point of the image processing apparatus. For example, H512
When image processing of a captured image of an image sensor having × V490 pixels (250,000 pixels) at a resolution of H256 × V256, as shown in blocks A to C of FIG. 6, H2 × V2
The arithmetic processing unit 27 of the addition processing unit 24 is configured so that the pixels of 1 are set as one block, and the luminance values of the respective pixels in the block are all added to be the sampling point of the image processing. By fixing the block to be added in this way, the calculation amount of the calculation processing unit 27 is reduced, the processing time can be shortened, and the real-time property can be improved.

Embodiment 3. 7 to 9 are a block diagram and an explanatory view of the vehicle-mounted camera according to the third embodiment of the present invention. In this embodiment, as shown in FIG. 7, coefficient processing for giving a feedback coefficient to the DNR processing unit 20 is performed. Means 30 is provided,
The coefficient processing means 30 sets a coefficient according to the output signal of the AGC 16 and gives it to the DNR processing section 20. FIG. 8 is a flowchart showing the processing procedure of the coefficient setting means 30.
From the two frame images input from the GC 16, the difference in the sum of the brightness values of each pixel is obtained. That is, the luminance value bij of each image of the frame image 1 and the luminance value cij of each pixel of the frame image 2 shown in FIG. 9 are obtained, and the difference between the two is obtained by the following equation (3).

In process 101, the feedback coefficient is set according to the difference in the brightness value of the image between each frame. Feedback coefficient (1-k)
Is set to be small when the difference in the sum of the brightness values is large, and is set to be large when the difference in the sum of the brightness values is small, and is given to the DNR processing unit 20 in process 102. By setting the feedback coefficient according to the difference in the total luminance of images between frames in this way, when the screen changes suddenly, the feedback coefficient becomes small, the previous screen information becomes small, and the normal screen may be omitted. When there is no correlation and the screen has a correlation in the time axis direction, the feedback coefficient becomes large and the noise can be effectively removed.

Fourth Embodiment FIG. 10 is a block diagram of a vehicle-mounted camera according to a fourth embodiment of the present invention, and FIG. 11 is a flowchart showing a processing method. In this embodiment, the coefficient setting means 30 outputs an output signal of the AGC 16 and a vehicle not shown. The steering angle of the steering wheel and the steering angle are input. In the vehicle-mounted camera of this embodiment configured as described above, the feedback coefficient changes in accordance with the steering angle, in addition to the feedback coefficient changing due to the difference in images between frames as in the third embodiment.

In the flowchart of FIG. 11, process 1
In step 10, the steering angle of the vehicle is read by the coefficient setting means 30, and in step 111, the amount of change in the steering angle is compared with the reference value to be obtained. If the steering angle is smaller than the reference value, it is determined that the vehicle is traveling normally, the feedback coefficient is set large in step 112 to enable noise removal, and if the steering angle is larger than the reference value, it is determined that the screen suddenly changes, In step 114, the feedback coefficient is reduced to prevent the normal screen from being omitted, and the result is output to the DNR processing unit 20 in step 114. In the process 111, it is effective to detect the steering speed by the differential value of the change amount of the steering wheel angle. As described above, also in the fourth embodiment, it is possible to obtain a highly reliable image with low noise without a lack of a normal screen.

Embodiment 5. 12 is a block diagram of an addition processing unit of the vehicle-mounted camera according to the fifth embodiment of the present invention. In this embodiment, an output adjusting means 31 is added to the addition processing section 24 described in the first embodiment. The output adjusting means 31 obtains the average value of the brightness of the output signal of the AGC 16 and sets the brightness value of the input video signal to 1 / N so as to approach the preset target value. Output adjusting means 31
The output signal of is converted into a digital signal by the A / D converter 25 and is stored in the memory 26, and the arithmetic processing unit 27 adds the brightness ground of the peripheral image. After that, the signal is converted into an analog signal and output as in the first embodiment. However, by adjusting the signal to a constant level in this way, a stable video signal can be obtained and the reliability is further improved. It is a thing.

[0023]

As described above, according to the vehicle-mounted camera of the present invention, the addition processing unit adds the luminance signal of the missing pixel around the pixel sampled during the image processing to the sampling signal. Therefore, it is possible to obtain a sensitized image signal, so that the sharpness of the processed image does not decrease, and the pixels of the image are divided into blocks according to the ratio of the number of pixels of the image and the resolution of the image processing device. Since the addition processing unit outputs the sum of the signals of each pixel in the block, an image with high real-time property can be obtained. Furthermore, the DNR circuit is provided with a coefficient setting unit to calculate the sum of the luminance values between fields. The feedback coefficient is set according to the difference and the output of the steering angle sensor, so that it is possible to obtain a highly reliable image, and an excellent vehicle-mounted camera can be obtained. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of the present invention.

FIG. 2 is a block diagram of an addition processing unit according to the first embodiment of the present invention.

FIG. 3 is an explanatory diagram illustrating pixels and sampling points of an image according to Embodiment 1 of the present invention.

FIG. 4 is an explanatory diagram of addition processing of an addition processing unit according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram of addition processing of an addition processing unit according to the first embodiment of the present invention.

FIG. 6 is an explanatory diagram of block processing according to the second embodiment of the present invention.

FIG. 7 is a block diagram showing a configuration of a third embodiment of the present invention.

FIG. 8 is a flowchart for explaining the operation of the coefficient setting means according to the third embodiment of the present invention.

FIG. 9 is an explanatory diagram of a setting process of a coefficient setting unit according to the third embodiment of the present invention.

FIG. 10 is a block diagram showing a configuration of a fourth embodiment of the present invention.

FIG. 11 is a flowchart for explaining the operation of the coefficient setting means according to the fourth embodiment of the present invention.

FIG. 12 is a block diagram of an addition processing unit according to a fifth embodiment of the present invention.

FIG. 13 is a block diagram of a conventional vehicle-mounted camera and image processing.

FIG. 14 is a block diagram showing a configuration of a conventional vehicle-mounted camera.

FIG. 15 is a block diagram showing a configuration of a DNR circuit.

[Explanation of symbols]

11 lenses, 12 iris, 13 iris control section, 14 image sensor, 15 CDS, 16 ACG, 1
7 signal processing part, 18 ACG control part, 19 timing pulse generating part, 20 DNR processing part, 27 arithmetic processing device, 30 coefficient setting means.

Front page continuation (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N 5/225 H04N 5/21 H04N 5/232 H04N 5/243 H04N 7/18

Claims (9)

(57) [Claims] 1. An image sensor for photoelectrically converting incident light, an image processing apparatus for forming an image by sampling a signal of a predetermined pixel for each pixel of an image captured by the image sensor, and the sampled pixel. An in-vehicle camera, which is provided with a luminance adding means for adding the signals of the peripheral pixels to the sampled pixel signals. 2. The luminance adding means adds the pixels imaged by the image pickup device in the horizontal and vertical directions, and the signals of all the pixels in the divided blocks are added by the luminance adding means. In-vehicle camera. 3. The method according to claim 1, wherein the number of peripheral pixels added by the brightness adding means is determined according to a ratio between the number of pixels of the image and the resolution of the image processing device. In-vehicle camera. 4. A DNR circuit which is a noise removing means is provided on the output side of the image pickup signal to which the luminance is added by the luminance adding means, and a coefficient setting means for giving a feedback coefficient to the DNR circuit is provided. Claim 1 to claim 3
The vehicle-mounted camera according to any one of items 1. 5. The vehicle-mounted camera according to claim 4, wherein the feedback coefficient of the coefficient determining means is set by the brightness difference of each pixel between the frames of the image. 6. The vehicle-mounted camera according to claim 4, wherein the feedback coefficient of the coefficient determining means is set according to the steering amount of the vehicle. 7. The vehicle-mounted camera according to claim 4, wherein the feedback coefficient of the coefficient determining means is set according to the steering speed of the vehicle. 8. The vehicle-mounted camera according to claim 6 or 7, wherein the steering amount or steering speed signal is detected by a steering wheel angle sensor. 9. The vehicle-mounted camera according to claim 1, further comprising an output adjusting unit for making the output signal constant to the brightness adding unit.