JP6693387B2 - Image processing device - Google Patents

Description

  The present disclosure relates to a technique of processing an image captured using a high dynamic range function.

  A short shutter image captured with a relatively short shutter time and a long shutter image captured with a relatively long shutter time are combined to create an image with a wide dynamic range with less overexposure and blackout. A high dynamic range (HDR) function for generating is known. However, general devices such as monitors that reproduce images have only a standard dynamic range. Therefore, when reproducing an image captured using the HDR function, it is necessary to perform tone mapping processing to compress the dynamic range. In this tone mapping process, Log characteristics that are close to human visual characteristics are often used (for example, see Patent Document 1).

Japanese Patent Publication No. 2008-511048

  However, the Log characteristic has a problem that the output value is abruptly changed with respect to the input value in the low-luminance region, so that the output value is excessively assigned to the low-luminance region and the limited output value is wasted. is there. Further, in the Log characteristic, the output value changes very gently with respect to the input value in the middle and high brightness regions, so that there is also a problem that a gradation of a sufficient output value cannot be assigned to the imaging target. In particular, in the case of an on-vehicle camera, the brightness of a road surface or a pedestrian, which is an imaging target, is often distributed in the middle-brightness region, and it is not desirable that the gradation of the output value in the middle-brightness region is insufficient.

  The present disclosure has been made in view of the above circumstances, and provides an image processing device capable of generating an output image in which an output value is efficiently assigned to a captured image captured by an in-vehicle camera using the HDR function. The main purpose is to provide.

  The present disclosure is an image processing apparatus, which includes a storage unit (22) and an image conversion unit (21). The storage unit stores a compression characteristic in which the luminance represented by the input value belonging to the first range showing the range of values is represented by the output value belonging to the second range showing the range of values narrower than the first range. The image conversion unit is configured to convert the captured image captured by the vehicle-mounted camera (10) using the high dynamic range function using the compression characteristic stored in the storage unit to generate an output image. There is.

  The compression characteristic is such that the range of the input value corresponding to the distribution range set as the range in which the brightness of the imaging target of the vehicle-mounted camera is distributed is the main range, the minimum value of the main range, and the output of the same value as the minimum value of the main range. The point represented by the value is the first singular point, the maximum value of the main range and the point represented by the output value set for the maximum value of the main range are the second singular point, the minimum value of the first range, and The point represented by the minimum value of the second range is the start point, and the point represented by the maximum value of the first range and the maximum value of the second range is the end point. In the compression characteristic, the output value is the same as the input value in the value range from the starting point to the first singular point. In the compression characteristic, the output value with respect to the input value is represented by the Log function in the main value range. Furthermore, the compression characteristic is represented by a straight line connecting the second singular point and the end point with respect to the output value with respect to the input value in the value range from the second singular point to the end point.

  According to the compression characteristics of the present disclosure, in the range from the starting point to the first singular point, the output value is assigned to the input value in a one-to-one manner, so the output value is not wasted. Further, in the main range, the larger the output value at the second singular point, the more output values can be assigned to the imaging target. Therefore, by using the above compression characteristics, it is possible to generate an output image in which output values are efficiently assigned from a captured image.

  It should be noted that the reference numerals in parentheses described in this column and the claims indicate the correspondence with the specific means described in the embodiments described later as one aspect, and the technical scope of the present invention. It is not limited.

It is a block diagram which shows the structure of the driving assistance system which concerns on this embodiment. It is a graph which shows the compression characteristic concerning this embodiment. It is a graph which shows a Log characteristic which is a general compression characteristic. FIG. 4 is an enlarged view of a low brightness portion R in FIGS. 2 and 3.

Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings.
[1. Constitution]
First, the configuration of the image recognition system 100 according to the present embodiment will be described with reference to FIG. The image recognition system 100 includes an in-vehicle camera 10 and a processing device 20.

  The vehicle-mounted camera 10 includes an image pickup device such as a CCD sensor or a CMOS sensor, a lens for forming an image of a subject on the image pickup device, and a signal processing circuit, and a predetermined range including a road surface in front of the vehicle serves as an image pickup range. Thus, for example, it is installed on the windshield behind the room mirror.

  The vehicle-mounted camera 10 has an HDR function and continuously performs short shutter imaging and long shutter imaging. Short-shutter imaging is imaging in which the shutter time is relatively short, that is, the exposure time is relatively short. The long shutter imaging is imaging in which the shutter time is relatively long, that is, the exposure time is relatively long.

  Then, the vehicle-mounted camera 10 combines the captured short shutter image and the long shutter image to generate an HDR composite image having a wider dynamic range than the short shutter image and the long shutter image. The dynamic range is the width of the brightness difference that can be projected in one image, and is determined by the sensitivity of the image sensor. The generated HDR composite image is an image in which the brightness is represented by the input value x belonging to the preset first value range Xr. The vehicle-mounted camera 10 transmits the generated HDR composite image to the processing device 20 at a predetermined cycle. In this embodiment, the HDR composite image corresponds to the captured image.

  The processing device 20 is mainly configured by a well-known microcomputer including a semiconductor memory such as a CPU, a ROM, a RAM, and a flash memory. The processing device 20 realizes various functions by the CPU executing the programs stored in the non-transitional physical recording medium. By executing this program, the method corresponding to the program is executed. The number of microcomputers forming the processing device 20 may be one or plural.

  The processing device 20 includes an image processing unit 23 and a recognition processing unit 25. The image processing unit 23 includes an image conversion unit 21 and a storage unit 22. In the present embodiment, the image processing unit 23 corresponds to the image processing device.

  The storage unit 22 is composed of a flash memory or the like, and uses the brightness represented by the input value x belonging to the first range Xr as the output value y belonging to the second range Yr indicating a range of values narrower than the first range Xr. The compression characteristic TC represented is stored. Details of the compression characteristic TC will be described later.

  The image conversion unit 21 converts the input value x representing the brightness of the HDR composite image generated by the vehicle-mounted camera 10 into the output value y using the compression characteristic TC stored in the storage unit 22, and outputs the output image. To generate. That is, the image conversion unit 21 performs so-called tone mapping processing.

  The recognition processing unit 25 performs a recognition process of recognizing an imaging target such as a white line, a pedestrian, or another vehicle from the generated output image. Since the recognition processing unit 25 can recognize only the dynamic range of the short shutter image and the long shutter image, the image conversion unit 21 needs to convert the HDR composite image into an output image. The result of the recognition processing by the recognition processing unit 25 is sent to the vehicle control device and used for traveling support such as steering control and brake control.

[2. Compression characteristics]
Next, the compression characteristic TC stored in the storage unit 22 will be described with reference to FIGS. FIG. 2 is a graph showing the compression characteristic TC according to this embodiment. The starting point A of the compression characteristic TC is a point represented by the minimum value xa of the first range Xr and the minimum value ya of the second range Yr. The end point D of the compression characteristic TC is a point represented by the maximum value xd of the first range Xr and the maximum value yd of the second range Yr. That is, the starting point A = (xa, ya) and the ending point D = (xd, yd). Further, the compression characteristic TC has two singular points B and C between the start point A and the end point D, which are points at which the characteristic changes. Here, the singular point B that is the first singular point on the low luminance side is represented by (xb, yb), and the singular point C that is the second singular point on the high luminance side is represented by (xc, yc).

  The compression characteristic TC is a characteristic in which the output value y is the same value as the input value x in the value range from the starting point A to the singular point B. That is, the compression characteristic TC is represented by a straight line L1 of y = x in xa ≦ x ≦ xb.

  Further, the compression characteristic TC is a characteristic in which the output value y with respect to the input value x is represented by the Log function in the value range from the singular point B to the singular point C. That is, the compression characteristic TC is a curve L2 of y = α × Log10 (x) −β when xb ≦ x ≦ xc. α is the slope of the Log function and corresponds to the Log slope.

  Further, the compression characteristic TC is a characteristic in which the output value y with respect to the input value x is represented by a straight line connecting the singular point C and the end point D in the value range from the singular point C to the end point D. That is, the compression characteristic TC is a straight line L3 of y = yc + ((yd-yc) / (xd-xc)) * (x-xc) in xc≤x≤xd.

  Next, a procedure for generating the compression characteristic TC will be described. As for the compression characteristic TC, first, the input value xb of the singular point B and the input value xc of the singular point C are set. Specifically, the minimum value of the range of the corresponding input value x is set as the input value xb, and the maximum value is set in the distribution range set as the range where the brightness of the vehicle-mounted camera 10, for example, the road surface or the pedestrian is distributed. Is the input value xc. The range of xb ≦ x ≦ xc thus installed is defined as a main range Xm, the range of xa ≦ x ≦ xb having a lower brightness than the main range Xm is defined as a low range X1, and the range of xc ≦ having a higher brightness than the main range Xm is defined. The range of x ≦ xd is defined as the high range Xh. By setting the main range Xm, the output value yb = xb of the singular point B and the straight line L1 in the low range Xl are also determined.

  Next, the slope α of the curve L2 is calculated. Specifically, it is calculated as follows. An arbitrary input value belonging to the range of the main range Xm is A1, and an input value belonging to the main range Xm and K times the input value A1 is A2. K is a positive real number, which is a preset value. Then, the output value of the Log function for the input value A1 is B1, and the output value of the Log function for the input value A2 is B2.

  At this time, B1−B2 = α × (LogA1−LogA2) = α × Log (K). That is, the ratio K of the two input values is the difference between the two output values with respect to the two input values. Here, if the value of the difference B2-B1 is too small, the contrast of the imaging target in the output image becomes low. For example, when the imaging target is a road surface, the contrast of the white line with respect to the asphalt portion is low. Therefore, the slope α is set so that the difference B2-B1 becomes larger than the preset difference threshold. The difference threshold is the minimum difference that can obtain the necessary contrast for the imaging target.

  It should be noted that as the inclination α is increased, the difference B2-B1 is increased, and the contrast of the imaging target in the output image is increased. However, the output value yc of the singular point C increases by that amount, the output value y assigned to the high-value range Xh decreases, and the high-luminance portion of the output image is crushed. Therefore, in the present embodiment, the inclination α is set to a value such that the difference B2-B1 is slightly larger than the difference threshold. In the present embodiment, for example, the gradient α is calculated with a difference threshold value of 3 for the ratio K = 1.1.

  Next, the intercept β of the Log function having the slope α passing through the singular point B (xb, yb) is calculated. As a result, the curve L2 in the main range Xm is determined. When the curve L2 is determined, the output value yc corresponding to the input value xc is determined on the curve L2. Then, the straight line L3 in the high range Xh is also determined.

  The compression characteristic TC generated in this way is stored in the storage unit 22. Specifically, the table of the output value y with respect to the input value x based on the compression characteristic TC, and the expressions representing the straight line L1, the curve L2, and the straight line L3 are stored. Further, a plurality of different compression characteristics TC may be stored in the storage unit 22 according to shooting scenes such as daytime, nighttime, and inside a tunnel. Further, the main range Xm may be changed according to the shooting scene to generate a new compression characteristic TC, and the compression characteristic TC stored in the storage unit 22 may be updated. The image conversion unit 21 refers to a table stored in the storage unit 22 to convert the input value x into an output value y, or uses an expression stored in the storage unit 22 to convert the input value x to the output value y. Or calculate.

  Next, one of the numerical examples of the compression characteristic TC will be shown. The first range Xr is 0 to 690860 and the second range Yr is 0 to 255. The first range Xr is 116 dB of data. Then, the main range Xm is set to 40 to 13824. That is, the input value xc = 13824 of the starting point (0, 0), the ending point (659860, 13824), the singular point B (40, 40), and the singular point C is set. At this time, when the slope α of the Log function is calculated so that the necessary contrast can be obtained in the main range Xm, the slope α = 81. As a result, the intercept β = 90 of the Log function is calculated. That is, the curve L2 becomes y = 81 × Log10 (x) −90. Then, the output value yc = 245 of the singular point C on the curve L2 is calculated. Thereby, the straight line L3 is calculated as a straight line passing through the singular point C (13824, 245) and the end point (695860, 13824).

[3. Comparison]
FIG. 3 shows a graph of Log characteristics (hereinafter referred to as general characteristics) which are general compression characteristics, and FIG. 4 shows an enlarged view of the compression characteristics TC according to the present embodiment and the low-luminance portion R of the general characteristics. .. As shown in FIG. 4, in the low value range Xl, the general characteristic is that the slope is large and an excessive output value y is assigned to the input value x, and the output value y is wasted. On the other hand, in the compression characteristic TC, the slope of the straight line L1 is smaller than the slope of the general characteristic, and the output value y is integral with the input value x, so the output value y is not wasted. ..

  In the main range Xm, the slope of the curve L2 is larger than the slope of the general characteristic, and the input value x of the compression characteristic TC is assigned a larger output value y than the input value x of the general characteristic. ing. That is, in the compression characteristic TC, a larger number of output values y are assigned to the imaging target of the vehicle-mounted camera 10 than in the general characteristic.

  On the other hand, in the high value range Xh, the slope of the curve L2 is smaller than the slope of the general characteristic, and the input value x of the compression characteristic TC is assigned the output value y smaller than the input value x of the general characteristic. ing. That is, in the compression characteristic TC, the compression rate of the information of the subject having a higher luminance than the imaging target is higher than that in the general characteristic.

[4. effect]
According to this embodiment described above, the following effects can be obtained.
(1) According to the compression characteristic TC, in the low value range Xl from the starting point A to the singular point B, the output value y is assigned to the input value x in a one-to-one relationship, so the output value y is wasted. Never.

  (2) In the main range Xm, by setting the difference between the two output values B1 and B2 with respect to the two input values A1 and A2 of the ratio K to a value that exceeds the difference threshold, the contrast of the imaging target is sufficiently high in the output image. Can be high. As a result, the recognition accuracy of the imaged object in the output image can be increased. For example, when the imaging target is a road surface, the recognition accuracy of the white line can be increased by sufficiently increasing the contrast between the white line and the asphalt portion.

  (3) In the main range Xm, the slope α of the Log function is set to a slope that provides the necessary contrast, so that the output value yc at the singular point C is sufficiently large, and many output values yc with respect to the main range Xm. Can be assigned. As a result, it is possible to prevent the output image from excessively including the information of the high-value range Xh that is out of the imaging target, and efficiently include the information of the imaging target in the output image.

(Other embodiments)
Although the embodiments for carrying out the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be carried out.

  (A) In the above embodiment, the output value yc of the singular point C is calculated after calculating the slope α of the curve L2, but the present invention is not limited to this. For example, the value range yb ≦ y ≦ yc of the output value y assigned to the main value range Xm may be set first, and the slope α and the intercept β of the curve L2 may be calculated so as to pass through the singular point B and the singular point C.

  (B) A plurality of functions of one constituent element in the above embodiment may be realized by a plurality of constituent elements, or one function of one constituent element may be realized by a plurality of constituent elements. .. Further, a plurality of functions of a plurality of constituent elements may be realized by one constituent element, or one function realized by a plurality of constituent elements may be realized by one constituent element. Moreover, you may omit a part of structure of the said embodiment. Further, at least a part of the configuration of the above-described embodiment may be added or replaced with respect to the configuration of the other above-described embodiment. It should be noted that all modes included in the technical idea specified only by the wording recited in the claims are the embodiments of the present invention.

  (C) In addition to the image processing apparatus described above, a system having the image processing apparatus as a constituent element, a program for causing a computer to function as the image processing apparatus, and a non-transitional physical recording such as a semiconductor memory recording the program The present invention can be implemented in various forms such as a medium and an image generation method.

  Reference numeral 10 ... In-vehicle camera, 21 ... Image conversion unit, 22 ... Storage unit.

Claims (3)

A storage unit (22) that stores the compression characteristic that represents the brightness represented by the input value belonging to the first range showing the range of values by the output value belonging to the second range showing the range of values narrower than the first range. )When,
An image conversion unit configured to convert an imaged image captured by the vehicle-mounted camera (10) using the high dynamic range function using the compression characteristic stored in the storage unit to generate an output image ( 21) and, the compression characteristic is
The range of the input value corresponding to the distribution range set as the range in which the brightness of the imaging target of the vehicle-mounted camera is distributed is the main range, the minimum value of the main range, and the output of the same value as the minimum value of the main range. A point represented by a value is a first singular point, a maximum value of the main range and a point represented by the output value set for the maximum value of the main range are a second singular point and the first range. Of the minimum value and the minimum value of the second range as a starting point, and the maximum value of the first range and the maximum value of the second range as an end point,
In the range from the starting point to the first singular point, the output value is the same value as the input value,
In the main range, the output value with respect to the input value is represented by a Log function,
In the value range from the second singular point to the end point, the output value with respect to the input value is represented by a straight line connecting the second singular point and the end point.
Image processing device. The arbitrary input value belonging to the main range is A1, the input value that is α (α is a positive real number) times A1 is A2, the output value of the Log function for A1 is B1, and the Log for A2 is The output value of the function is B2,
The slope of the Log function is a Log slope that is a slope at which the difference between B2 and B1 becomes larger than a preset difference threshold.
The image processing apparatus according to claim 1. The Log function is a function having the Log slope and passing through the first singular point,
The output value representing the second singular point is a value corresponding to the maximum value of the main range on the Log function,
The image processing apparatus according to claim 2.

Images