JP4114752B2 - Imaging device - Google Patents

Description

  The present invention performs an abnormality determination using both the charge accumulation time control unit and the gain control unit, so that the appropriate exposure control performed according to the subject is accurately interrupted without interruption. The present invention relates to an imaging apparatus capable of performing abnormality determination. The present invention also relates to an imaging apparatus that can accurately determine an abnormality of an exposure control unit even in an imaging apparatus having nonlinear input / output characteristics.

  In general, in an imaging apparatus, exposure control for determining a final video signal is performed by controlling the charge accumulation time of a light receiving element, or by gain control (amplification) for a photoelectrically converted analog video signal. Therefore, when this exposure control is abnormal, an appropriate image output cannot be obtained.

  In order to solve such an abnormal exposure, a selection means for selecting a predetermined area from a screen obtained by the imaging means, a luminance detection means for detecting the luminance of the predetermined area, and the luminance of the predetermined area are predetermined. There is an apparatus that includes an exposure control unit that adjusts the iris or shutter speed so that the luminance of a predetermined area becomes the target luminance based on the target luminance (see, for example, Patent Document 1).

JP-A-8-240833 (particularly paragraph 0018)

In conventional imaging devices, it is necessary to temporarily interrupt exposure control according to the subject for shooting in order to determine an exposure abnormality by adjusting the iris or shutter speed and increasing or decreasing the exposure amount. was there. Specifically, in addition to the process of obtaining the image output at the shutter speed optimized according to the subject, the exposure control is temporarily interrupted to obtain the image output when the shutter speed is the fastest and the slowest. It was determined whether the average luminance in the image output corresponds to the shutter speed.
For this reason, the conventional imaging apparatus has a problem that the exposure control process for shooting must be temporarily interrupted, or the exposure control abnormality determination must be performed when the exposure control process for shooting is not performed. It was.

  In addition, in an imaging device that captures images with nonlinear input / output characteristics having at least one characteristic switching point in the entire imaging range, even if the exposure amount is increased or decreased from the current value, the exposure amount is increased or decreased. There was a problem that output could not be obtained.

  Further, since exposure control is performed by controlling the charge accumulation time of the light receiving element or by gain control (amplification) with respect to the analog video signal subjected to photoelectric conversion, it is necessary to determine whether the gain control unit is abnormal. In particular, in an imaging apparatus that captures images with nonlinear input / output characteristics that do not have a characteristic switching point over the entire imaging range, the charge accumulation time is not controlled, and gain control (amplification) is performed on an analog video signal that has been photoelectrically converted. Therefore, the abnormality determination of the gain control unit is essential.

  The present invention has been made to solve the above-described problems, and performs an abnormality determination using both the charge accumulation time control unit determined by the shutter speed and the gain control unit, so that it can respond to the subject. It is an object of the present invention to provide an imaging apparatus capable of accurately determining an abnormality of an exposure control unit without interrupting proper exposure control performed in this manner.

  In addition, even in an imaging device that captures an image with a nonlinear input / output characteristic having at least one characteristic switching point in the entire imaging range, it is possible to accurately determine the abnormality for each characteristic region delimited by the characteristic switching point. An object of the present invention is to provide an imaging apparatus capable of performing an abnormality determination of an exposure control unit.

  Furthermore, by determining the abnormality of the gain control unit, it is possible to accurately determine the abnormality of the exposure control unit even in an imaging apparatus that captures images with nonlinear input / output characteristics that do not have characteristic switching points over the entire imaging range. An object is to provide an imaging device.

The imaging device of the present invention includes a photoelectric conversion unit that photoelectrically converts light received from a subject,
A signal processing unit that performs control processing of a time for accumulating charges in the photoelectric conversion unit and a gain to be given to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an exposure or gain abnormality of the image output processed by the signal processing unit, the photoelectric conversion unit has a linear output signal characteristic with respect to the incident light intensity, and the abnormality determination unit includes: For the variables m and n having a relationship of m × n = 1, the image output obtained by the accumulation time C and the gain G is compared with the image output obtained by the n times accumulation time Cn and the m times gain Gm. Thus, an abnormality in exposure or gain is determined.

The imaging device of the present invention includes a photoelectric conversion unit that photoelectrically converts light received from a subject,
A signal processing unit that performs control processing of a time for accumulating charges in the photoelectric conversion unit and a gain to be given to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an exposure or gain abnormality of the image output processed by the signal processing unit, the photoelectric conversion unit has a linear output signal characteristic with respect to the incident light intensity, and the abnormality determination unit includes: For the variables m and n having a relationship of m × n = 1, the image output obtained by the accumulation time C and the gain G is compared with the image output obtained by the n times accumulation time Cn and the m times gain Gm. Thus, since an abnormality in exposure or gain is determined, it is possible to determine an abnormality in exposure using an image output obtained by an accumulation time (shutter speed) and gain optimally controlled according to the subject. In a system that recognizes a white line, when comparing the image output when the shutter speed is the fastest, optimal, and slowest as in the past, the white line and the road surface become an image with a white line. It does not occur inconvenience can not be detected. Further, in the white line recognition system using the imaging apparatus of the present invention, it is not necessary to temporarily stop the white line recognition process for detecting an exposure abnormality.

Embodiment 1 FIG.
Hereinafter, an imaging apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram schematically showing a configuration of an imaging apparatus according to Embodiment 1 of the present invention.
FIG. 2 is a diagram conceptually showing an example of attachment of the image pickup apparatus according to Embodiment 1 of the present invention to a vehicle.
FIG. 3 is a diagram showing an example of an image taken by the imaging apparatus according to Embodiment 1 of the present invention.
FIG. 4 shows an example of image output at an optimum accumulation time (shutter speed) in the image pickup apparatus, image output at the longest accumulation time (the shutter speed is slowest), and image output at the shortest accumulation time (the shutter speed is the fastest). FIG.
FIG. 5a is a flowchart showing a process including an exposure control abnormality determination process in the imaging apparatus according to Embodiment 1 of the present invention.
FIG. 5b is a flowchart showing a process including an exposure control abnormality determination process in the imaging apparatus according to Embodiment 1 of the present invention.
FIG. 6 is a diagram showing input / output characteristics (signal level characteristics with respect to image plane illuminance) of the image sensor used in the imaging apparatus according to Embodiment 1 of the present invention.

  As illustrated in FIG. 1, the imaging apparatus 100 includes a photoelectric conversion unit 201 configured by, for example, a photodiode, an analog signal processing unit 202 that processes an output signal of the photoelectric conversion unit 201, and an output of the analog signal processing unit 202. An A / D conversion unit 206 that converts a signal into a digital signal, an image memory 207 that stores an output signal of the A / D conversion unit 206 as image data, and a CPU 208 connected to the image memory 207 are provided.

Here, the photoelectric conversion unit 201, the analog signal processing unit 202, and the A / D conversion unit 206 constitute an imaging element. Note that the imaging element does not necessarily include the A / D conversion unit 206.
The imaging apparatus 100 having such a configuration captures an image of a subject using a single linear input / output characteristic of the imaging element over the entire imaging range.

  The photoelectric conversion unit 201 includes a light receiving element (not shown) that receives reflected light or light emitted from the subject, and converts an output signal of the light receiving element into an analog video signal having a voltage corresponding to the amount of light. That is, the luminance of the subject is converted into a voltage signal by the photoelectric conversion unit 201 and input to the analog signal processing unit 202.

  The analog signal processing unit 202 includes an accumulation time control unit 203 that controls the charge accumulation time in the photoelectric conversion unit 201, and an exposure control unit that includes a gain control unit 204 that amplifies the analog signal output from the photoelectric conversion unit 201. 205, and further includes a CDS circuit, a γ correction circuit, and the like.

The analog signal processing unit 202 processes the analog video signal output from the photoelectric conversion unit 201 and transmits the processed video signal component to the A / D conversion unit 206.
The analog video signal input to the A / D converter 206 is converted to a digital video signal by the A / D converter 206 and stored in the image memory 207 as image data. Further, the image data stored in the image memory 207 is subjected to image processing by the CPU 208.

  For example, the exposure control unit 205 in the analog signal processing unit 202 performs feedback control of charge accumulation time and gain so that the average luminance in a predetermined area included in the image data becomes a target value, and accumulates according to the subject. Set time and gain.

  In the imaging apparatus according to Embodiment 1 of the present invention, the CPU 208 includes a feedback control unit (not shown) and an abnormality determination unit 209 that determines an exposure or gain abnormality of the image output processed by the analog signal processing unit 202. The charge accumulation time and the gain necessary for controlling the exposure control unit 205 can be set in the exposure control unit 205 from the CPU 208.

In the following, when it is simply described as exposure control or exposure, it includes not only exposure but also gain unless otherwise specified. Therefore, reference to exposure control or abnormality includes gain control or abnormality.
The predetermined area for calculating the average luminance of the image data is an area surrounded by R (R area) shown in FIG. The average luminance in the R region may be calculated using either the video signal component after the analog video signal processing or the digital video signal after A / D conversion.

In the first embodiment, the abnormality determination unit 209 is provided in the CPU 208 at the subsequent stage of the A / D conversion in order to perform the exposure abnormality determination using the digital video signal. However, the video signal after the analog signal processing is processed. It may be arranged before the A / D conversion, as used.
In the first embodiment, the CPU 208 performs various types of image processing. However, a block dedicated to image processing may be provided separately.

  Further, in the first embodiment, as shown in FIG. 2, the imaging device 100 is arranged in the vicinity of the rear confirmation mirror in the vehicle interior of the vehicle, and is configured to capture a front road image. Accordingly, as illustrated in FIG. 3, the imaging device 100 captures an image output Ji including a white line on the road surface, and the CPU 208 in the imaging device 100 recognizes the captured white line.

FIG. 4 shows an image output J1 obtained at the currently set accumulation time (shutter speed), an image output J2 obtained at the longest accumulation time (latest shutter speed), and the shortest accumulation time (fastest shutter). It is a figure which shows image output J3 obtained by (speed).
As shown in FIG. 4, when the exposure control is normally performed, the accumulation time becomes long (
As the shutter speed decreases), the brightness of the image increases. That is, the luminance of the image is image output J3 <image output J1 <image output J2.

FIG. 5A is a flowchart illustrating an outline of processing of an example of an imaging apparatus including processing for determining abnormality in exposure control.
In step S400, the average brightness B (for example, the average brightness in the region R) of the image data used for controlling the exposure according to the subject is acquired. S401 is a step of performing image processing performed by the imaging apparatus, and S402 is a step of determining whether abnormality determination is being performed. If it is determined in S402 that the abnormality determination is not being performed, the process proceeds to S403 to determine whether or not to perform the abnormality determination process. If no abnormality determination is made, the process proceeds to S404, and exposure control corresponding to the subject is performed based on the luminance B obtained in S400. When performing abnormality determination, it progresses to S405 and starts abnormality determination processing. After S404 and S405, the process returns to S400 and repeats the steps. If the abnormality determination process is started in S403, it is determined in S402 in the subsequent process that the abnormality determination is being performed, and thus the process proceeds to S405 and the abnormality determination process is continued. The abnormality determination process is repeated until the abnormality determination process ends in S405.

Hereinafter, the process after the abnormality determination process is started will be described with reference to FIG.
FIG. 5B is a flowchart illustrating an outline of an example of processing for determining an abnormality in exposure control performed by the imaging apparatus.
In step S501, the storage time C1 optimally controlled according to the subject is set. The accumulation time is a time during which the image sensor accumulates charges by receiving light from the subject, and is inversely proportional to the shutter speed.
In step S502, the optimally controlled gain G1 is set according to the subject.
(At this time, exposure control is assumed to be stable.)

For example, in the optimum control of the accumulation time and gain for obtaining the optimum image of the subject, the CPU 208 compares the target value of the preset image output with the image output obtained with the preset accumulation time and gain, and the CPU 208 performs feedback control. Is done by doing
In step S503, the average luminance B (for example, the average luminance in the region R) in the case of the accumulation time C1 and the gain G1 optimally controlled according to the subject is stored as B1. That is, the average luminance in the region R of the image output J1 shown in FIG. 4 is stored.

In step S504, an accumulation time C1n that is n times the accumulation time C1 optimally controlled in accordance with the subject in step S501 is set.
In step S505, a gain G1m that is m times the gain G1 optimally controlled in accordance with the subject in step S502 is set.
Thereafter, the process returns to step S404, and the average luminance when the accumulation time is C1n and the gain is G1m is acquired. Since the process proceeds from S402 to S506 during the abnormality determination, in step S506, the average luminance B when the accumulation time is C1n and the gain is G1m is set to B2.

  In step S505, in the first embodiment, the accumulation time and the gain are set such that C1 × G1 = (C1 × n) × (G1 × m) = C1n × G1m. That is, in an image sensor having linear characteristics, n × m = 1 (n and m are arbitrary constants satisfying this relationship). For this reason, the final video signal including the accumulation time and the gain is set to be the same as when the accumulation time and the gain optimally controlled according to the subject are used.

  Here, in FIG. 6, an input / output characteristic 601 indicates the input / output characteristic of the image sensor when the accumulation time is C1n and the gain is G1m. Similarly, the input / output characteristic 602 indicates the input / output characteristic when the accumulation time is C1 and the gain is G1. An input / output characteristic 603 indicates the input / output characteristic when the accumulation time is C1n and the gain is G1. The input / output characteristic 602 is an input / output characteristic when exposure is optimally controlled according to the subject.

That is, the brightness B1 acquired in step S503 is obtained by the input / output characteristic 602.
The input / output characteristic 603 is a characteristic obtained when the accumulation time of the input / output characteristic 602 is multiplied by n, and the input / output characteristic 601 is an input / output characteristic obtained when the gain of the input / output characteristic 603 is multiplied by m. .
In the first embodiment, since n × m = 1, if the exposure control is performed normally, the output when the accumulation time is C1n and the gain is G1m shows the same characteristics as the input / output characteristics 602. It should be.

In step S507, the average brightness B1 and the average brightness B2 are compared to determine whether or not the difference between these values is within a predetermined range in order to perform the exposure abnormality determination.
Specifically, in step S507, the determination is made based on, for example, whether or not the following expression holds.
B1 to B2 ratio: Db = B1 / B2> k1 or Db = B1 / B2 <k2
Here, the constants K1 and k2 are thresholds that can be arbitrarily set.
Here, since n × m = 1, if the exposure control is normally performed, the average luminance B1 and the average luminance B2 have substantially the same value.

Therefore, for example, in the case of an image sensor in which the gain can be set in 40 levels, when a gain difference of 3 levels or more occurs between the average brightness B1 and the average brightness B2, the average brightness B1 and the average brightness B2 are obtained. What is necessary is just to determine with a difference more than a predetermined value.
In this case, if “a state in which there is a difference in three stages of gain” = “a state in which the gain is 1.3 times”, the threshold values k1 and k2 in the above equation are set as k1 = 1.3 and k2 = 0. .77 (1 / 1.3).

Here, the case where it is determined that the exposure is abnormal when a difference of three or more levels occurs in the gain has been described. However, the determination criterion (gain difference) is not limited to this, and other comparison arithmetic expressions are appropriately used. A comparison may be made using.
That is, for example, the comparison may be performed based on the difference between the average luminances instead of the comparison based on the ratio between the average luminance B1 and the average luminance B2.
Moreover, you may comprise so that the difference of average brightness | luminance B1 and average brightness | luminance B2 may be determined with the difference of the stage number of exposure time instead of a gain.

If it is determined in step S507 that the difference between the average luminance B1 and the average luminance B2 is less than the predetermined value, it is determined that the exposure control is normally performed, and the abnormality determination ends.
On the other hand, if it is determined in step S507 that the difference between the average luminance B1 and the average luminance B2 is greater than or equal to a predetermined value, it is determined that the exposure control is abnormal, and the flow proceeds to step S508.
In the subsequent step S508, a warning that the exposure is abnormal is issued to the user of the imaging apparatus.
In step S508, a warning is issued. However, the operation of the white line recognition system using such an imaging apparatus may be stopped.

  As described above, according to the imaging apparatus of the present invention, it is determined whether or not the exposure is abnormal using the image output (corresponding to the image output J1 in FIG. 4) obtained by the accumulation time (shutter speed) and gain optimally controlled according to the subject. In a system that recognizes a white line on the road surface, when comparing the image output when the shutter speed is the fastest (accumulation time is shortest), optimal and slowest (longest) (as in the past). The white line and the road surface are overexposed, and the white line cannot be detected, and the white line recognition system using the conventional imaging device does not detect the white line in order to detect the exposure abnormality. Although it was necessary to temporarily stop, in the white line recognition system using the imaging apparatus of the present invention, it is not necessary to pause the white line recognition process for detecting the exposure abnormality.

In the first embodiment, the exposure abnormality determination is performed during the system operation. For example, when the system is turned on, or this imaging device is mounted on a vehicle and used for recognition of a traveling lane on the road surface. In this case, the abnormality determination unit may be configured to execute the abnormality determination process when the vehicle enters the accessory mode, when the ignition is turned on, or immediately after the imaging device is turned on. Alternatively, the abnormality determination unit may be configured to execute the abnormality determination process when the vehicle speed is equal to or lower than a predetermined value.
In addition, here, the case where the imaging device is applied to a white line recognition system mounted on a vehicle has been described, but the application range of the imaging device of the present invention is not limited to such a system, and is applicable to other systems. Is also applicable.
The same applies to the imaging apparatuses according to Embodiments 2 to 8 described below.

Embodiment 2. FIG.
Hereinafter, an image pickup apparatus according to Embodiment 2 of the present invention will be described.
The configuration of the imaging apparatus according to Embodiment 2 of the present invention is similar to the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
The outline of an example of the imaging apparatus including the exposure control abnormality determination process according to the second embodiment of the present invention conforms to the flowchart (see FIG. 5A) of the imaging apparatus according to the first embodiment.
However, in the present embodiment, since the optimum exposure control according to the subject is not used, the image processing shown in S401 is not performed during the abnormality determination.
FIG. 7 is a flowchart showing the contents of exposure control abnormality determination processing in the imaging apparatus according to Embodiment 2 of the present invention.
FIG. 8 is a diagram showing input / output characteristics (signal level characteristics with respect to image plane illuminance) of the image sensor used in the imaging apparatus according to Embodiment 2 of the present invention.

In FIG. 7, in step S701, the accumulation time reference C2 is selected as the shortest accumulation time.
In step S702, the gain reference G2 is selected to be the smallest value.
In step S703, an accumulation time C2x that is a target of abnormality determination is set. Here, C2x is set to the longest accumulation time. In S710, the gain is set to G2.
Thereafter, the process returns to step S400, and the average luminance when the accumulation time is C2x and the gain is G2 is acquired. Since the process proceeds from S402 to S712 during the abnormality determination, it is determined whether or not B3 has been stored. If B3 has not been stored, in step S704, the average brightness B (when the accumulation time is C2x and the gain is G2). For example, the average luminance in the region R) is stored as B3.

In step S711, an accumulation time reference C2 is set.
In step S705, a gain G2y that is a target of abnormality determination is set. Here, G2y is set to a value satisfying G2y = (C2x / C2) × G2.
Thereafter, the process returns to step S404, and the average luminance when the accumulation time is C2 and the gain is G2y is acquired. Since the process proceeds from S402 to S712 during the abnormality determination, it is determined whether or not B3 has been stored. If B3 has been stored, the average in the image output when the accumulation time is C2 and the gain is G2y in step S706. The luminance B is B4.

  In FIG. 8, an input / output characteristic 604 is an input / output characteristic of the image sensor when the accumulation time is C2 and the gain is G2. The input / output characteristic 605 is an input / output characteristic when the accumulation time is C2x and the gain is G2. The input / output characteristic 606 is an input / output characteristic when the accumulation time is C2 and the gain is G2y.

The average luminance B3 acquired in step S704 is due to the input / output characteristic 605.
The average luminance B4 acquired in step S706 is due to the input / output characteristic 606.
As described above, in the imaging apparatus according to Embodiment 2 of the present invention, G2y is G2y = (C2x / C) × G, and therefore the output in the case of the input / output characteristic 803 is the same as the input / output characteristic 605. Show the characteristics.

In step S707, exposure abnormality determination is performed based on whether or not the following expression is satisfied.
B3 to B4 ratio: Db = B3 / B4> k1, or Db = B3 / B4 <k2
Here, the constants K1 and k2 are thresholds that can be arbitrarily set.
Here, since G2y = (C2x / C) × G, if the exposure control is normally performed, the average luminance B3 and the average luminance B4 should be almost the same value.

Therefore, for example, if the image sensor can set the gain in 40 steps, it can be set so that an abnormality in exposure is determined when a gain difference of 3 steps or more occurs between the average luminance B1 and the average luminance B2. .
In this case, if “difference in three stages of gain” = “gain is 1.3 times”, the threshold values k1 and k2 in the above equation are set to k1 = 1.3, k2 = 0.77 (1 / 1.3 ) Can be set.

Here, the case where it is determined that the exposure is abnormal when a difference of three or more levels occurs in the gain has been described. However, the determination criterion (gain difference) is not limited to this, and other comparison arithmetic expressions are appropriately used. You may go on.
For example, the comparison may be performed based on the difference between the average luminances instead of the comparison based on the ratio between the average luminance B3 and the average luminance B4.
Moreover, you may comprise so that the difference of average brightness | luminance B1 and average brightness | luminance B2 may be determined with the difference of the stage number of exposure time instead of a gain.

In step S707, when it is determined that the difference between the average luminance B3 and the average luminance B4 is less than the predetermined value, it is determined that the exposure control is normally performed, and the abnormality determination ends.
On the other hand, if it is determined in step S707 that the difference between the average luminance B3 and the average luminance B4 is greater than or equal to a predetermined value, it is determined that the exposure control is abnormal, and the flow proceeds to step S708.
In a subsequent step S708, a warning that the exposure is abnormal is issued to the user of the imaging apparatus.
In step S708, a warning is issued. However, the operation of the white line recognition system using such an imaging apparatus may be stopped.

As described above, in the imaging device according to Embodiment 2 of the present invention, instead of using the optimally controlled gain and accumulation time according to the subject as in Embodiment 1, using any accumulation time and gain, It is possible to make an exposure abnormality determination.
That is, by setting C2x and G2y to arbitrary values, it is possible to perform abnormality determination at a plurality of points or a plurality of times in succession, and the accuracy of abnormality determination can be improved.

In the second embodiment, G2y is set to be Gy = (C2x / C) × G. However, weighting using a constant L is performed and G2y = L × ((C2x / C) × G) is set. Then, the exposure abnormality determination may be performed.
However, in that case, it is necessary to convert B3 or B4 according to the value of L and determine abnormality, or to set the thresholds k1 and k2 according to the value of L.

Further, according to the imaging apparatus according to Embodiment 2 of the present invention, since the image accumulation time and gain optimized according to the subject are not required, when the white line recognition system using this imaging apparatus is turned on Alternatively, it is possible to determine the abnormality of exposure even when a vehicle equipped with such a system is ignited or when a system operation such as white line recognition such as a vehicle speed of a predetermined speed or less is not performed.
In addition, in an imaging apparatus that does not perform white line recognition processing, such as a camera, it is possible to notify the user that exposure abnormality determination is being performed and to perform exposure abnormality determination at an arbitrary timing.

Embodiment 3 FIG.
In Embodiments 1 and 2, an imaging apparatus having an imaging element having linear input / output characteristics has been described. However, the present invention can also be applied to an imaging apparatus using an imaging element having nonlinear input / output characteristics. .

FIG. 9 is a graph representing the input / output characteristics (signal level characteristics with respect to image plane illuminance) of the image sensor used in the imaging apparatus according to the third embodiment of the present invention.
In FIG. 9, an imaging element having nonlinear input / output characteristics has a linear characteristic portion 801 in which incident light-output characteristics are linear in low illuminance and a logarithmic characteristic in incident light-output characteristics in high illuminance. And a logarithmic characteristic portion 802. An inflection point 803 is a switching point of input / output characteristics of the imaging apparatus, and its signal level is CHB.
Here, the case where the imaging apparatus having nonlinear input / output characteristics has the characteristics shown in FIG. 9 will be described as an example. However, since there are various methods for realizing nonlinear input / output characteristics, only the characteristics shown in FIG. It is not limited to.

Hereinafter, an image pickup apparatus according to Embodiment 3 of the present invention using an image pickup element having nonlinear input / output characteristics will be described.
The configuration of the imaging apparatus according to Embodiment 3 of the present invention is in accordance with the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
The outline of the process of an example of the imaging apparatus including the exposure determination abnormality determination process according to the third embodiment of the present invention is in accordance with the flowchart (see FIG. 5A) of the imaging apparatus according to the first embodiment.
The processing content of the exposure control abnormality determination applied to the imaging apparatus according to Embodiment 3 of the present invention basically conforms to the processing content of Embodiment 1 (see FIG. 5b). The judgment method is different.

FIG. 10 is a flowchart showing an outline of processing of an example of exposure control abnormality determination in the imaging apparatus according to Embodiment 3 of the present invention.
FIG. 11 is a diagram showing the principle of input / output characteristic conversion in exposure abnormality determination according to Embodiment 3 of the present invention.

In step S1001, an accumulation time C3 that is optimally controlled according to the subject is set.
In step S1002, the optimally controlled gain G3 is set according to the subject. For example, the optimum control of the accumulation time and gain for obtaining the optimal image of the subject is set with a preset target value for image output. The image output obtained by the accumulation time and the gain is compared, and the CPU 208 performs feedback control.

  In step S1003, the average brightness B (for example, the average brightness in the region R) in the case of the accumulation time C3 and gain G3 optimally controlled according to the subject is acquired as B5. That is, the average luminance in the region R of the image output J1 shown in FIG. 4 is acquired.

In step S1004, it is determined whether the acquired average luminance B5 is greater than the signal level CHB at the switching point.
If the average luminance B5 is greater than the signal level CHB at the switching point, it is determined that the input / output characteristic is the input / output characteristic 802 in the nonlinear region, and the flow proceeds to step S1005.
In step S1005, a conversion function for the input / output characteristic 802 is used. The conversion will be described later.

On the other hand, if the average luminance B5 is smaller than the signal level CHB at the switching point in step S1004, it is determined that the input / output characteristic is the input / output characteristic 801 in the linear region, and the flow proceeds to step S1006.
In step S1006, a conversion function for the input / output characteristic 801 is used. The conversion will be described later.
In step S1015, the converted average luminance is stored as B5 ′.

In step S1007, an accumulation time C3n that is n times the accumulation time C3 optimally controlled according to the subject in step S1001 is set.
In step S1008, a gain G3m that is m times the gain G3 optimally controlled in accordance with the subject in step S1001 is set.
In step S1008, as in the first embodiment, the accumulation time and gain are set such that C3 × G3 = (C3 × n) × (G3 × m) = C3n × G3m. That is, in an image sensor having linear characteristics, n × m = 1 (n and m are arbitrary constants satisfying this relationship).
Thereafter, the process returns to step S400, and the average luminance when the accumulation time is C3n and the gain is G3m is acquired. Since the process proceeds from S402 to S1009 during the abnormality determination, in step S1009, the average luminance B when the accumulation time is C3n and the gain is G3m is acquired as B6.
Thereafter, the process returns to step S400, and the average luminance when the accumulation time is C1n and the gain is G1m is acquired. Since the process proceeds from S402 to S506 during the abnormality determination, in step S506, the average luminance B when the accumulation time is C1n and the gain is G1m is set to B2.

Next, in step S1010, as in step S1005, it is determined whether or not the average luminance B6 is greater than the signal level CHB at the switching point 803.
If the average luminance B6 is greater than the signal level CHB at the switching point 803, it is determined that the input / output characteristic is the input / output characteristic 802, and the flow proceeds to step S1011.
In step S1011, a conversion function for the input / output characteristic 802 is used. The conversion process will be described later.

On the other hand, if it is determined in step S1010 that the average luminance B6 is smaller than the signal level CHB at the switching point 803, the input / output characteristic is determined to be the input / output characteristic 801, and the flow proceeds to step S1012.
In step S1012, a conversion function for the input / output characteristic 801 is used. The conversion process will be described later.
In step S1016, the average luminance after conversion is set to B6 ′.

The conversion process in step S1006, step S1007, step S1011 and step S1012 described above will be described.
The processing in these steps is processing for converting the nonlinear input / output characteristics 801 and the input / output characteristics 802 so as to be linear input / output characteristics with respect to the input.
For example, the input / output characteristics are image plane illuminance E [lx], digital video signal B [LSB] after A / D conversion (luminances B5 and B6 are also average values in a predetermined area of the digital video signal in this embodiment. When there is an accumulation time C3n [s], a gain G3m, and an image sensor sensitivity S [LSB / lx · s], the input / output characteristic 801 is expressed by the following equation, for example.
B = E × C3n × G3m × S

The input / output characteristic 802 is expressed by the following equation, for example.
B = (CHB + K × log (E × C3n × S-CHB)) × G3m
When the output after B is converted by the above equation is B ′, in this embodiment, since the input / output characteristic 801 used in step S1007 and step S1012 is a linear region characteristic, the conversion formula of the characteristic 801 is The following formula.
B '= B
Further, since the input / output characteristics 802 used in steps S1006 and S1011 are characteristics in a nonlinear region, the following expression is used.
B ′ = (10 ^ (((B / G3m) −CHB) / K) + CHB) × G3m

As a result, the input / output characteristics after conversion have output characteristics that are continuously linear from the input / output characteristics 801 as shown by the input / output characteristics 804 of FIG.
Accordingly, as in the first embodiment, the final video signal including the accumulation time and the gain is the same as when the accumulation time and the gain optimally controlled according to the subject are used.

The exposure control abnormality determination based on the luminance comparison in step S1013 is performed using the converted average luminance B5 ′ and average luminance B6 ′.
The comparison itself between the average luminances B5 ′ and B6 ′ is in accordance with the comparison process in the first embodiment.
B5 ′ to B6 ′ ratio: Db = B5 ′ / B6 ′> k1, or Db = B5 ′ / B6 ′ <k2
Here, the constants K1 and k2 are thresholds that can be arbitrarily set.
Here, since C3 × G3 = (C3 × n) × (G3 × m) = C3n × G3m, if exposure control is performed normally, the average luminance B5 ′ and the average luminance B6 ′ are It should be almost the same value.

Therefore, for example, if the image sensor can set the gain in 40 steps, it can be set so that an abnormality in exposure is determined when a gain difference of 3 steps or more occurs between the average luminance B1 and the average luminance B2. .
In this case, if “difference in three stages of gain” = “gain is 1.3 times”, the threshold values k1 and k2 in the above equation are set to k1 = 1.3, k2 = 0.77 (1 / 1.3 ) Can be set.

Here, the case where it is determined that the exposure is abnormal when a difference of three or more levels occurs in the gain has been described. However, the determination criterion (gain difference) is not limited to this, and other comparison arithmetic expressions are appropriately used. You may go on.
For example, not the comparison based on the ratio between the average luminance B5 ′ and the average luminance B6 ′, but the comparison may be performed based on the difference between the average luminances.
Moreover, you may comprise so that abnormality may be determined by the difference in the number of steps of exposure time.

In step S1013, when it is determined that the difference between the average luminance B5 ′ and the average luminance B6 ′ is less than the predetermined value, it is determined that the exposure control is normally performed, and the abnormality determination ends.
On the other hand, if it is determined in step S1013 that the difference between the average luminance B5 ′ and the average luminance B6 ′ is greater than or equal to a predetermined value, it is determined that the exposure control is abnormal, and the flow proceeds to step S1014.

In step S1014, when it is determined that the exposure is abnormal, an abnormality is reported to the user.
In step S1014, a warning is issued. However, the operation of the white line recognition system using such an imaging apparatus may be stopped.

  As described above, according to the imaging apparatus according to Embodiment 3 of the present invention, the image output (corresponding to the image output J1 in FIG. 4) obtained by the accumulation time (shutter speed) and gain optimally controlled according to the subject is used. Therefore, in a system that recognizes the white line on the road surface, the image with the fastest shutter speed (shortest accumulation time), optimum and slowest (longest) (as in the past) is used. When comparing the output, the white line and the road surface are overexposed, and there is no problem that the white line cannot be detected.In addition, in the white line recognition system using the conventional imaging device, in order to detect an abnormal exposure Although it is necessary to pause processing such as white line recognition, in the white line recognition system using the imaging apparatus of the present invention, there is no need to pause white line recognition processing for detecting an exposure abnormality.

Further, as in the second embodiment, the exposure abnormality determination may be performed using an arbitrary accumulation time and gain instead of the image accumulation time and gain optimized according to the subject. By setting the accumulation time and the gain to arbitrary values, it is possible to perform abnormality determination at a plurality of points or a plurality of times in succession, and the accuracy of abnormality determination can be improved.
In particular, when the system is turned on or mounted in a vehicle, it is possible to determine the exposure abnormality even when the ignition is turned on or even when the vehicle speed is below a predetermined level and the system operation such as white line recognition is not performed.

  In the third embodiment, there is one input / output characteristic switching point and the input / output characteristic includes one type each of the linear region and the nonlinear region. Any number of linear regions or non-linear regions can be processed in the same manner. Similarly, for an image sensor having non-linear input / output characteristics without a switching point, the input / output characteristics are linear. By using the conversion formula, the exposure abnormality determination unit shown in the present embodiment can be used.

Embodiment 4 FIG.
Hereinafter, an image pickup apparatus according to Embodiment 4 of the present invention will be described.
The configuration of the imaging apparatus according to Embodiment 4 of the present invention is similar to the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
The outline of an example of the imaging apparatus including the exposure control abnormality determination process according to the fourth embodiment of the present invention is in accordance with the flowchart (see FIG. 5A) of the imaging apparatus according to the first embodiment.
FIG. 12 is a flowchart illustrating an outline of an example of processing for determining abnormality in exposure control performed by the imaging apparatus.
Further, the imaging apparatus according to Embodiment 4 of the present invention has the input / output characteristics shown in FIG. 9 in the same manner as the imaging element according to Embodiment 3.

  FIG. 12 is a flowchart showing processing details of exposure control abnormality determination in the imaging apparatus according to Embodiment 4 of the present invention.

In step S1201, an accumulation time C4 that is optimally controlled according to the subject is set.
In step S1202, a gain G4 optimally controlled according to the subject is set.
For example, in the optimum control of the accumulation time and gain for obtaining the optimum image of the subject, the CPU 208 compares the target value of the preset image output with the image output obtained with the preset accumulation time and gain, and the CPU 208 performs feedback control. Is done by doing

  In step S1203, the average luminance B (for example, the average luminance in the region R) in the case of the accumulation time C4 and gain G4 optimally controlled according to the subject is acquired as B7. That is, the average luminance in the region R of the image output J1 shown in FIG. 4 is acquired.

In step S1204, it is determined whether or not the acquired average luminance B7 is greater than the signal level CHB at the switching point.
If the average luminance B7 is larger than the signal level CHB at the switching point, it is determined that the input / output characteristic is the input / output characteristic 802 in the nonlinear region, and the flow proceeds to step S1205.
In step S1205, a conversion function for the input / output characteristic 802 is used. The conversion is the same as in the third embodiment.

On the other hand, if the average luminance B7 is smaller than the signal level CHB at the switching point in step S1204, it is determined that the input / output characteristic is the input / output characteristic 801 in the linear region, and the flow proceeds to step S1206.
In step S1206, a conversion function for the input / output characteristic 801 is used. The conversion is the same as in the third embodiment.
In step S1215, the converted average luminance is stored as B7 ′.

In step S1207, an arbitrary accumulation time C different from the accumulation time C4 optimally controlled according to the subject in step S1201 is set. Here, the gain remains G4 and need not be specified.
After the arbitrary accumulation time C is set in step S1207, the steps of S400 → S402 → S1216 → S1217 are repeated to perform exposure control using only gain. The gain G is a value obtained as a result of exposure control, and varies depending on the situation. Here, the exposure control is performed for a predetermined time when the exposure control is stabilized or considered to be stable.
After the predetermined time has elapsed, in step S1209, the average luminance B is acquired as B8.

  Next, in step S1210, as in step S1204, it is determined whether or not the average luminance B8 is greater than the signal level CHB at the switching point 803.

If the average luminance B8 is greater than the signal level CHB at the switching point 803, it is determined that the input / output characteristic is the input / output characteristic 802, and the flow proceeds to step S1211.
In step S1211, a conversion function for the input / output characteristic 802 is used. The conversion process is the same as in the third embodiment.

On the other hand, if the average luminance B8 is smaller than the signal level CHB at the switching point in step S1210, it is determined that the input / output characteristic is the input / output characteristic 801 in the linear region, and the flow proceeds to step S1212.
In step S1212, a conversion function for the input / output characteristic 801 is used. The conversion is the same as in the third embodiment.
In step S1218, the converted luminance B8 is set to B8 ′.

The exposure control abnormality determination based on the luminance comparison in step S1213 is performed using the converted average luminance B7 ′ and average luminance B8 ′.
The average luminances B7 ′ and B8 ′ are both image outputs obtained by performing optimal exposure control. That is, in the present embodiment, the optimal exposure control is controlled so that the image output becomes a preset target value. Therefore, if the exposure control is normally performed, the average brightness B7 ′ and the average brightness B8 ′ are Should be almost the same value.

  Therefore, for example, the average brightness 7 'and the average brightness 8' are compared, and when there is a difference of 20 [LSB] or more, an abnormality in exposure can be determined. At this time, there is a high possibility that the exposure control using only the gain for obtaining the average luminance of 8 'is not optimal, and it is determined that the gain is abnormal rather than the accumulation time.

In the optimum exposure control, since a target value is set in advance, when the difference between the average luminance 8 ′ and the target value is greater than or equal to a predetermined value, it may be determined that there is an abnormality.

  In step S1207, the accumulation time is fixed to an arbitrary value C different from the optimum exposure control accumulation time C4. However, the gain may be fixed to an arbitrary value G different from the optimum exposure control gain G4. In such a case, the optimum exposure control is performed only for the accumulation time, not for the gain, and an abnormality in the accumulation time can be determined.

If it is determined in step S1213 that the exposure control is normally performed, the abnormality determination ends.
On the other hand, if it is determined in step S1213 that the exposure control is abnormal, the flow proceeds to step S1214.

In step S1214, when it is determined that the exposure is abnormal, an abnormality is reported to the user.
In step S1214, a warning is issued. However, the operation of the white line recognition system using such an imaging apparatus may be stopped.

As described above, according to the imaging apparatus according to Embodiment 4 of the present invention, the abnormality of exposure is determined using the image output (corresponding to the image output J1 in FIG. 4) obtained while continuing the optimal control according to the subject. Therefore, in a system that recognizes a white line on the road surface, when comparing the image output when the shutter speed is the fastest (accumulation time is the shortest), the optimum and the slowest (the longest) as in the past, The white line and the road surface are overexposed, and there is no inconvenience that the white line cannot be detected.In the white line recognition system using the conventional imaging device, processing such as white line recognition is temporarily performed in order to detect the exposure abnormality. Although it was necessary to stop, in the white line recognition system using the image pickup apparatus of the present invention, it is not necessary to temporarily stop the white line recognition process for detecting the exposure abnormality.
Furthermore, in Embodiments 1 to 3, since the accumulation time and gain for abnormality determination are set as fixed values, when there is a large variation in one step of gain (accumulation time), it is a variation as to whether it is abnormal. It becomes difficult to determine whether or not. In general, since the value of the accumulation time has a very small variation compared to the gain, in this embodiment, the exposure control is performed in a state where either the accumulation time or the gain is fixed. Abnormality determination can be performed in a state in which variations in the above are absorbed.

  In the fourth embodiment, there is one input / output characteristic switching point, and the input / output characteristic includes one type of linear region and nonlinear region. However, the number of input / output characteristic switching points, Any number of linear regions or non-linear regions can be processed in the same manner. Similarly, for an image sensor having non-linear input / output characteristics without a switching point, the input / output characteristics are linear. By using the conversion formula, the exposure abnormality determination unit shown in the present embodiment can be used.

Embodiment 5. FIG.
Hereinafter, an imaging apparatus according to Embodiment 5 of the present invention will be described.
The configuration of the imaging apparatus according to Embodiment 5 of the present invention is similar to the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
The outline of the processing of an example of the imaging device including the processing for determining the abnormality of exposure control according to the fifth embodiment of the present invention is in accordance with the flowchart (see FIG. 5A) of the imaging device according to the first embodiment.
FIG. 13 is a flowchart illustrating an outline of an example of processing for determining abnormality of exposure control performed by the imaging apparatus.

FIG. 13 is a flowchart showing processing details of exposure control abnormality determination in the imaging apparatus according to Embodiment 5 of the present invention.
As shown in FIG. 13, in step S1402, an arbitrary accumulation time C5 different from the accumulation time optimally controlled according to the subject is set. The gain need not be specified here.
After the arbitrary accumulation time C is set in step S1402, the steps of S400 → S402 → S1403 → S1404 → S1405 are repeated to perform exposure control using only gain. The gain G is a value obtained as a result of exposure control, and varies depending on the situation. Here, the exposure control is performed for a predetermined time when the exposure control is stabilized or considered to be stable.
After the predetermined time has elapsed, the average luminance B is acquired as B9 in step S1406.

In step S1409, it is determined whether or not the acquired average luminance B9 is higher than the signal level CHB of the switching point CH.
If the average luminance B9 is greater than the signal level CHB at the switching point, it is determined that the input / output characteristics are the input / output characteristics 802 in the nonlinear region, and the flow proceeds to step S1410.
In step S1410, conversion processing is performed using a conversion function for the input / output characteristics 802. The conversion process is the same as in the third embodiment.

On the other hand, if the average luminance B9 is smaller than the signal level CHB at the switching point in step S1409, it is determined that the input / output characteristic is the input / output characteristic 801 in the linear region, and the flow proceeds to step S1411.
In step S1411, conversion processing is performed using a conversion function for the input / output characteristic 801. The conversion process is the same as in the third embodiment.
In step 1412, the converted luminance B9 is stored as B9 ′.

As shown in FIG. 13, in step S1413, the gain is set to G5. Here, the accumulation time is an arbitrary value C and does not need to be specified.
After the arbitrary gain G5 is set in step S1413, the steps of S400, S402, S1403, S1414, and S1415 are repeated to perform exposure control using only the accumulation time. The accumulation time C is a value obtained as a result of exposure control, and varies depending on the situation. Here, the exposure control is performed for a predetermined time when the exposure control is stabilized or considered to be stable.
After the predetermined time has elapsed, in step S1416, the average luminance B (for example, the average luminance in the region R) is acquired as B10.
Next, in step S1417, it is determined whether or not the average luminance B10 is greater than the signal level CHB at the switching point 803.
If it is determined that the average brightness B10 is greater than the signal level at the switching point 803, the input / output characteristic is determined to be 802, and the flow proceeds to step S1418.
In step S1418, conversion processing is performed using a conversion function for 802. The conversion process is the same as in the third embodiment.

On the other hand, if it is determined in step S1417 that the average luminance B10 is lower than the signal level CHB at the switching point, it is determined that the input / output characteristic is 801, and the flow proceeds to step S1419.
In step S1419, conversion processing is performed using a conversion function for the input / output characteristic 801. The conversion process is the same as in the third embodiment.
In step 1420, the converted luminance B10 is set to B10 ′.

After step S1420, step S1421 and subsequent steps shown in FIG. 13 are performed.
The exposure control abnormality determination based on the luminance comparison in step S1421 is performed in the same manner as in the fourth embodiment, and in this embodiment, the average luminance B9 ′ and the average luminance B10 ′ after conversion are used.
The average luminances B9 ′ and B10 ′ are both image outputs obtained by performing optimal exposure control. That is, in the present embodiment, the optimal exposure control is controlled so that the image output becomes a preset target value. Therefore, if the exposure control is normally performed, the average brightness B9 ′ and the average brightness B10 ′ are Should be almost the same value.

Therefore, for example, the average brightness 9 ′ and the average brightness 10 ′ can be compared, and when there is a difference of 20 [LSB] or more, an abnormality in exposure can be determined.
In the optimum exposure control, since the target value is set in advance, when the difference between the average luminance 9 ′ or the average luminance 10 ′ and the target value is greater than or equal to a predetermined value, it may be determined that there is an abnormality.

If it is determined in step S1421 that the difference between the average luminance B9 ′ and the average luminance B10 ′ is less than the predetermined value, it is determined that the exposure control is normally performed, and the abnormality determination ends.
On the other hand, if it is determined in step S1421 that the difference between the average luminance B9 ′ and the average luminance B10 ′ is greater than or equal to a predetermined value, it is determined that the exposure control is abnormal, and the flow proceeds to step S1422.

In step S1422, if it is determined that the exposure is abnormal, the user is notified of the abnormality.
Although the warning is issued in step S1422, the operation of the white line recognition system using such an imaging apparatus may be stopped.

As described above, according to the imaging apparatus according to Embodiment 5 of the present invention, the abnormality of exposure is determined using the image output (corresponding to the image output J1 in FIG. 4) obtained while continuing the optimum control according to the subject. Therefore, in a system that recognizes a white line on the road surface, when comparing the image output when the shutter speed is the fastest (accumulation time is the shortest), the optimum and the slowest (the longest) as in the past, The white line and the road surface are overexposed, and there is no inconvenience that the white line cannot be detected.In the white line recognition system using the conventional imaging device, processing such as white line recognition is temporarily performed in order to detect the exposure abnormality. Although it was necessary to stop, in the white line recognition system using the image pickup apparatus of the present invention, it is not necessary to temporarily stop the white line recognition process for detecting the exposure abnormality.
Furthermore, in Embodiments 1 to 3, since the accumulation time and gain for abnormality determination are set as fixed values, when there is a large variation in one step of gain (accumulation time), it is a variation as to whether it is abnormal. It becomes difficult to determine whether or not. In this embodiment, since exposure control is performed in a state in which one of the accumulation time and the gain is fixed, abnormality determination can be performed in a state where variations in gain (accumulation time) are absorbed.

  In the fifth embodiment, the case where there is one input / output characteristic switching point and the input / output characteristic includes one type each of the linear region and the nonlinear region is described. Any number of linear regions or non-linear regions can be processed in the same manner. Similarly, for an image sensor having non-linear input / output characteristics without a switching point, the input / output characteristics are linear. By using the conversion formula, the exposure abnormality determination unit shown in the present embodiment can be used.

Embodiment 6 FIG.
In the sixth embodiment, a case of an imaging device having a nonlinear input / output characteristic having a plurality of switching points will be described.
FIG. 14 is a diagram illustrating input / output characteristics (signal level characteristics with respect to image plane illuminance) of an image sensor used in an imaging apparatus according to Embodiment 6 of the present invention.
As shown in FIG. 14, the imaging apparatus according to the sixth embodiment has a nonlinear input / output characteristic having a plurality of switching points, a linear characteristic unit 1601 having a linear incident light-output characteristic in the case of low illuminance, and a high The first non-linear characteristic unit 1602 has a nonlinear incident light-output characteristic in the case of illuminance, and the second non-linear characteristic unit 1603 has a non-linear incident light-output characteristic in the case of high illuminance.

CH1 and CH2 are switching points of the input / output characteristics of this imaging apparatus, and the signal levels thereof are CHB1 and CHB2, respectively.
Here, the signal level start point is CH0 and the end point is CH3.
Here, as an example of an imaging apparatus having nonlinear input / output characteristics having a plurality of switching points, the case of having the characteristics of FIG. 14 will be described as an example, but a method for realizing nonlinear input / output characteristics having a plurality of switching points Since there are various types, there is no limitation to the characteristics shown in FIG.

Hereinafter, an image pickup apparatus according to Embodiment 6 of the present invention using an image pickup element having a nonlinear input / output characteristic having a plurality of switching points will be described.
The configuration of the imaging apparatus according to Embodiment 6 of the present invention is similar to the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
An outline of processing of an example of an imaging apparatus including processing for determining abnormality in exposure control according to Embodiment 6 of the present invention is in accordance with the flowchart (see FIG. 5A) of the imaging apparatus according to Embodiment 1.
FIG. 15 is a flowchart showing processing details of exposure control abnormality determination applied to the imaging apparatus according to Embodiment 6 of the present invention.

  In FIG. 15, in step S1703, a first switching point CHj for determining an input / output characteristic section for performing abnormality determination is set in an imaging apparatus having a plurality of switching points for input / output characteristics. (J is a natural number from 0 to n-1 where n is the number of switching points) Here, it is set to the start point CH1 of the section shown in FIG.

In step S1704, a second switching point CHj + 1 is set. Here, the end point CH2 of the section shown in FIG. 14 is set.
In step S1705, a target value for exposure control is set within the range of the abnormality determination section set in steps S1703 and S1704. Here, it is TGT shown in FIG.
After the target value TGT for exposure control is set in step S1705, the steps of S400 → S402 → S1706 → S1707 are repeated to perform exposure control. The accumulation time C6 and the gain G6 are values obtained as a result of exposure control, and differ depending on the situation. Here, the exposure control is performed for a predetermined time when the exposure control is stabilized or considered to be stable.
After the predetermined time has elapsed, in step S1710, the average luminance B (for example, the average luminance in the region R) is acquired as B11.

In steps S1711, S1712, it is determined whether or not the average luminance B11 is within the range of the signal levels CHBj to CHBj + 1 of the switching points CHj to CHj + 1 set in steps S1703 and S1704.
If the average luminance B11 is within the range of the signal level CHBj to CHBj + 1 at the switching point, it is determined that the input / output characteristics are H0j + 1, and a conversion function for H0j + 1 is used.
In step S1714, the converted luminance B11 is stored as B11 ′.
On the other hand, if the average luminance B11 is out of the range of the signal levels CHBj to CHBj + 1 of the switching points CHj to CHj + 1, it is determined that it is outside the abnormality determination section, and the process is terminated.
In this case, although the TGT is set within the range of the signal levels CHBj to CHBj + 1 of the switching points CHj to CHj + 1, the average luminance B11 does not fall within the range of the replacement points CHj to CHBj + 1 even after a predetermined time has elapsed. In the case, it is determined as abnormal.
Further, when it is not determined that there is an abnormality, the abnormality determination ERR may be set and the process may be restarted from S400.

In step S1715, an accumulation time C6n that is n times the accumulation time C6 optimally controlled in accordance with the subject in step S1708 is set.
In step S1716, a gain G6m that is m times the gain G6 optimally controlled in accordance with the subject in step S1709 is set.
In step S1718, the average luminance B when the accumulation time is C6n and the gain is G6m is acquired as B12.
Next, in steps S1719 and S1720, it is determined whether or not the average luminance B12 is within the range of the signal levels CHBj to CHBj + 1 of the switching points CHj to CHj + 1 set in step S1703.

If average luminance B12 is within the range of signal level CHBj to CHBj + 1 at the switching point, it is determined that the input / output characteristics are H0j + 1, and the flow proceeds to step S1721.
In step S1721, the conversion process is performed using the conversion function for H0j + 1. This conversion process is a process similar to the conversion process in steps S1006, S1007, S1011 and S1012 of the third embodiment.
As a result, the converted input / output characteristics have a single continuous input / output characteristic with respect to the input, as in the third embodiment.
In step S1722, the converted luminance B12 is set to B12 ′.

The comparison of the average luminance in step S1724 is performed using the converted luminances B11 ′ and B12 ′.
When it is determined that the exposure is abnormal as a result of the comparison of the average luminance in step S1724, the flow proceeds to step S1725.
If it is determined that the exposure is normal, the process proceeds to S1726, and the abnormality determination ends.

In step S1725, when it is determined that the exposure is abnormal, an abnormality is reported to the user.
In step S1725, a warning is issued. However, the operation of the white line recognition system using such an imaging apparatus may be stopped.

On the other hand, if the average luminance B12 is outside the range of the signal levels CHBj to CHBj + 1 of the switching points CHj to CHj + 1 in steps S1719 and S1720, it is determined that the area is outside the abnormality determination section, and the process proceeds to S1723. In this case, if the average luminance 12 does not fall within the range of the replacement points CHj to CHBj + 1 even after a predetermined time has elapsed, it is determined that there is an abnormality.
Further, when it is not determined that there is an abnormality, the abnormality determination ERR may be set and the process may be restarted from S400.

Note that the switching point set in step S1703 and step S1704 may be fixed, and the exposure abnormality in the same one section may always be determined.
Further, it may be configured to perform the exposure abnormality determination in the entire range of the input / output characteristics while switching the sections in time.
Further, as in the second embodiment, the exposure abnormality determination may be performed using an arbitrary accumulation time and gain instead of the image accumulation time and gain optimized according to the subject. By setting the accumulation time and the gain to arbitrary values, it is possible to perform abnormality determination at a plurality of points or a plurality of times in succession, and the accuracy of abnormality determination can be improved.
In particular, when the system is turned on or mounted in a vehicle, it is possible to determine the exposure abnormality even when the ignition is turned on or even when the vehicle speed is below a predetermined level and the system operation such as white line recognition is not performed.

  As described above, according to the imaging apparatus according to Embodiment 6 of the present invention, by limiting the input / output characteristic section in which abnormality determination is performed, the time for stopping the system operation by shortening one abnormality determination operation is reduced. Can be reduced.

Further, when the abnormality determination section set in step S1703 and step S1704 is limited to a section including the original target value of exposure control, step S1705 can be omitted, and the entire subject can be imaged optimally. Judgment is possible. In the case of an image pickup apparatus used in a road white line recognition system, the optimum control means an exposure value at which the road surface brightness is an optimum value in the image.
When the processing is further simplified, the input / output characteristic conversion processing in steps S1713 and S1721 may be omitted, and the abnormality may be detected by simply comparing the average luminance B11 and the average luminance B12.

In an imaging device having both linear input / output characteristics and nonlinear input / output characteristics, if the abnormality determination section set in steps S1703 and S1704 is limited to sections having linear input / output characteristics, step S1713 and step S1713 are performed. The conversion process in S1721 can be omitted, and an abnormal exposure can be detected by a simplified flow.
As for the abnormality determination, in S1711, S1712 or S1719, S1720, the luminance B is set as a determination target only within the set section, and for those outside the set section, the abnormality or ERR determination is performed in S1717 or S1723. ,
Similarly to the fourth and fifth embodiments, the section of the luminance B may be determined, converted with characteristics according to the section of the luminance B, and determined by comparison with the target value.

Embodiment 7 FIG.
In the imaging apparatus according to Embodiment 7, the abnormality determination is performed at once for a plurality of sections in the input / output characteristics of the imaging element. This is because the imaging apparatus according to the sixth embodiment performs an exposure control abnormality determination in one specific section of the input / output characteristics of the image sensor, or switches the determination target section in terms of time to change all of the input / output characteristics. In contrast to performing an exposure control abnormality determination for a range, an exposure control abnormality determination is performed at once for a plurality of sections of input / output characteristics of the image sensor.

Hereinafter, an exposure abnormality determination process in an imaging apparatus using an imaging element having a plurality of switching points and having nonlinear input / output characteristics will be described.
The configuration of the imaging apparatus according to Embodiment 7 of the present invention is similar to the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
The outline of an example of the imaging apparatus including the exposure control abnormality determination process according to the seventh embodiment of the present invention is in accordance with the flowchart (see FIG. 5A) of the imaging apparatus according to the first embodiment.
FIG. 16 is a flowchart showing processing details of exposure control abnormality determination applied to an imaging apparatus according to Embodiment 7 of the present invention.

The processing contents in steps S1903 to S1923 are in accordance with the processing contents in steps S1703 to S1723 in the sixth embodiment.
In FIG. 16, in step S1902, i for setting the abnormality determination target section is initialized. In step S1903, in an imaging apparatus having a plurality of switching points in the input / output characteristics, the first switching that is the start point of the section of the input / output characteristics to be subjected to abnormality determination is performed in order to perform abnormality determination of the exposure control of the image sensor. Set point CHi. In step S1904, a second switching point CHi + 1 that is the end point of the predetermined section is set. (I is a natural number from 0 to n-1 where n is the number of switching points)
In step S1905, the target value TGTi + 1 for exposure control is set within the range of the abnormality determination section set in steps S1903 and S1904.

After the target value TGT for exposure control is set in step S1905, the steps of S400 → S402 → S1906 → S1907 are repeated to perform exposure control. The accumulation time C7i and the gain G7i are values obtained as a result of exposure control, and differ depending on the situation. Here, the exposure control is performed for a predetermined time when the exposure control is stabilized or considered to be stable.
After the predetermined time has elapsed, in step S1910, the average luminance B (for example, the average luminance in the region R) is acquired as B13i.

In steps S1911, S1912, it is determined whether or not the average luminance B13i is within the range of the signal levels CHBi to CHBi + 1 of the switching points CHi to CHi + 1 set in steps S1903 and S1904.
If the average luminance B13i is within the signal level CHBi to CHBi + 1 at the switching point, it is determined that the input / output characteristics are H0i + 1, and a conversion function for H0i + 1 is used.
In step S1914, the converted luminance B13i is stored as B13i ′.
On the other hand, if the average luminance B13i is outside the range of the signal levels CHBi to CHBi + 1 of the switching points CHi to CHi + 1, it is determined that it is outside the abnormality determination section, and the process ends.
In this case, although the TGTi + 1 is set within the range of the signal levels CHBi to CHBi + 1 of the switching points CHi to CHi + 1, the average luminance B13i does not fall within the range of the replacement points CHi to CHBi + 1 even if a predetermined time elapses. In the case, it is determined as abnormal.
Further, when it is not determined that there is an abnormality, the abnormality determination ERR may be set and the process may be restarted from S400.

In step S1915, an accumulation time C7in that is n times the accumulation time C7i optimally controlled in accordance with the subject in step S1908 is set.
In step S1916, a gain G7im that is m times the gain G7i optimally controlled in accordance with the subject in step S1909 is set.
In step S1918, the average luminance B when the accumulation time is C7in and the gain is G7im is acquired as B14.
Next, in steps S1919 and S1920, it is determined whether or not the average luminance B14 is within the range of the signal levels CHBi to CHBi + 1 of the switching points CHi to CHi + 1 set in step S1903.

If average luminance B14 is within the range of signal level CHBi to CHBi + 1 at the switching point, it is determined that the input / output characteristic is H0i + 1, and the flow proceeds to step S1921.
In step S1921, the conversion process is performed using the conversion function for H0i + 1. This conversion process is a process similar to the conversion process in steps S1006, S1007, S1011 and S1012 of the third embodiment.
As a result, the converted input / output characteristics have a single continuous input / output characteristic with respect to the input, as in the third embodiment.
In step S1922, the converted luminance B14 is set to B14 ′.

The comparison of the average luminance in step S1924 is performed using the converted luminances B13 ′ and B14 ′.
As a result of the comparison of the average luminance in step S1924, when it is determined that the exposure is abnormal, the flow proceeds to step S1925, and the abnormality determination is terminated without performing the subsequent abnormality determination processing. However, at this time, the target value TGT for exposure control is reset to the original target value.
If it is determined that the exposure is normal, the process proceeds to S1926.

In step S1925, when it is determined that the exposure is abnormal, an abnormality is reported to the user.
Although a warning is issued in step S1925, the operation of the white line recognition system using such an imaging apparatus may be stopped at the same time.

On the other hand, if the average luminance B14 is outside the range of the signal levels CHBi to CHBi + 1 of the switching points CHi to CHi + 1 in steps S1919 and S1920, it is determined that the area is outside the abnormality determination section, and the process proceeds to S1923. In this case, if the average luminance 14 does not fall within the range of the replacement points CHi to CHBi + 1 even after a predetermined time has elapsed, it is determined that there is an abnormality.
Further, when it is not determined that there is an abnormality, the abnormality determination ERR may be set and the process may be restarted from S400.

In step S1926, it is determined whether i indicating the start switching point of the abnormality determination section is greater than n-1.
If it is larger, it is assumed that abnormality determination has been performed for all the characteristic sections, and the process proceeds to S1928. If it is smaller, i = i + 1 is set in order to determine abnormality in the next characteristic section, and the process returns to S1903.
In this way, the process of comparing the average luminances B13i and B14i is performed for all input / output characteristic sections.

  As described above, according to the imaging apparatus according to Embodiment 7 of the present invention, even when an imaging element having a plurality of switching points exists in the input / output characteristics, it is determined whether there is an abnormality in exposure control in the entire input / output characteristics. Can do. In addition, by switching the input / output characteristic interval for performing the abnormality determination in time, one abnormality determination operation can be shortened, and the time for stopping the system operation can be reduced.

  As for the abnormality determination, in S1911, S1912 or S1919, S1920, the luminance B is set as the determination target only in the set section, and for those outside the set section, the abnormality or ERR determination is made in S1717 or S1723. Similarly to the fourth and fifth embodiments, the determination of the section of the luminance B may be performed, the characteristic may be converted according to the section of the luminance B, and the determination may be made by comparison with the target value.

Embodiment 8 FIG.
In the imaging device according to Embodiment 8, abnormality determination is performed at once for a plurality of sections in the input / output characteristics of the imaging device. This is because the imaging apparatus according to the sixth embodiment performs an exposure control abnormality determination in one specific section of the input / output characteristics of the image sensor, or switches the determination target section in terms of time to change all of the input / output characteristics. In contrast to performing an exposure control abnormality determination for a range, an exposure control abnormality determination is performed at once for a plurality of sections of input / output characteristics of the image sensor.

Hereinafter, an exposure abnormality determination process in an imaging apparatus using an imaging element having a plurality of switching points and having nonlinear input / output characteristics will be described.
The configuration of the imaging apparatus according to Embodiment 8 of the present invention is in accordance with the configuration of the imaging apparatus according to Embodiment 1 (see FIG. 1).
The outline of the process of an example of the imaging apparatus including the process for determining the abnormality in exposure control according to the eighth embodiment of the present invention is in accordance with the flowchart (see FIG. 5A) of the imaging apparatus according to the first embodiment.
FIG. 17 is a flowchart showing processing details of exposure control abnormality determination applied to the imaging apparatus according to Embodiment 8 of the present invention.

In FIG. 17, in step S2001, i for setting the determination target section is initialized.
In step S2002, the accumulation time reference C9 is selected as the shortest accumulation time.
In step S2003, the gain reference G9 is selected to be the smallest value.
In step S2004, an accumulation time C2xi that is a target of abnormality determination is set. Here, C2xi is a value obtained by multiplying C2 by a preset coefficient xi when the abnormality determination sections CHi to CHi + 1 are targeted. (Xi may be a continuous real number.)
In S2005, the gain is set to G9.
Thereafter, the process returns to step S404, and the average luminance when the accumulation time is C9xi and the gain is G9 is acquired. Since the process proceeds from S402 to S2006 during the abnormality determination, it is determined whether or not B17i has been stored. If B17i has not been stored, in step S2007, the average brightness B (when the accumulation time is C9xi and the gain is G9). For example, the average luminance in the region R) is acquired as B17i.

In step S2009, B17i is compared with the signal levels CHB0 to CHBn of the switching points CH0 to CHn to determine which section the value of B17i is.
In S2004, since xi is set to a preset value, step S2009 for determining which section B17i is in may be limited to an expected range instead of the entire section.
After the section is determined, in step S2010, input / output characteristic conversion processing is performed using the conversion function H0i + 1 of the section determined at the determined first switching point CHi and second switching point CHi + 1.
If B17i is smaller than CHB0 or larger than CHBn, the input / output characteristic section of the image sensor is assumed to be off, and the process proceeds to S2019.
Further, when it is not determined that there is an abnormality, the abnormality determination ERR may be set and the process may be restarted from S400.
In step S2011, the converted luminance B17i is stored as B17i ′.

In step S2012, an accumulation time reference C9 is set.
In step S2013, a gain G9yi that is a target of abnormality determination is set. Here, G9yi is set to a value satisfying G9yi = (C9xi / C9) × G9.
Thereafter, the process returns to step S404, and the average luminance when the accumulation time is C9 and the gain is G9yi is acquired. Since the process proceeds from S402 to S2006 during the abnormality determination, it is determined whether or not B17i has been stored. If B17i has been stored, in step S2014, the average in image output when the accumulation time is C9 and the gain is G9yi The brightness B is set to B18i.

In step S2015, B17i is compared with the signal levels CHB0 to CHBn of the switching points CH0 to CHn, and it is determined in which section the value B18i is.
In S2004, in order to set xi to a preset value, step S2015 for determining which section B18i is in may be narrowed down to an expected range instead of the entire section as in S2009.
After the section is determined, in step S2016, input / output characteristic conversion processing is performed using the conversion function H0i + 1 of the section determined at the determined first switching point CHi and second switching point CHi + 1.
If B18i is smaller than CHB0 or larger than CHBn, the input / output characteristic section of the image sensor is assumed to be off, and the process proceeds to S2019.
Further, when it is not determined that there is an abnormality, the abnormality determination ERR may be set and the process may be restarted from S400.
In step S2017, the converted luminance B18i is set to B18i ′.

The comparison of the average luminance in step S2018 is performed using the converted luminances B17i ′ and B18i ′.
As a result of the comparison of the average luminance in step S2018, when it is determined that the exposure is abnormal, the flow proceeds to step S2019, and the abnormality determination is terminated without performing the subsequent abnormality determination processing.
If it is determined that the exposure is normal, the process proceeds to S2020.

In step S2019, when it is determined that the exposure is abnormal, the user is notified of the abnormality.
In step S2019, a warning is issued. However, the operation of the white line recognition system using such an imaging apparatus may be stopped.

In step S2020, it is determined whether i indicating the start switching point of the abnormality determination section is greater than n-1. (When xi is a continuous real number, it may be determined whether or not the determination of all the sections has been completed based on the value of the luminance B17i or 18i.)
If it is larger, it is assumed that abnormality determination has been performed for all characteristic sections, and the process proceeds to S2022. If it is smaller, i = i + 1 is set in order to determine abnormality in the next characteristic section, and the process returns to S2004.
Thus, the process of comparing the average luminances B17i ′ and B18i ′ is performed for all the input / output characteristic sections.

  As described above, according to the imaging apparatus according to Embodiment 8 of the present invention, even when an imaging element having a plurality of switching points exists in the input / output characteristics, it is determined whether there is an abnormality in exposure control in the entire input / output characteristics. Can do. In addition, by switching the input / output characteristic interval for performing the abnormality determination in time, one abnormality determination operation can be shortened, and the time for stopping the system operation can be reduced.

  As for the abnormality determination, in S1911, S1912 or S1919, S1920, the luminance B is set as the determination target only in the set section, and for those outside the set section, the abnormality or ERR determination is made in S1717 or S1723. Similarly to the fourth and fifth embodiments, the determination of the section of the luminance B may be performed, the characteristic may be converted according to the section of the luminance B, and the determination may be made by comparison with the target value.

Embodiment 9 FIG.
In the ninth embodiment, a description will be given of a mode of determining an abnormality in exposure control in an imaging apparatus in which the input / output characteristics of the imaging element are nonlinear input / output characteristics having no switching points.
FIG. 18 is a diagram illustrating input / output characteristics (signal level characteristics with respect to image plane illuminance) of an imaging element used in an imaging apparatus according to Embodiment 9 of the present invention.

As shown in FIG. 18, the characteristic E01 of the imaging device having non-linear input / output characteristics having no switching point includes the characteristic E02 of the imaging device having linear input / output characteristics, and the nonlinear input / output characteristics having the switching point. Compared with the characteristic E03 of the image pickup apparatus, it has a characteristic represented by a straight line. However, the horizontal axis (image plane illuminance) in FIG. 18 is logarithmic display.
It is assumed that the imaging element having non-linear input / output characteristics that does not have a switching point in this embodiment does not have an accumulation time adjustment function.

The ninth embodiment of the present invention will be described below.
FIG. 19 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 9 of the present invention.
FIG. 20 is a flowchart showing an overview of processing of an example of an imaging apparatus including processing for determining abnormality in exposure control according to Embodiment 9 of the present invention.

The configuration of the imaging apparatus according to Embodiment 9 of the present invention shown in FIG. 19 is obtained by removing the accumulation time control unit 204 and the exposure control unit 205 from the imaging apparatus according to Embodiment 1 shown in FIG.
The configuration is basically the same except that the accumulation time is not controlled.

In the gain control unit 204 in the analog signal processing unit 202, for example, the gain is controlled so that the average luminance in a predetermined area of the image data becomes a target value, and the result of performing the feedback control is reflected on the subject. Set the gain accordingly.
In the ninth embodiment, the feedback control unit (not shown) and the gain abnormality determination unit 210 are arranged in the CPU 208, and the gain setting necessary for controlling the gain control unit 204 is controlled by the CPU 208. The configuration can be transmitted to the unit 204.
The predetermined area for calculating the average brightness of the image data is an area surrounded by R shown in FIG. The average luminance in the R region may be calculated using either a video signal component after analog video signal processing or a digital video signal after A / D conversion.

  In the ninth embodiment, the gain abnormality determination unit 210 is provided in the CPU 208 subsequent to the A / D conversion to determine the gain abnormality control using the digital video signal. Gain abnormality determination may be performed using a video signal. In this case, the gain abnormality determination unit 210 may be disposed before the A / D conversion unit 206. Although the embodiment has been described here in which the CPU 208 performs various image processing, a block dedicated to image processing may be provided separately from the CPU 208.

In FIG. 20, in step S2301, a gain G8 optimally controlled according to the subject is set.
In step S2302, the average luminance B (for example, the average luminance in the region R) when the gain is G8 is acquired as B15.
In step S2303, the average luminance B15 is converted to B15 ′ using the conversion function for E01 of the input / output characteristics of the image sensor shown in FIG.
In step S2309, the converted luminance B15 is stored as B15 ′.

In step S2304, a gain G8y that is a target of abnormality determination is set.
In step S2305, the average luminance B when the gain is G8y is acquired as B16.
In step S2306, similar to step S2303, the average luminance B16 is converted to B16 ′ using the conversion function for E01.
The average luminance is converted in step S2303 and step S2306, for example, so that the characteristic E01 becomes a linear input / output characteristic with respect to the input.
In step S2310, the converted luminance B16 is set to B16 ′.

Here, for example, the input / output characteristics are image plane illuminance E [lx], A / D-converted digital video signal B [LSB] (luminances B15 and B16 are also specified in a predetermined region of the digital video signal in this embodiment. When the average value), the gain G8y, and the sensitivity S [LSB / lx · s] of the image sensor, the characteristic E01 is expressed by the following equation, for example.
B = K × log (E × S) × G8y (K is an arbitrary constant)
In the ninth embodiment, when the output after converting B is B ′, the conversion equation of the characteristic E01 is the following equation.
B ′ = (10 ^ (B / (G8y × K))
As a result, the input / output characteristics after conversion have a single input / output characteristic with respect to the input, as indicated by E02 in FIG.

Therefore, when determining the gain abnormality by comparing the average luminances B15 ′ and B16 ′ after conversion in S2307, for example, it is determined that there is an abnormality when the following equation is satisfied.
Here, the average luminances B15 ′ and B16 ′ are compared using the ratio, but the comparison method is not limited to the method based on the ratio, and even if the comparison is performed using the difference between the average luminances B15 and B16. Of course, the comparison may be made using other arithmetic expressions.
Db = B15 ′ / B16 ′> G / Gm × k
Or, Db = B16 ′ / B15 ′> Gm / G × k (k> 1)

In step S2308, a warning that the exposure is abnormal is issued to the user of the imaging apparatus.
Although a warning is issued in step S2308, the operation of the white line recognition system using such an imaging apparatus may be stopped at the same time.
In this embodiment, G8 is an optimum gain according to the subject, but an arbitrary gain may be set.

  As described above, according to the imaging device according to the ninth embodiment of the present invention, the exposure amount can be obtained in the imaging device that performs only gain control without controlling the accumulation time, as in the imaging devices in the first to seventh embodiments. In order to determine an abnormality in exposure by increasing or decreasing the value, it is possible to determine an abnormality in gain control without temporarily interrupting the exposure control performed according to the subject for photographing.

1 is a block diagram schematically showing the configuration of an imaging apparatus according to Embodiment 1 of the present invention. It is a figure which shows notionally the example of attachment to the vehicle of the imaging device which concerns on Embodiment 1 of this invention. It is a figure which shows the example of an image | video image | photographed with the imaging device which concerns on Embodiment 1 of this invention. FIG. 6 is a diagram illustrating an example of image output at an optimum shutter speed in the imaging apparatus, image output at the slowest shutter speed, and image output at the fastest shutter speed. It is a flowchart which shows the process including the process of the abnormality determination of exposure control in the imaging device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the process including the process of the abnormality determination of exposure control in the imaging device which concerns on Embodiment 1 of this invention. It is a figure which shows the input-output characteristic (characteristic of the signal level with respect to image surface illumination intensity) of the image pick-up element used for the imaging device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the content of the abnormality determination process of exposure control in the imaging device which concerns on Embodiment 2 of this invention. It is a figure which shows the input-output characteristic (signal level characteristic with respect to image surface illumination intensity) of the image pick-up element used for the imaging device which concerns on Embodiment 2 of this invention. It is the figure which represented the input-output characteristic (characteristic of the signal level with respect to image surface illumination intensity) of the image pick-up element used for the imaging device concerning Embodiment 3 of this invention to a logarithmic graph. It is a flowchart which shows the processing content of abnormality determination of exposure control in the imaging device which concerns on Embodiment 3 of this invention. It is a figure which shows the principle of the input / output characteristic conversion in the exposure abnormality determination which concerns on Embodiment 3 of this invention. It is a flowchart which shows the processing content of abnormality determination of exposure control in the imaging device which concerns on Embodiment 4 of this invention. It is a flowchart which shows the content of the abnormality determination process of the imaging device which concerns on Embodiment 5 of this invention. It is a figure showing the input-output characteristic (signal level characteristic with respect to image surface illumination intensity) of the image pick-up element used for the imaging device concerning Embodiment 6 of this invention. It is a flowchart which shows the first half part of the processing content of the abnormality determination of exposure control applied to the imaging device which concerns on Embodiment 6 of this invention. It is a flowchart which shows the processing content of the abnormality determination of exposure control applied to the imaging device which concerns on Embodiment 7 of this invention. It is a flowchart which shows the processing content of the abnormality determination of exposure control applied to the imaging device which concerns on Embodiment 8 of this invention. It is a figure showing the input-output characteristic (characteristic of the signal level with respect to image surface illumination intensity) of the image pick-up element used for the imaging device concerning Embodiment 9 of this invention. It is a block diagram which shows schematic structure of the imaging device which concerns on Embodiment 9 of this invention. It is a flowchart which shows roughly the processing content of the abnormality determination of exposure control applied to the imaging device which concerns on Embodiment 9 of this invention.

Explanation of symbols

  100, 100A imaging device, 201 photoelectric conversion unit, 202 analog signal processing unit, 203 gain control unit, 204 accumulation time control unit, 205 exposure control unit, 206 A / D conversion unit, 207 image memory, 208 CPU, 209 abnormality determination Part, 210 Gain abnormality determination part.

Claims (21)

A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit, and the photoelectric conversion unit has an output signal characteristic linear with respect to an incident light intensity, and the abnormality determination The unit outputs an image output obtained by an accumulation time C and a gain G with respect to variables m and n having a relationship of m × n = 1, and an image output obtained by an n-times accumulation time Cn and an m-times gain Gm. An imaging device characterized by determining an abnormality in exposure or gain by comparing.   The output signal characteristic of the photoelectric conversion unit further includes a non-linear section with respect to incident light intensity, and the abnormality determination unit converts the non-linear output signal characteristic section into a linear characteristic, and after the conversion For the variables m and n having the relationship of m × n = 1, the image output obtained by the accumulation time C and the gain G, and the image output obtained by the n times accumulation time Cn and the m times gain Gm The imaging apparatus according to claim 1, wherein an abnormality in exposure or gain is determined by comparing with. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has an output signal characteristic linear with respect to incident light intensity,
The abnormality determination unit is configured so that the signal processing unit applies the image output obtained by the accumulation time C and the gain G appropriately controlled according to the subject by the signal processing unit and the gain is fixed to a predetermined value to the subject by the signal processing unit. An image output obtained by an appropriately controlled accumulation time, or an image output obtained by a gain appropriately controlled by the signal processing unit in a state where the accumulation time is fixed to a predetermined value; An imaging device characterized by determining an abnormality in exposure or gain by comparing. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has an output signal characteristic linear with respect to incident light intensity,
In the state where the abnormality determination unit fixes the gain to a predetermined value, the image output obtained by the accumulation time appropriately controlled according to the subject by the signal processing unit, and the accumulation time fixed to a predetermined value, An imaging apparatus, wherein an abnormality in exposure or gain is determined by comparing an image output obtained with a gain appropriately controlled according to a subject by the signal processing unit.   The output signal characteristic of the photoelectric conversion unit further includes a non-linear section with respect to the incident light intensity, and the abnormality determination unit converts the non-linear output signal characteristic section into a linear characteristic, The signal processing unit appropriately controls the image output obtained by the accumulation time C and the gain G appropriately controlled according to the subject by the signal processing unit and the gain is fixed to the predetermined value by the signal processing unit. By comparing the image output obtained by the accumulated accumulation time, or the image output obtained by the gain appropriately controlled according to the subject by the signal processing unit with the accumulation time fixed to a predetermined value. The imaging apparatus according to claim 3, wherein an abnormality in exposure or gain is determined.   The output signal characteristic of the photoelectric conversion unit further includes a non-linear section with respect to the incident light intensity, and the abnormality determination unit converts the non-linear output signal characteristic section into a linear characteristic, The image output obtained by the accumulation time appropriately controlled according to the subject by the signal processing unit with the gain fixed to a predetermined value, and the signal processing unit with the accumulation time fixed to the predetermined value. The imaging apparatus according to claim 4, wherein an abnormality in exposure or gain is determined by comparing an image output obtained with a gain controlled appropriately according to a subject. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit, and the photoelectric conversion unit has an output signal characteristic that is nonlinear with respect to incident light intensity, and the abnormality determination The unit converts the nonlinear output signal characteristic into a linear characteristic, and obtains the variable m and n having a relationship of m × n = 1 by the accumulation time C and the gain G in the characteristic after the conversion. An image pickup apparatus that determines an abnormality in exposure or gain by comparing an image output to be obtained and an image output obtained by an n times accumulation time Cn and an m times gain Gm. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has an output signal characteristic that is nonlinear with respect to incident light intensity,
The abnormality determining unit converts the non-linear output signal characteristic section into a linear characteristic, and the obtained characteristic is obtained by the accumulation time C and the gain G appropriately controlled according to the subject by the signal processing unit. With the image output and the gain fixed to a predetermined value, the image output obtained by the accumulation time appropriately controlled according to the subject by the signal processing unit, or the accumulation time fixed to a predetermined value, An imaging apparatus characterized by determining an exposure or gain abnormality by comparing an image output obtained by a gain appropriately controlled according to a subject by a signal processing unit. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating charges in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has an output signal characteristic that is nonlinear with respect to incident light intensity,
The abnormality determining unit converts the nonlinear output signal characteristic section into a linear characteristic, and is appropriately controlled according to the subject by the signal processing unit in a state where the gain is fixed to a predetermined value in the converted characteristic. By comparing the image output obtained by the accumulation time with the image output obtained by the gain appropriately controlled according to the subject by the signal processing unit in a state where the accumulation time is fixed to a predetermined value, exposure or An imaging apparatus characterized by determining an abnormality in gain. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has input / output characteristics divided into a plurality of linear sections by a plurality of switching points,
The abnormality determination unit converts any section of the linear output signal characteristic into a standard linear characteristic, and accumulates variables m and n having a relationship of m × n = 1 in the characteristic after conversion. An image pickup characterized by determining an exposure or gain abnormality by comparing an image output obtained by time C and gain G with an image output obtained by n times accumulation time Cn and m times gain Gm. apparatus.   The abnormality determination unit determines an exposure or gain abnormality in a standard linear section that does not require characteristic conversion of input / output characteristics divided into a plurality of linear sections by a plurality of switching points. The imaging device according to claim 10.   The imaging apparatus according to claim 10, wherein the abnormality determination unit determines an abnormality in exposure or gain for all sections of input / output characteristics divided into a plurality of linear sections by a plurality of switching points. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit for determining an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has input / output characteristics divided into a plurality of linear sections by a plurality of switching points,
The abnormality determination unit converts the non-linear nonlinear output signal characteristic section into a standard linear characteristic, and in the converted characteristic, the accumulation time C and the gain appropriately controlled according to the subject by the signal processing unit With the image output obtained by G and the gain fixed to a predetermined value, the image output obtained by the accumulation time appropriately controlled according to the subject by the signal processing unit, or the accumulation time is fixed to the predetermined value An imaging apparatus characterized by determining an exposure or gain abnormality by comparing an image output obtained by a gain appropriately controlled according to a subject by the signal processing unit in a state. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating charges in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has input / output characteristics divided into a plurality of linear sections by a plurality of switching points,
The abnormality determining unit converts the non-linear nonlinear output signal characteristic section into a standard linear characteristic, and in the characteristic after conversion, the gain is fixed to a predetermined value according to the subject by the signal processing unit. Comparing the image output obtained by appropriately controlled accumulation time with the image output obtained by appropriately controlled gain according to the subject by the signal processing unit with the accumulation time fixed at a predetermined value. To determine an abnormality in exposure or gain. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an exposure or gain abnormality of the image output processed by the signal processing unit, and the photoelectric conversion unit has input / output characteristics that are divided into a plurality of nonlinear sections by a plurality of switching points. The abnormality determination unit logarithmically transforms any section of the nonlinear output signal characteristic into a linear characteristic, and a variable m having a relationship of m × n = 1 in the section after conversion, By comparing the image output obtained by the accumulation time C and the gain G with respect to n and the image output obtained by the n-times accumulation time Cn and the m-times gain Gm, an abnormality in exposure or gain is determined. An imaging apparatus characterized by that.   16. The imaging apparatus according to claim 15, wherein the abnormality determination unit determines an abnormality in exposure or gain for all sections of input / output characteristics divided into a plurality of nonlinear sections by a plurality of switching points. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has input / output characteristics divided into a plurality of nonlinear sections by a plurality of switching points,
The abnormality determination unit converts the non-linear nonlinear output signal characteristic section into a standard linear characteristic, and in the converted characteristic, the accumulation time C and the gain appropriately controlled according to the subject by the signal processing unit With the image output obtained by G and the gain fixed to a predetermined value, the image output obtained by the accumulation time appropriately controlled according to the subject by the signal processing unit, or the accumulation time is fixed to the predetermined value An imaging apparatus characterized by determining an exposure or gain abnormality by comparing an image output obtained by a gain appropriately controlled according to a subject by the signal processing unit in a state. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit that performs control processing on a time for accumulating electric charge in the photoelectric conversion unit and a gain applied to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an exposure or gain abnormality of the image output processed by the signal processing unit, and converts the nonlinear non-linear output signal characteristic section into a standard linear characteristic. ,
An image output obtained by an accumulation time appropriately controlled according to the subject by the signal processing unit with the gain fixed at a predetermined value, and an object output by the signal processing unit with the accumulation time fixed at a predetermined value. An imaging apparatus characterized by determining an abnormality in exposure or gain by comparing with an image output obtained with a gain controlled appropriately according to the above. A photoelectric conversion unit that photoelectrically converts light received from the subject;
A signal processing unit for controlling and processing a gain to be given to the output of the photoelectric conversion unit;
An abnormality determination unit that determines an abnormality in exposure or gain of the image output processed by the signal processing unit,
The photoelectric conversion unit has an output characteristic signal that is nonlinear with respect to incident light intensity,
The abnormality determination unit determines an abnormality in exposure or gain by comparing an image output obtained by a gain G and an image output obtained by a gain Gm that is m times the gain G. .   The imaging device is mounted on a vehicle and used for road lane recognition on the road surface, and the abnormality determination unit is configured to turn on the imaging device when the vehicle enters the accessory mode, the ignition is turned on, or the imaging device is turned on. The imaging apparatus according to any one of claims 1 to 19, wherein an abnormality determination process is performed immediately after being performed.   2. The imaging device according to claim 1, wherein the imaging device is mounted on a vehicle and used for recognition of a traveling lane on a road surface, and the abnormality determination unit executes an abnormality determination process when the speed of the vehicle is equal to or lower than a predetermined value. The imaging device according to claim 19.

Images