JP6238576B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  In a conventional imaging device, an optical image formed by a photographing lens is converted into an electrical signal by an imaging device and output. The image signal obtained from the image sensor is subjected to AD conversion and then subjected to signal processing such as clamp processing. In order to correct the black level (pedestal level) of the image signal after AD conversion, the image signal is composed of an optical black region (generally, a photodiode shielded with metal, hereinafter referred to as OB). The difference from the output signal of the (region) is taken. By such an optical black clamp process (hereinafter referred to as OB clamp process) of the imaging signal, an imaging signal with a stable black level can be obtained even when the black level of the imaging signal varies due to dark current noise or the like. .

  FIG. 7B is a diagram illustrating the configuration of the image sensor and the clamp signal. An imaging element 301 formed of a phototransistor includes an invalid area 302 that is not electrically guaranteed, an optical black area (OB area) 303, and an effective pixel area 304 that converts an optical image formed by a lens into an imaging signal and outputs the imaging signal. It consists of FIG. 8 is a diagram showing a general OB clamp circuit. An imaging signal subjected to AD conversion is input to the clamp circuit 501. The filter unit 502 selectively receives an output signal (OB signal) of the OB region from among the imaging signals, and performs filter processing to remove noise. In the subtractor 503, the difference between the image pickup signal and the reference signal (clamp signal) after the filter processing on the OB signal by the filter unit 502 is taken, and the OB clamp processing is performed. The OB signal output from the OB region 303 in FIG. 7B is input to the filter unit 502 of the clamp circuit 501 shown in FIG. 8, and high frequency components are removed.

  FIG. 7A shows the filter output characteristics when the dark current suddenly increases from a certain frame due to an increase in the amplification factor of the amplifier and an OB signal having a high signal level is input to the filter circuit 502. The clamp signal 305 output from the filter circuit 502 converges in a time corresponding to the time constant of the filter circuit 502. Here, convergence is shown at the timing indicated by the arrow 306. When the dark current suddenly decreases from a certain frame due to a decrease in the amplification factor of the amplifier or the like, the black level after clamping is once shifted to minus and then converges to zero according to the convergence time of the filter.

  As described above, the filter unit 502 of the clamp circuit 501 is configured by a filter having a certain time constant. When the OB signal input to the filter unit 502 fluctuates, the filter unit 502 outputs a clamp signal at a target level. Requires time corresponding to the time constant characteristics of the filter. Thus, a technique for switching a clamp signal to an imaging signal according to the changed OB level when the OB level changes rapidly due to switching of the shooting mode or the like is disclosed (Patent Document 1). That is, the clamp signal level assumed in each photographing mode is held in advance in a memory table or the like in order to cause the clamp process to rapidly follow the rapid change in the OB level. Then, the image pickup signal is clamped with a clamp signal level that is suitable when the shooting mode is switched.

JP 2009-77184 A

  However, since the technique disclosed in Patent Document 1 employs a clamp signal level that is assumed in advance, an unexpected OB level may be output depending on photographing conditions such as exposure time and temperature. In addition, with this technology, it is impossible to follow different dark shading characteristics or the like for each image sensor, and appropriate OB clamping processing cannot be performed. Furthermore, a circuit configuration including a memory table for holding a plurality of clamp signal levels is required.

  As described above, the position indicated by the arrow 306 in FIG. 7 indicates the position where the black level is converged by the clamping process. That is, it indicates that a predetermined time is required for convergence of the black level, and the convergence of the black level of a part of the line located at the upper part in the effective area 304 of the image sensor 301 is not in time. Therefore, as shown in FIG. 9, the upper part of the captured image of the subject cannot be clamped at an appropriate clamp level, resulting in unevenness.

  An object of the present invention is to provide an imaging device capable of clamping an imaging signal at an appropriate clamp level even in a situation where the black level of the imaging signal output from the imaging device fluctuates rapidly. .

  In order to achieve the above object, an imaging apparatus of the present invention has an effective pixel area and an optical black area, outputs an electrical signal, and performs predetermined filtering on the output signal of the optical black area. Filter processing means for generating a clamp signal by performing, clamping means for clamping the output signal of the effective pixel area based on the clamp signal, and a threshold value preset in the output signal of the optical black area Control means for controlling so that a signal of a level exceeding the level is not input to the filter processing means, and setting means for setting the threshold and the time constant of the filter processing according to the photographing conditions. .

  According to the present invention, it is possible to perform an appropriate clamping process even when the photographing condition changes significantly.

1 is a block diagram illustrating a configuration of an imaging apparatus according to Embodiment 1. FIG. 1 is a schematic configuration diagram of a clamp processing circuit according to Embodiment 1. FIG. FIG. 3 is a block diagram of a scratch processing unit according to the first embodiment. 3 is a flowchart illustrating a clamping process according to the first embodiment. It is a figure explaining the signal input into a flaw correction threshold value and a clamp circuit. 10 is a flowchart illustrating a clamping process according to the second embodiment. It is a figure which shows the structural example and clamp signal of an image pick-up element. It is a figure which shows the schematic structural example of a clamp circuit. It is a figure which shows an example of the picked-up image in which a clamp process is not appropriate.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Example 1
FIG. 1 is a schematic configuration block diagram of an imaging apparatus according to Embodiment 1 of the present invention. In FIG. 1, an optical lens 201 forms an optical image of a subject on an image sensor 203. The lens driving unit 211 is configured by a DC motor or the like and drives the optical lens 201. The diaphragm mechanism 202 adjusts the amount of light that passes through the lens and enters the image sensor 203, and is driven and controlled by the diaphragm driver 210. An image sensor 203 such as a CMOS sensor or a CCD sensor converts a subject image formed by an optical lens into an electrical signal and outputs the electrical signal. In this embodiment, four color filters (R, Gr, Gb, B) in a Bayer array are arranged on the phototransistor of the image sensor 203. The configuration of the image sensor 203 is the same as that shown in FIG.

  The amplifier 204 amplifies the electrical signal output from the image sensor 203. The AD conversion unit 205 converts the signal amplified by the amplifier 204 from analog to digital. The clamp processing unit 206 outputs the output signal of the optical black region (OB region) and the image signal output from the effective pixel region in order to adjust the black level (pedestal level) of the image signal after AD conversion processing to the target level. OB clamp processing is performed to take the difference between the two. Even when the black level of the imaging signal fluctuates due to dark current noise or the like by the OB clamping process, an imaging signal with a stable black level can be obtained.

  The video signal processing unit 207 generates a predetermined video signal by performing signal processing such as development processing, white balance processing, and noise reduction processing. The video signal output unit 208 converts the output from the video signal processing unit 207 into a video signal of a predetermined format such as a video signal and outputs the video signal to the outside. The arithmetic control unit 209 is a central processing unit such as a microcomputer, calculates a luminance evaluation value from the video signal from the video signal processing unit 207 by calculation, and adjusts the lens driving unit 210 and the aperture so as to obtain an appropriate exposure level. A drive signal is transmitted to the drive unit 210.

  Further, the arithmetic control unit 209 transmits a control signal to the image sensor 210 so as to have a predetermined exposure time, and transmits a control signal to the amplifier 204 so as to adopt a predetermined amplification factor. Further, a control signal for setting the time constant of the internal filter circuit to a predetermined value and position information of the OB area in the image sensor 203 are transmitted to the clamp processing unit 206. The arithmetic control unit 209 receives the video signal from the video signal processing unit 209, performs a calculation to detect a moving object by tracking the difference between the frames of the video, or to track the face, and to store information on the video signal. To superimpose.

  The timing generator (TG) 212 is a module that generates a reference clock for driving each unit of the imaging apparatus, and generates a vertical synchronization signal, a horizontal synchronization signal, and a pixel-unit drive signal (pixel clock) for each module. Supply. The clamp processing unit 206 performs OB clamp processing based on the output signal of the OB area of the image sensor in accordance with the timing signal from the TG 212 and the information from the arithmetic processing unit 209. The temperature sensor 213 is a thermistor disposed in the vicinity of the image sensor 203 and measures the temperature of the image sensor.

  FIG. 2 is a detailed view of the clamp processing unit 206 of FIG. An AD-converted image signal is input to the clamp processing unit 206 in units of pixels for each readout line. Here, before the output signal of the OB area 303 subjected to AD conversion is input to the filter circuit 102, it is input to the scratch processing unit 101. The scratch processing unit 101 detects a scratch pixel (defective pixel) existing in the OB area so that the output signal of the pixel is not input to the filter circuit 102. The filter unit 102 receives an output signal from a pixel other than a flawed pixel in the OB area of the image sensor 203, and performs a filtering process to remove noise. The signal in the OB region filtered by the filter unit 102 is input to the subtractor 103 as a reference signal (clamp signal), and the subtracter 103 obtains a difference from the image pickup signal, thereby OB clamping based on the clamp signal. Processing is performed.

The scratch processing unit 101 compares the threshold Th for scratch identification specified by the scratch processing control unit 105 in the arithmetic control unit 209 with the output signal S of the OB area 303 for each pixel. When the following expression (1) is satisfied, it is determined that the output signal is a defect (defective) pixel due to a pixel defect or the like existing in the OB region, and the output signal S from the corresponding pixel in the OB region 303 is Control is performed so as not to input to the filter circuit 102.
Th <S (1)

  The scratch processing unit 101 operates with parameters according to a control signal from the scratch processing control unit 105 located inside the arithmetic processing unit 104, and is synchronized with the drive timing from the TG 212. The scratch processing control unit 105 is located inside the calculation control unit 104, but may be configured independently of the calculation control unit 104. Although omitted in FIG. 2, the driving timing inside the clamp processing unit 206 is synchronized with the pixel clock, vertical synchronizing signal, and horizontal synchronizing signal supplied from the TG 212 in FIG.

The filter unit 102 has a primary IIR type LPF configuration. When the time constant of the IIR filter is M, the input of the filter is Xn, and the output is Yn, the filter characteristic is expressed by the following equation (2).
Yn = (1 / M) * {M * Xn + (1-M) * Yn-1} (2)

  Here, when a defective pixel due to a pixel defect or the like exists in the OB region 303 and the output signal is input to the filter circuit 102, the filter output Yn obtained by the expression (2) is a value deviated from the original clamp level. . If the time constant M is increased, the influence of the output signal of the scratch pixel on the filter output is reduced, but the output response characteristic with respect to the input Xn is delayed. In order to speed up the output response characteristics with respect to the input Xn, the time constant M can be decreased to easily follow the fluctuation of the OB level. However, the filter output Yn greatly fluctuates due to the influence of the scratch pixel, and the line Phenomena such as streaking in the direction may occur. Therefore, when the scratch processing unit 101 determines that the output signal is a scratch pixel, the signal can be clamped at an appropriate clamp level by controlling so that the signal is not input to the filter circuit 102.

  FIG. 3 is a diagram showing details of the scratch processing unit 101 in FIG. The scratch determination unit 902 receives the output signal S from the OB area 303. The scratch determination unit 902 compares the output signal S of the OB area with the threshold value Th stored in the scratch threshold storage unit 901 for each pixel. When Expression (1) is satisfied, the switch 903 is opened so that the output signal of the pixel is not input to the subsequent filter circuit 102. Further, when the expression (1) is not satisfied, it is determined that the output signal S of the corresponding OB area is a normal pixel output, and the switch 903 is closed so that the output signal of the pixel is sent to the filter circuit 102 in the subsequent stage. Control to input.

FIG. 4 is a flowchart showing a clamping process when the amplification factor set in the amplifier 204 is changed as an example of the imaging condition. First, the amplification factor of the amplifier 204 in the next frame is confirmed based on the level of the image signal up to the current frame (S701). It is determined whether or not the amplification factor setting of the amplifier 204 has been changed in the next frame compared to the current frame (S702). When it is determined that the setting of the amplification factor of the amplifier 204 in the next frame is not different from that of the current frame, the filter processing time constant (filter time constant) is set to Mnormal (S703), and the threshold for scratch identification is set to Normal. The setting is made (S704). If it is determined that the amplifier 204 has an amplification factor larger than that of the current frame in the next frame, the filter time constant is set to Mfast (S705), and the threshold for scratch identification is set to Thfast (S706). The filter time constant Mfast and the threshold value Thfast have the following relationship with the filter time constant Mnormal and the threshold value Normal set at the normal time.
Mnormal> Mfast (3)
Normal> Thfast (4)

  In other words, a filter time constant smaller than the normal time is set as shown in Expression (3) in order to respond quickly to changes in the black level. Further, by lowering the threshold for scratch identification from the normal level, even if there is a scratch pixel at a level that is not so conspicuous even if it is amplified with the normal gain setting, the scratch determination unit 902 ensures that the scratch pixel is a scratch pixel. Can be determined. Since the output signal of the pixel is controlled not to be input to the filter circuit 102, the output Yn of the filter circuit 102 greatly fluctuates, and a phenomenon such as a streak in the line direction does not occur. With the above processing, even when the amplification factor of the amplifier changes significantly, it is possible to perform appropriate clamping processing without delaying the black level convergence.

  Here, FIG. 5 is a diagram showing the relationship between the threshold Th for scratch identification and the histogram of the output signal of the OB area 303. A graph 601 is a histogram of the output signal of the OB region 303 when the normal threshold value Normal is used. A distribution having a certain spread with respect to the clamp level 604 at the normal time is shown. On the other hand, the graph 602 shows a histogram of the output signal of the OB area 303 when the threshold Th for scratch identification is set to Thfast lower than normal. When the threshold Th is at a level lower than normal, the number of pixels that satisfy the condition of Expression (1) increases. The clamp level 603 is lower than the clamp level 604 when the normal threshold Th is used. For this reason, the black level after the clamping process becomes higher than the original level, and the black level of the entire screen is raised.

  Therefore, in this embodiment, not only the threshold Th for identifying scratches is set low, but also the filter time constant M is set to Mfast that is smaller than the normal time in order to respond to the change in black level at high speed. By reducing the filter time constant in this way, the level fluctuation of the image signal after the amplification factor of the amplifier is changed converges within one frame after the change of the amplification factor. Even when the black level is raised by lowering the threshold value for identifying scratches, it is possible to perform appropriate clamping processing without delaying convergence. When the amplification factor of the amplifier 204 decreases, the threshold Th for identifying scratches may be set larger than usual, and the filter time constant M may be set larger than usual. .

(Example 2)
In the above-described first embodiment, the clamp processing when the amplification factor set in the amplifier is changed is described as an example in which the photographing condition in which the black level of the image is significantly changed is described. In the second embodiment, a clamp process when the exposure time is changed as an imaging condition will be described.

FIG. 6 is a flowchart showing the clamping process when the exposure time is changed. First, when the exposure time of a certain frame is T1 and the exposure time set after F frames from that frame is changed to T2, the change rate of the exposure time is obtained from the equation (5) (S801).
Rate of change = (| T1-T2 |) / F (5)

Next, it is determined whether or not the rate of change obtained by Expression (5) satisfies Expression (6) (S802). If it is determined that Expression (6) is not satisfied, the filter time constant is set to Normal ( S803), a threshold for scratch identification is set to Normal (S804).
Rate of change> X (6)

  If it is determined that the change rate of the exposure time satisfies Expression (6), the filter time constant is set to Mfast, and the threshold for scratch identification is set to Thfast. Note that the filter time constant Mfast and the threshold value Thfast have the same relationship as the expressions (3) and (4) described in the first embodiment with respect to the filter time constant Mnormal and the threshold value Normal set in the normal time. The value of the change rate X of the exposure time is set to an optimum value for each system.

  With the above processing, even when the exposure time changes significantly within a predetermined time, it is possible to perform appropriate clamping processing without delaying convergence. In other words, a filter time constant smaller than the normal time is set as shown in Expression (3) in order to respond quickly to changes in the black level. Further, assuming that the exposure time becomes longer, the threshold value for identifying scratches is lowered below the normal level, so that even if scratch pixels at a level that is not so noticeable in the normal exposure time are in the OB area, the scratch determination unit At 902, it can be determined as a defective pixel. Since the output signal of the pixel is controlled so as not to be input to the filter circuit 102, the output Yn of the filter circuit 102 greatly fluctuates, and a phenomenon such as a streak in the line direction does not occur.

  Also in this embodiment, not only the threshold Th for identifying scratches is set low, but also the filter time constant M is set to Mfast that is smaller than the normal time in order to respond to the black level change at high speed. By reducing the filter time constant in this way, the level fluctuation of the image signal after the exposure time is changed converges within one frame after the exposure time is changed. Even when the black level is raised by lowering the threshold value for identifying scratches as shown in FIG. 5, it is possible to perform an appropriate clamping process without delaying convergence.

  When the exposure time is shortened, the threshold Th for identifying scratches may be set larger than usual, and the filter time constant M may be set larger than usual. Further, the setting of the threshold value Th and the filter time constant M for identifying scratches according to the amplification factor set in the amplifier described in the first embodiment and the control according to the exposure time described in the second embodiment are combined. Control may be performed so as to set the threshold Th and the filter time constant M.

(Example 3)
In the third embodiment, a clamp process when the temperature of the image sensor changes will be described as an example of a shooting environment. When the temperature T of the image sensor 203 measured by the temperature sensor 213 increases, dark current noise and the like increase, so the average value of the level S of the output signal of the OB unit 303 increases. For this reason, the number of pixels that satisfy the condition of Expression (1) increases, and it may not be possible to clamp normally. Here, as described in the first and second embodiments, the threshold Thnormal and threshold Thfast set according to the amplification factor of the amplifier and the change rate of the exposure time are set according to the temperature T of the image sensor. It changes with correlation with temperature T as shown in the following formula | equation (8) and (9).
T ∝ Normal (8)
T∝Thfast (9)

  In addition to the control for setting the threshold Thnormal and threshold Thfast for identifying scratches according to the amplification factor of the amplifier and the change rate of the exposure time described in the first and second embodiments, Expressions (8) and (9) are shown. Control to be correlated. That is, control is performed such that the threshold Thnormal and the threshold Thfast for scratch identification are proportional to the temperature T of the image sensor measured by the temperature sensor 213, and as the temperature T increases, the values of the threshold Normal and the threshold Thfast are increased. . By such control, even when the temperature of the image sensor changes, appropriate clamping processing can always be performed.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 Scratch Processing Unit 102 Filter Unit 103 Subtraction Unit 105 Scratch Processing Control Unit 206 Clamp Circuit 209 Operation Control Unit

Claims (12)

An image sensor that has an effective pixel area and an optical black area and outputs an electrical signal;
Filter processing means for generating a clamp signal by performing predetermined filter processing on the output signal of the optical black region;
Clamping means for clamping the output signal of the effective pixel region based on the clamping signal;
Control means for controlling so that a signal having a level exceeding a preset threshold value among the output signals of the optical black region is not input to the filter processing means;
Setting means for setting the threshold value and the time constant of the filtering process according to the shooting conditions;
An imaging device comprising:   Furthermore, it has an amplifying means for amplifying the electric signal output from the image sensor, and the setting means sets the threshold and the time constant of the filtering process according to the amplification factor set in the amplifying means. The imaging apparatus according to claim 1.   3. The imaging apparatus according to claim 2, wherein the setting unit reduces the threshold and the time constant of the filter processing unit when the amplification factor set in the amplification unit changes to a large value. .   The imaging apparatus according to claim 3, wherein the setting unit increases the threshold and the time constant of the filter processing unit when the amplification factor set in the amplification unit changes to a small value. .   5. The imaging apparatus according to claim 1, wherein the setting unit sets the threshold value and a time constant of the filtering process according to an exposure time of the imaging element.   The imaging apparatus according to claim 5, wherein the setting unit decreases the threshold and the time constant of the filter processing unit when the exposure time of the imaging device changes.   7. The setting unit according to claim 6, wherein when the change rate of the exposure time of the image sensor changes more than a predetermined amount, the setting unit decreases the threshold value and the time constant of the filter processing unit, respectively. Imaging device.   The image pickup apparatus according to claim 7, wherein the setting unit increases the threshold value and the time constant of the filter processing unit when the exposure time of the image sensor changes short.   The imaging apparatus according to claim 1, wherein the setting unit sets the threshold value and a time constant of the filtering process according to a shooting condition and a shooting environment.   The imaging apparatus according to claim 9, wherein the imaging environment includes a temperature of the imaging element.   The imaging apparatus according to claim 10, wherein the setting unit sets the threshold value and a time constant of the filtering process in proportion to a temperature of the imaging element. A method for controlling an imaging apparatus having an effective pixel area and an optical black area and having an imaging element that outputs an electrical signal,
A filtering step for generating a clamp signal by performing a predetermined filtering process on the output signal of the optical black region;
A clamping step of clamping the output signal of the effective pixel region based on the clamping signal;
A control step of controlling so as not to be filtered level signal advance beyond the set are threshold Te in the filtering process step among the output signal of the optical black region,
A control method for an imaging apparatus, comprising: a setting step for setting the threshold value and a time constant of the filtering process according to a shooting condition.

Images