JP4720118B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus including an imaging sensor that generates an electrical signal corresponding to an incident light quantity, and in particular, as a photoelectric conversion characteristic, a linear characteristic in which the electrical signal is linearly converted with respect to the incident light quantity and output. An image sensor having a region and a logarithmic characteristic region in which the electrical signal is logarithmically converted with respect to the incident light amount (the operation of the linear characteristic and the operation of the logarithmic characteristic can be switched) The present invention relates to an imaging apparatus using an imaging sensor that exhibits photoelectric conversion characteristics having two different characteristic areas.

  Conventionally, by adding a logarithmic conversion circuit equipped with a MOSFET or the like to a solid-state imaging device in which photoelectric conversion elements such as photodiodes are arranged in a matrix, and utilizing the subthreshold characteristics of the MOSFET, the solid-state imaging device 2. Description of the Related Art An imaging sensor (also referred to as “LOG sensor”) is known in which an electrical signal is logarithmically converted with respect to an incident light amount as an output characteristic. In such an imaging sensor, it is possible to switch between the linear operation state in which an electrical signal is linearly converted and output according to the original output characteristics of the solid-state imaging device, that is, the incident light quantity, and the logarithmic operation state described above. An image sensor is known.

  For example, Patent Document 1 discloses an imaging apparatus that can automatically switch from a linear operation state to a logarithmic operation state by applying a specific reset voltage to a MOSFET, and that makes the switching points of the pixels equal. Yes. Also, Patent Document 2 discloses an imaging apparatus that can automatically switch from a linear operation state to a logarithmic operation state and that can adjust the potential state of the MOSFET by adjusting the reset time of the MOSFET.

By the way, when the imaging sensor is used in a linear operation state, an output proportional to the amount of electric charge generated by the photoelectric conversion element can be obtained, so that even a low-luminance subject has a high contrast (high gradation) image signal. However, there is an inconvenience that the dynamic range becomes narrow. On the other hand, when the image sensor is used in a logarithmic operation state, an output that is naturally logarithmically converted with respect to the amount of incident light can be obtained. Therefore, there is an inconvenience that the contrast is deteriorated.
JP 2002-77733 A JP 2002-300476 A

  The imaging devices according to Patent Literature 1 and Patent Literature 2 merely disclose that the imaging sensor can be automatically switched from the linear operation state to the logarithmic operation state. However, in view of the advantages and disadvantages of the above-described linear operation state and logarithmic operation state, the image capturing operation is performed not only by automatic switching but also by actively utilizing the advantages of each operation state. Is desirable. For example, even in the case of performing automatic exposure control, if the control is performed by associating the luminance of the target subject with the switching point from the linear operation state to the logarithmic operation state of the image sensor, each operation state has the advantages. There is a possibility that the optimum automatic exposure control using the can be performed.

  Therefore, according to the present invention, exposure control of the imaging device is performed in association with the photoelectric conversion characteristics of the imaging sensor included in the imaging device, so that when moving image shooting is performed, the exposure is controlled according to the amount of light from the subject (subject brightness). Even if the subject brightness changes suddenly or frequently with an optimal exposure state and a predetermined dynamic range, a sudden (frequent) exposure change corresponding to this is avoided. Thus, an object of the present invention is to provide an imaging apparatus capable of smoothly controlling a photographing exposure state and capable of performing more optimal exposure control.

An imaging apparatus according to a first aspect of the present invention generates an electrical signal corresponding to the amount of incident light, and the photoelectric conversion characteristic of the imaging device is output after the electrical signal is converted into the first characteristic with respect to the amount of incident light. An imaging sensor comprising: one characteristic region; and a second characteristic region in which the electrical signal is converted into a second characteristic whose output change with respect to the incident light amount is smaller than the first characteristic with respect to the incident light amount; A moving image photographing means for photographing a moving image composed of a plurality of frame images; an exposure evaluation value detecting means for detecting an exposure evaluation value when photographing a subject based on luminance information of the subject obtained from the frame image; determine the exposure amount setting luminance as a subject luminance for exposure setting based on the exposure evaluation value detected by the exposure evaluation value detecting means, the image sensor with respect to the determined exposure amount setting luminance Output level calculates the control value for controlling the amount of exposure to a predetermined output level, based on the control value for controlling the exposure amount, the exposure amount control means for controlling the exposure amount for the image sensor And determining the dynamic range setting luminance as the subject luminance for exposure setting based on the exposure evaluation value detected by the exposure evaluation value detecting means, and the output level of the imaging sensor with respect to the determined dynamic range setting luminance is A dynamic range control means for calculating a control value for controlling the dynamic range so as to achieve a predetermined output level, and controlling the photoelectric conversion characteristics of the image sensor based on the control value for controlling the dynamic range ; control value Tokoro for controlling the exposure amount calculated by the exposure control unit based on the exposure evaluation value When reaching the level, the control value for controlling the exposure amount perform stepwise alters change control by a predetermined change amount, or calculated by the dynamic range control means based on the exposure evaluation value the Change control means for performing change control for changing the control value for controlling the dynamic range step by step by a predetermined change amount when the control value for controlling the dynamic range reaches a predetermined level. And

  According to the above configuration, moving image shooting is performed by the moving image shooting unit, and an exposure evaluation value is detected by the exposure evaluation value detection unit based on the luminance information of the subject obtained from the frame image constituting the moving image. Using the evaluation value, exposure control is performed by performing exposure amount control by the exposure amount control unit and dynamic range control by the dynamic range control unit based on the photoelectric conversion characteristics. When the predetermined control value for controlling the exposure amount or the dynamic range calculated by the exposure control unit based on the exposure evaluation value reaches a predetermined level, that is, when the change control unit reaches a predetermined level, or the predetermined level Change control is performed to change the control value step by step by a predetermined change amount.

  As described above, the exposure amount control by the exposure amount control unit and the dynamic range control by the dynamic range control unit can be performed in association with the photoelectric conversion characteristics of the image sensor included in the imaging apparatus, and the exposure amount or the dynamic range is controlled. Therefore, when the control value reaches a predetermined level, for example, a level higher than a normal shooting level, even if the control value actually obtained changes (fluctuates) abruptly (frequently), Instead of immediately reflecting this change in actual exposure control, it is possible to control the change of the control value by changing the control value step by step by a predetermined amount, so that the subject is optimally exposed according to the subject brightness. In a state where a predetermined dynamic range is ensured, and the subject brightness rapidly or frequently during moving image shooting Even when made into, rapid (frequent) exposure changes can smoothly control the photographing exposure state to avoid the occurrence, it is possible to perform a more optimal exposure control according thereto.

  Here, the definition relating to the concept of “exposure control” (hereinafter referred to as “AE control” as appropriate) in the present invention will be described with reference to FIG. Unlike so-called silver-salt cameras, in imaging devices such as digital cameras and digital movies, the control element for AE control is related to the photoelectric conversion characteristics of the image sensor (by changing the photoelectric conversion characteristics artificially). There is a method of controlling and a method of adjusting the total amount of light reaching the imaging surface of the imaging sensor and the integration time of the photoelectric conversion current after photoelectric conversion. In the present specification, the former is called “dynamic range control” and the latter is called “exposure amount control”. The “dynamic range control” is executed, for example, by controlling a switching point (hereinafter referred to as an inflection point) between a linear characteristic region and a logarithmic characteristic region of the imaging sensor. The “exposure amount control” is executed, for example, by adjusting the aperture amount of the aperture, adjusting the shutter speed of the mechanical shutter, or controlling the integration time of charges by controlling the reset operation of the image sensor.

According to a second aspect of the present invention, there is provided the imaging apparatus according to the first aspect, wherein the change control means uses a stepped control value obtained by changing a control value for controlling the exposure amount stepwise according to the predetermined amount of change. In order to follow the level of the control value for controlling the exposure amount calculated by the exposure amount control means based on the above, the step control value is changed by the predetermined change amount to determine a new step control value , or A control value for controlling the dynamic range calculated by the dynamic range control means based on a photographed image using a step control value in which a control value for controlling the dynamic range is changed stepwise by the predetermined change amount. In order to follow this level, the step control value is changed by the predetermined change amount to determine a new step control value . According to this configuration, in the change control of the control value by the change control unit, the step control is performed so as to follow the control value level calculated based on the step control value (approaching gradually to the control value level step by step). A new step control value is set by changing the value by a predetermined change amount, that is, a temporary control value is calculated based on the step control value, and a new step control value is set with reference to the temporary control value. Since the method of changing the control value stepwise by sequentially repeating the above is adopted, the smooth control of the photographing exposure state can be realized with a simpler configuration.

According to a third aspect of the present invention, there is provided the imaging apparatus according to the first or second aspect, wherein a control value for controlling the exposure amount that changes with time as the moving image is captured reaches a first threshold value indicating a start level of the change control. It is determined whether or not the control value for controlling the exposure amount has reached a second threshold value indicating the end level of change control , or time is taken along with the moving image shooting. It is determined whether or not the control value for controlling the changing dynamic range has reached a first threshold value indicating the start level of the change control, and the control value for controlling the dynamic range is changed control. further comprising a control value level discriminating means for discriminating whether or not reached the second threshold value indicating the end level, the change control unit, a control value for controlling the exposure amount by the control value level judgment means first The change control when the initiate said change control if it is judged to have reached the threshold value, the control value for controlling the exposure amount by the control value level judgment means is judged to have reached the second threshold value Or when the control value level determining means determines that the control value for controlling the dynamic range has reached the first threshold, the change control is started, and the control value level determining means When it is determined that the control value for controlling the range has reached the second threshold value, the change control is terminated . According to this configuration, when it is determined by the control value level determining unit that the control value that changes over time with moving image shooting has reached the first threshold value, change control is started by the change control unit (change). Control is executed), and when it is determined that the control value has reached the second threshold value, the change control is terminated.

  Thus, the change control is not started until the first threshold value is reached, and is changed to a so-called insensitivity so that the change control is executed after reaching the first threshold value and then reaching the second threshold value. Since control is started, and further (until the first threshold value is reached from the first threshold value), the control value is changed to insensitive by the change control. As the level increases, the change control is not started immediately, and the change control makes it possible to obtain a more optimal shooting exposure state according to the actually required moving image shooting.

  According to a fourth aspect of the present invention, there is provided the imaging apparatus according to the third aspect, wherein the predetermined control value for controlling the exposure amount is such that the output level of the imaging sensor with respect to the subject luminance (Lt1) for setting the exposure amount is the first characteristic region. The gain (Gain) for changing the exposure amount so as to be a predetermined target output level (Vtarget), and the change control means, the control value level discrimination means, the gain is set to the first threshold value (Gain). When it is determined that GainTh1 or GainTh2) has been reached, change control for changing the amplification factor in a stepwise manner according to the predetermined change amount (ΔGain) is started, and the amplification factor is controlled by the control value level determination unit. When it is determined that the threshold value (GainTh10 or GainTh20) is reached, the change control is terminated. According to this configuration, the control value for controlling the exposure amount is an amplification factor, and when the amplification factor reaches the first threshold value, change control in which the amplification factor changes stepwise according to the predetermined change amount is started. Since this change control is terminated when the threshold value of 2 is reached, rapid (frequent) exposure is performed by performing the change control on the amplification factor even when the subject brightness changes suddenly or frequently during moving image shooting. It is possible to prevent the occurrence of a change in amount, and thus to perform more optimal exposure control.

  According to a fifth aspect of the present invention, in the third aspect, the predetermined control value for controlling the dynamic range is such that the output level of the imaging sensor with respect to the subject luminance (Lm20) for setting the dynamic range is a predetermined saturated output level. An inflection point change amount (ΔVth) indicating an output level change amount at the inflection point of the photoelectric conversion characteristic for changing the dynamic range to be (Vmax), wherein the change control means has a control value When it is determined by the level determining means that the inflection point change amount has reached the first threshold value (ΔVthTh1 or ΔVthTh2), the inflection point change amount is changed stepwise by the predetermined change amount (ΔCVth). Change control is started, and the inflection point change amount is set to the second threshold value (ΔVthTh10 or ΔVt by the control value level determining means. When it is judged to have reached the TH20), characterized in that ends the change control. According to this configuration, the control value for dynamic range control is the inflection point change amount, and when the inflection point change amount reaches the first threshold value, the inflection point change amount is stepwise according to the predetermined change amount. Since the change control is started and this change control is terminated when the second threshold value is reached, even if the subject brightness changes suddenly or frequently during moving image shooting, the change with respect to the inflection point change amount By performing the control, it is possible to prevent a sudden (frequent) dynamic range (photoelectric conversion characteristic) change, and thus it is possible to perform more optimal exposure control.

The imaging device according to claim 6 generates an electrical signal corresponding to the amount of incident light, and a photoelectric conversion characteristic of the first characteristic region is output after the electrical signal is converted into a first characteristic with respect to the amount of incident light. A second characteristic region in which the electrical signal is converted into a second characteristic whose output change with respect to the incident light quantity is smaller than the first characteristic with respect to the incident light quantity, and a plurality of frames A moving image photographing means for photographing a moving image composed of images, an exposure evaluation value detecting means for detecting an exposure evaluation value when photographing a subject based on luminance information of the subject obtained from the frame image, and the exposure evaluation determine the exposure amount setting luminance as a subject luminance for exposure setting based on the detected exposure evaluation value by the value detection means, output level of the image sensor with respect to the determined exposure amount setting luminance Le calculates a control value for controlling the amount of exposure to a predetermined output level, based on the control value for controlling the exposure amount, the exposure amount control means for controlling the exposure amount for the image sensor Then, a dynamic range setting luminance as a subject luminance for exposure setting is determined based on the exposure evaluation value detected by the exposure evaluation value detecting means, and an output level of the image sensor with respect to the determined dynamic range setting luminance is predetermined. A dynamic range control means for calculating a control value for controlling the dynamic range so as to achieve an output level of the image sensor, and controlling a photoelectric conversion characteristic of the imaging sensor based on the control value for controlling the dynamic range ; control value is a predetermined level range for controlling the exposure amount calculated by the exposure amount control means based on the exposure evaluation value In the case where there is performed a control for prohibiting a change in the control value for controlling the Symbol exposure, or for controlling the dynamic range calculated by the dynamic range control means based on the exposure evaluation value And a change control means for performing control for prohibiting a change in the control value for controlling the dynamic range when the control value is within a predetermined level range .

  According to the above configuration, moving image shooting is performed by the moving image shooting unit, and an exposure evaluation value is detected by the exposure evaluation value detection unit based on the luminance information of the subject obtained from the frame image constituting the moving image. Using the evaluation value, exposure control is performed by performing exposure amount control by the exposure amount control unit and dynamic range control by the dynamic range control unit based on the photoelectric conversion characteristics. If the predetermined control value for controlling the exposure amount or the dynamic range calculated by the exposure control means based on the exposure evaluation value is within the predetermined level range, the change control means is within the predetermined level range. The control for prohibiting the change of the control value at is performed.

  As described above, the exposure amount control by the exposure amount control unit and the dynamic range control by the dynamic range control unit can be performed in association with the photoelectric conversion characteristics of the image sensor included in the imaging apparatus, and the predetermined level range can be set, for example, as a target. If the range including the control value is set, the control value for controlling the exposure amount or the dynamic range is maintained in this level range (does not change rapidly and frequently), and as a result Since insensitivity control is performed, it is possible to image the subject in an optimal exposure state and a predetermined dynamic range according to the subject brightness, and further, the target exposure during moving image shooting The subject brightness changes so as to deviate from the state (exposure amount, dynamic range) (desirably or frequently). It is avoided such would immediately transition to Zone exposed state, it is possible to perform thus optimal (smooth) exposure control.

  According to the imaging apparatus of the first aspect, the exposure amount control by the exposure amount control unit and the dynamic range control by the dynamic range control unit can be performed in association with the photoelectric conversion characteristics of the imaging sensor included in the imaging device, and the exposure can be performed. When the control value for controlling the amount or the dynamic range reaches a predetermined level, for example, a level higher than a normal shooting level, the actually obtained control value changes (fluctuates) abruptly (frequently). Even in this case, instead of immediately reflecting this change in actual exposure control, it is possible to control the change of the control value by changing the control value step by step by a predetermined change amount. Accordingly, the subject can be imaged in an optimal exposure state and with a predetermined dynamic range secured. Even if the image changes suddenly or frequently, it is possible to smoothly control the shooting exposure state by avoiding sudden (frequent) exposure changes corresponding to this, and to perform more optimal exposure control Is possible.

  According to the imaging apparatus of the second aspect, in the change control of the control value by the change control means, the control value level calculated based on the step control value is made to follow (approaching gradually to this control value level step by step). Therefore, the step control value is changed by a predetermined change amount to set a new step control value, that is, a temporary control value is calculated based on the step control value, and a new control value is calculated with reference to the temporary control value. Since the method of changing the control value stepwise by sequentially setting the step control value is adopted, the smooth control of the photographing exposure state can be realized with a simpler configuration.

  According to the imaging device of the third aspect, the change control is not started until the first threshold value is reached, and is executed until the second threshold value is reached after the first threshold value is reached. In this way, the control is started by so-called insensitivity, and the control value is changed insensitively by the change control (until the first threshold is reached from the first threshold) (insensitive control). Therefore, when the control value level increases, the change control is not started immediately, and the change control makes it possible to obtain a more optimal shooting exposure state according to the actually required moving image shooting. it can.

  According to the imaging apparatus of the fourth aspect, the control value for controlling the exposure amount is an amplification factor. When the amplification factor reaches the first threshold value, change control in which the amplification factor changes stepwise according to the predetermined change amount. Since the change control is terminated when the second threshold value is reached, even if the subject brightness changes suddenly or frequently during moving image shooting, the change control for the gain is performed rapidly. Occurrence of (frequent) exposure amount changes can be prevented, and as a result, more optimal exposure control can be performed.

  According to the imaging device of the fifth aspect, the control value for dynamic range control is the inflection point change amount, and when the inflection point change amount reaches the first threshold value, the inflection point change amount changes by a predetermined amount. The change control that changes stepwise depending on the amount is started, and when the second threshold value is reached, this change control is terminated. Therefore, even if the subject brightness changes suddenly or frequently during moving image shooting, the inflection point By performing the change control with respect to the change amount, it is possible to prevent a sudden (frequent) change in the dynamic range (photoelectric conversion characteristics), and thus it is possible to perform more optimal exposure control.

  According to the imaging device of the sixth aspect, the exposure amount control by the exposure amount control unit and the dynamic range control by the dynamic range control unit can be performed in association with the photoelectric conversion characteristics of the imaging sensor included in the imaging device. If the predetermined level range is set as a range including the target control value, for example, the control value for controlling the exposure amount or the dynamic range is maintained in this level range (does not change rapidly and frequently). As a result, the insensitivity control is performed, so that the subject can be imaged in an optimum exposure state and a predetermined dynamic range is ensured according to the subject luminance, and further, moving image shooting is performed. The subject is exposed (rapidly or frequently) so that it is not within the target exposure state (exposure amount, dynamic range). Brightness is changed, be immediately avoided such would migrate to undesired exposure state, it is possible to perform thus optimal (smooth) exposure control.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Description of external structure of imaging device)
1A and 1B are views showing an external appearance of a small digital camera 1 to which an imaging apparatus according to the present invention is preferably applied. FIG. 1A is a top view, FIG. 1B is a front view, and FIG. Respectively. The digital camera (imaging device) 1 includes a power switch 101 and a release switch 102 on the top surface of the camera body 10, a flash light emitting unit 103 and a photographing lens window 104 on the front side, and a back side. Various operation buttons such as a mode setting switch 105 and an LCD display unit 106 including a liquid crystal monitor (LCD) are arranged. In addition to the various main body devices, a bent lens barrel 20 is disposed inside the main body 10.

  The power switch 101 is a push switch for turning on (starting) and turning off (starting and stopping) the power of the camera 1, and the camera power is turned on and off sequentially by the pressing operation. The mode setting switch 105 is for setting various shooting modes such as a still image shooting mode for shooting a still image and a moving image shooting mode for shooting a moving image.

  The release switch 102 is a push switch that can be operated in a “half-pressed state” that is pressed halfway and further in a “full-pressed state”. For example, in the still image shooting mode, when the release switch 102 is pressed halfway, a preparatory operation for capturing a still image of the subject (preparation operations such as automatic exposure control and automatic focus control described later) is executed, and the release switch When 102 is fully pressed, a photographing operation (a series of operations for exposing an image sensor described later, performing predetermined image processing on an image signal obtained by the exposure, and recording the image signal on a memory card or the like) is performed. In addition, in the moving image shooting mode, when the release switch 102 is fully pressed, a predetermined moving image shooting operation is executed, and when the release switch 102 is fully pressed again, the moving image shooting operation is ended.

  The flash light emitting unit 103 emits light to illuminate the subject when the subject image is dark when the release switch 102 is half-pressed (still image shooting mode). The photographic lens window 104 is an opening for taking an optical image of a subject into the bending lens barrel 20 disposed inside the main body 10. The display unit 106 reproduces and displays a recorded image recorded on a built-in recording medium, and displays a through image (live view image) of a subject that has been video-recorded during shooting standby or in the video shooting mode. is there. In addition to the mode setting switch 105, a push switch group such as a zoom switch, a menu selection switch, and a selection determination switch is provided.

  The bendable lens barrel 20 constitutes a photographing lens system that takes a subject image through the photographing lens window 104 and guides it to the image sensor 30 disposed inside the main body 10. The lens barrel 20 is a lens barrel whose length does not vary even during zooming or focusing driving, that is, does not protrude outward from the main body 10. Inside the lens barrel 20 are a lens group 21 (see FIG. 2) constituting a photographing optical system composed of a zoom lens block and a fixed lens block arranged in series along the optical axis, and the lens group 21. And a diaphragm 22 arranged in place. Further, a shutter 23 is disposed at an appropriate position of the lens group 21, and the optical path of the photographing optical system is shielded or transmitted by the opening / closing operation of the shutter 23. That is, the exposure amount of the image sensor 30 is controlled by the degree of opening area setting of the diaphragm 22 and the opening / closing operation control of the shutter 23.

(Overall description of the electrical configuration of the imaging device)
FIG. 2 is an imaging process block diagram in the digital camera 1. The digital camera 1 includes an operation unit 100, the above-described bending lens barrel 20, an image sensor 30, a signal processing unit 40, an overall control unit 50, a drive unit 60, and the like. The operation unit 100 includes the power switch 101, the release switch 102, the mode setting switch 105, and the like described above.

  The image sensor 30 is an image signal (image sensor 30) of each component of R (red), G (green), and B (blue) according to the amount of light of the subject light image formed by the lens group 21 in the lens barrel 20. The signal is a photoelectric conversion into a signal consisting of a signal sequence of pixel signals received by each pixel of (1) and output to the signal processing unit 40. In the present embodiment, the imaging sensor 30 is a logarithmic conversion type solid-state imaging device that outputs an output pixel signal (an output electrical signal generated by photoelectric conversion) with respect to an incident light amount, not linearly but logarithmically. Is used. Note that the imaging sensor 30 has a characteristic in which an output pixel signal is linearly converted and output when the amount of incident light is small, and an area where the photoelectric conversion characteristic is linear (linear characteristic area = dark And a logarithmic area (logarithmic characteristic area = light time) (that is, a second characteristic area having a smaller change in output relative to the incident light amount than the linearly changing first characteristic area) ing). Further, the switching point (inflection point) between the linear characteristic region and the logarithmic characteristic region can be arbitrarily controlled by a specific control signal (dynamic range control signal). The configuration and operation of the image sensor 30 will be described in detail later.

  The timing generation circuit (timing generator) 31 controls a photographing operation (charge accumulation based on exposure, reading of accumulated charge, etc.) in the image sensor 30. The timing generation circuit 31 generates predetermined timing pulses (pixel drive signal, horizontal synchronization signal, vertical synchronization signal, horizontal scanning circuit drive signal, vertical scanning circuit drive signal, etc.) based on the imaging control signal from the overall control unit 50. In the moving image shooting mode (through image display mode), for example, a frame image is captured every 1/30 (seconds) and sequentially output to the signal processing unit 40. Further, during exposure in the still image shooting mode, charges are accumulated in conjunction with the exposure operation of the image sensor 30 (that is, the subject light image is photoelectrically converted into an image signal), and the accumulated charges are stored in the signal processing unit 40. Output. Further, the timing generation circuit 31 also generates an A / D conversion clock used in an A / D converter 402 described later.

  The signal processing unit 40 performs predetermined analog signal processing and digital signal processing on the image signal sent from the image sensor 30, and the signal processing of the image signal is performed for each pixel signal constituting the image signal. Done. The signal processing unit 40 includes an analog signal processing unit 401, an A / D converter 402, a black reference correction unit 403, an FPN correction unit 404, an evaluation value detection unit 405, a white balance (WB) control unit 406, and a color interpolation unit 407. A 3 × 3 color correction unit 408, a gradation conversion unit 409, a noise cancellation unit 410, and an image memory 411 are provided.

  The analog signal processing unit 401 performs predetermined analog signal processing on the image signal output from the image sensor 30 (analog signal group received by each pixel of the image sensor 30), and includes reset noise included in the analog image signal. A CDS circuit (correlated double sampling circuit) that reduces the image quality, and an AGC circuit (auto gain control circuit) that corrects the level of the analog image signal. Note that the AGC circuit amplifies the analog image signal with an appropriate amplification factor so as to match the input voltage range of the A / D converter 402 in the subsequent stage when appropriate exposure cannot be obtained, etc. It also has an amplifier function to compensate for this.

  The A / D converter 402 serves to convert the analog image signal output from the analog signal processing unit 401 into, for example, a 12-bit digital image signal (image data). The A / D converter 402 converts an analog image signal into a digital image signal based on an A / D conversion clock input from the timing generation circuit 31.

The black reference correction unit 403 uses the black level (image signal level in the dark) of the digital image signal input from the A / D converter 402 as a reference value (for example, 0 at the digital signal level after A / D conversion). When the image signal level input from the A / D converter 402 is SD1 and the image signal level in the dark is SD2,
SD1-SD2
Perform the operation. Such black reference correction is performed based on imaging dynamic range information corresponding to the photoelectric conversion characteristics of the imaging sensor 30 input from the overall control unit 50. This is because, in the digital camera 1 according to the present embodiment, the photoelectric conversion characteristics of the image sensor 30 can be controlled, and therefore the image signal level in the dark of the digital image signal input from the A / D converter 402. However, since it fluctuates due to a change in the photoelectric conversion characteristics of the image sensor 30, it is possible to perform accurate black reference correction following the fluctuation.

An FPN (Fixed Pattern Noise) correction unit 404 is for removing FPN (fixed pattern noise) of the image signal input from the black reference correction unit 403. The fixed pattern noise is noise caused by variations in output values of image signals generated by the respective pixels, which are caused by variations in the threshold values of the FETs included in the respective pixel circuits of the image sensor 30. When the image signal level input from the black reference correction unit 403 is SD3 and the fixed pattern component of the image signal input from the black reference correction unit 403 is SD4, the FPN correction unit 404
SD3-SD4
Perform the operation.

The evaluation value detection unit 405 performs automatic exposure control (AE), automatic focus control (AF; autofocus), white balance control (WB control), or the like from an image signal actually captured by the image sensor 30. An evaluation value serving as a base value, that is, an AE evaluation value, an AF evaluation value, a white balance evaluation value (hereinafter referred to as a WB evaluation value), and the like are detected. For example, when performing AE control,
(1) Measure the luminance level and luminance range of the subject to be imaged,
(2) A necessary exposure control amount is calculated so that an output conforming to the luminance level and luminance range can be obtained from the imaging sensor,
(3) The exposure amount and the like are specifically adjusted based on the calculation result, and the main imaging is started.
In this evaluation value detection unit 405, the luminance level and luminance range of the subject are determined from the image signal actually captured by the image sensor 30 in order to perform the role of step (1). This is obtained and output to the overall control unit 50 as an AE evaluation value, and provided for AE control operation in the subsequent step.

  In the case of AF control, for example, a focus lens in which the contrast of an image obtained by the imaging operation is maximized while alternately driving the focus lens (lens group 21) in the optical axis direction and imaging operation by the imaging sensor 30. Is obtained (so-called hill-climbing detection method), which is output to the overall control unit 50 as an AF evaluation value, and used for the AE control operation in the subsequent step. Further, the white balance control is intended to correct the color of the output image to a color suitable for the light source color of the subject. In this case, R, R based on the image signal input from the FPN correction unit 404 in the previous stage. The evaluation value detection unit 405 calculates the luminance ratio and the luminance difference between the G and B colors, and outputs them to the overall control unit 50 as WB evaluation values. A specific method for obtaining these evaluation values will be described later.

  Based on the dynamic range information given from the overall control unit 50 and the WB evaluation value, the white balance control unit 406 sets each color component R, G, B so that the color balance of the image signal becomes a predetermined color balance. Correction for converting the level of the pixel data is performed. In this embodiment, since the image sensor 30 includes a linear characteristic region and a logarithmic characteristic region, a WB evaluation value is acquired for each linear characteristic region and logarithmic characteristic region, and a white balance suitable for each region is obtained. It is desirable to make corrections.

  The color interpolation unit 407 interpolates data of pixel positions where the frame image is insufficient for each color component R, G, B of the image signal input from the white balance control unit 406. That is, the color filter structure of the logarithmic conversion type image sensor 30 used in the present embodiment lacks color information because of the so-called Bayer system in which G is a checkered pattern and R and B are line-sequentially arranged. Therefore, the color interpolation unit 407 interpolates pixel data at non-existing pixel positions using a plurality of actual pixel data.

  Specifically, for a G color component frame image having pixels up to a high band, the color interpolation unit 407 masks the image data constituting the frame image with a predetermined filter pattern, and then applies a median (intermediate value) filter. The average value of the pixel data from which the maximum value and the minimum value are removed from the pixel data actually existing around the pixel position to be interpolated is calculated, and the average value is interpolated as the pixel data at the pixel position. For the R and B color components, after the image data constituting the frame image is masked with a predetermined filter pattern, an average value of pixel data actually existing around the pixel position to be interpolated is calculated, and the average value is calculated. Is interpolated as pixel data at the pixel position.

FIG. 3 shows an example of the color filter structure of the image sensor 30. In such a color filter structure, the image signals of the color components R, G, and B in each pixel by the color interpolation are generated as follows, for example.
(A) Color interpolation formula of address 11 (B11) R11 = (R00 + R20 + R02 + R22) / 4
G11 = (Gr10 + Gb01 + Gb21 + Gr12) / 4
B11 = B11
(B) Color interpolation formula of address 11 (Gr12) R12 = (R02 + R22) / 2
G12 = Gr12
B12 = (B11 + B13) / 2
(C) Color interpolation formula of address 21 (Gb21) R21 = (R20 + R22) / 2
G21 = Gb12
B21 = (B11 + B31) / 2
(D) Color interpolation formula for address 22 (R22) R22 = R22
G22 = (Gb21 + Gr12 + Gr32 + Gb23) / 4
B22 = (B11 + B31 + B13 + B33) / 4

The 3 × 3 color correction unit 408 corrects the saturation of the image signals of the color components R, G, and B input from the color interpolation unit 407 (corrects the hue). The 3 × 3 color correction unit 408 has three types of conversion coefficients that convert the level ratio of the image signals of the color components R, G, and B, and converts the level ratio with a conversion coefficient according to the shooting scene to generate an image. Correct the saturation of the data. For example, using a total of nine transform coefficients a1 to c3, the image signal is linearly transformed as follows.
R ′ = a1 * R + a2 * G + a3 * B
G '= b1 * R + b2 * G + b3 * B
B ′ = c1 * R + c2 * G + c3 * B

  The gradation converting unit 409 changes the level of the image signal for each color component to a predetermined gamma characteristic so that the image signals of the color components R, G, and B input from the 3 × 3 color correcting unit 408 have an appropriate output level. Is used for nonlinear conversion and offset adjustment. That is, the tone conversion unit 409 corrects the tone characteristics (γ curve and digital gain) of the image signal that has been subjected to white balance adjustment and color correction to the tone characteristics of the LCD display unit 106, an externally output monitor TV, and the like. To do. The tone conversion unit 409 changes the tone characteristics of the image signal based on the dynamic range information input from the overall control unit 50, the AE evaluation value detected by the evaluation value detection unit 405, and the like.

  The noise cancellation unit 410 corrects the sharpness of the image to a good state by removing the noise component of the image signal input from the gradation conversion unit 409 and extracting and enhancing only the edge component. Based on the dynamic range information input from the overall control unit 50, the noise canceling unit 410 changes the coring coefficient (the coefficient for extracting only the noise component of the image signal and extracting the edge component and the coefficient to be enhanced). Then, make an appropriate correction.

  The image memory 411 includes a memory such as a ROM or a RAM, and temporarily stores the image data after the signal processing by the signal processing unit 40. The image memory 411 has a capacity capable of storing, for example, one frame of image data. is doing.

  The memory card I / F unit 412 is an interface for outputting the memory card recording image data generated by the signal processing unit 40 so as to be recorded on the memory card 107. The memory card 107 is a memory for recording and storing image data such as still images and moving images. The memory card 107 is detachable from the digital camera 1 and can be used as an image with an external recording medium. Data exchange is possible. The LCD display I / F unit 413 is an interface for converting the LCD display image data generated by the signal processing unit 40 into, for example, an NTSC or PAL image signal and outputting the image signal to the LCD display unit 106.

  The overall control unit 50 includes a CPU (Central Processing Unit) and the like, and centrally controls the photographing operation of the digital camera 1. That is, the overall control unit 50 generates signals based on information sent from each unit of the signal processing unit 40 (the above-described AE evaluation value, AF evaluation value, WB evaluation value, etc.) and the operation mode of the digital camera 1. The operation of each processing unit is controlled by calculating and transmitting operation information such as parameters required by each unit of the processing unit 40. In addition, the overall control unit 50 controls the timing generation circuit 31 for photographing operation, zooming and focusing drive of the lens group 21, and control of the drive unit 60 for driving the aperture 22 and the shutter 23, and output of image signals. Control and so on.

  FIG. 4 is a functional block diagram for explaining functions of the overall control unit 50. The overall control unit 50 includes an information reception unit 501, an information transmission unit 502, a calculation unit 510 including a memory unit 515, a control signal generation unit 520, a level determination unit 530, and an input / output unit 540.

  The information receiving unit 501 acquires the AE evaluation value, the AF evaluation value, and the WB evaluation value detected by the evaluation value detection unit 405 of the signal processing unit 40, and distributes them to each parameter calculation unit included in the calculation unit 510. Send. On the other hand, the information transmission unit 502 appropriately extracts information (photoelectric conversion characteristic information, coring coefficient, etc.) required in the signal processing unit 40 from the memory unit 515, and distributes the information to each processing unit of the signal processing unit 40 in a timely manner for transmission. To do.

  The calculation unit 510 performs an operation of calculating a control parameter based on the evaluation value given from the information reception unit 501, and calculates an AE control parameter including an exposure amount control parameter calculation unit 511 and a dynamic range control parameter calculation unit 512. A unit 5110, an AF control parameter calculation unit 513, a white balance (WB) control parameter calculation unit 514, and a memory unit 515.

  The memory unit 515 includes a ROM, a RAM, and the like, and information on photoelectric conversion characteristics of the image sensor 30 (information for obtaining desired photoelectric conversion characteristics at the time of photographing), that is, an exposure time setting value and an aperture setting value described later, Alternatively, a photoelectric conversion characteristic information storage unit 516 that stores a photoelectric conversion characteristic setting value (dynamic range information corresponding to the photoelectric conversion characteristic) and a coring coefficient storage unit that stores an installation position of a coring coefficient used in the noise cancellation unit 410 517, from the LUT storage unit 518 or the like for storing conversion information for performing data conversion (mutual conversion) on the data obtained in the linear characteristic region and logarithmic characteristic region of the image sensor 30, that is, LUT (Look Up Table) and the like. It is configured.

  The photoelectric conversion characteristic information storage unit 516 may be configured to store the photoelectric conversion characteristic itself (a photoelectric conversion characteristic curve as shown in FIG. 10 described later). In addition to the above LUT, the LUT storage unit 518 includes an LUT that performs data conversion between an exposure time (T1) and an aperture area (S2), which will be described later, and an exposure time setting value and an aperture setting value. LUT that performs data conversion between inflection point value (output level) of characteristic and photoelectric conversion characteristic setting value, LUT that outputs photoelectric conversion characteristic setting value from maximum luminance output level, or photoelectric conversion from change amount of maximum luminance output level Various LUTs for data conversion are stored, such as an LUT that outputs a change amount of the characteristic setting value, or an LUT that converts data between the value of the saturation pixel number and the value of the inflection point change amount (ΔVth). Further, as described above, the data values stored in the photoelectric conversion characteristic information storage unit 516, the coring coefficient storage unit 517, and the LUT storage unit 518 are appropriately transmitted from the information transmission unit 502 to the appropriate position of the signal processing unit 40. It is like that.

  The AE control parameter calculation unit 5110 calculates control parameters for setting the optimal exposure amount at the time of shooting and the photoelectric conversion characteristics of the image sensor 30 in order to perform exposure control (AE control) according to the luminance of the subject. To do. That is, the exposure amount control parameter calculation unit 511 of the AE control parameter calculation unit 5110 calculates a control parameter for optimizing the exposure time and aperture, and the exposure amount control parameter calculation unit 511 is an evaluation value detection unit. Based on the AE evaluation value detected at 405 and the photoelectric conversion characteristic information of the imaging sensor 30 at the time of acquisition of the AE evaluation value stored in the photoelectric conversion characteristic information storage unit 516, exposure according to subject luminance is performed. Calculate the time setting value and aperture setting value.

  The dynamic range control parameter calculation unit 512 calculates a control parameter for optimizing the photoelectric conversion characteristics of the image sensor 30 according to the subject brightness. The dynamic range control parameter calculation unit 512 calculates a photoelectric conversion characteristic setting value such that, for example, the subject luminance for dynamic range setting becomes a desired saturation output level in the imaging sensor 30. Also in this calculation, the photoelectric conversion characteristic information of the image sensor 30 at the time of obtaining the AE evaluation value stored in the photoelectric conversion characteristic information storage unit 516 is referred to.

  The AE control parameter calculation unit 5110 controls the exposure amount or dynamic range calculated by the overall control unit 50 (exposure amount control parameter calculation unit 511 or dynamic range control parameter calculation unit 512) based on the AE evaluation value. When the predetermined control value, that is, an amplification factor Gain related to exposure amount control, which will be described later, and an output level change amount ΔVth of the inflection point of photoelectric conversion characteristics related to dynamic range control reach a predetermined level, the control value is predetermined. A change control unit 5111 is provided that performs change control of the control value so as to change step by step (amplification factor variation ΔGain or variation variation ΔCVth described later).

  In the change control, the change control unit 5111 performs actual imaging under the exposure setting of the imaging apparatus using a control value obtained by changing the control value stepwise by a predetermined change amount (this control value is referred to as a step control value). From the image (AE evaluation value extracted from the captured image), a temporary control value for the next AE control is calculated, and a new step control value is set based on the temporary control value in order. The control value is changed step by step.

  The AF control parameter calculation unit 513 calculates a control parameter for setting an optimum focal length for shooting a subject based on the AF evaluation value detected by the evaluation value detection unit 405. In calculating the control parameter, the AF evaluation value to be referred to is acquired in each of the logarithmic characteristic area and the linear characteristic area of the imaging sensor 30, and rough ranging (obtained from the logarithmic characteristic area by utilizing the characteristics of each characteristic area). (AF evaluation value) and control parameters for detailed distance measurement (AF evaluation value obtained from the linear characteristic region) are preferably calculated.

  The white balance control parameter calculation unit 514 calculates a control parameter for setting the color balance of the image signal to a predetermined color balance based on the WB evaluation value detected by the evaluation value detection unit 405. In calculating the control parameter, the AF evaluation value to be similarly referred to is acquired in each of the logarithmic characteristic area and the linear characteristic area of the imaging sensor 30, and the control parameter corresponding to each characteristic area is calculated. preferable.

  The control signal generator 520 generates a control signal for driving each control operation element according to various control parameters calculated by the arithmetic unit 510. The control signal generator 520 has a dynamic range. A control signal generator 521, a sensor exposure time control signal generator 522, a shutter control signal generator 523, a zoom / focus control signal generator 524, and an aperture control signal generator 525 are configured.

  The dynamic range control signal generation unit 521 outputs an output level point at which the photoelectric conversion characteristic is switched from the linear characteristic region to the logarithmic characteristic region according to the photoelectric conversion characteristic setting value of the imaging sensor 30 calculated by the dynamic range control parameter calculation unit 512. A drive signal for the image sensor 30 for adjusting (inflection point) is generated and transmitted to the timing generation circuit 31. The timing generation circuit 31 generates a timing signal for controlling the dynamic range of the image sensor 30 according to the input drive signal, and drives the image sensor 30. Specifically, as will be described later, the photoelectric conversion characteristic of the image sensor 30 is controlled by controlling the signal φVPS (the height of the voltage VPH at φVPS or the length of time ΔT) for the image sensor 30. Therefore, the dynamic range control signal generation unit 521 controls the driving signal for the timing generation circuit 31 for controlling the signal φVPS according to the photoelectric conversion characteristic setting value, so that the dynamics of the image sensor 30 can be changed. Control the range to suit the brightness of the subject.

  The sensor exposure time control signal generation unit 522 generates a control signal for controlling the exposure time (integration time) of the imaging sensor 30 by an electronic circuit control operation without depending on the mechanical operation of the diaphragm 22 and the shutter 23. To do. The sensor exposure time control signal generation unit 522 generates a drive signal (specifically, the image sensor 30 so as to ensure an intended exposure time based on the optimum exposure amount calculated by the exposure amount control parameter calculation unit 511. In this case, as will be described later, a signal for controlling the time ΔS when the signal φVPS to the image sensor 30 becomes the medium potential M) is generated and transmitted to the timing generation circuit 31. The timing generation circuit 31 generates a timing signal for controlling the exposure time of the image sensor 30 according to the input drive signal, and drives the image sensor 30.

  Similarly, the shutter control signal generation unit 523 sets the shutter speed (shutter opening time) of the shutter 23 in accordance with the exposure time based on the optimum exposure amount calculated by the exposure amount control parameter calculation unit 511. Is generated. The zoom / focus control signal generation unit 524 generates a control signal for driving the lens group 21 based on the optimum focal length calculated by the AF control parameter calculation unit 513. Further, the aperture control signal generation unit 525 generates a control signal for setting the aperture area of the aperture 22 based on the optimum exposure amount calculated by the exposure amount control parameter calculation unit 511. The control signals generated by the shutter control signal generation unit 523, the zoom / focus control signal generation unit 524, and the aperture control signal generation unit 525 are transmitted to corresponding portions of the drive unit 60, respectively.

  The level determination unit 530 constantly detects the level (level fluctuation) of control values (amplification gain and a change amount ΔVth, which will be described later) for controlling the exposure amount or dynamic range during shooting driving (during driving by AE control). In addition, a threshold value (GainTh1 and GainTh2, which will be described later) indicating the start level of the control value change control (which will be described later, insensitive control) in which the control value is changed step by step by a predetermined change amount (ΔGain or ΔCVth which will be described later), or Whether or not (ΔVthTh1 or ΔVthTh2) has been reached, and whether or not this control value has reached a threshold (GainTh10 and GainTh20 described later, or ΔVthTh10 or ΔVthTh20) indicating the end level of the change control is determined. It is.

  The input / output unit 540 is connected to the memory card I / F unit 412 and the LCD display I / F unit 413, and performs predetermined image processing on the photographed image in response to an instruction signal from the operation unit 100 or the like. Thereafter, the captured image signal is recorded on the memory card 107, displayed on the LCD display unit 106, or conversely, an image signal is taken in from the memory card 107.

  Returning to FIG. 2, the driving unit 60 actually operates the mechanical driving unit included in the digital camera 1 based on the control signal generated by the control signal generating unit 520. The shutter driving unit 61, A zoom / focus drive unit 62 and an aperture drive unit 63 are provided.

  The shutter drive unit 61 opens and closes the shutter 23 according to the control signal supplied from the shutter control signal generator 523 so that the shutter 23 is opened for a predetermined time. The zoom / focus control unit 62 operates a zoom lens block or a motor that operates the focus lens block of the lens group 21 in accordance with a control signal supplied from the zoom / focus control signal generation unit 524, and focuses the lens block. Move to position. Further, the diaphragm driving unit 63 drives the diaphragm 22 in accordance with a control signal given from the diaphragm control signal generating unit 525 and sets the diaphragm to a predetermined opening amount.

(Description of overall flow of operation)
The overall flow of the operation of the digital camera 1 configured as described above will be described first. FIG. 5 is a flowchart showing an example of the overall operation of the digital camera 1. As shown in the figure, when the operation is roughly divided, an evaluation value detecting step (step S1) for detecting an evaluation value such as an AE evaluation value, an AF evaluation value, and a WB evaluation value, and various parameters based on the obtained evaluation value. Control parameter calculation step (step S2) for calculating the parameters, and the calculated parameters are set in the respective sections of the corresponding digital camera 1, and the control parameters for driving the respective sections of the digital camera 1 so as to obtain a shooting state corresponding to the parameters are set. And a control parameter setting step (step S3) to be set.

Specifically, in each of the steps S1 to S3, the following processing is performed.
First, in the evaluation value detection step S1, evaluation value information serving as a base for various controls is acquired, and an evaluation value is calculated based on the evaluation value information. In the case of AE control, the luminance level of the subject that is the imaging target is measured (detected), and the AE evaluation value is calculated from the measured value. In the detection of the luminance level and the luminance range, it is reasonable to obtain from a captured image actually captured by the imaging sensor 30, and the imaging sensor 30 can capture still images and moving images.
(Step S1-1) Detection from a still image: The image sensor 30 acquires an image for evaluation value detection as a still image before actual photographing, and measures a luminance level and a luminance range.
(Step S1-2) Measurement from a moving image: The image sensor 30 acquires an image for evaluation value detection as a moving image before actual photographing, and measures a luminance level and a luminance range.
The two types of luminance information acquisition steps can be exemplified. After that,
(Step S1-3) Calculation of Evaluation Value: The evaluation value detecting unit 405 calculates various evaluation values including the AE evaluation value based on the acquired luminance information of the image.
The steps are performed.

Next, in step S2, various parameters are calculated based on the evaluation value. In the case of AE control, since the exposure amount or the dynamic range is an element of AE control, these control parameters are converted into the AE evaluation value. Calculated based on That is, as step S2,
(Step S2-1) Calculation of exposure amount control parameter: The overall control unit 50 calculates an exposure amount control parameter based on the AE evaluation value.
(Step S2-2) Calculation of dynamic range control parameter: The overall control unit 50 calculates the dynamic range control parameter based on the AE evaluation value.
Two parameter calculation steps can be exemplified.

Finally, in step S3, control parameters for driving each part of the digital camera 1 are set. In the case of AE control, control parameters are set based on (Step S2-1) or (Step S2-2), that is, as Step S3,
(Step S3-1) Setting of exposure amount control parameter: Based on the calculated exposure amount control parameter, the parameter is set in the memory unit 515, the control signal generation unit 520, etc., and the timing generation circuit 31 and the drive unit 60 are operated. Let
(Step S3-2) Setting of dynamic range control parameter: Based on the calculated dynamic range control parameter, the parameter is set in the memory unit 515, the control signal generation unit 520, etc., and the timing generation circuit 31 is operated.
The following two parameter setting steps can be exemplified.

(Basic characteristics of the image sensor used in this embodiment)
Hereinafter, each step described above will be described in detail, but in this embodiment, as the photoelectric conversion characteristics, a linear characteristic region in which the electric signal is linearly converted with respect to the incident light amount and output, and the incident light amount. On the other hand, since it is assumed that an image sensor 30 having a logarithmic characteristic region in which the electrical signal is logarithmically converted and output is used, first, basic characteristics of the image sensor 30 used in the present embodiment are used. A specific example will be described in detail.

  FIG. 7 is a schematic configuration diagram of a two-dimensional MOS solid-state imaging device which is an example of the imaging sensor 30. In the figure, G11 to Gmn indicate pixels arranged in a matrix. A vertical scanning circuit 301 and a horizontal scanning circuit 302 are disposed in the vicinity of the outer peripheral edge of the pixel unit composed of the pixels G11 to Gmn. The vertical scanning circuit 301 sequentially scans row lines (signal lines) 304-1, 304-2,... 304-n (collectively referred to as row lines 304). The horizontal scanning circuit 302 applies the photoelectric conversion signal derived from each pixel to the output signal lines 306-1, 306-2,... 306-m (collectively referred to as the output signal line 306) in the horizontal direction for each pixel. Read sequentially. Each pixel is supplied with power by the power supply line 305. Each pixel is connected not only to the lines and output signal lines but also to other lines (for example, a clock line), but is not shown in FIG.

  The output signal lines 306-1, 306-2,... 306-m are respectively connected to constant current sources 307-1, 307-2,. 307-m (these are collectively referred to as a constant current source 307) is provided. However, as this amplifier circuit, a resistor or a transistor (MOS transistor) may be provided instead of the constant current source 307. The image data at the time of imaging and the correction data at the time of reset of each pixel output via the output signal line 306 are sequentially selected as selection circuits (sample hold circuits) 308-1, 308-2,... 308-m. (They are collectively referred to as a selection circuit 308). The selection circuit 308 outputs image data and correction data for each row and samples and holds them. The sampled and held image data and correction data are output to the correction circuit 309 for each column, and the correction circuit 309 corrects the image data based on the correction data so that noise components due to sensitivity variations are removed. Is called. Then, the image data in which the sensitivity variation of each pixel is corrected is output serially from the correction circuit 309 for each pixel.

  FIG. 8 is a circuit diagram illustrating a configuration example of each of the pixels G11 to Gmn illustrated in FIG. As shown in the figure, the pixel includes a photodiode PD, transistors T1 to T6 as MOSFETs (Metal Oxide Semiconductor Field Effect Transistors), and a capacitor C as an integrating capacitor. Here, P-channel MOSFETs are employed as the transistors T1 to T6. φVD, φV, φVPS, φRST, φS, and RSB indicate signals (voltages) to the respective transistors and capacitors C, and GND indicates ground.

  The photodiode PD is a photosensitive part (photoelectric conversion part) and outputs an electrical signal (photocurrent IPD) corresponding to the amount of incident light from the subject. The transistor T5 is paired with the constant current source 307 shown in FIG. 7 to constitute an amplification circuit (source follower amplifier) for source follower amplification, and performs amplification (current amplification) on the voltage VOUT described later. The transistor T6 is a signal readout transistor that operates as a switch that is turned on and off in accordance with the voltage applied to the gate. That is, the source of the transistor T6 is connected to the output signal line 306 shown in FIG. 7, and when turned on, the current amplified by the transistor T5 is led to the output signal line 306 as an output current.

  The transistor T2 generates a voltage obtained by linear conversion or logarithmic conversion with respect to the photocurrent IPD at the gate of the transistor T2. By the way, in the MOSFET, a minute current called a subthreshold current flows when the gate voltage is equal to or lower than a threshold value. The transistor T2 performs the linear conversion or logarithmic conversion using this subthreshold characteristic.

  Specifically, when the luminance of the subject to be imaged is low (the subject is dark), that is, when the amount of incident light incident on the photodiode PD is small, the gate potential of the transistor T2 becomes higher than the source potential of the transistor. The transistor T2 is in a so-called cut-off state, the subthreshold current does not flow through the transistor T2 (the transistor T2 does not operate in the subthreshold region), and the photocurrent generated in the photodiode PD becomes a parasitic capacitance of the photodiode PD. The charge flows and accumulates, and a voltage corresponding to the amount of accumulated charge is generated. At this time, since T1 is turned on, a voltage corresponding to the amount of charge accumulated in the parasitic capacitance is generated as the voltage VG at the gates of the transistors T2 and T3. The voltage VG causes a current to flow through the transistor T3, and an amount of charge proportional to the voltage VG is accumulated in the capacitor C (the transistor T3 and the capacitor C constitute an integration circuit). A voltage linearly proportional to the integral value of the photocurrent IPD appears at the connection node a between the transistor T3 and the capacitor C, that is, the output VOUT. At this time, the transistor T4 is in an off state. When the transistor T6 is turned on, the electric charge accumulated in the capacitor C is led to the output signal line 306 as an output current through the transistor T5. This output current is a value obtained by linearly converting the integrated value of the photocurrent IPD. This is the operation of the imaging sensor 30 in the linear characteristic region.

  On the other hand, when the luminance of the subject to be imaged is high (the subject is bright) and the amount of incident light incident on the photodiode PD is large, the gate potential of the transistor T2 becomes equal to or lower than the source potential of the transistor T2, and the subthreshold current is supplied to the transistor T2. Flows (transistor T2 operates in the subthreshold region), and a voltage VG having a value obtained by natural logarithm conversion of photocurrent IPD is generated at the gates of transistors T2 and T3. The voltage VG causes a current to flow through the transistor T3, and a charge equivalent to a value obtained by natural logarithmically converting the integrated value of the photocurrent IPD is accumulated in the capacitor C. As a result, a voltage proportional to a value obtained by natural-logarithmically converting the integral value of the photocurrent IPD is generated at the connection node a (output VOUT) between the capacitor C and the transistor T3. At this time, the transistor T4 is in an off state. When the transistor T6 is turned on, the electric charge accumulated in the capacitor C is led to the output signal line 306 as an output current through the transistor T5. This output current is a value obtained by natural logarithm conversion of the integral value of the photocurrent IPD. This is the operation of the imaging sensor 30 in the logarithmic characteristic region. As described above, each pixel outputs a voltage that is linearly or naturally logarithmically proportional to the amount of incident light (subject luminance).

  The transistor T1 is a switch used for extracting noise data at reset (noise signal generated due to manufacturing variation of the transistor T2). The transistor T1 is turned on except during reset, and a photocurrent IPD flows between the transistor T2 (the drain thereof) and the photodiode PD. At the time of reset, the photocurrent IPD of the photodiode PD is cut off and only the variation is taken out. The extracted variation (noise signal) is subtracted from the video signal described later.

  The transistor T4 is a transistor for resetting the capacitor C, which operates as a switch that is turned on and off according to a voltage applied to the gate of the transistor T4. When the transistor T4 is turned on, a reset voltage (the voltage of the signal RSB) is applied, and the charge (charge amount) accumulated in the capacitor C is returned to the original state, that is, the state before the start of integration.

  FIG. 9 is an example of a timing chart regarding the imaging operation of the imaging sensor 30 (pixel). Here, due to the polarity of the P-channel MOSFET, it is turned off at Hi (High) and turned on at Low (Low) as follows. First, the signal φV becomes Low at the position indicated by reference numeral 311, the transistor T 6 is turned on, and the video signal is read, that is, the charge accumulated in the capacitor C is led to the output signal line 306 as an output current (video signal). The Next, the signal φS becomes Hi at the position indicated by reference numeral 312, the transistor T1 is turned off, and the photodiode PD is disconnected. Next, the signal φVPS becomes Hi at the position indicated by reference numeral 313, and the transistor T2 is reset. At the same time when the transistor T2 is reset, the signal φRST becomes Low at the position indicated by reference numeral 314, the transistor T4 is turned on, and the reset voltage by the signal RSB is applied to the capacitor C (connection node a) (connection node). The potential of a becomes the potential of RSB (VRSB)), and the capacitor C is reset (charged). After the transistor T2 and the capacitor C are thus reset, the signal φV becomes Low again at the position indicated by reference numeral 315, the transistor T6 is turned on, and a noise signal is derived to the output signal line 306.

  Next, the signal φS becomes Low at the position indicated by reference numeral 316 (the transistor T1 is turned on), and the separation of the photodiode PD is released. Then, the signal φVPS becomes the intermediate potential M at the position indicated by reference numeral 318, and the parasitic capacitance of the photodiode PD is reset to reduce the afterimage. Further, in order to make the integration start voltage of the next frame constant, the signal φRST becomes Low again at the position indicated by reference numeral 317, the transistor T4 is turned on, and the capacitor C is reset again.

  Thereafter, the signal φVPS changes from M to Low at the position indicated by reference numeral 319, and the reset of the parasitic capacitance of the photodiode PD is completed. At the same time, the signal φRST changes from Low to Hi, and the reset operation of the capacitor C is also completed. The integration of the capacitor C is started from time t1 at this time, and the integration continues until the position indicated by reference numeral 311 where the signal φV is changed from Hi to Low, that is, until the time t2 when reading of the video signal in the next frame is started. Is done. The time between the times t1 and t2 is the integration time of the capacitor C, that is, the exposure time in imaging. This exposure time is controlled by controlling the time ΔS (length) for applying the signal φVPS at the medium potential M. This time ΔS is controlled by the sensor exposure time control signal generation unit 522 via the timing generation circuit 31.

  The signal φVD is used to perform a potential operation in order to adjust to the operating range of the amplifier circuit (source follower amplifier) or to adjust an offset generated in the video signal or noise signal. Vh, Vm, and Vl of the signal φVD indicate high potential, medium potential, and low potential, respectively.

  As described above, the image sensor 30 can obtain an output signal that is linearly or logarithmically converted according to the luminance of the subject, and has a photoelectric conversion characteristic 320 as shown in FIG. As shown in the figure, the photoelectric conversion characteristic 320 is divided into a linear characteristic region and a logarithmic characteristic region with an inflection point 321 as a boundary. The inflection point 321 is a point where the linear characteristic region is switched to the logarithmic characteristic region, and the value of the sensor output at the inflection point 321 is indicated by Vth. In general, in a linear characteristic region, it is impossible to image a subject with a wide luminance range (the dynamic range is narrow), but the gradation of the entire image can be increased (high contrast can be obtained). In addition, a high-quality image with rich gradation can be obtained even for a dark subject (for example, a subject in cloudy weather or shade). On the other hand, in the logarithmic characteristic region, gradation at high luminance is poor, but it is possible to image a subject with a wide luminance range (wide dynamic range) and a bright subject (for example, exposed to direct sunlight or direct sunlight). Even if it is a subject in which sunlight is present behind), a high-quality image having a deep depth can be obtained including a dark part.

  By the way, this photoelectric conversion characteristic 320 (inflection point 321) can be changed (moved) by changing the difference between the Hi and Low voltages of the signal φVPS input to the source of the transistor T2. That is, assuming that the voltage at the time of Hi is VPH and the voltage at the time of Low is VPL, by changing the voltage difference ΔVPS (= VPH−VPL) (see FIG. 9), as shown in FIG. The characteristic 320 (inflection point 321) can be arbitrarily changed to the photoelectric conversion characteristic 322 (inflection point 324) or the photoelectric conversion characteristic 323 (inflection point 325). As the photoelectric conversion characteristics change in this manner, the ratio between the linear characteristic area and the logarithmic characteristic area changes, and as shown in the photoelectric conversion characteristics 322, the photoelectric conversion characteristics or the photoelectric conversion characteristics 323 have a large ratio of the linear characteristic areas. As shown in FIG. 4, photoelectric conversion characteristics having a large ratio of the logarithmic characteristic region can be obtained. In this case, you may change so that all the photoelectric conversion characteristics may become a linear characteristic area | region or a logarithmic characteristic area | region.

  In the present embodiment, ΔVPS is changed by changing the voltage VPH, and the photoelectric conversion characteristics of the image sensor 30 are changed. In FIG. 11, as the VPH increases (the ΔVPS increases), the ratio of the linear characteristic region increases and changes to the photoelectric conversion characteristic 322 side. Thus, the photoelectric conversion characteristic 323 changes. This voltage VPH is controlled by a dynamic range control signal generation unit 521 via the timing generation circuit 31.

  In order to change the photoelectric conversion characteristics as described above, the time ΔT for applying the signal φVPS that becomes the voltage VPH may be changed. In this case, the photoelectric conversion characteristics change so that the proportion of the linear characteristic region increases as the time ΔT increases, and the proportion of the logarithmic characteristic region increases as the time ΔT decreases. In FIG. 11, the case where the time ΔT is long corresponds to the photoelectric conversion characteristic 322, and the case where the time ΔT is short corresponds to the photoelectric conversion characteristic 323.

(Evaluation Value Detection Step S1)
Next, a specific method for obtaining an evaluation value such as an AE evaluation value in the evaluation value detection unit 405 of the signal processing unit 40 will be described.
(Step S1-1) Example of Detection of Evaluation Value from Still Image FIG. 12 is a flowchart showing an operation example when the above-described imaging sensor 30 detects an AE evaluation value or the like of a subject from a still image actually captured. . That is, when a still image is shot (main shooting) with the digital camera 1 according to the present embodiment, a still image for acquiring an AE evaluation value is shot (preliminary shooting) before the main shooting, and the preliminary shooting is performed. The flow which calculates AE evaluation value based on an image is shown. This evaluation value detection method is a method suitable for an imaging apparatus such as a digital single-lens reflex camera or the like in which a subject light image is incident on the optical viewfinder and does not enter the imaging sensor 30 in the shooting preparation stage.

  First, in the state where the power switch 101 of the digital camera 1 is pressed and the power is turned on, it is confirmed whether an operation for starting shooting has been performed (step S111), and the release switch 102 is operated (for example, half-pressed). (YES in step S111), the preliminary shooting preparation operation is started (step S112).

  In step S112, the dynamic range control for the preliminary shooting is performed when the preliminary shooting is performed in order to calculate the AE evaluation value. In this dynamic range control, the imaging sensor 30 is controlled to have the maximum dynamic range so that the luminance of the subject can be sensed in a wide range. That is, in a digital single-lens reflex camera or the like, since there is only one chance of preliminary shooting before the main shooting, a wide dynamic range is set so that the luminance of any subject can be reliably detected.

  For this reason, the operation state of the image sensor 30 is controlled so that the image sensor 30 performs a logarithmic conversion output operation in the entire region. Specifically, when the release switch 102 is half-pressed, an instruction to shift to the preliminary photographing mode is issued from the overall control unit 50 to each unit, and the dynamic range control signal generation unit 521 receives the instruction, for example, as shown in FIG. The signal φVPS input to the source of the transistor T2 shown in FIG. 9 changes the difference between the Hi and Low voltages (in this case, ΔVPS is reduced, see FIG. 9). Control is performed so that the ratio of the logarithmic characteristic region is increased. Note that, from the viewpoint of securing a wide dynamic range, it is desirable that the entire region be a logarithmic characteristic region, but it is not always necessary to convert the entire region to a logarithmic characteristic region, and a linear characteristic region may remain to some extent. .

  Subsequently, exposure control for preliminary shooting is performed, and preliminary shooting is performed (step S113). Specifically, for example, the sensor exposure time control signal generation unit 522 generates a drive signal that sets the length of the time ΔS when the signal φVPS becomes the medium potential M in accordance with a predetermined exposure time, and generates the timing thereof. By sending it to the circuit 31, exposure control (exposure amount control) for preliminary photographing of the image sensor 30 is performed. In addition, adjustment of the shutter speed of the shutter 23 by the shutter driving unit 61 based on the control signal generated by the shutter control signal generation unit 523 and the aperture control unit 63 based on the control signal generated by the aperture control signal generation unit 525 Exposure control is also performed by adjusting the aperture 22. After such exposure control is performed, a preliminary image of a still image is performed. Based on the photographed preliminary photographed image, the AE evaluation value is calculated by the evaluation value detecting unit 405 (step S114). This AE evaluation value calculation step will be described in detail later. When the AE evaluation value is calculated, the preliminary shooting is finished (step S115), and exposure control based on the AE evaluation value is performed, and then the main shooting is started (step S116). Although the case where the AE evaluation value is acquired has been described above, the AF evaluation value and the white balance evaluation value can be acquired in the same manner.

(Step S1-2) Evaluation Value Detection Example from Moving Image FIG. 13 is a flowchart showing an operation example in the case of detecting an AE evaluation value or the like of a subject from a moving image continuously captured by the image sensor 30. . That is, when the digital camera 1 is in a shooting standby mode, in a moving image shooting mode, or when the imaging apparatus of the present embodiment is applied to a digital movie, all frame images captured by the imaging sensor 30 are captured. The flow which calculates AE evaluation value using is shown.

  First, it is confirmed whether or not an operation for starting shooting has been performed (step S121). For example, when the mode setting switch 105 is operated to enter the moving image shooting mode and the start of shooting is confirmed (YES in step S121), shooting of a moving image is performed. The process is started (step S122). Control values such as an imaging dynamic range, an exposure time, and an aperture at the start of imaging are set as initial setting values.

  Subsequently, an AE evaluation value is calculated by the evaluation value detection unit 405 based on the image captured in step S122 (step S123). Based on the detected AE evaluation value, the dynamic range control signal generation unit 521 changes the setting of the signal φVPS to control the dynamic range, and the shutter drive signal generation unit 523 and the aperture control signal generation unit 525 generate the dynamic range. The predetermined imaging AE control is performed by controlling the diaphragm by the control signal to be performed (step S124).

  Then, it is confirmed whether or not the photographing has been completed (step S125). If there is no photographing termination command (NO in step S125), the process returns to step S123 and the same AE evaluation value calculation and imaging AE control in step S124 are repeated. It is what That is, when moving image shooting is being performed, all of the captured images are used as evaluation images for AE evaluation value detection, and imaging AE control for the next shooting is performed based on the obtained AE evaluation values. The cycle of being performed is repeated. Note that not all of the photographed images are evaluated images, but a part of the photographed images (for example, a ratio of one to several frames of the photographed images) may be used as an evaluation image, and an AE evaluation value may be acquired from the evaluation image. .

  By the way, at the time of moving image shooting in the step S1-2, in order to smoothly control the shooting exposure state (exposure amount) or photoelectric conversion characteristic state (dynamic range) even when the luminance of the subject changes suddenly and frequently, it will be described later. When the gain (Gain) or the change amount (ΔVth) of the output level at the inflection point position of the photoelectric conversion characteristics reaches a predetermined value, the degree of change of the gain or change amount is controlled. Instead of using the actual gain and change values as they are, so-called hysteresis control is used to gradually change the gain and change amount so as to follow these actual gain and change changes. Is controlled insensitive (hereinafter referred to as insensitive control). The present embodiment is characterized in that such insensitivity control is performed during moving image shooting, which will be described in detail later.

(Step S1-3) Calculation of Evaluation Value Next, the step of calculating the evaluation value in the above flow (the above steps S114 and S123) will be described in detail. FIG. 14 is a functional block diagram for explaining the function of the evaluation value detection unit 405. The evaluation value detection unit 405 includes a divided photometry unit 4051, a histogram calculation unit 4052, and a saturation determination unit 4055.

  The split photometry unit 4051 performs photometry on a subject using a split photometry (multi-pattern photometry) method. That is, the division photometry unit 4051 divides a captured image obtained by imaging by the imaging sensor 30 into a predetermined number of regions (and sections), and the luminance in the captured image (each region or section) is an image signal (image data). It is to detect from. FIG. 15 is a schematic diagram illustrating a division state of an imaging region (photometric range) when performing division photometry with the imaging sensor. Reference numeral 330 denotes an imaging region (imaging region 330) obtained by imaging by the imaging sensor 30, and a subject is photographed (captured) in the imaging region 330. The imaging region 330 includes a large number of pixel information corresponding to the imaging elements constituting the imaging sensor 30, that is, luminance information of the subject. The imaging region 330 is divided into, for example, a central region that is the central portion of the imaging region 330 and a peripheral region that is a peripheral portion of the central region, and each of the central region and the peripheral region has a predetermined number of detection blocks (detection Divided into compartments). The central area is divided into 36 detection blocks such as A, B, C,... Z, AA, AB,... AJ blocks (A to AJ blocks). -It is divided into 16 detection blocks such as the 16th block. In the present embodiment, the subject imaged in the central area is referred to as the main subject (hereinafter, the central area is referred to as the main subject area 331), and the subject imaged in the peripheral area is referred to as the peripheral subject (hereinafter referred to as the peripheral object). The area is referred to as a peripheral subject area 332). Note that an area formed from the O, P, U, and V blocks in the central portion of the main subject area 331 is an AF area (AF area 333) where an AF evaluation value for focus control is detected. The luminance in the main subject area 331 (taken image) is referred to as main subject luminance, and the luminance in the peripheral subject area 332 is referred to as peripheral subject luminance.

  The histogram calculation unit 4052 calculates a main subject luminance histogram (distribution) for each of the A to AJ blocks, and uses the main subject luminance histogram for each of the A to AJ blocks, as shown in FIG. A main subject overall luminance histogram in the entire subject region 331 is calculated. Further, a peripheral subject luminance histogram for each of the first to sixteenth blocks is calculated, and a peripheral subject region 332 as shown in FIG. 16B is used by using the main subject luminance histogram for each of the first to sixteenth blocks. A luminance histogram of the entire surrounding subject is calculated.

  In addition, the histogram calculation unit 4052 calculates the luminance range of the entire main subject and the entire luminance range of the peripheral subject using the calculated main subject overall luminance histogram and surrounding subject overall luminance histogram. In this calculation, a cut-off is performed using a predetermined threshold, that is, luminance data having a frequency smaller than this threshold is not used. As shown in FIG. 16A, the main subject is cut off at the threshold D1, and the range of the minimum value L1 to the maximum value L8 of the luminance having a frequency equal to or higher than D1 is set as the entire main subject luminance range. Similarly, in the peripheral subject, as shown in FIG. 16 (b), the threshold D2 is cut off, and the range from the minimum luminance value L12 to the maximum value L19 having a frequency equal to or higher than D2 is defined as the entire main subject luminance range. To do. The “foot cut” by this threshold is performed in order to reduce an error due to noise or the like. Note that the brightness (image (brightness) data) of each brightness histogram shown in FIG. 16 is set to L1 to L19 and the like here for convenience of explanation, but actually, for example, when handling image data of 8 bits, for example. 256 levels (gradations), for example, L1 to L256.

  The histogram calculation unit 4052 includes an average luminance calculation unit 4053 and a maximum / minimum luminance calculation unit 4054. The average luminance calculation unit 4053 calculates the average luminance of the main subject for each of the A to AJ blocks and the average luminance of the peripheral subjects for each of the first to sixteenth blocks. This average luminance is calculated for each of R, G, and B colors. In the calculation of the average luminance, the main subject luminance histogram and the peripheral subject luminance histogram are calculated for each of the A to AJ blocks and the first to sixteenth blocks, respectively, and a predetermined threshold is set in the same manner as described above. The average luminance is obtained by performing “cutting” and averaging the luminance values after the cut.

  The maximum / minimum luminance calculation unit 4054 calculates the maximum / minimum luminance of the main subject for each of the A to AJ blocks and the maximum / minimum luminance of the peripheral subject for each of the first to sixteenth blocks. Similarly, in this case, the main subject luminance histogram or the peripheral subject luminance histogram calculated for each block is “cut off” at a predetermined threshold value, and the maximum or Calculate the minimum luminance.

  The histogram calculation unit 4052 calculates an entire region luminance histogram in the entire imaging region (imaging region 330) including the main subject region 331 and the peripheral subject region 332 to be used for saturation determination by a saturation determination unit 4055 described later. However, the whole area luminance histogram may be calculated by combining the main subject whole luminance histogram and the peripheral subject whole luminance histogram.

  The saturation determination unit 4055 determines whether or not the output of the image sensor 30 is saturated at the time of detecting the AE (AF, WB) evaluation value, based on the whole region luminance histogram calculated by the histogram calculation unit 4052. is there.

  FIG. 17 is a graph showing an example of the entire region luminance histogram when the output of the image sensor is saturated. Pmax in the figure indicates the sensor incident luminance (saturated luminance) when the image sensor 30 is at the saturation output level Vmax (the maximum value of the output level of the image sensor 30), and Pmaxth is saturated / unsaturated. The sensor incident luminance (luminance threshold value) with respect to the sensor output Vmaxth set as a predetermined threshold value for discrimination is shown. Similarly, Dth indicates a frequency (frequency threshold) that is set in advance as a threshold value for performing the determination of saturation / non-saturation. However, the Vmax may be a saturated output level as an actual maximum output level, or may be an arbitrarily set saturated output level (for example, an output level set somewhat lower than the actual maximum output level). Good.

  The saturation determination unit 4055 has a total frequency in the shaded area (referred to as a saturated area 342) indicated by reference numeral 342 in the entire area luminance histogram 341 that is equal to or higher than the luminance threshold Pmaxth and the frequency threshold Dth, that is, the total number of pixels (saturation). When the number of saturated pixels is equal to or greater than a predetermined number, it is determined that the output level of the image sensor 30 is saturated (if less than the predetermined number) It is not saturated). Saturation / non-saturation may be determined using only the frequency (number of pixels) of the saturation luminance Pmax.

  As described above, the evaluation value detection unit 405 performs split photometry, and calculates the average luminance, maximum / minimum luminance, luminance histogram, luminance range, etc. from luminance information (image data) in each detection block of the main subject and the surrounding subject area. Is detected as an AE (AF, WB) evaluation value. This evaluation value data is sent to a parameter calculation unit corresponding to various evaluation values of the calculation unit 510 via the information receiving unit 501, for example, an AE control parameter calculation unit 5110 if it is an AE evaluation value, and AF control if it is an AF evaluation value. If it is a WB evaluation value, it is output to the parameter calculation part 513 to the white balance control parameter calculation part 514, and various control parameters are calculated based on this evaluation value in each said calculation part.

(AE control parameter calculation step S2)
Subsequently, AE control by exposure amount control and dynamic range control based on the photoelectric conversion characteristics of the image sensor 30 according to the present embodiment will be described in detail below. FIG. 18 is a graph showing how the photoelectric conversion characteristics of the image sensor change when AE control is performed. FIG. 18A shows changes when exposure amount control is performed, and FIG. 18B shows dynamics. It is a figure which shows the change in the case of performing range control. 18A and 18B, the horizontal axis represents sensor incident luminance, the vertical axis represents sensor output, and the horizontal axis represents logarithmic coordinates (logarithmic value of sensor incident luminance). However, the sensor incident luminance indicates the luminance of the subject incident on the image sensor 30, and is hereinafter simply referred to as luminance.

Specifically, the exposure amount control and the dynamic range control are performed based on the following controls (A) and (B).
(A) Exposure amount control based on controlling the exposure time in the shutter 23 and / or the image sensor 30, that is, the opening time of the shutter 23 and / or the integration time of the image sensor 30, and / or the opening area of the aperture 23.
(B) Dynamic range control based on control of photoelectric conversion characteristics of the image sensor 30 (specifically, control of the inflection point position of the photoelectric conversion characteristics; see FIG. 22).

(Step S2-1) Calculation of Exposure Amount Control Parameter First, exposure amount control in the case of (A) will be described with reference to FIG. The photoelectric conversion characteristic 601 is the photoelectric conversion characteristic of the image sensor 30 stored in the photoelectric conversion characteristic information storage unit 516 at the time of obtaining the AE evaluation value. The photoelectric conversion characteristic 601 is divided into a linear characteristic region and a logarithmic characteristic region with an inflection point 603 (sensor output at this time is Vth) as a boundary. This photoelectric conversion characteristic 601 is an exposure amount control parameter for obtaining an exposure amount that changes to a photoelectric conversion property 602 that provides a predetermined sensor output with respect to a predetermined luminance for exposure amount setting (brightness for exposure amount setting). Exposure amount setting value), that is, an exposure time setting value for controlling the exposure time and an aperture setting value for controlling the aperture area of the aperture are calculated by the exposure amount control parameter calculation unit 511.

  Here, the sensor output value (sensor output at point 605) with respect to a predetermined luminance Lt1 (corresponding to the exposure amount setting luminance) in the linear characteristic region of the photoelectric conversion characteristic 601 is Vtarget (sensor output at point 606). In other words, the photoelectric conversion characteristic 601 is changed in the direction indicated by the arrow 608 (the direction of the arrow 608) so that the photoelectric conversion characteristic 601 becomes the photoelectric conversion characteristic 602 passing through the point 606 ( (At this time, the inflection point 603 is translated to the inflection point 604 and the value of the sensor output Vth does not change). Here, Vtarget is a target output that is a certain target of sensor output, and is a preset value. This Vtarget is stored in the exposure amount control parameter calculation unit 511 or the like.

  In this case, the sensor output at the luminance Lt1 increases from the sensor output at the point 605 of the photoelectric conversion characteristic 601 to the sensor output (Vtarget) at the point 606 of the photoelectric conversion characteristic 602, that is, a sensor for the same luminance. An exposure time setting value and an aperture setting value that can increase the exposure amount are calculated so that the output increases. From another viewpoint, the luminance corresponding to Vtarget changes from Lt2 (luminance at the point 607) to Lt1, that is, the exposure amount is such that the luminance for obtaining the sensor output of Vtarget is Lt1 smaller than Lt2. Increased exposure time setting values and aperture setting values are calculated by the exposure amount control parameter calculation unit 511. At this time, the opening time of the shutter 23 or the integration time by the image sensor 30 is increased based on the exposure time setting value and the aperture setting value, and the opening area of the aperture 22 is controlled to be increased.

  When the photoelectric conversion characteristic 601 changes to the photoelectric conversion characteristic 602, the luminance at Vmax changes (decreases) from Lm2 to Lm1, and the dynamic range decreases. Vmax is the maximum value of the sensor output in the image sensor 30, that is, the saturation output level. However, the Vmax may be a saturation output level as a physical maximum output level, or an arbitrarily set saturation output level (for example, an output level set somewhat lower than the physical maximum output level). It is good.

  In the case of FIG. 18A, the photoelectric conversion characteristic is changed in the direction of the arrow 608 in order to obtain Vtarget with respect to the exposure amount setting luminance (Lt1). (Direction) may be changed (moved). Further, when the photoelectric conversion characteristic at the time of obtaining the AE evaluation value has already obtained Vtarget with respect to the exposure amount setting luminance as described above, the photoelectric conversion characteristic is not changed (moved). However, in this case, even if the exposure time setting value and the aperture setting value at the time of the previous acquisition of the AE evaluation value become the same value, the current exposure time setting value and the aperture setting value are calculated. Good.

(Step S2-2) Calculation of Dynamic Range Control Parameter Next, dynamic range control in the case of (B) will be described with reference to FIG. The photoelectric conversion characteristic 701 is the photoelectric conversion characteristic of the imaging sensor 30 stored in the photoelectric conversion characteristic information storage unit 516 when the AE evaluation value is acquired. The photoelectric conversion characteristic 701 is divided into a linear characteristic region and a logarithmic characteristic region with an inflection point 703 (sensor output at this time is Vth1) as a boundary. The dynamic range control parameter (photoelectric conversion characteristic setting value) is a photoelectric conversion characteristic 702 in which the photoelectric conversion characteristic 701 provides a predetermined sensor output with respect to predetermined dynamic range setting luminance (dynamic range setting luminance). The control value of the photoelectric conversion characteristic that can be changed, specifically, the setting relating to the position of the inflection point (inflection point 704) in the photoelectric conversion characteristic (702) after the change (output level for the inflection point). Calculated as a value. This photoelectric conversion characteristic setting value is calculated by the dynamic range control parameter calculation unit 512.

  Here, the luminance Lm20 set as the maximum luminance in the dynamic range is set as the subject luminance for dynamic range setting, and the value of the sensor output with respect to the luminance Lm20 is the saturation output level Vmax (shown in FIG. 18A). The photoelectric conversion characteristic curve 702 is calculated so as to be the same as Vmax. In other words, the photoelectric conversion characteristic 701 is changed in the direction indicated by the arrow 705 (in the direction of the arrow 705) so as to become the photoelectric conversion characteristic 702 passing through the point 704. At this time, the inflection point 703 moves to the inflection point 704, and the sensor output for the inflection point also changes from Vth1 to Vth2.

  In this case, a photoelectric conversion characteristic setting value is calculated such that the sensor output at luminance Lm20 is reduced from the sensor output at point 706 (Vover: sensor output value exceeding Vmax) to the sensor output at point 707 (Vmax). . From another point of view, the photoelectric conversion characteristic setting value such that the maximum luminance capable of obtaining the sensor output Vmax increases from Lm10 (luminance at the point 708) to Lm20 (expands the dynamic range) Calculated by the dynamic range control parameter calculation unit 512.

  Note that when the photoelectric conversion characteristic 701 changes to the photoelectric conversion characteristic 702, the luminance in Vtarget changes (increases) from Lt10 to Lt20, and the exposure amount decreases. However, the Vtarget shown in FIG. 18B is set to explain the change in the exposure amount, and may be different from the value of Vtarget shown in FIG.

  In the case of FIG. 18B, in order to obtain Vmax with respect to the dynamic range setting luminance (Lm20), the photoelectric conversion characteristic is changed in the arrow 705 direction. The direction may be changed. In addition, when the photoelectric conversion characteristic at the time of obtaining the AE evaluation value is such that Vmax is obtained with respect to the dynamic range setting luminance as described above, the photoelectric conversion characteristic is not changed. However, in this case, even if it becomes the same value as the photoelectric conversion characteristic setting value at the time of the previous AE evaluation value acquisition, the configuration in which the current photoelectric conversion characteristic setting value is calculated may be used.

  As described above, the exposure amount setting luminance is photographed in the linear characteristic region of the photoelectric conversion characteristic and output at a predetermined sensor output level by the AE control by the exposure amount control of (A) and the dynamic range control of (B). At the same time, it is possible to shoot with the dynamic range setting brightness (here, the maximum brightness of the subject; the maximum brightness in the dynamic range) being equal to or lower than the sensor saturation output level.

(Detailed explanation of exposure amount control parameter calculation method)
Here, in the case of the exposure amount control of FIG. 18A, the exposure amount control parameter (exposure time setting value and the exposure time control value) by the exposure amount control parameter calculation unit 511 based on the AE evaluation value detected by the evaluation value detection unit 405. The calculation of (aperture setting value) will be described more specifically.

  FIG. 19 is a diagram for explaining an example of a calculation method for causing the sensor output value to be Vtarget with respect to the luminance Lt1 (exposure amount setting luminance) in FIG. A photoelectric conversion characteristic α1 in the figure is a photoelectric conversion characteristic at the time of obtaining the AE evaluation value, and is divided into a linear characteristic region 612 and a logarithmic characteristic region 613 with an inflection point 611 (sensor output corresponding thereto is Vth) as a boundary. . The photoelectric conversion characteristic β1 indicates a photoelectric conversion characteristic when the logarithmic characteristic region 613 in the photoelectric conversion characteristic α1 is converted into a linear characteristic (linear characteristic region 614), that is, all of which become a linear characteristic region.

  LtLin at point A shown in FIG. 19 is the average luminance (linear characteristic region average luminance) in the linear characteristic region 612 of the photoelectric conversion characteristic α1, and the sensor output for this luminance LtLin is VtLin. LtLog at point B is the average luminance (logarithmic characteristic region average luminance) in the logarithmic characteristic region 613 of the photoelectric conversion characteristic α1, and the sensor output for this luminance LtLog is VtLog. First, the point B for LtLog in the logarithmic characteristic region 613 of the photoelectric conversion characteristic α1 moves to the point C on the linear characteristic region 614, that is, the sensor output value (VtLog) for LtLog in the logarithmic characteristic region 613 is a linear characteristic. Data conversion is performed so as to be the value (VtLogLin) in the area 614 (this allows each data in the photoelectric conversion characteristic α1 to be handled in a unified manner in the linear characteristic area). Data conversion from the logarithmic characteristic region 613 (photoelectric conversion characteristic α1) to the linear characteristic region 614 (photoelectric conversion characteristic β1) is performed using the LUT stored in the LUT storage unit 518. Then, the sensor output VtAve at the point D is calculated from the VtLin at the point A and the VtLogLin at the point C by the following formula. Note that the luminance LtAve in VtAve corresponds to Lt1 as the exposure amount setting luminance shown in FIG.

VtAve = (VtLin * k1) + (VtLogLin * (1-k1))
However, k1 = m / (m + n)
m: the total number of pixels used when calculating the luminance LtLin of the point A n: the total number of pixels used when calculating the luminance LtLog of the B point In this way, the values of VtLin and VtLogLin are calculated from the values of LtLin and LtLog, VtAve is calculated from the values of VtLin and VtLogLin.

  Next, the exposure gain A gain such that this VtAve becomes the V target shown in FIG. 18A, the exposure time gain Gt based on the exposure gain A and the aperture gain Gs. Further, the exposure time T2 and the aperture area S2 of the stop based on the amplification factors Gt and Gs are calculated by the following equations. However, the calculation of the values of Gt and Gs using the respective equations is determined by case classification according to the flowchart shown in FIG.

Gain = Vtarget / VtAve
Gt * Gs = Gain

≪Formula for calculating amplification factor for exposure time≫
Tmax / T1 = Gtmax (maximum amplification factor of exposure time)
Tmin / T1 = Gtmin (minimum gain of exposure time)
Gain / Gtmax = GGtmax (amplification factor to compensate for the shortage at the maximum amplification factor)
Gain / Gtmin = GGtmin (amplification factor to compensate for the shortage at the minimum amplification factor)
T2 = T1 * Gt
T1: Exposure time when AE evaluation value is detected
T2: Exposure time after AE correction
Tmax: Maximum exposure time of the image sensor 30
Tmin: Minimum exposure time of the image sensor 30

≪Formula for calculating the gain related to the aperture≫
Smax / S1 = Gsmax (maximum aperture gain)
Smin / S1 = Gsmin (minimum aperture gain)
Gain / Gsmax = GGsmax (amplification factor to compensate for the shortage at the maximum amplification factor)
Gain / Gsmin = GGsmin (amplification factor to compensate for the shortage at the minimum amplification factor)
S2 = S1 * Gs
However, S1: Aperture aperture area when AE evaluation value is detected
S2: Aperture aperture area after AE correction
Smax: Maximum aperture ratio of the diaphragm 23
Smin: Minimum aperture ratio of the diaphragm 23

  As shown in the flowchart of FIG. 20, first, when VtAve is the same value as the value of Vtarget, that is, the exposure amount gain Gain = 1.0, the exposure amount control (change of the exposure amount control parameter) is performed. If not necessary (YES in step S11), the exposure time gain Gt = 1.0 and the aperture gain Gs = 1.0 (step S12), and the exposure time and aperture aperture area are not changed. When the gain Gain is not 1.0 (NO in step S11), and Gain> 1.0 (YES in step S13) and Gain> Gtmax is not satisfied (NO in step S14), that is, Gain is If the exposure amount control is larger than 1.0 and the exposure amount gain Gain can be dealt with by the exposure time gain Gt (the gain Gt below the maximum gain Gtmax), Gt = Gain. And Gs = 1.0 (step S15).

  In Step S13, if Gain> 1.0 is not satisfied (NO in Step S13) and Gain <Gtmin is not satisfied (NO in Step S16), Gain is smaller than 1.0 as in Step S15. Since the exposure amount needs to be controlled and the exposure amount amplification factor Gain can be handled by the exposure time amplification factor Gt (amplification factor Gt greater than or equal to the minimum amplification factor Gtmin), Gt = Gain and Gs = 1.0. (Step S17).

When Gain> Gtmax is satisfied in step S14 (YES in step S14) and Gsmax> GGtmax is satisfied (YES in step S18), Gt = Gtmax and Gs = GGtmax are satisfied (step S19). In this step S19, the amplification factor Gain of the exposure amount becomes a value larger than the maximum amplification factor Gtmax of the exposure time, and without changing the aperture amplification factor Gs (Gs = 1.0), Since the amplification rate Gt alone cannot be used, the shortage of the amplification rate of Gt relative to the gain is handled (supplemented) by changing the aperture amplification rate Gs. However, as the value of the aperture gain Gs, the gain GGtmax for compensating for the shortage of the exposure time with the maximum gain Gtmax is used. This is because the amplification factor GGtmax is smaller than the maximum aperture amplification factor Gsmax (it is not necessary to use the maximum aperture amplification factor Gsmax), and therefore the amplification factor GGtmax related to the exposure time is used. This saves the effort of calculating the value (amplification factor) for controlling the diaphragm using the formula for calculating the amplification factor for the diaphragm.

  If Gain <Gtmin in step S16 (YES in step S16) and Gsmin <GGtmin (YES in step S21), Gt = Gtmin and Gs = GGtmin are satisfied (step S22). In this case as well, as in step S19, the gain of the exposure amount is smaller than the minimum gain Gtmin of the exposure time, and the aperture gain Gs is not changed (Gs = 1.0). ) Since it is not possible to deal with only the gain Gt of the exposure time, the shortage of the gain of Gt with respect to Gain is dealt with by changing the gain Gs of the aperture. However, an amplification factor GGtmin for compensating for the shortage of the minimum amplification factor Gtmin of the exposure time is used as the value of the amplification factor Gs of the diaphragm. This is because the gain GGtmin is smaller than the minimum aperture gain Gsmin (it is not necessary to use the minimum aperture gain Gsmin), and therefore the gain GGtmin related to the exposure time is used. In this case as well, the effort to calculate the value (amplification factor) for controlling the diaphragm using the formula for calculating the amplification factor for the diaphragm is saved.

  In step Se8, if Gsmax> GGtmax is not satisfied (NO in step S18), Gt = Gtmax and Gs = Gsmax are satisfied (step S20). In step S21, when Gsmin <GGtmin is not satisfied (NO in step S21), Gt = Gtmin and Gs = Gsmin are satisfied (step S23). In step S20, when the gain GGtmax is equal to or larger than the maximum aperture gain Gsmax, the maximum gain Gsmax is used as the aperture gain Gs. Similarly, in step S23, when the gain GGtmin is equal to or less than the minimum gain Gsmin of the aperture, the minimum gain Gsmin is used as the value of the aperture gain Gs.

  In the present embodiment, as shown in the flowchart of FIG. 20, when selecting a control parameter for obtaining the exposure amount gain Gain, priority is given to the exposure time gain Gt (exposure time control). (Prioritized) may be configured to prioritize the aperture gain Gs (prioritize aperture control). In this embodiment, the amplification factors Gt and Gs are calculated for one exposure amount setting luminance (Lt1), but the same calculation is performed for two or more exposure amount setting luminances. In this case, the average value or the maximum value or the minimum value of the amplification factors (Gt, Gs) calculated respectively may be used.

  In this way, the amplification factors Gt and Gs are calculated, and the exposure time T2 after AE correction and the aperture area S2 of the stop after AE correction are calculated from the Gt and Gs, respectively. Then, a setting value (exposure time setting value) for the image sensor 30 and the shutter 23 according to T2 and S2, or a setting value (aperture setting value) for the aperture 22 is calculated by data conversion using the LUT, respectively. Then, the exposure time setting value and the aperture setting value obtained by the data conversion are stored in the photoelectric conversion characteristic information storage unit 516 (or the exposure time setting value and the aperture setting value at the time of the previous AE evaluation value acquisition are The configuration may be such that it is updated with the newly obtained same set value (the same applies to the following photoelectric conversion characteristic set values).

  Note that the shutter control signal generation unit 523 and the aperture control signal generation unit 525 are based on the exposure time setting value and the aperture setting value calculated by the exposure amount control parameter calculation unit 511, respectively. A control signal for the shutter drive unit 61 and the aperture drive unit 63 is generated such that (integration time) becomes T2 or the aperture area of the aperture 22 becomes S2.

  Here, a specific calculation method of the sensor output level VtLin corresponding to the linear characteristic region average luminance LtLin and the logarithmic characteristic region average luminance LtLog shown in FIG. 19 will be described. First, a method for calculating the sensor output level VtLin corresponding to the linear characteristic region average luminance LtLin will be described. Based on the luminance information of the subject detected by each detection block (A to AJ blocks) in the main subject region 331 shown in FIG. 15, the average luminance (referred to as block linear average luminance) in the linear characteristic region for each detection block is calculated. calculate. The calculation of the block linear average luminance is performed using an average value (referred to as a color linear average value) of linear characteristic regions of the three colors R, G, and B. That is, the color linear average values of R colors obtained from the A to AJ blocks are calculated as AveRA, AveRB,... ... AveGAJ, AveBA, AveBB, ... AveBAJ are calculated. Then, using these color linear average values of RGB colors, the block linear average luminance for each of the A to AJ blocks is calculated by the following equation for performing color space conversion. For example, if the block linear average luminance for the A block is AveYA, AveYA can be obtained by the following equation.

AveYA = AveRA * K1 + AveGA * K2 + AveBA * K3
However, K1, K2, K3: coefficients used for color space conversion from RGB to YCbCr, for example, K1 = 0.22989, K2 = 0.5866, and K3 = 0.1145.

  The same calculation is performed for the other B to AJ blocks, and as a result, block linear average luminances AveYA, AveYB,... AveYAJ for each of the A to AJ blocks are calculated. Further, an average value of these block linear average luminances AveYA, AveYB,... AveYAJ is calculated. When this average value is MainY, MainY becomes the sensor output level VtLin corresponding to the linear characteristic region average luminance LtLin.

  On the other hand, the calculation of the sensor output level VtLog corresponding to the logarithmic characteristic region average luminance LtLog is performed in the same manner as in the case of the linear characteristic region average luminance LtLin. That is, based on the luminance information of the subject detected by the A to AJ blocks in the main subject region 331 shown in FIG. 15, the average luminance (referred to as block logarithmic average luminance) in the logarithmic characteristic region for each detected block is calculated. This block logarithmic average luminance is calculated using the average value (referred to as the color logarithmic average value) of the logarithmic characteristic areas of the three colors R, G and B. That is, the average logarithm values of the R colors obtained from the A to AJ blocks are respectively calculated as AveRLogA, AveRLogB,... ... AveGLogAJ, AveBlogA, AveBlogB, ... AveBlogAJ.

  By the way, the color logarithm average value of each color of RGB in these logarithmic characteristic areas is temporarily converted into values in the linear characteristic area using the LUT stored in the LUT storage unit 518, and is set as linear data. Using the color space conversion formula as described above, the block logarithmic average brightness AveYLogA, AveYLogB,... AveYLogAJ is calculated for each of the A to AJ blocks. Further, the average value of these block logarithmic average luminances AveYLogA, AveYLogB,... AveYLogAJ is calculated. When this average value is MainYLog, MainYLog becomes the sensor output level VtLogLin corresponding to the linear characteristic region average luminance LtLog. The color linear average value (color logarithm average value) of each color in each of the A to AJ blocks calculates a luminance histogram in the linear characteristic region (logarithmic characteristic region) for each of the A to AJ blocks and also calculates the luminance histogram. May be calculated by averaging the luminance values after the cut.

  By the way, the specific calculation method of the gain Gain (exposure amount control parameter) described in FIGS. 19 and 20 may be as follows. First, the maximum luminance value (referred to as the color maximum value) for each RGB color in each of the A to AJ blocks in the main subject area 331 is calculated. That is, the maximum value of R color is calculated as MaxRA, MaxRB,... MaxRAJ, and similarly, the maximum value of G color and B color is set as MaxGA, MaxGB,.・ ・ Calculated as MaxBAJ. Then, by using the maximum value of the luminance for each color, the block maximum luminance for each of the A to AJ blocks is calculated by the following equation for performing color space conversion. For example, when the block maximum luminance for the A block is MaxYA, MaxYA can be obtained by the following equation.

MaxYA = MaxRA * K1 + MaxGA * K2 + MaxBA * K3
However, similarly to the above, for example, K1 = 0.22989, K2 = 0.5866, and K3 = 0.1145. The same calculation is performed for the other B to AJ blocks, and as a result, the block maximum brightness MaxYA, MaxYB,... MaxYAJ for each of the A to AJ blocks is calculated. Further, the maximum value of these block maximum luminances MaxYA, MaxYB,... MaxYAJ (referred to as the maximum luminance value in the main subject region 331) is calculated. When this maximum luminance value is MaxY, MaxY becomes a sensor output level VtAve2 corresponding to the luminance Ltmax shown in FIG.

  Similarly, the minimum luminance value (color minimum value) of each RGB color in each of the A to AJ blocks is set to MinRA, MinRB, ... MinRAJ, and MinGA, MinGB, ... MinGAJ, and MinBA, MinBB,. ... Calculated as MinBAJ, and using these minimum values, block minimum luminance MinYA, MinYB,... Further, the minimum value of the entire block minimum brightness MinYA, MinYB,... MinYAJ (referred to as the minimum brightness value in the main subject area a2) is calculated. When this minimum luminance value is MinY, MinY becomes a sensor output level VtAve1 corresponding to the luminance Ltmin shown in FIG.

  However, among the maximum color value and the minimum color value, the values in the logarithmic characteristic region are similarly converted to the linear characteristic region using the LUT and then subjected to color space conversion. The maximum color value and the minimum color value of each color in each A to AJ block are calculated for each A to AJ block, and the luminance histogram is “cut off”. You may calculate from a luminance value.

  Then, as shown in FIG. 21, an amplification factor (Vtarget1 / VtAve1; referred to as a first amplification factor) is calculated such that the sensor output value VtAve1 with respect to the luminance Ltmin becomes Vtarget1 that is a preset target output value. An amplification factor (Vtarget2 / VtAve2; referred to as a second amplification factor) is calculated so that the sensor output value VtAve2 with respect to the luminance Ltmax becomes Vtarget2, which is a preset target output value, and the smaller of the two amplification factors. 20 is selected, and the selected amplification factor is set as the amplification factor Gain of the exposure amount, as shown in FIG. It may be a configuration.

  As described above, the larger one of the first and second amplification factors may be selected, or the first or second amplification factor is not selected by comparing the first and second amplification factors. A configuration may be used in which only two amplification factors are calculated and used. Moreover, the structure using what averaged these 1st and 2nd amplification factors may be used. The values of Vtarget1 and Vtarget2 are stored in the exposure amount control parameter calculation unit 511 and the like.

  Further, the minimum luminance value MinY and the maximum luminance value MaxY may be calculated from the entire luminance histogram of the entire A to AJ block obtained by collecting the luminance histograms of the respective A to AJ blocks. In this case, the “luminance range” in the entire luminance histogram is calculated by similarly performing “cutting off”, and the minimum luminance value MinY and the maximum luminance value MaxY are calculated from the luminance range. At the time of the calculation, the minimum luminance value MinY = maximum luminance value MaxY−luminance range, or the maximum luminance value MaxY = minimum luminance value MinY + luminance range. The other luminance value may be obtained.

(Detailed explanation of dynamic range control parameter calculation method)
Subsequently, the dynamic range control parameter (photoelectric conversion characteristic setting value) by the dynamic range control parameter calculation unit 512 based on the AE evaluation value detected by the evaluation value detection unit 405 in the case of the exposure amount control in FIG. ) Will be described more specifically.

  FIG. 22 is a graph for explaining a method of calculating the position of the inflection point in the photoelectric conversion characteristic when calculating the dynamic range control parameter. FIG. 22A shows a predetermined sensor output with respect to the luminance Lmax. It is a figure at the time of changing a photoelectric conversion characteristic, (b) is a figure at the time of modeling a photoelectric conversion characteristic. First, in FIG. 22A, the luminance Lmax is the dynamic range setting luminance, and corresponds to, for example, Lm20 shown in FIG. A photoelectric conversion characteristic α2 indicates a photoelectric conversion characteristic before change having an inflection point 711, and a photoelectric conversion characteristic β2 indicates a photoelectric conversion characteristic after change having an inflection point 712. Vmax2 indicates the saturation output level (maximum output level) of the image sensor 30. The photoelectric conversion characteristic is changed so that the value of the sensor output with respect to the luminance Lmax shifts from Vmax1 at the point E on the photoelectric conversion characteristic α2 to Vmax2 at the point F on the photoelectric conversion characteristic β2. In this case, the photoelectric conversion characteristic α2 changes from the sensor output Vth1 at the inflection point 711 to the state of the photoelectric conversion characteristic β2 according to the inflection point change amount ΔVth that becomes the sensor output Vth2 at the inflection point 712.

  By the way, the Vth2 is calculated from the sensor output ΔVmax (= Vmax2−Vmax1) between the points E and F. This will be described. As shown in FIG. 22B, when the linear characteristic region and the logarithmic characteristic region in the photoelectric conversion characteristics α2 and β2 are modeled (graphed) and expressed by a function (formula), they are expressed as follows.

Function modeling the linear characteristic region: V = K2 * L (common to photoelectric conversion characteristics α2 and β2)
A function modeling the logarithmic characteristic region: V = K1 * ln (L) + Wα (photoelectric conversion characteristic α2)
: V = K1 * ln (L) + Wβ (photoelectric conversion characteristic β2)
However, K1 and K2 are constants, L is the sensor incident luminance (horizontal axis coordinates in FIG. 22), and Wα and Wβ are intercepts.

  Here, since ΔVmax is expressed as ΔVmax = Wβ−Wα, the equation of V = K1 * ln (L) + Wβ is expressed as V = K1 * ln (L) + Wα + ΔVmax. The sensor output value at the intersection 713 between this equation and the equation V = K2 * L is Vth2. Therefore, the simultaneous expression of the two formulas for calculating the intersection point 713 (coordinates thereof); a value of “L” that satisfies K1 * ln (L) + Wα + ΔVmax = K2 * L, that is, the luminance L shown in FIG. Vth2 of the sensor output corresponding to the luminance L is calculated. However, when the calculated output level Vth2 is higher than the saturated output level Vmax2, the imaging sensor 30 has a photoelectric conversion characteristic only in a linear characteristic region without a logarithmic characteristic region.

  Then, a setting value for the imaging sensor 30 corresponding to Vth2 calculated as described above, that is, a setting value (photoelectric conversion characteristic) for changing the photoelectric conversion characteristic so that the inflection point of the photoelectric conversion characteristic becomes the position of Vth2. Setting value) is calculated by data conversion of the Vth2 using the LUT. Then, the photoelectric conversion characteristic setting value corresponding to the inflection point Vth2 obtained by the data conversion is stored in the photoelectric conversion characteristic information storage unit 516. The dynamic range control signal generation unit 521 has the photoelectric conversion characteristic (the inflection point position) of the imaging sensor 30 based on the photoelectric conversion characteristic setting value calculated by the dynamic range control parameter calculation unit 512 as described above. A control signal for the timing generation circuit 31 that changes is generated.

  Incidentally, a specific calculation method of the sensor output level Vmax1 corresponding to the luminance Lmax which is the dynamic range setting luminance shown in FIG. 22 is as follows. First, similar to the calculation of the sensor output level VtLog corresponding to the logarithmic characteristic area average luminance LtLog in FIG. 19, the logarithmic characteristic area average luminance in the main subject area 331 (A to AJ blocks) shown in FIG. 15 is calculated. The logarithmic characteristic area average luminance in the peripheral subject area 332 (first to sixteenth blocks) is calculated in the same manner as in the main subject area 331. Then, the logarithmic characteristic area average luminance calculated in the main subject area 331 and the peripheral subject area 332 is compared to select the larger logarithmic characteristic area average luminance, and the sensor output level corresponding to the selected logarithmic characteristic area average luminance is selected. Is Vmax1.

  Note that, in the main subject region 331 and the peripheral subject region 332, not only the logarithmic characteristic region average luminance but also the sensor output corresponding to the linear characteristic region average luminance similar to the linear characteristic region average luminance LtLin in FIG. The total characteristic area average luminance obtained by averaging the linear characteristic area average luminance and the logarithmic characteristic area average luminance is calculated for each of the subject area 331 and the peripheral subject area 332, and the larger of the total characteristic area average luminance in each of these areas is calculated. The sensor output corresponding to the luminance Lmax may be used. However, if the luminances are the same, any luminance value may be used as a sensor output corresponding to the luminance Lmax (and so on).

  In addition, the sensor output corresponding to the luminance Lmax may be obtained from the logarithmic characteristic area average luminance (or the total characteristic area average luminance combined with the linear characteristic area average luminance) only in the main subject area 331, or a peripheral object. The sensor output corresponding to the luminance Lmax may be obtained from the logarithmic characteristic area average luminance only in the area 332 (or the total characteristic area average luminance combined with the linear characteristic area average luminance).

  Further, a specific method for calculating the sensor output corresponding to the luminance Lmax may be as follows. That is, first, similarly to the calculation of the sensor output corresponding to the maximum luminance value Ltmax (MaxY) in FIG. 21, the maximum luminance value in the main subject region 331 is calculated, and the maximum luminance value in the peripheral subject region 332 is calculated. In the same manner as in the main subject area 331, the calculation is performed. Then, the maximum luminance values calculated in the main subject region 331 and the peripheral subject region 332 are compared to select a larger maximum luminance value, a sensor output level corresponding to the selected maximum luminance value is obtained, and this output level The luminance corresponding to is defined as luminance Lmax. The sensor output corresponding to the luminance Lmax may be obtained from the maximum luminance value only in the main subject region 331, or the sensor output corresponding to the luminance Lmax may be obtained from the maximum luminance value only in the peripheral subject region 332. It may be a configuration.

  Incidentally, the control of the photoelectric conversion characteristic (inflection point position) based on the photoelectric conversion characteristic setting value as shown in FIG. 22 is actually the saturation determination unit 4055 (see FIG. 14) as described in FIG. ) Is performed when it is determined that the output level of the image sensor 30 is not saturated, and when it is determined that the output level is saturated, according to the number of saturated pixels, FIG. As shown in FIG. 5, the photoelectric conversion characteristics are changed so that the output level at the inflection point is lowered by ΔVth, that is, the dynamic range is expanded by the image sensor 30 to enable shooting on the higher luminance side. Let Here, the photoelectric conversion characteristic α3 is changed to the photoelectric conversion characteristic β3. However, the value of ΔVth is set so as to increase as the number of saturated pixels increases as shown in FIG. 23B, and ΔVth corresponding to the number of saturated pixels is calculated by data conversion by LUT. It is the structure performed by. Then, the calculated ΔVth is subtracted from the output level Vth1 at the inflection point 721 of the photoelectric conversion characteristic α3 at the time of calculation at the time of the AE evaluation value, so that the photoelectric conversion characteristic β3 after the change (compared with the photoelectric conversion characteristic α3). Thus, the output level Vth2 at the inflection point 722 of the characteristic that the dynamic range becomes wide) is obtained. Further, a set value (photoelectric conversion characteristic set value) for the image sensor 30 according to the obtained Vth2 is calculated by data conversion by the LUT. Then, the photoelectric conversion characteristic setting value corresponding to the inflection point Vth2 obtained by the data conversion is stored in the photoelectric conversion characteristic information storage unit 516.

(Step S3-1) Setting of exposure amount control parameter If the exposure amount control parameter for AE control is calculated by the method described in step S2-1 above, AE control is performed based on the exposure amount control parameter. And an actual imaging operation is performed. That is, when shooting a still image, the main shooting is performed after AE control is performed based on the AE evaluation value acquired from the preliminary shooting image. When shooting a moving image, for example, moving image shooting is performed sequentially after AE control based on the AE evaluation value acquired from the shot image shot immediately before is performed.

  Specifically, the exposure amount control parameter calculated by the exposure amount control parameter calculation unit 511 of the overall control unit 50 is input to the control signal generation unit 520 to generate a drive signal for performing an actual exposure amount control operation. A control signal for operating the generation circuit 31 and the drive unit 60 is generated by each unit of the control signal generation unit 520. That is, the sensor exposure time control signal generation unit 522 of the control signal generation unit 520 generates a control signal of the image sensor 30 so as to ensure an intended exposure time according to the exposure amount control parameter, and timings this. Transmit to the generation circuit 31. For example, in the timing chart shown in FIG. 9, the control signal here is a signal that sets the time ΔS when the signal φVPS to the image sensor 30 becomes the medium potential M to an appropriate time according to the exposure amount control parameter (that is, photo Signal for setting the integration time from the time t1 at which the parasitic capacitance of the diode PD is reset to the time t2 at which the readout of the video signal of the next frame is started to an appropriate time). The timing generation circuit 31 generates a timing signal for controlling the exposure time of the image sensor 30 according to the input drive signal, and drives the image sensor 30.

  Similarly, the shutter control signal generation unit 523 generates a control signal for setting the shutter speed (shutter opening time) of the shutter 23 in accordance with the exposure time based on the exposure amount control parameter. This control signal is sent to the shutter drive unit 61 of the drive unit 60. The shutter drive unit 61 generates a drive signal for the shutter 23 based on the control signal, and performs the shutter opening operation of the shutter 23 according to the exposure amount control parameter. Let it be done.

  Further, the aperture control signal generator 525 similarly generates a control signal for setting the aperture area of the aperture 22 based on the exposure amount control parameter. This control signal is sent to the aperture drive unit 63, and the aperture drive unit 63 generates a drive signal for the aperture 22 based on the control signal, and performs an aperture area setting operation of the aperture 22 according to the exposure amount control parameter.

  As described above, the exposure amount control (integration time control) includes the drive control of the image sensor 30, the shutter speed control, and the aperture control by the timing generation circuit 31, and these three controls are all performed. However, from the viewpoint of achieving control at high speed, it is desirable to prioritize the electronic circuit exposure amount control by the timing generation circuit 31 as described based on the flowchart of FIG.

(Step S3-2) Setting of Dynamic Range Control Parameter On the other hand, when performing AE control according to the present embodiment by dynamic range control, the dynamic range control parameter calculated by the dynamic range control parameter calculation unit 512 of the overall control unit 50 is The control signal generation unit 520 inputs the control signal, and the dynamic range control signal generation unit 521 generates a control signal for performing an actual dynamic range control operation.

  That is, the dynamic range control signal generation unit 521 outputs an output level point at which the photoelectric conversion characteristic is switched from the linear characteristic region to the logarithmic characteristic region according to the photoelectric conversion characteristic setting value of the imaging sensor 30 calculated by the dynamic range control parameter calculation unit 512. A control signal for the image sensor 30 for adjusting (inflection point) is generated and transmitted to the timing generation circuit 31. The control signal here is, for example, a control signal for appropriately setting the signal φVPS for the imaging sensor 30 in accordance with the calculated dynamic range control parameter in the timing chart shown in FIG.

  That is, the inflection point is changed by controlling the height of the voltage VPH in the φVPS or the length of the time ΔT. A control signal for controlling is generated and transmitted to the timing generation circuit 31. The timing generation circuit 31 generates a timing signal for controlling the dynamic range of the image sensor 30 in accordance with the input control signal, and actually drives the image sensor 30 as a predetermined photoelectric conversion characteristic state. is there.

  Here, the insensitive control with respect to the change of the gain (Gain) relating to the exposure amount control at the time of moving image shooting and the change amount (ΔVth) of the inflection point of the photoelectric conversion characteristic relating to the dynamic range control will be described in detail below.

(About insensitivity control of gain (Gain) regarding exposure setting)
FIG. 24 is a graph showing a change in amplification factor related to exposure amount control during moving image shooting. GainTh1 shown on the vertical axis of FIG. 24 is a predetermined value (threshold value) of the gain that is the insensitive control start level when the gain (gain) increases, and GainTh2 is when the gain decreases. This is a predetermined value (threshold value) of the amplification factor, which is the insensitive control start level. Similarly, GainTh10 shown on the vertical axis is a predetermined value (threshold value) of an amplification factor that is an end level of the insensitive control starting from the above GainTh1, and GainTh20 is an end level of the insensitive control starting from the above GainTh2. Is a predetermined value (threshold value) of the amplification factor. However, GainTh10 is positioned at a low level with respect to GainTh1, and GainTh20 is positioned at a high level with respect to GainTh2 (when viewed in a direction in which the amplification factor decreases). Suppose that

  In FIG. 24, when the amplification factor changes with time (on the horizontal axis), the insensitivity control is started when the amplification factor becomes larger than GainTh1 at time t1, and when the amplification factor becomes smaller than GainTh10 at time t2. This insensitivity control is terminated (insensitivity control is performed in the change section 801 at times t1 to t2 in FIG. 24). Further, the insensitivity control starts when the gain changes with time and becomes an amplification factor smaller than GainTh2 at time t3, and the insensitivity control ends when the amplification factor becomes larger than GainTh20 at time t4. (Insensitive control is performed in the change section 802 at times t3 to t4 in FIG. 24).

  Here, an actual operation in the insensitive control from when the amplification factor exceeds GainTh1 (GainTh2) to GainTh10 (GainTh20) will be described by taking the case of the change interval 801 as an example.

FIG. 26 is an enlarged view showing an example of a change with time of the gain value in the change section 801. However, the number of points (plots) indicating the change in amplification factor between times t1 and t2 or between GainTh10 and GainTh1 in this figure (interval between points) is shown smaller (larger) for convenience of explanation. Actually, a larger number (small interval) may be used.
As shown in FIG. 26, the gain obtained when the exposure amount control parameter (exposure time setting value and aperture setting value) is calculated by the exposure amount control parameter calculation unit 511 is represented by a change point 811, a change point 812,. When the gain becomes a gain indicated by a change point 813 exceeding GainTh1, instead of the gain at the change point 813, a gain Th1 corresponding to a predetermined change amount ΔGain of the gain indicated by reference numeral 814 is obtained. The amplification factor at the increased change point 815 is set (adopted) as an amplification factor for actual exposure control. Hereinafter, the predetermined change ΔGain in the amplification factor will be referred to as amplification factor variation ΔGain as appropriate.

  Next, it is calculated based on the AE evaluation value acquired from the photographed image (evaluation image) at the amplification point (based on the setting of the exposure time and the aperture setting value) at the change point 815, and the amplification obtained by this calculation If the rate reaches the change point 816, the gain at the change point 817, which is further increased from the gain at the change point 815 by the gain variation ΔGain, is set so as to approach the change point 816. Similarly, the amplification factor indicated by the change point 818 is calculated based on the amplification factor of the change point 817, and the amplification factor at the change point 819 increased by the amplification factor variation ΔGain so as to approach this is obtained.

  Further, when the amplification factor calculated based on the currently set amplification factor becomes a value smaller than the currently set amplification factor, similarly to the above, from the currently set amplification factor, The amplification factors reduced by the amplification factor variation ΔGain are sequentially set (adopted). For example, in FIG. 26, when the gain at the change point 822 is calculated based on the gain at the change point 821, the gain variation ΔGain (the gain variation indicated by reference numeral 823) is greater than the gain at the change point 821. An amplification factor indicated by a change point 824 that is reduced by the amount is set. The gain set in this way gradually decreases step by step, and when the value becomes smaller than GainTh10 as indicated by a change point 829, the insensitive control is terminated, and the change point 811 or Exposure amount control based on the normal amplification factor (change) at the change point 812 is performed.

  Note that the gain value calculated based on the currently set gain is within a gain variation ΔGain with respect to the currently set gain, as indicated by a change point 825 and a change point 826. In other words, in other words, the amplification factor calculated based on the currently set amplification factor is changed from the currently set amplification factor (in the increasing or decreasing direction in the figure). When there is no increase / decrease change larger than ΔGain, the amplification factor changes at the currently set amplification factor as indicated by a change point 827 or a change point 828 (the set amplification factor does not change from the previous state). .

  Similarly, the gain at the change point 813 is smaller than the gain at the change point 815 (the change point 813 is located between the position of GainTh1 and the position of the change point 815). In this case, that is, when the gain increase beyond GainTh1 is smaller than the gain variation ΔGain indicated by reference numeral 814, the gain at that time is not the change point 815 but the value of GainTh1. Adopted (even if the gain obtained by the calculation exceeds GainTh1, the gain remains in the state of GainTh1).

  As described above, in the change section 801, as indicated by the black points such as the change points 815, 817, and 819, with respect to the rapid and frequent change of the amplification factor indicated by the white points such as the change points 813, 816, and 818. An amplification factor (exposure amount control parameter based on this amplification factor) that gradually changes stepwise (stepwise) with a predetermined amplification factor variation ΔGain is set. The change control of the amplification factor in the case of the change section 802 is similarly performed.

  In this insensitive control, when the amplification factor increases (decreases) in this way, the insensitive control is not started from the time when GainTh10 (GainTh20) is reached, but when the gain exceeds GainTh1 (GainTh2). The start of the insensitive control, i.e., insensitive detection of the start of the insensitive control, and the amplification factor (inflection point) in each of the change sections 801 and 802 from the GainTh1 (GainTh2) to the GainTh10 (GainTh20). Insensitive control is performed in two stages, such as insensitive control so as not to cause rapid and frequent changes in the change amount of

  FIG. 27 is a flowchart showing an example of a flow for setting an amplification factor during exposure amount control related to the insensitivity control in FIG. First, based on an AE evaluation value obtained from an image (frame image; evaluation image) taken with a moving image at a currently set amplification factor (exposure time setting value and aperture setting value calculated based on the amplification factor). Thus, an amplification factor Gain is calculated such that the sensor output at the subject luminance (Lt1) for setting the exposure amount becomes a desired output level (Vtarget) in the linear characteristic region of the photoelectric conversion characteristic (FIG. 18A ) And FIG. 20) (step S31). Then, the level determination unit 530 determines whether or not the calculated gain Gain is greater than GainTh1, which is a predetermined threshold for insensitive control start when the gain increases (whether GainTh1 has been exceeded). If it is determined that the gain is greater than GainTh1 (YES in step S32), the gain change control using the gain variation ΔGain is started, and the gain variation ΔGain is changed (from the gain GainTh1). The amplification factor is calculated (including the case where it does not change) (step S33). Note that the gain obtained by controlling the gain change using the gain variation ΔGain is defined as gain Gain ′. The gain variation ΔGain in the gain change control when it is determined that the gain is greater than GainTh1 is referred to as gain variation ΔGain1.

  Then, the exposure time setting value and the aperture setting value calculated from the gain Gain ′ calculated based on this change control are set, and the image sensor 30, the shutter 23, the aperture 22 and the like are driven according to the setting. (Step S34). However, when the calculated amplification factor is less than the amplification factor increased from GainTh1 by the amplification factor variation ΔGain1, the value of GainTh1 is used as the amplification factor Gain 'at this point.

  Next, the next shooting is performed in the state of being driven with the amplification factor (each setting) in step S34, and the amplification factor Gain (exposure time setting value and aperture value) is obtained based on the AE evaluation value acquired from this captured image. (Set value) is calculated (step S35). Then, the level determination unit 530 determines whether or not the gain Gain calculated in step S35 is smaller than GainTh10 which is a predetermined threshold for the end of the insensitive control (whether or not it exceeds GainTh10), and is not smaller than GainTh10. (NO in step S36), the process returns to step S33, the gain Gain 'is calculated, and the gain change control is continued. If it is determined that the gain is smaller than GainTh10 (YES in step S36), the insensitivity control is terminated, and the exposure time and aperture are set based on the gain Gain obtained by normal control (step S42). If shooting is not completed (NO in step S43), the process returns to step S31 and the above processing is repeated. If shooting is completed (YES in step S43), the flow ends.

  On the other hand, if it is determined in step S32 that the gain calculated in step S31 is not greater than GainTh1 (NO in step S32), the process proceeds to step S37, where the gain is decreased. If the level determination unit 530 determines whether or not it is less than a predetermined threshold GainTh2 (whether or not it exceeds GainTh2) with respect to the start of the insensitivity control in the case of performing, and if it is determined that it is less than GainTh2 (YES in step S37) In the same manner as in step S33, the gain variation ΔGain2 (the gain variation ΔGain in the gain change control when determined to be smaller than GainTh2 is referred to as gain variation ΔGain2). The change control of the amplification factor used is started, and the amplification factor variation ΔGain A gain Gain ′ that has been changed only (including the case where it does not change from the gain GainTh2) is calculated (step S38), and the exposure time setting value and the aperture setting value calculated from the gain Gain ′ are set. The image sensor 30, the shutter 23, the aperture 22 and the like are driven according to the setting (step S39).

  Then, the next shooting is performed while being driven at the amplification factor (each setting) in step S39, and the amplification factor Gain (exposure time setting value and aperture setting) is based on the AE evaluation value acquired from this captured image. Value) is calculated (step S40), and the level determination unit 530 determines whether the gain Gain calculated in step S40 is greater than GainTh20, which is a predetermined threshold for the end of insensitive control (whether it exceeds GainTh20). If it is determined that the gain is not greater than GainTh20 (NO in step S41), the process returns to step S38, and the gain Gain '(exposure time setting value and aperture setting value) is calculated, and the gain change Control continues. If it is determined that it is greater than GainTh20 (YES in step S41), the insensitivity control is terminated, and the exposure time and aperture setting in step S42 are performed. If shooting is not completed (NO in step S43), the process returns to step S31 and the above processing is repeated. If shooting is completed (YES in step S43), the flow ends.

  If it is determined in step S37 that the gain Gain is not smaller than GainTh2 (NO in step S37), that is, even if the gain Gain is changed, the gain Gain is not greater than GainTh1. If the gain is not smaller than GainTh2, the insensitivity control is not started, and the process returns to step S31 to continue the exposure amount control based on the normal amplification factor calculation.

(About insensitive control to change in inflection point (ΔVth) of photoelectric conversion characteristics)
FIG. 25 is a graph showing a change in the amount of change at the inflection point of the photoelectric conversion characteristics related to dynamic range control during moving image shooting. ΔVthTh1 shown on the vertical axis in FIG. 25 is a predetermined value (threshold value) of the change amount ΔVth, which is the insensitive control start level when the change amount ΔVth of the inflection point output level (see ΔVth shown in FIG. 22) increases. ΔVthTh2 is a predetermined value (threshold value) of the change amount ΔVth, which is the insensitive control start level when the change amount ΔVth decreases. Similarly, ΔVthTh10 shown on the vertical axis is a predetermined value (threshold value) of the change amount ΔVth that is the end level of the insensitive control starting from the above-described ΔVthTh1, and ΔVthTh20 is the end of the insensitive control starting from the above-described ΔVthTh2. This is a predetermined value (threshold value) of the amount of change ΔVth, which is a level. However, it is assumed that ΔVthTh10 is positioned at a low level relative to ΔVthTh1, and ΔVthTh20 is at a high level relative to ΔVthTh2 (low level when viewed in a direction in which the inflection point change amount decreases). Suppose that it is located. Hereinafter, the change amount ΔVth of the output level at the inflection point in the photoelectric conversion characteristics will be referred to as the inflection point change amount.

  In FIG. 25, the inflection point change amount changes with the passage of time (on the horizontal axis), and when the inflection point change amount is larger than ΔVthTh1 at time t1, insensitivity control is started and is smaller than ΔVthTh10 at time t2. This insensitivity control is terminated when the inflection point change amount is reached (insensitivity control is performed in the change section 841 at times t1 to t2 in FIG. 25). Further, the inflection point change amount changes with time, and when the inflection point change amount is smaller than ΔVthTh2 at time t3, the insensitive control is started, and the inflection point change amount becomes larger than ΔVthTh20 at time t4. This insensitivity control is terminated (the insensitivity control is performed in the change section 842 at times t3 to t4 in FIG. 25).

  The actual operation in the change control of the inflection point change amount from when the inflection point change amount exceeds ΔVthTh1 (ΔVthTh2) to ΔVthTh10 (ΔVthTh20), that is, the actual operation in the insensitive control is shown in FIG. In this case, for the change section 841 in FIG. 25, GainTh1 and GainTh10 shown in FIG. 26 indicate ΔVthTh1 and ΔVthTh10, respectively. In the change section 841, the change points 813, 816, With respect to the rapid and frequent change of the inflection point change amount ΔVth indicated by a white point such as 818, as indicated by black points such as change points 815, 817, and 819, the change amount variation ΔCVth is stepwise ( Inflection point change that gradually changes in a staircase shape (dynamics based on this inflection point change) Is set (adopted). The change control (insensitivity control) of the inflection point change amount in the case of the change section 842 is performed in the same manner. The variation variation ΔCVth indicates the predetermined variation amount when the inflection point variation amount (ΔVth) is changed stepwise by a predetermined variation amount as indicated by reference numeral 814 in FIG. Is.

  FIG. 28 is a flowchart showing an example of a flow in setting the inflection point change amount at the time of dynamic range control regarding the insensitive control in FIG. First, an AE evaluation obtained from an image (frame image; evaluation image) taken with a moving image at the currently set inflection point change amount (a photoelectric conversion characteristic setting value calculated based on the inflection point change amount). Based on the value, the inflection point change amount ΔVth is calculated so that the sensor output at the subject luminance (luminance Lm20) for setting the dynamic range becomes a desired output level (saturated output level Vmax of the image sensor 30). (See the description in FIG. 18B) (step S51). Then, the level determination unit 530 determines whether or not the calculated inflection point change amount ΔVth is greater than ΔVthTh1, which is a predetermined threshold for starting the insensitive control when the inflection point change amount increases, and from ΔVthTh1 When it is determined that the value is large (YES in step S52), the change control of the inflection point change amount using the change amount variation ΔCVth is started, and the change amount variation ΔCVth is changed (the inflection point). An inflection point change amount is calculated (including a case where the change amount ΔVthTh1 does not change) (step S53). The inflection point change amount obtained by the change control of the inflection point change amount using the change amount variation ΔCVth is defined as an inflection point change amount ΔVth ′. Further, the variation variation ΔCVth in the variation control of the inflection point variation when it is determined that it is greater than ΔVthTh1 is referred to as a variation variation ΔCVth1.

  Then, the photoelectric conversion characteristic setting value calculated from the inflection point change amount ΔVth ′ calculated based on this change control is set, and the image sensor 30 has a photoelectric conversion characteristic (dynamic range) according to the setting. Driven (step S54). However, when the calculated inflection point change amount is less than the inflection point change amount increased by ΔVVTh1 from ΔVthTh1, the value of ΔVthTh1 is the inflection point change amount at this time. Used as ΔVth ′.

  Next, the next shooting is performed while being driven by the inflection point change amount ΔVth ′ (photoelectric conversion characteristic setting value) in step S54, and the inflection is performed based on the AE evaluation value acquired from the captured image. A point change amount (photoelectric conversion characteristic setting value) is calculated (step S55). When the level determination unit 530 determines whether the inflection point change amount ΔVth calculated in step S55 is smaller than ΔVthTh10, which is a predetermined threshold for the end of the insensitive control, and when it is determined that it is not smaller than ΔVthTh10 (NO in step S56), the process returns to step S53, the inflection point change amount ΔVth ′ is calculated, and change control of the inflection point change amount is continued. If it is determined that it is smaller than ΔVthTh10 (YES in step S56), the insensitive control is terminated, and the photoelectric conversion characteristics are set based on the inflection point change amount ΔVth obtained by normal control (step S62). ). If shooting is not completed (NO in step S63), the process returns to step S51 and the above processing is repeated. If shooting is completed (YES in step S63), the flow ends.

  On the other hand, if it is determined in step S52 that the inflection point change amount ΔVth calculated in step S51 is not greater than ΔVthTh1 (NO in step S52), the process proceeds to step S57, and the inflection point change amount ΔVth. Is determined to be smaller than ΔVthTh2 which is a predetermined threshold value for the start of insensitive control when the inflection point change amount decreases, and when it is determined that it is smaller than ΔVthTh2 (in step S57) YES) As in the case of step S53, the variation variation ΔCVth2 (change amount variation ΔCVth2 in the variation control of the inflection point variation when it is determined that the variation variation ΔCVth2 is smaller than ΔVthTh2) Change control of the inflection point change amount using the change amount variation ΔCVth2 is started. The inflection point change amount ΔVth ′ (including the case where the inflection point change amount ΔVthTh2 does not change) is calculated (step S58), and the photoelectric conversion characteristic setting value calculated from the inflection point change amount ΔVth ′ is calculated. Setting is performed, and the image sensor 30 is driven with a photoelectric conversion characteristic (dynamic range) corresponding to the setting (step S59).

  Then, the next shooting is performed while being driven by the inflection point change amount ΔVth ′ (photoelectric conversion characteristic setting value) in step S59, and the inflection point is based on the AE evaluation value acquired from the captured image. A change amount (photoelectric conversion characteristic setting value) is calculated (step S60), and a level determination is made as to whether or not the inflection point change amount ΔVth calculated in step S60 is greater than ΔVthTh20, which is a predetermined threshold for the end of the insensitive control. If it is determined by the unit 530 and is not greater than ΔVthTh20 (NO in step S61), the process returns to step S58, and the inflection point change amount ΔVth ′ (photoelectric conversion characteristic setting value) is calculated. The change control of the point change amount is continued. If it is determined that it is greater than ΔVthTh20 (YES in step S61), the insensitivity control is terminated, and the photoelectric conversion characteristics are set in step S62. If shooting is not completed (NO in step S63), the process returns to step S51 and the above processing is repeated. If shooting is completed (YES in step S63), the flow ends.

  If it is determined in step S57 that the inflection point change amount ΔVth is not smaller than ΔVthTh2 (NO in step S57), that is, even if the inflection point change amount ΔVth is changed, this inflection point is changed. If the change amount ΔVth is not greater than ΔVthTh1 and less than ΔVthTh2, the insensitive control is not started, and the process returns to step S51 to continue the dynamic range control based on the normal inflection point change amount calculation.

  Although the AE control has been mainly described above, the actual digital camera 1 also performs AF control, WB control, and the like. Similarly, in the case of AF control, control can be performed based on an AF evaluation value acquired from a captured image of the image sensor 30. For example, the luminance histogram detected from the O, P, U, and V blocks of the main subject region 331 shown in FIG. 14 is used, and the evaluation value detection unit 405 obtains a point at which the contrast with the adjacent luminance is maximized. The AF evaluation value can be calculated by the “hill climbing method” or the like. In this case, the AF evaluation value is detected from each of the linear characteristic region and the logarithmic characteristic region of the image sensor 30, and the characteristics of the respective characteristic regions are utilized. It is desirable that the AF evaluation value obtained from the linear characteristic region is used for detailed distance measurement.

  The AF evaluation value detected by the evaluation value detection unit 405 in this way is sent to the AF control parameter calculation unit 513 of the overall control unit 50. The AF control parameter calculation unit 513 calculates an AF control parameter corresponding to the AF evaluation value, and transmits this to the zoom / focus control signal generation unit 523. The zoom / focus control signal generation unit 523 generates a control signal corresponding to the input AF control parameter, and transmits this to the zoom / focus drive unit 62. Then, the zoom / focus drive unit 62 generates a drive signal corresponding to the control signal, and the lens group 21 of the lens barrel 20 is focus-driven by the drive signal.

  The WB control can also be performed based on the WB evaluation value acquired from the captured image of the image sensor 30. Also in this case, it is desirable to detect the WB evaluation value from each of the linear characteristic region and the logarithmic characteristic region of the image sensor 30. That is, WB evaluation value detection is performed based on a captured image based on a neutral image, and each RGB level (R-Log, G-Log) is based on two types of images of a linear characteristic area and a logarithmic characteristic area as the neutral image. , B-Log, R-Lin, G-Lin, B-Lin). The evaluation value detection unit 405 detects such a WB evaluation value, sends the WB evaluation value to the white balance control unit 406, and white balance correction is performed by the white balance control unit 406 so as to obtain an appropriate color balance. is there.

  As described above, according to the imaging apparatus (digital camera 1) of the present embodiment, moving image shooting is performed by the overall control unit 50 (moving image shooting means), and the subject obtained from the frame image constituting the moving image is captured. An AE evaluation value is detected by the evaluation value detection unit 405 based on the luminance information, and the exposure amount control by the exposure amount control parameter calculation unit 511 (exposure amount control unit) is performed based on the photoelectric conversion characteristics using the AE evaluation value. AE control is performed by performing dynamic range control by the dynamic range control parameter calculation unit 512 (dynamic range control means). When the predetermined control value for controlling the exposure amount or the dynamic range calculated by the overall control unit 50 (exposure control means) based on the AE evaluation value reaches a predetermined level, When the predetermined level is reached or exceeds the predetermined level, change control is performed in which the control value is changed stepwise by a predetermined change amount.

  As described above, in association with the photoelectric conversion characteristics of the image sensor 30 included in the imaging apparatus, exposure amount control by the exposure amount control parameter calculation unit 511 and dynamic range control by the dynamic range control parameter calculation unit 512 can be performed, and exposure is performed. When the control value for controlling the amount or the dynamic range reaches a predetermined level, for example, a level higher than a normal shooting level, the actually obtained control value changes (fluctuates) abruptly (frequently). Even in this case, this change is not reflected in actual exposure control as it is, but control value change control (insensitive control) in which the control value is changed step by step by a predetermined change amount becomes possible. The subject can be imaged in an optimal exposure state and with a predetermined dynamic range according to the subject brightness. In the case of moving image shooting, even if the subject brightness changes suddenly or frequently, it is possible to smoothly control the shooting exposure state by avoiding sudden (frequent) exposure changes corresponding to this change. This makes it possible to perform more optimal exposure control.

  In the change control of the control value (amplification factor Gain or inflection point change amount ΔVth) by the change control unit 5111, the control value level calculated based on the step control value (Gain ′ or ΔVth ′) is followed (this In order to gradually approach the control value level stepwise, the step control value is changed by a predetermined change amount (ΔGain or ΔCVth) to set (calculate) a new step control value (Gain ′ or ΔVth ′). A temporary control value (for example, a change point 818 for the change point 817) is calculated based on the step control value (for example, the change point 817 shown in FIG. 26), and a new step control value ( By sequentially repeating the setting of the change point 819), the method of changing the control value stepwise is adopted, so that the smooth control of the shooting exposure state can be performed more easily. It can be realized with an easy configuration.

  If the level determination unit 530 determines that the control value (amplification factor Gain or inflection point change amount ΔVth) that changes with time in moving image shooting has reached the first threshold, the change control unit 5111. The change control is started (change control is executed), and when it is determined that the control value has reached the second threshold value, the change control is ended. In this way, the change control is not started until the first threshold value is reached, and is executed until the second threshold value is reached after the first threshold value is reached. Further, since the control value is changed to insensitive by the change control (until the first threshold value is reached from the first threshold value), the control value level is reduced. The change control does not immediately start the change control, and the change control makes it possible to obtain a more optimal shooting exposure state according to the actually required moving image shooting.

  Further, the predetermined control value for controlling the exposure amount means that the output level of the image sensor 30 with respect to the subject luminance Lt1 for setting the exposure amount is a predetermined target output level Vtarget in the linear characteristic region (first characteristic region). The gain Gain for changing the exposure amount so that the change control unit 5111 determines that the gain determination gain has reached GainTh1 or GainTh2 (each a first threshold value) by the level determination unit 530. Then, change control for changing the amplification factor Gain stepwise by the amplification factor variation ΔGain (ΔGain1, ΔGain2) is started, and the level determination unit 530 sets the amplification factor Gain to GainTh10 or GainTh20 (each a second threshold). When it is determined that the change has been reached, the change control is terminated. Even when the subject brightness changes suddenly or frequently during image shooting, by performing the change control for the gain Gain, it is possible to prevent the occurrence of a sudden (frequent) exposure amount change, and thus more optimal exposure control. Can be done.

  The predetermined control value for controlling the dynamic range is a photoelectric value for changing the dynamic range so that the output level of the imaging sensor 30 with respect to the subject luminance Lm20 for setting the dynamic range becomes a predetermined saturated output level Vmax. The inflection point change amount ΔVth with respect to the conversion characteristic, and the change control unit 5111 determines that the inflection point change amount ΔVth has reached ΔVthTh1 or ΔVthTh2 (each a first threshold value) by the level determination unit 530. Then, change control for changing the inflection point change amount ΔVth stepwise by the change amount variation ΔCVth (ΔCVth1, ΔCVth2) is started, and the inflection point change amount ΔVth is set to ΔVthTh10 or ΔVthTh20 by the level determination unit 530 (each second If it is determined that the threshold of Since the change control is terminated, even when the subject brightness changes suddenly or frequently during moving image shooting, the change control for the inflection point change amount ΔVth is performed, so that the rapid (frequent) dynamic range (photoelectric (Conversion characteristics) Changes can be prevented from occurring, and more optimal exposure control can be performed.

(Another embodiment)
In the present embodiment, the insensitive control of the control value is realized by giving a predetermined width to the saturation output level Vmax that is the target of the dynamic range control and the Vtarget that is the sensor output level that is the target of the exposure amount control. Is.

  In the present embodiment, the target control values for the exposure amount control and the dynamic range control are Vtarget and Vmax. In the present embodiment, the dynamic range control luminance Lmax is within a predetermined level range including the saturation output level Vmax. If it is within (for example, Vmax1 to Vmax2), the change of the control value is prohibited and the dynamic range control is not performed. Further, if the exposure control brightness Lt1 is within a predetermined level range including Vtarget (for example, Vtarget1 to Vtarget2), the change of the control value is also prohibited and the exposure control is not performed.

  When each control value is outside the predetermined level range, it can be performed by normal dynamic range control and exposure amount control (that is, control that directly reflects an evaluation value instead of insensitivity control), and has been described in the previous embodiment. It can also be performed by the insensitive control. In any case, the insensitive control can be performed as a result by prohibiting the exposure amount control and the dynamic range control when the control value exists within a predetermined level range including the target control value.

  That is, according to the imaging apparatus of the present embodiment, moving image shooting is performed by the overall control unit 50 (moving image shooting means), and evaluation value detection is performed based on the luminance information of the subject obtained from the frame images constituting the moving image. The AE evaluation value is detected by the unit 405, and the exposure amount control parameter calculation unit 511 (exposure amount control unit) performs exposure amount control and dynamic range control parameter calculation using the AE evaluation value based on the photoelectric conversion characteristics. AE control is performed by performing dynamic range control by the unit 512 (dynamic range control means). If the predetermined control value for controlling the exposure amount or dynamic range calculated by the overall control unit 50 (exposure control means) based on the AE evaluation value is within the predetermined level range, the change control unit By 5111, control for prohibiting the change of the control value within the predetermined level range is performed.

  As described above, the exposure amount control by the exposure amount control parameter calculation unit 511 and the dynamic range control by the dynamic range control parameter calculation unit 512 can be performed in association with the photoelectric conversion characteristics of the image sensor 30 included in the imaging device. If the predetermined level range is set as a range including the target control value, for example, the control value for controlling the exposure amount or the dynamic range is maintained in this level range (does not change rapidly and frequently). As a result, the insensitivity control is performed, so that the subject can be imaged in an optimum exposure state and a predetermined dynamic range is ensured according to the subject luminance, and further, moving image shooting is performed. So that it is not within the target exposure state (exposure amount, dynamic range). There often) the subject brightness is changed, be immediately avoided such would migrate to undesired exposure state, it is possible to perform thus optimal (smooth) exposure control.

In addition, this invention can take the following aspects.
(A) In the above embodiment, a P-channel MOSFET is employed in each pixel of the image sensor 30, but an N-channel MOSFET may be employed.

  (B) In the above embodiment, the subject brightness is detected by the image sensor 30, but a photometric element provided separately from the image sensor 30 (element that measures the subject brightness by dividing the brightness of the subject with a plurality of light receiving elements), etc. May be used to detect the subject brightness (AE evaluation value). However, from the viewpoint of simplifying the mechanism, it is desirable to detect subject luminance (AE evaluation value) based on an image signal obtained from an actual captured image captured by the imaging sensor 30.

  (C) In the above embodiment, the exposure time is controlled by the shutter 23 and the image sensor 30, but the exposure time may be controlled by only one of the shutter 23 and the image sensor 30.

  (D) In the above embodiment, the setting values for the imaging sensor 30, the shutter 23, and the aperture 22 obtained by converting the exposure time T2 and the aperture area S2 of the aperture by data conversion using the LUT are set as the exposure time setting value and the aperture setting value, respectively. However, the amplification factors Gt and Gs (or T exposure time T2 and aperture opening area S2) may be used as the exposure time setting value and the aperture setting value, respectively. Similarly, the output level Vth2 (or ΔVth) at the inflection point may be set as the photoelectric conversion characteristic setting value.

  (E) In the above embodiment, RGB primary color data is used as image data for evaluation value calculation. However, other image data, for example, captured image data, monochrome luminance data, or the like may be used.

  (F) The division (area division) of the imaging region 330 may not be a division by the division photometry method, and may be any method such as a spot photometry method or a central partial photometry method. Further, the block configuration of the main subject region 331 and the peripheral subject region 332 of the imaging region 330 may not be the configuration shown in FIG. In addition, although the evaluation value is calculated for each block, the evaluation value is calculated in two areas (2 blocks; each area is defined as 1 block) of the main subject area 331 and the surrounding subject area 332. May be. In addition, the imaging area 330 is not divided into two areas such as the main subject area 331 and the peripheral subject area 332, but is divided into three or more areas, and evaluation values are calculated from luminance information in each area (block). The AE control based on this may be performed. Alternatively, the evaluation value may be similarly calculated for the one imaging area without dividing the imaging area, and the AE control may be performed. Further, the area setting of each area (block) as described above of the imaging area 30 may be performed. May be configured to be performed at any time according to a user operation instruction.

  (G) It is good also as a structure which performs the said insensitive control by making the start level of insensitive control (change control) the same level as the end level of insensitive control. For example, in FIG. 24, GainTh1 and GainTh10 may be set to the same level (GainTh2 and GainTh20).

  (H) Parameters other than the gain Gain and the inflection point change amount ΔVth may be used as control values for controlling the exposure amount or dynamic range for insensitive control.

  (I) In the above embodiment, a linear characteristic region in which an electric signal is linearly converted with respect to the incident light amount and output, and a logarithmic characteristic in which the electric signal is logarithmically converted with respect to the incident light amount and output. An image sensor that exhibits photoelectric conversion characteristics including a region is used. Such an image sensor has a very wide dynamic range, and by using the sub-threshold characteristic of a MOS transistor, a sensor that generates a pixel output having the photoelectric conversion characteristic as described above can be realized relatively easily. However, the present invention is not limited to this, and various sensors can be used as long as the image sensor includes two characteristic regions having different output changes with respect to the amount of incident light. For example, it may be an imaging sensor having a photoelectric conversion characteristic including a first linear characteristic region and a second linear characteristic region in which an output change with respect to the amount of incident light is smaller.

  (J) In the above embodiment, the control value change control is performed when the amplification factor or the inflection point change amount exceeds the threshold value, but either one or both of the amplification factor and the inflection point change amount are used. Control value change control may be performed when is equal to or greater than a threshold value.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the external appearance of the small digital camera with which the imaging device concerning this invention is applied suitably, (a) is the top view, (b) is a front view, (c) is a rear view. It is an imaging process block diagram in the said digital camera. It is a schematic diagram which shows an example of the color filter structure of the image sensor used for the said digital camera. It is a functional block diagram for demonstrating the function of the whole control part with which the said digital camera is provided. It is a flowchart which shows an example of the whole operation | movement of the said digital camera. It is a table format for demonstrating the definition of the term regarding exposure control. It is a schematic block diagram of the two-dimensional MOS type solid-state imaging device which is an example of an imaging sensor. It is a circuit diagram which shows the structural example of each pixel G11-Gmn shown in FIG. It is a figure which shows an example of the timing chart regarding the imaging operation of an imaging sensor. It is a figure which shows an example of the photoelectric conversion characteristic of an imaging sensor. It is a graph for demonstrating the change operation | movement of the said photoelectric conversion characteristic. It is a flowchart which shows the operation example in the case of detecting the AE evaluation value etc. of a to-be-photographed object from the still image which the imaging sensor actually imaged. It is a flowchart which shows the operation example in the case of detecting the AE evaluation value etc. of a to-be-photographed object from the moving image which the imaging sensor is imaging continuously. It is a functional block diagram for demonstrating the function of an evaluation value detection part. It is a schematic diagram which shows the division | segmentation state of the imaging area (photometry range) at the time of the division | segmentation photometry by an imaging sensor. It is a graph which shows an example of the brightness | luminance histogram by the said division | segmentation photometry, (a) is a figure which shows a main subject whole luminance histogram, (b) is a figure which respectively shows a surrounding subject whole histogram. It is a graph which shows an example of the whole area histogram at the time of the output saturation of an imaging sensor. FIG. 6 is a graph showing how the photoelectric conversion characteristics of the image sensor change when performing AE control, where (a) shows the change when performing exposure amount control, and (b) performs dynamic range control. It is a figure which shows the change in a case. It is a graph for demonstrating the linear conversion process at the time of exposure amount control parameter calculation. It is a flowchart which shows an example of the flow of exposure amount control parameter calculation. It is a graph for demonstrating the process at the time of exposure amount control parameter calculation. It is a graph explaining the calculation method of the position of the inflection point in the photoelectric conversion characteristic when calculating the dynamic range control parameter, and (a) shows the photoelectric conversion characteristic so that a predetermined sensor output is obtained with respect to the luminance Lmax. It is a figure at the time of changing, (b) is a figure at the time of modeling a photoelectric conversion characteristic. It is a graph for demonstrating the process at the time of dynamic range control parameter calculation. It is a graph which shows the mode of the gain change regarding the exposure amount control at the time of video recording. It is a graph which shows the mode of the change of the variation | change_quantity of the inflexion point of the photoelectric conversion characteristic regarding the dynamic range control at the time of video recording. FIG. 26 is an enlarged view showing an example of a change with time of the value of the amplification factor or the inflection point change amount in the change section shown in FIG. 24 or FIG. 25. It is a flowchart which shows an example of the flow in the setting of the gain at the time of exposure amount control regarding the insensitivity control in FIG. It is a flowchart which shows an example of the flow in the setting of the inflection point variation | change_quantity at the time of dynamic range control regarding the insensitive control in FIG.

Explanation of symbols

1 Digital camera (imaging device)
DESCRIPTION OF SYMBOLS 100 Operation part 20 Lens barrel 21 Lens group 22 Aperture 23 Shutter 30 Imaging sensor 31 Timing generation circuit 40 Signal processing part 405 Evaluation value detection part (Exposure evaluation value detection means)
50 Overall control unit (exposure control means / moving image photographing means)
510 arithmetic unit 5110 AE control parameter calculation unit 5111 change control unit (change control means)
511 exposure amount control parameter calculation unit (exposure amount control means)
512 Dynamic range control parameter calculation unit (dynamic range control means)
516 Photoelectric conversion characteristic information storage unit 518 LUT storage unit 520 Control signal generation unit 530 Level determination unit (control value level determination unit)
Reference Signs List 60 drive section 320 photoelectric conversion characteristic 321 inflection point 330 imaging area 331 main subject area 332 peripheral subject area

Claims (6)

An electrical signal corresponding to the amount of incident light is generated, and a photoelectric conversion characteristic of the first characteristic region in which the electric signal is converted into a first characteristic with respect to the amount of incident light is output. An image sensor comprising: a second characteristic region in which a signal is converted into a second characteristic having a smaller change in output with respect to the amount of incident light than the first characteristic and output;
Moving image shooting means for shooting a moving image composed of a plurality of frame images;
Exposure evaluation value detection means for detecting an exposure evaluation value when imaging a subject based on luminance information of the subject obtained from the frame image;
Based on the exposure evaluation value detected by the exposure evaluation value detecting means , an exposure amount setting luminance is determined as a subject luminance for exposure setting, and the output level of the image sensor with respect to the determined exposure amount setting luminance is a predetermined level. An exposure amount control means for calculating a control value for controlling the exposure amount so as to be an output level, and for controlling the exposure amount for the image sensor based on the control value for controlling the exposure amount ;
Based on the exposure evaluation value detected by the exposure evaluation value detection means, a dynamic range setting luminance is determined as a subject luminance for exposure setting, and an output level of the image sensor with respect to the determined dynamic range setting luminance is a predetermined level. A dynamic range control means for calculating a control value for controlling the dynamic range so as to be an output level, and controlling a photoelectric conversion characteristic of the imaging sensor based on the control value for controlling the dynamic range ;
When a control value for controlling the exposure amount calculated by the exposure amount control means based on the exposure evaluation value reaches a predetermined level, the control value for controlling the exposure amount is incremented by a predetermined change amount. The dynamic range is controlled when the control value for controlling the dynamic range calculated by the dynamic range control means based on the exposure evaluation value reaches a predetermined level. An image pickup apparatus comprising: change control means for performing change control for changing a control value for performing stepwise change by a predetermined change amount . The change control unit is configured to calculate the exposure amount calculated by the exposure amount control unit based on a photographed image using a step control value in which a control value for controlling the exposure amount is changed stepwise by the predetermined change amount. In order to follow the level of the control value for controlling , the step control value is changed by the predetermined change amount to determine a new step control value , or the control value for controlling the dynamic range is set to the predetermined change amount. In order to follow the level of the control value for controlling the dynamic range calculated by the dynamic range control unit based on the captured image using the step control value changed stepwise by the step control value, the step control value is set to the predetermined value. The imaging apparatus according to claim 1, wherein a new step control value is determined by changing the amount of change . In order to determine whether or not the control value for controlling the exposure amount that changes with the moving image shooting has reached the first threshold value indicating the start level of the change control, and to control the exposure amount It is determined whether or not the control value has reached a second threshold value indicating the end level of change control , or the control value for controlling the dynamic range that changes with time as the moving image is captured is the change control. It is determined whether or not the first threshold value indicating the start level of the current control range has been reached, and whether or not the control value for controlling the dynamic range has reached the second threshold value indicating the end level of the change control. A control value level determining means for
The change control means starts the change control when the control value level determining means determines that the control value for controlling the exposure amount has reached a first threshold, and the control value level determining means performs exposure. When it is determined that the control value for controlling the amount has reached the second threshold, the change control is terminated , or the control value for controlling the dynamic range by the control value level determining means is the first control value. The change control is started when it is determined that the threshold value has been reached, and when the control value for controlling the dynamic range is determined by the control value level determination means to reach the second threshold value, the change control is started. The imaging apparatus according to claim 1, wherein the imaging device is terminated . The predetermined control value for controlling the exposure amount is such that the output level of the imaging sensor with respect to the subject luminance (Lt1) for setting the exposure amount becomes a predetermined target output level (Vtarget) in the first characteristic region. Is an amplification factor (Gain) for changing
When the control value level determining means determines that the gain has reached the first threshold value (GainTh1 or GainTh2), the change control means determines the gain according to the predetermined change amount (ΔGain). Change control is started, and when it is determined by the control value level determination means that the amplification factor has reached the second threshold (GainTh10 or GainTh20), the change control is terminated. The imaging device according to claim 3. The predetermined control value for controlling the dynamic range is for changing the dynamic range so that the output level of the image sensor with respect to the subject luminance (Lm20) for setting the dynamic range becomes a predetermined saturation output level (Vmax). An inflection point change amount (ΔVth) indicating an output level change amount at the inflection point of the photoelectric conversion characteristics,
When the control value level determining means determines that the inflection point change amount has reached the first threshold (ΔVthTh1 or ΔVthTh2), the change control means determines the inflection point change amount as the predetermined change amount. When change control that changes stepwise by (ΔCVth) is started, and the control value level determining means determines that the inflection point change amount has reached the second threshold (ΔVthTh10 or ΔVthTh20), the change 4. The imaging apparatus according to claim 3, wherein the control is terminated. An electrical signal corresponding to the amount of incident light is generated, and a photoelectric conversion characteristic of the first characteristic region in which the electric signal is converted into a first characteristic with respect to the amount of incident light is output. An image sensor comprising: a second characteristic region in which a signal is converted into a second characteristic having a smaller change in output with respect to the amount of incident light than the first characteristic and output;
Moving image shooting means for shooting a moving image composed of a plurality of frame images;
Exposure evaluation value detection means for detecting an exposure evaluation value when imaging a subject based on luminance information of the subject obtained from the frame image;
Based on the exposure evaluation value detected by the exposure evaluation value detecting means , an exposure amount setting luminance is determined as a subject luminance for exposure setting, and the output level of the image sensor with respect to the determined exposure amount setting luminance is a predetermined level. An exposure amount control means for calculating a control value for controlling the exposure amount so as to be an output level, and for controlling the exposure amount for the image sensor based on the control value for controlling the exposure amount;
Based on the exposure evaluation value detected by the exposure evaluation value detection means, a dynamic range setting luminance is determined as a subject luminance for exposure setting, and an output level of the image sensor with respect to the determined dynamic range setting luminance is a predetermined level. A dynamic range control means for calculating a control value for controlling the dynamic range so as to be an output level, and controlling a photoelectric conversion characteristic of the imaging sensor based on the control value for controlling the dynamic range ;
When the control value for controlling the exposure amount calculated by the exposure amount control means based on the exposure evaluation value is within a predetermined level range, the change of the control value for controlling the exposure amount is prohibited. Control for controlling the dynamic range when the control value for controlling the dynamic range calculated by the dynamic range control means based on the exposure evaluation value is within a predetermined level range An imaging apparatus comprising: a change control unit that performs control for prohibiting a change in value .

Images