JP4886335B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image pickup apparatus using an image pickup device such as a CCD or a CMOS image sensor, and more particularly to a technique for improving the image quality of an image pickup apparatus that reads a screen by dividing it into n channels.

  In an imaging apparatus such as a digital camera or a video camera, a CCD or a CMOS image sensor is generally used as an imaging element.

  In recent years, an increase in the number of pixels of an image sensor has been promoted, and a higher reading speed of pixel signals has been demanded. For this reason, there has been proposed an imaging apparatus that has a plurality of readout circuits so that pixel signals can be read out in parallel.

  In the imaging apparatus as described above, if there is an offset difference or gain difference between a plurality of readout circuits, there is a problem that the difference significantly deteriorates the image quality.

As methods for solving this problem, methods as described in Patent Document 1 and Patent Document 2 have been proposed.
JP-A-7-38814 JP 2000-253305 A

  In the above-mentioned Patent Document 1, the same signal is input to all of the plurality of readout circuits, and the offset difference and gain difference between the readout circuits are detected from the output signals of the plurality of readout circuits with respect to the same input signal. The difference is corrected.

  However, when the number of read circuits is large (for example, m), it is necessary to read the same signal by m read circuits, and there is a problem that the read time increases accordingly.

  In Patent Document 2, an output signal from a readout circuit is corrected based on a correlation between output signals from a plurality of readout circuits. Specifically, the screen is divided into a plurality of areas, signals are read out by individual readout circuits for each area, and a correlation operation is performed with the signal of the adjacent area near the boundary with the adjacent area. In this method, correction is performed so that the difference from the adjacent region is eliminated.

  In this method, there is a problem that it is necessary to obtain a correlation with an adjacent region for each divided region, and the calculation becomes very complicated. In addition, since the screen is divided into planes, the average of a predetermined range of the divided area is obtained and correlation calculation is performed, so the screen is divided into lines for each column (divided into a skewer shape), etc. However, there was a problem that it was difficult to cope with averaging.

  In the above two conventional techniques, correction is performed in real time by calculating the output read for each photographing, so that correction can be performed regardless of environmental changes (for example, ambient temperature). However, since the time for shifting to the next shooting operation is delayed by the time required for the calculation for each shooting, it is also a problem to reduce the rapid shooting property in continuous shooting.

  Therefore, the present invention has been made in view of the above-described problems, and an object of the present invention is to suppress degradation in image quality when an image is divided into a plurality of areas and each area is read by a separate readout circuit. Is to do.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging element in which a plurality of pixels are arranged in a matrix, and image signals from the imaging element at predetermined row intervals or column intervals. Readout means for reading out divided into a first channel and a second channel, and a second region which is a relatively dark region and a first region which is a relatively bright region in the image signal of the first channel A first difference value that is a difference between the respective integral values and a fourth area that is a relatively dark area and a third area that is a relatively bright area in the image signal of the second channel. Correction means for correcting a level difference between the image signals of the first channel and the second channel based on a second difference value that is a difference between the integral values of the first channel and the second channel.

An image pickup apparatus control method according to the present invention is an image pickup apparatus control method including an image pickup device in which a plurality of pixels are arranged in a matrix, and image signals are output from the image pickup device at predetermined row intervals or column intervals. A reading step of reading the first channel and the second channel separately, and a first region which is a relatively bright region and a second region which is a relatively dark region in the image signal of the first channel. A first difference value that is a difference between the respective integral values and a fourth area that is a relatively dark area and a third area that is a relatively bright area in the image signal of the second channel. And a correction step of correcting a level difference between the image signals of the first channel and the second channel based on a second difference value that is a difference between the integral values of the first channel and the second channel.

  According to the present invention, it is possible to suppress deterioration in image quality when an image is divided into a plurality of areas and each area is read out by separate readout circuits.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram for explaining a first embodiment of an imaging device of the present invention, and is a diagram for explaining details of a CMOS sensor that is an imaging device and its peripheral circuits. Although an actual image sensor has a larger number of pixels, such as millions of pixels, in FIG. 1, in order to simplify the description, the image sensor has pixels of m rows in the vertical direction and 8 columns in the horizontal direction. Are arranged and configured. Further, it is assumed that the pixel signal of m rows and 8 columns is divided into two lines every other column (at intervals of two columns) and read by two readout circuits.

  In FIG. 1, 11 is a pixel portion, 12 is a vertical scanning circuit for sending out pixel signals in the vertical direction, 21a to 21d and 22a to 22d are selection switches for selecting horizontal lines of the pixel signals, and 23 and 24, respectively. Is a horizontal scanning circuit that sequentially selects horizontal lines. In this configuration, the selection switches 21a to 21d, 22a to 22d, and horizontal scanning circuits 23 and 24 divide the screen into alternate rows (intervals between two rows) to divide the screen upward (hereinafter CH1) and downward (hereinafter CH2). The pixel signal is read out.

  25 is an amplification amplifier that amplifies the pixel signal read from CH1, 26 is an amplification amplifier that amplifies the pixel signal read from CH2, and 27 and 28 are for correcting the gain characteristic difference between the amplification amplifiers 25 and 26. This is a variable gain amplifier (hereinafter PGA). Reference numerals 29 and 30 denote A / D converters for A / D converting the output signals from the PGAs 27 and 28, and are connected to a subsequent digital signal processing circuit (DSP, see FIG. 7).

  Here, the vertical scanning circuit 12, the horizontal scanning circuits 23 and 24, the PGAs 27 and 28, and the A / D converters 29 and 30 are driven by a timing generation circuit (hereinafter referred to as TG) (not shown).

  2A and 2B are diagrams showing the relationship between the output of the A / D converters 29 and 30 and the exposure amount in the imaging apparatus described with reference to FIG.

  FIG. 2A shows a case where the settings of the PGAs 27 and 28 and the A / D converters 29 and 30 are set to default settings (for example, gain of 1 and offset correction 0). As can be seen from FIG. 2A, the gain and the offset are shifted between CH1 and CH2. Most of this is because the gain and offset of the amplifiers 25 and 26 are different due to manufacturing errors.

  In the manufacturing process of the imaging device, the gain setting of the PGAs 27 and 28 and the offset correction setting of the A / D converters 29 and 30 are adjusted in order to correct such a shift between the gains and offsets of CH1 and CH2. . Therefore, actually, the gain and offset are corrected as shown in FIG.

  By the way, in general, the gain and offset error of the amplification amplifier change depending on the ambient temperature state, and there are individual differences in the amount of change. In other words, if the ambient temperature is the same as the ambient temperature at the time of manufacture, the gain and offset error of the amplifier are sufficiently corrected, but the gain and offset error of the amplifier are not sufficiently corrected under different temperature environments. .

  3A and 3B are diagrams illustrating this phenomenon using the image output state of the imaging apparatus. In the figure, (2) shows an image obtained by extracting only CH1, (3) shows an image obtained by extracting only CH2, and (1) shows an image when CH1 and CH2 are synthesized. Here, an example of photographing a gray stair chart will be described.

  FIG. 3A is an image at normal temperature that is the same temperature as the ambient temperature at the time of manufacture, and the gain and offset error of the amplifier are corrected at the time of manufacture. Therefore, the image (2) from which only CH1 is extracted and the image (3) from which only CH2 is extracted are shifted in the horizontal direction by n columns (here, two columns), but are identical at any luminance level. It has become. For this reason, the state of the gray stair chart is also accurately reproduced in the final image (1) obtained by combining CH1 and CH2.

  On the other hand, FIG. 3B is an image when the ambient temperature is 0 ° C., and the gain and offset error of the amplification amplifier are not sufficiently corrected as compared with the normal temperature. Therefore, the image (2) from which only CH1 is extracted and the image (3) from which only CH2 is extracted are shifted in the horizontal direction by n columns (here, two columns), and there is a difference in any luminance level. It is an image. For this reason, the final image (1) obtained by combining CH1 and CH2 cannot accurately reproduce the gray step chart, and the difference between CH1 and CH2 appears as a vertical linear contrast, resulting in a greatly deteriorated image. End up.

  FIG. 4 is an image when the ambient temperature is 0 ° C., as in FIG. 3B, but “region k” that is a relatively bright portion and “region j” that is a relatively dark portion in the image. Show.

  FIG. 5 shows the relationship between the integrated values of the outputs CH1 and CH2 in the region k as CH1k and CH2k, and the integrated values of the outputs CH1 and CH2 in the region j as CH1j and CH2j, respectively. It is a broken line. In FIG. 5, it can be seen that the straight line indicated by the broken line is shifted from the ideal straight line (indicated by a solid line) of CH1 = CH2. The relationship between CH1 and CH2 at this time is expressed as follows.

CH1 × {(CH2k−CH2j) / (CH1k−CH1j)} + p = CH2
... (Formula 1)
From (Equation 1), it can be seen that the gain of CH1 can be matched to CH2 by multiplying the output value of CH1 by (CH2k-CH2j) / (CH1k-CH1j).

  FIG. 6 is a diagram showing the relationship between the output of the A / D converters 29 and 30 and the exposure amount when the ambient temperature is 0 ° C. Here, the output values of the A / D converters 29 and 30 when the exposure amount is 0 are the offset deviation. The output when the exposure amount is 0 corresponds to the output of optical black (hereinafter referred to as OB) in the image sensor, and the offset deviation is as follows.

CH1 offset deviation = CH1OB−CH1OB (room temperature) (Formula 2)
CH2 offset deviation = CH2OB-CH2OB (room temperature) (Equation 3)
Here, CH1OB (normal temperature) and CH2OB (normal temperature) are integral values of the OB region at the time of manufacturing the imaging device. CH1OB and CH2OB are integral values of the OB area at the time of shooting. Note that the temperature (normal temperature) at the time of manufacturing the imaging device, CH1OB (normal temperature), and CH2OB (normal temperature) are both stored in the ROM 106 shown in FIG. 7 or the like when the imaging device is manufactured.

  That is, in order to correct the gain and offset error of the amplification amplifiers CH1 and CH2, the output of CH1 may be corrected by the following equation.

CH1 × {(CH2k−CH2j) / (CH1k−CH1j)} + CH1 offset deviation (Expression 4)
Moreover, what is necessary is just to correct | amend the output of CH2 by following Formula.

CH2 + CH2 offset deviation (Formula 5)
Here, in order to prevent the vertical linear contrast from being generated due to the output difference between CH1 and CH2 and causing the image to greatly deteriorate, the change in absolute gain is ignored and the relative gain difference between CH1 and CH2 is corrected. I am doing so. Further, although the CH1 gain is corrected to match CH2, the CH2 gain may be adjusted to CH1, or both CH1 and CH2 may be corrected to match each other. .

  Here, the reason why the gain and offset error of the amplification amplifiers of CH1 and CH2 can be corrected by the method described so far is as follows. In other words, since the lines are divided and read at intervals of n columns, as described with reference to FIGS. 3A and 3B, the image obtained by extracting only CH1 and the image obtained by extracting only CH2 are shifted by n columns. This is because almost the same image is output except for.

  Next, if the correction calculation as described above is performed for each shooting operation, the quickness in continuous shooting is deteriorated. In this embodiment, the gain and offset correction values for the ambient temperature are tabulated in the imaging apparatus. To do. This will be described with reference to FIGS.

  FIG. 7 is a diagram showing a configuration of an imaging apparatus such as a digital camera or a digital video camera. Reference numeral 101 denotes an imaging element, which uses a CCD or CMOS sensor. Here, the portions indicated by 11 to 28 in FIG.

  Reference numeral 102 denotes an A / D converter that performs analog / digital conversion on an output signal from the image sensor. The A / D converter 102 includes the A / D converters 29 and 30 shown in FIG. Reference numeral 103 denotes a DSP (Digital Signal Processor), which performs various correction processes and development processes on the data from the A / D converter 102. The DSP 103 controls various memories such as the ROM 106 and the RAM 107 and controls the writing of image data to the recording medium 108.

  Reference numeral 104 denotes a TG (Timing Generator) that is a timing generation circuit that supplies a clock signal and a control signal to the image sensor 101, the A / D converter 102, and the DSP 103, and is controlled by the CPU 105.

  Reference numeral 105 denotes a CPU that controls the DSP 103 and the TG 104, and controls camera functions using various parts (not shown) such as photometry and distance measurement. In addition, each of the switches 109 to 111 is connected, and processing corresponding to each state is executed.

  A ROM 106 stores a camera control program, a correction table described later, and the like, and can be rewritten by the CPU 105. Reference numeral 107 denotes a RAM that temporarily stores image data and correction data processed by the DSP 103. The RAM 107 can be accessed at a higher speed than the ROM 106.

  Reference numeral 108 denotes a recording medium such as a compact flash (registered trademark) card (hereinafter referred to as CF) for storing the photographed image, and is connected to the camera via a connector (not shown).

  Reference numeral 109 denotes a power switch for starting the camera, and reference numeral 110 denotes a shutter switch SW1 for instructing start of photographing preparation operations such as photometry processing and distance measurement processing. Reference numeral 111 denotes a shutter switch SW2 that drives a mirror and a shutter (not shown) and instructs the start of a series of imaging operations to write a signal read from the imaging element 101 to the recording medium 108 via the A / D converter 102 and the DSP 103. . Reference numeral 112 denotes a temperature sensor that detects the temperature of the camera or the image sensor at the time of shooting.

  FIG. 8 is a flowchart showing the control of the camera shown in FIG. 7, and the operation of the camera will be described with reference to this figure.

  First, in step S401, it is determined whether or not the power switch 109 is ON. If it is OFF here, step S401 is repeated, and if it is ON, it is determined in step S402 whether SW1, which is a shooting preparation operation start switch, is ON. If it is OFF here, step S402 is repeated. If it is ON, in step S403, the temperature sensor 112 measures the temperature of the camera or the image sensor at the time of shooting. Subsequently, in step S404, a well-known photometric process and a distance measuring process are performed, and the process proceeds to step S405.

  In step S405, it is determined whether or not SW2 that is an imaging operation start switch is ON. If it is OFF, the process returns to step S402, and steps S402 to S404 are repeated. If SW2 is ON, the image sensor 101 is exposed by driving a shutter (not shown) in step S406, and a signal from the image sensor 101 is read in step S407 and temporarily stored in the RAM 107.

  Next, in step S408, the shooting temperature measured in step S403 is compared with the camera manufacturing temperature (normal camera is manufactured at room temperature around 25 ° C., for example, 25 ° C.). The temperature at the time of manufacturing the camera is stored in the ROM 106 in advance as described above. If the temperature difference between the shooting temperature and the camera manufacturing temperature is equal to or smaller than a predetermined temperature difference (for example, ± 10 ° C.), the process proceeds to step S414 without performing subsequent correction processing, and the image in the RAM 107 is stored as CF. Recording on the medium 108.

  If it is determined in step S408 that the difference between the shooting temperature and the camera manufacturing temperature exceeds the predetermined temperature difference, the process proceeds to step S409, and a table indicating the relationship between the shooting temperature and the correction value, which will be described later, is completed. Determine if. If the table is not yet completed, the process proceeds to step S410, the correction value is calculated from the image signal using (Equation 1) to (Equation 3) as described above, and the process proceeds to step S411.

  In step S411, the relationship between the photographing temperature obtained in step S403 and the correction value obtained in step S410 is summarized as in the example of FIG. Referring to FIG. 9, the temperature range at the time of photographing is divided in increments of 10 ° C., and correction values for 10 times are stored for each temperature range. When correction values are obtained 10 times in all temperature ranges, the average value of the correction values is calculated for each temperature range, and then the relationship between the shooting temperature and the correction value is approximated by a linear expression or a curve as shown in FIG. To make a table. By storing the data tabulated here in the ROM 106, it can be used as a table, so that in subsequent corrections, it is possible to perform optimal correction without bothering calculating correction values from the image. Here, only the gain correction value is described in FIGS. 9 and 10, but the offset correction value can be similarly tabulated.

  In step S411, the relationship between the temperature and the correction value at the time of shooting is summarized, and then the process proceeds to step S413. In step S413, based on the correction value obtained in step S410, the image in the RAM 107 is corrected using (Equation 4) and (Equation 5), stored again in the RAM 107, and the process proceeds to step S414. Then, the image in the RAM 107 is recorded on a storage medium 108 such as a CF.

  If it is determined in step S409 that the table indicating the relationship between the shooting temperature and the correction value has already been completed, the process proceeds to step S412 to calculate the correction value based on the table as shown in FIG. Thereafter, in step S413, based on the correction value obtained in step S412, the image in the RAM 107 is corrected using (Equation 4) and (Equation 5), stored again in the RAM 107, and the process proceeds to step S414. Then, the image in the RAM 107 is recorded on a storage medium 108 such as a CF.

  As can be seen from the above description, the correction value is initially calculated from the captured image for each shooting operation, but after the table is completed, the correction value can be obtained from the table simply by measuring the temperature at the time of shooting. become able to. For this reason, it is possible to provide an imaging apparatus that does not reduce the speed of continuous shooting.

(Second Embodiment)
FIG. 11 is a diagram for explaining a second embodiment of the imaging apparatus of the present invention, and is a diagram for explaining details of a CMOS sensor that is an imaging device and its peripheral circuits. The actual image sensor has a larger number of pixels, such as millions of pixels, but in FIG. 11, in order to simplify the description, the image sensor has pixels of m rows in the vertical direction and 8 columns in the horizontal direction. Are arranged and configured. Further, the pixel signal is read out by four readout circuits by dividing the line into four at intervals of four columns. Although the detailed description of FIG. 11 is omitted, the difference from FIG. 1 is the number of line divisions in the horizontal direction. Even with the configuration as shown in FIG. 11, it is possible to easily apply the same operation as that of the first embodiment as shown in FIGS.

(Third embodiment)
1 and 11 illustrate an example in which lines are divided in the horizontal direction, the present invention can be applied even when lines are similarly divided in the vertical direction.

  As described above, by performing the correction according to the first to third embodiments described above, even in an imaging apparatus that reads out a screen by dividing the screen into n channels, the offset difference or gain difference between a plurality of readout circuits. Therefore, it is possible to provide a high-quality image.

  In addition, since correction can be performed in real time, correction can be performed regardless of environmental changes (for example, ambient temperature).

  Further, since the calculation time can be shortened by learning the result for each temperature, it is possible to provide an imaging apparatus that does not deteriorate the speed of continuous shooting after learning.

  The present invention relates to image quality improvement in digital cameras and digital video cameras. In particular, in an apparatus configured to read out pixel signals of an image sensor with a plurality of readout circuits, image quality deterioration due to environmental changes is prevented.

It is a figure explaining 1st Embodiment of the imaging device of this invention, and is a figure explaining the detail of the CMOS sensor which is an image pick-up element, and its peripheral circuit. FIG. 2 is a diagram illustrating a relationship between an output of an A / D converter and an exposure amount in the imaging apparatus described in FIG. 1. It is a figure explaining an image when ambient temperature state changes. It is a figure explaining an image when ambient temperature state changes. It is a figure explaining output correction when ambient temperature changes. It is a figure explaining output correction when ambient temperature changes. It is a figure which shows the structure of the imaging device in the 1st Embodiment of this invention. It is a flowchart explaining operation | movement of an imaging device. It is a figure explaining the table of temperature and a correction value. It is a figure explaining the table of temperature and a correction value. It is a figure explaining 2nd Embodiment of the imaging device of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Pixel part 12 Vertical scanning circuit 21a-d, 22a-d Horizontal direction line selection switch 23, 24 Horizontal scanning circuit 25, 26 Amplification amplifier 27, 28 Variable gain amplifier (PGA)
29, 30 A / D converter 101 Image sensor 102 A / D converter 103 DSP
104 Timing generation circuit 105 CPU
106 ROM
107 RAM
108 Recording medium 109, 110, 111 Switch 112 Temperature sensor

Claims (4)

An imaging device in which a plurality of pixels are arranged in a matrix;
Reading means for reading out an image signal from the image sensor by dividing it into a first channel and a second channel at a predetermined row interval or column interval ;
A first difference value that is a difference between integrated values of a first area that is a relatively bright area and a second area that is a relatively dark area in the image signal of the first channel; Based on the second difference value, which is the difference between the integral values of the third area, which is a relatively bright area, and the fourth area, which is a relatively dark area, in the image signal of the second channel. Correcting means for correcting a level difference between the image signals of the first channel and the second channel;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, further comprising: a temperature detection unit that detects a temperature; and a control unit that controls a correction operation by the correction unit according to a temperature detected by the temperature detection unit. .   The apparatus further comprises storage means for storing a relationship between the temperature detected by the temperature detection means and the correction value of the correction means, and the control means includes the temperature stored in the storage means and the correction value. The imaging apparatus according to claim 2, wherein a correction operation by the correction unit is controlled based on a relationship. A method for controlling an imaging apparatus having an imaging element in which a plurality of pixels are arranged in a matrix,
A reading step of reading out an image signal from the imaging device by dividing it into a first channel and a second channel at a predetermined row interval or column interval ;
A first difference value that is a difference between integrated values of a first area that is a relatively bright area and a second area that is a relatively dark area in the image signal of the first channel; Based on the second difference value, which is the difference between the integral values of the third area, which is a relatively bright area, and the fourth area, which is a relatively dark area, in the image signal of the second channel. A correction step of correcting a level difference between the image signals of the first channel and the second channel;
An image pickup apparatus control method comprising:

Images