JP5963542B2 - Image processing apparatus, control method thereof, and program - Google Patents

Description

  The present invention relates to an image processing apparatus having variable optical parameters such as a focal length of a zoom lens, a focus lens position, and an optical aperture amount, and a control method and program thereof.

  In an imaging device such as a video camera or digital camera, a zoom lens for shooting a distant subject up, a focus lens for focusing on the subject to be photographed, and an appropriate exposure state for subjects with various amounts of light A lens unit composed of an optical diaphragm or the like for adjusting the lens is mounted. The optical image projected by the lens unit is photoelectrically converted by a solid-state imaging device such as a CCD or CMOS to generate a video signal, which is displayed on a monitor or recorded on a recording medium.

In such a lens unit, it is also well known that characteristic deterioration called aberration or light quantity drop due to physical phenomena such as light diffraction and lens refractive index occurs.
Aberrations particularly noted for lens units such as video cameras are lateral chromatic aberration, axial chromatic aberration, and distortion. In addition, with regard to the drop in the amount of light, attention is paid to the drop in the amount of peripheral light that is particularly noticeable at the four corners of the shooting angle of view. In general, these aberrations and peripheral light loss are designed so as not to occur more than a predetermined amount in the lens unit design.

  However, in recent years, such aberration and peripheral light loss correction have been corrected by signal processing to make it inconspicuous in the final output image, thereby ensuring the freedom of lens unit design and aiming at cost reduction, Techniques have been proposed to realize lens unit design aimed at improving lens performance. Among them, the chromatic aberration of magnification, distortion, and peripheral light loss hardly depend on the subject and are point-symmetric with respect to the center of the optical axis according to three optical parameters such as the focal length of the zoom lens, the focus lens position, and the optical aperture amount. It becomes a phenomenon. Therefore, a technique has been proposed in which correction characteristics corresponding to a predetermined optical parameter state are stored in advance as a correction database, and point-symmetric correction processing is performed with respect to the optical axis center in accordance with the optical parameter state.

  The peripheral light amount drop is a phenomenon in which the light amount attenuates point-symmetrically with respect to the center of the optical axis, and thus is realized by performing point-symmetric gain processing with respect to the center of the optical axis. Further, since the chromatic aberration of magnification is a phenomenon of image-forming position deviation for each wavelength of light that is point-symmetric with respect to the optical axis center, coordinate conversion processing that is point-symmetric with respect to the optical axis center is performed by, for example, RGB (red green blue) three primary colors. This is performed for R and B of the image, and the image positions of RGB are aligned. Further, since distortion is a phenomenon called image position distortion that is point-symmetric with respect to the center of the optical axis, it can be realized by performing coordinate transformation processing that is point-symmetric with respect to the center of the optical axis, similar to correction of lateral chromatic aberration.

  Further, when realizing the peripheral light amount drop correction, the magnification chromatic aberration correction, and the distortion aberration correction at the same time, there arises a problem of how the correction order is appropriate as the final correction image quality. In order to deal with this problem, in Japanese Patent Application Laid-Open No. H11-228707, correction without coordinate conversion such as peripheral light amount drop correction is performed before performing correction with coordinate conversion such as magnification chromatic aberration correction or distortion aberration correction. Is good. The reason is that if the correction accompanied by the coordinate conversion is performed first, the correction gain of the peripheral light amount drop correction without the coordinate conversion cannot be correctly applied to the image after the coordinate conversion.

JP 2002-190979 A

  The characteristics of peripheral light loss, lateral chromatic aberration, and distortion inherently exhibit various aspects depending on the state of the optical parameters described above. Furthermore, when another correction is performed after any of these corrections, the accuracy of the correction performed later is reduced due to the influence of the correction performed earlier. Therefore, in order to make the final correction image quality appropriate, it is necessary to consider which correction is emphasized as the correction accuracy according to the optical parameters described above. No consideration is given.

  According to a study by the present inventor, the peripheral light amount drop has a characteristic that changes relatively depending on the focal length of the zoom lens and the optical aperture amount. Specifically, as the focal length is shorter (wide), the distance from the center of the optical axis (hereinafter referred to as the image height) where the peripheral light amount starts to decrease is farther (hereinafter referred to as the higher image height), In addition, the decrease in the amount of peripheral light often has a characteristic of sharply bending with respect to the image height. Conversely, the longer the focal length (telephoto), the lower the image height at which the peripheral light amount starts to drop, and the peripheral light amount drop often has a characteristic of bending gently with respect to the image height.

  Further, according to the study by the present inventor, the characteristics of the chromatic aberration of magnification and distortion change relatively depending on the focal length of the zoom lens and the focus lens position. Specifically, aberrations often increase particularly when the focal length and the focus lens position are close. However, aberrations do not necessarily occur even at other focal lengths or focus lens positions.

Further, according to a study by the present inventor, when the correction is excessive or insufficient, the influence on the corrected image quality increases in the order of chromatic aberration of magnification, distortion, and decrease in peripheral light amount. In particular, the chromatic aberration of magnification is the most noticeable because it is a phenomenon in which a color that is not supposed to be originally developed on the edge of an achromatic object. Distortion is a phenomenon in which the subject edge is distorted, but is less noticeable than lateral chromatic aberration in that the color does not develop. The decrease in the amount of peripheral light becomes conspicuous only in the case of a flat subject such as the sky.
This is not a problem related only to an imaging apparatus capable of correcting aberrations and peripheral light loss. A similar problem may occur in an arithmetic device having an image processing function, such as a personal computer provided with an application that acquires an image obtained by the imaging device and corrects the aberration and peripheral light loss of the obtained image.

  The present invention has been made in view of the above points. For example, when performing optical correction such as magnification chromatic aberration correction, distortion aberration correction, and light quantity drop correction, the final correction image quality can be made appropriate. With the goal.

The image processing apparatus of the present invention includes a first optical correction unit that performs light amount drop correction on an image signal, and a second optical correction that performs at least one of magnification chromatic aberration correction and distortion aberration correction on the image signal. And when the optical parameter of the imaging optical system used when the image signal is generated by the image sensor is set to the first value, aberration correction is performed by the second optical correction unit. Then, when the light quantity drop correction is performed by the first optical correction means and the optical parameter is set to the second value, the light quantity drop correction is performed by the first optical correction means , and then the second light correction is performed. Control means for performing aberration correction by the optical correction means, and when the optical parameter is set to the first value, compared to the case where the optical parameter is set to the second value, The second optical auxiliary Wherein the correction amount of the aberration correction by means increases.

  According to the present invention, the final correction image quality can be made appropriate when performing optical correction by switching the correction order of optical correction according to the optical parameters.

It is a figure which shows the structure of the video camera which concerns on 1st Embodiment. It is a figure which shows the detailed structure of a magnification chromatic aberration correction value production | generation part. It is a figure which shows the detailed structure of a magnification chromatic aberration correction part. It is a figure for demonstrating optical correction plot data. It is a figure which shows the concept which correct | amends phase shift. It is a figure for demonstrating the process by a magnification chromatic aberration correction part. It is a control diagram for correction | amendment order switching control. It is a flowchart which shows the correction order switching control in 1st Embodiment. It is a figure which shows the structure of the video camera which concerns on 2nd Embodiment. It is a flowchart which shows the correction | amendment order switching control in 2nd Embodiment.

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.
(First embodiment)
FIG. 1 is a diagram showing a configuration of a video camera according to the first embodiment, which is an example of an image processing apparatus to which the present invention is applied. Reference numeral 101 denotes an imaging optical system, which can set the focal length of the zoom lens, the focus lens position, and the optical aperture amount as control targets as variable optical parameters. Reference numeral 102 denotes an RGB (red green blue) three-plate image area sensor which is an image sensor. The image sensor 102 may be a single-plate image area sensor in which pixels corresponding to a plurality of color filters are arranged in a predetermined pattern like a Bayer array. Reference numeral 103 denotes an analog front end (hereinafter abbreviated as AFE) composed of a correlated double sampling and analog-to-digital converter.

  Reference numeral 104 denotes a first switching unit. A light amount drop correction unit 105 functions as a first optical correction unit. Reference numeral 106 denotes a second switching unit. A magnification chromatic aberration correction unit 107 functions as a second optical correction unit. Reference numeral 108 denotes a third switching unit. Reference numeral 109 denotes a camera signal processing unit.

  Reference numeral 110 denotes an optical system driving unit. Reference numeral 111 denotes a sensor driving unit. Reference numeral 112 denotes a light quantity drop correction value generation unit. Reference numeral 113 denotes a magnification chromatic aberration correction value generation unit. Reference numeral 114 denotes a correction order switching determination unit. Reference numeral 115 denotes a synchronization signal generator. Reference numeral 116 denotes an image height calculation unit. Reference numeral 117 denotes a microcomputer. Reference numeral 118 denotes an optical correction database. Reference numeral 119 denotes an AE (automatic exposure) evaluation value generation unit. Reference numeral 120 denotes an AF (autofocus) evaluation value generation unit. Reference numeral 121 denotes a zoom operation input unit.

  As a signal and data shown in FIG. 1, S101 is a RAW image signal. S102 is a first switching unit output image signal. S103 is a light quantity drop corrected image signal. S104 is a second switching unit output image signal. S105 is a magnification chromatic aberration corrected image signal. S106 is a third switching unit output image signal. S107 is a camera signal processed image signal. In the first embodiment, the RAW image signal S101, the first switching unit output image signal, the light amount drop corrected image signal S103, the second switching unit output image signal S104, the magnification chromatic aberration corrected image signal S105, and The third switching unit output image signal S106 is an RGB3 channel image signal corresponding to each color of the RGB3 plate image area sensor 102.

  S108 is an imaging optical system drive signal for driving a zoom / focus / aperture motor. S109 is a sensor driving signal for accumulating / reading the photocharge. S110 is a light quantity drop correction value. S111 is a magnification chromatic aberration correction value. S112 is a correction order switching signal. S113 is a horizontal and vertical synchronizing signal. S114 is image height data. S115 is imaging optical system drive setting data for instructing the zoom / focus / aperture position. S116 is sensor drive setting data for instructing an electronic shutter amount or the like. S117 is center coordinate data that indicates the relative position between the sensor center position and the optical center position. S118 is light quantity drop characteristic data corresponding to the zoom / focus / aperture position. S119 is magnification chromatic aberration characteristic data corresponding to the zoom / focus / aperture position.

  S121 is camera signal processing setting data such as white balance gain. S122 is optical characteristic data. S123 is an AE evaluation signal. S124 is an AF evaluation signal. S125 is AE evaluation data. S126 is AF evaluation data. S127 is a zoom control signal.

  FIG. 2 is a diagram illustrating a detailed configuration of the magnification chromatic aberration correction value generation unit 113. Although details will be described later, 241 is a function coefficient calculation circuit, 242 is a correction value calculation circuit, 243 is an XY vector coefficient calculation circuit, and 244 and 245 are multipliers.

  FIG. 3 is a diagram showing a detailed configuration of the magnification chromatic aberration correction unit 107. Although details will be described later, 301 and 302 are interpolation control circuits, 303 is a buffer memory, 304 to 311 are horizontal interpolation circuits, and 312 is a vertical interpolation circuit.

Next, the operation of the video camera according to the first embodiment will be described.
An optical image formed on the light receiving surface of the image area sensor 102 through the imaging optical system 101 is photoelectrically converted by the image area sensor 102 and generated as a sensor output video signal. The sensor output video signal generated by the image area sensor 102 is correlated double sampled and A / D converted by the AFE 103 to generate a RAW image signal S101.

  In the light quantity drop correction unit 105, correction is performed to increase the gain of a peripheral light quantity drop region expressed by the imaging optical system 101 with respect to the image signal S <b> 102 (image signal S <b> 101 or S <b> 105) switched by the first switching unit 104. Made. As a result, the light quantity drop corrected image signal S103 is generated. This correction is performed by the light amount drop correction value S110 calculated by the light amount drop correction value generation unit 112 based on the image height data S114 generated by the image height calculation unit 116 and the light amount drop characteristic data S118 set by the microcomputer 117. Based on.

The lateral chromatic aberration correction unit 107 corrects lateral chromatic aberration generated by the imaging optical system 101 by local scaling processing on the image signal S104 (image signal S101 or S103) switched by the second switching unit 106. Is made. As a result, a magnification chromatic aberration corrected image signal S105 is generated. This correction is performed on the magnification chromatic aberration correction value S111 calculated by the magnification chromatic aberration correction value generation unit 113 based on the image height data S114 generated by the image height calculation unit 116 and the magnification chromatic aberration characteristic data S119 set by the microcomputer 117. Based on.
Details of the operation of generating the magnification chromatic aberration correction value S111 by the magnification chromatic aberration correction value generating unit 113 and the details of the operation of correcting the chromatic aberration of magnification by the magnification chromatic aberration correcting unit 107 will be described later with reference to FIGS. Details of the operation of generating the image height data S114 by the image height calculation unit 116 will also be described later.

  The camera signal processing unit 109 performs well-known camera signal processing such as gamma processing, white balance processing, and matrix processing on the image signal S106 (image signal S103 or S105) switched by the third switching unit 108. The As a result, a camera signal processed image signal S107 is generated. This processing is performed based on camera signal processing setting data S121 set by the microcomputer 117. The camera signal processed image signal S107 is recorded or displayed as a moving image in a recording unit or a display unit (not shown). Further, the camera signal processing unit 109 uses, from the input third switching unit output image signal S106, an AE evaluation signal S123 for use in automatic exposure control, and an AF evaluation signal S124 for use in automatic focus control. Are transmitted to the AE evaluation value generation unit 119 and the AF evaluation value generation unit 120, respectively.

  The image area sensor 102 is driven by a sensor drive signal S109 generated by the sensor drive unit 111, thereby reading a sensor output video signal. In the sensor driving unit 111, the sensor driving signal S109 is generated in synchronization with the horizontal and vertical synchronizing signal S113 generated by the synchronizing signal generating unit 115. Therefore, the sensor output video signal is synchronized with the horizontal and vertical synchronizing signal S113. Will be. Further, the accumulation time is controlled by the sensor drive setting data S116 set by the microcomputer 117.

  The image height calculation unit 116 generates a horizontal coordinate and a vertical coordinate from the horizontal and vertical synchronization signal S113 generated by the synchronization signal generation unit 115, and performs a process of performing a polar coordinate conversion, thereby obtaining a distance from the center position. A certain image height data is generated. At this time, by providing the central coordinate data S117 by the microcomputer 117, it is also possible to generate image height data S114 following the optical camera shake correction when the imaging optical system 101 includes an optical camera shake correction unit. . If the center coordinate data S117 includes an optical center position Cx in the horizontal direction and an optical center position Cy in the vertical direction, the center position (Cx, Cy) is the origin from the horizontal coordinate Xt and the vertical coordinate Yt. Polar coordinates in the polar coordinate system can be expressed by the following equations (1) and (2). The image height calculation unit 116 obtains polar coordinate values Rt and θt by the calculation of the above formula, and outputs the image height data S114 with these values. Square root calculation represents known methods such as dichotomy and square root extraction with finite word length accuracy, and atan calculation is a high-order function approximation or a piecewise low-order function approximation in which the XY ratio is divided into predetermined ranges. Therefore, it can also be realized by hardware.

  The AE evaluation value generation unit 119 generates AE evaluation data S125 by performing a well-known photometric integration operation on the AE evaluation signal S123. The microcomputer 117 performs an operation of reading the AE evaluation data S125 at a predetermined timing. Further, the microcomputer 117 generates the imaging optical system drive setting data S115 based on the AE evaluation data S125 and controls the optical system drive unit 110. Thereby, the image forming optical system drive signal S108 is controlled, the optical aperture amount included in the image forming optical system 101 is adjusted, or the sensor drive setting data S116 is generated to control the sensor drive unit 111. By adjusting the accumulation time of the RGB three-plate image area sensor 102 by generating the sensor drive signal S109, or by adjusting the gain in the camera signal processing unit 109 by generating the camera signal processing setting data S121. AE operation is realized.

  The AF evaluation value generation unit 120 generates AF evaluation data S126 by performing a known high-frequency detection integration operation on the AF evaluation signal S124. The microcomputer 117 performs an operation of reading the AF evaluation data S126 at a predetermined timing. Further, the microcomputer 117 generates the imaging optical system drive setting data S115 based on the AE evaluation data S125 and controls the optical system drive unit 110. Accordingly, the AF operation is realized by controlling the imaging optical system drive signal S108 and adjusting the position of the focus lens included in the imaging optical system 101.

The microcomputer 117 detects the zoom control signal S127 from the zoom operation input unit 121 by the user operation, generates the imaging optical system drive setting data S115, and controls the optical system drive unit 110. Thus, the imaging optical system drive signal S108 is controlled to adjust the focal length of the zoom lens included in the imaging optical system 101.
Further, the microcomputer 117 is based on the optical correction data (optical correction characteristic information) stored in advance in the optical correction database 118, based on the imaging optical system drive setting data S115, and the focal length of the zoom lens, the focus lens position, and the optical lens. Optical correction data that matches or is close to the aperture amount is selected and read. Then, light quantity drop characteristic data S118 and magnification chromatic aberration characteristic data S119 are generated that represent the light quantity drop and magnification chromatic aberration of the imaging optical system 101, which are generated by the focal length of the zoom lens, the focus lens position, and the optical aperture amount. Generation of the light quantity drop characteristic data S118 and the magnification chromatic aberration characteristic data S119 will be described.

The optical correction data held in the optical correction database 118 is, for example, plot data directly indicating the magnification chromatic aberration correction amount or light amount drop correction amount at a predetermined image height position. The coefficient term of the function approximating these correction amount characteristics. May be held. In the optical correction database 118, the information amount may be determined in advance according to the memory capacity allowed by the video camera and the tendency of the characteristic change amount of the optical system. However, since the memory capacity is finite, the information is discrete I can only have it. Therefore, the optical parameter area without information, the image height position, and the angle are compensated by interpolation from previous and subsequent information.
First, the microcomputer 117 selects, for example, two Oa and OB near the current optical parameter from the optical correction characteristic information held in the optical correction database 118. As shown in FIG. 4A, the optical correction database 118 holds optical correction plot data cfa1 to cfa4 for the optical parameter Oa and optical correction plot data cfb1 to cfb4 for the optical parameter Ob. The microcomputer 117 weights the interpolation processing of the two correction amount plot data given to the image height according to the degree of deviation between the current optical parameter and Oa and Ob. Thereby, four correction plot data of cm1 to cm4 as shown in FIG. 4B are calculated and output as a correction amount plot data set Cm. The correction amount plot data set Cm generated in this way is generated as the light quantity drop characteristic data S118 and the magnification chromatic aberration correction data S119.

Next, the correction operation of the light quantity drop correction unit 105 will be described.
The light quantity drop characteristic data S118 is data indicating what correction gain should be applied according to the image height. The light amount drop correction unit 105 determines a correction gain based on the input light amount drop characteristic data S118 and the polar coordinate value Rt of the image height data S114, and performs gain correction on the first switching unit output image signal S102. Realize light loss correction.

Next, the correction operation of the magnification chromatic aberration correction value generation unit 113 will be described.
FIG. 2 is a diagram illustrating a configuration of the magnification chromatic aberration correction value generation unit 113. Image height data S114 indicating polar coordinate values Rt and θt at the current pixel of interest and correction amount plot data sets Cm1 to Cm4 output from the microcomputer 117 are input to the function coefficient calculation circuit 241.
As shown in FIG. 4C, the function coefficient calculation circuit 241 obtains a plot section to which the polar coordinate value Rt of the current pixel of interest belongs (thick black line section), and approximates coefficients a and b corresponding to the plot section. , C are output. Each plot section in the above processing is approximated by, for example, a quadratic function (3) below. It is also possible to make this a linear function (polygonal line approximation) or a higher-order function of third order or higher.

The correction value calculation circuit 242 calculates the correction amount Zt from the polar coordinate value Rt for the current pixel of interest based on the above formula. The correction amount Zt indicates a correction amount when viewed in the θt direction in the polar coordinate system. As will be described later, the magnification chromatic aberration correction unit 107 performs interpolation processing in a horizontal and vertical manner. For this reason, the lateral chromatic aberration correction value generation unit 113 performs vector decomposition on the obtained correction amount Zt in the horizontal and vertical directions (abbreviated as XY directions).
The XY vector coefficient calculation circuit 243 obtains vector coefficients Vx and Vy for vector decomposition of the correction amount Zt in the XY direction from the polar coordinate values Rt and θt at the target pixel t by the following expression (4).

The cos θ and sin θ operations can be realized by hardware by high-order function approximation or by piecewise low-order function approximation in which the polar coordinate value θ is divided into sections in a predetermined range.
The vector coefficients Vx and Vy are multiplied by the correction amount Zt by the multipliers 244 and 245, and the correction amounts ZtH and ZtV in the XY directions are obtained by the following equation (5).

The correction amounts ZtH and ZtV in the XY directions obtained in this way are independent of the RGB3 channel image signal corresponding to each color of the RGB3 plate type image area sensor 102 in the case of lateral chromatic aberration correction, and are shifted in phase in the horizontal and vertical directions. This is the amount of phase shift separated as a component. In the case of distortion correction, the correction amounts ZtH and ZtV are common to the RGB three-channel image signals corresponding to the respective colors of the RGB three-plate image area sensor 102 and are phase shift amounts separated as horizontal and vertical phase shift components. is there.
In this example, the optical correction database 118 holds plot data that directly indicates the optical correction amount, but may hold coefficient terms of a function that approximates the correction characteristics. The correction amount plot data is replaced with the order of the approximation function.

Next, the correction operation of the magnification chromatic aberration correction unit 107 will be described in detail.
In the case of lateral chromatic aberration correction and distortion aberration correction, the optical correction value is a phase shift amount obtained by separating the distortion amount at the target pixel position as a phase shift component in the horizontal and vertical directions. FIGS. 5A to 5C are diagrams showing the concept of correcting this phase shift, and for the sake of simplicity, the interpolation calculation is performed on four pixels in the vicinity of the target pixel position.
The black pixel in FIG. 5A indicates the phase that the target pixel S should be originally, and the pixel represented by a dot is a position where the target pixel S is imaged with a phase shift due to the influence of lateral chromatic aberration or distortion. Is a virtual pixel S ′.
In order to correct lateral chromatic aberration and distortion, a virtual pixel S ′ whose Hp is shifted in the horizontal direction and Vp phase is shifted in the vertical direction may be obtained and rearranged at the position of the pixel of interest S. As shown in FIG. 5B, the virtual pixel S ′ is a pixel of pixels s1, s2, s3, s4 and a virtual pixel S ′ from pixels s1, s2, s3, s4 that are actually imaged in the vicinity. The distances c1, c2, c3, and c4 can be generated by performing weighted interpolation calculation.
As shown in FIG. 5C, the generated virtual pixel S ′ is replaced with the position of the pixel of interest S, and the lateral chromatic aberration and distortion are corrected.

FIG. 3 is a diagram illustrating a configuration of the magnification chromatic aberration correction unit 107, and illustrates a configuration in which the processing described with reference to FIGS. 5A to 5C is implemented by performing an interpolation operation on 64 pixels near the target pixel.
The correction amounts ZtH and ZtV input to the lateral chromatic aberration correction unit 107 are input to the interpolation control circuits 301 and 302 and output integer horizontal and vertical phase shift components as Hp and Vp, respectively. The interpolation control circuits 301 and 302 output decimal horizontal and vertical phase shift components as interpolation coefficients ch0 to ch7 and cv0 to cv7.
The buffer memory 303 holds the input signal S over a plurality of pixels sequentially in the horizontal and vertical directions, and the vicinity of the virtual pixel S ′ around the address shifted by Hp and Vp from the target pixel position S as shown in FIG. The reference pixels s00 to s77 are read out simultaneously. Note that a combination of left and right pixel positions 0 to 7 and upper and lower pixel positions 0 to 7 in FIG. 6 indicates a pixel number (s00 or the like) indicating 64 pixels near the virtual pixel S ′.
The reference pixels s00 to s77 output from the buffer memory 303 are input to horizontal interpolation circuits 304 to 311 provided for each line, and interpolation coefficients ch0 to ch7 corresponding to each reference pixel are input to the horizontal direction. The reference pixels s0 ′ to s7 ′ subjected to the interpolation processing are output. The horizontal interpolation circuits 304 to 311 all have the same configuration. For example, the reference pixels s00 to s07 are respectively multiplied by the interpolation coefficients ch0 to ch7, the average value thereof is taken, and the reference pixel s0 ′ subjected to horizontal interpolation processing is obtained. It becomes. The same applies to s1 ′ to s7 ′.
The reference pixels s0 ′ to s7 ′ output from the horizontal interpolation circuits 304 to 311 are input to the vertical interpolation circuit 312, and interpolation coefficients cv0 to cv7 corresponding to the respective reference pixels are input. As a result, the reference pixels s0 ′ to s7 ′ and the interpolation coefficients cv0 to cv7 are respectively multiplied, the average value is taken, and the virtual pixel S ′ that has been subjected to interpolation processing in the vertical direction is output.
The virtual pixel S ′ generated in this way is the result of correcting the lateral chromatic aberration and distortion as described above.

The imaging optical system drive setting data S115 is also input to the correction order switching determination unit 114. The correction order switching determination unit 114 determines the correction order switching signal S112 according to FIG. 7A or FIG. 7B showing a control diagram corresponding to the value of the imaging optical system drive setting data S115 and the flowchart shown in FIG. To do.
FIG. 7A is a control diagram of the correction order switching signal S112 according to the focal length of the zoom lens (denoted as Zoom) and the aperture amount of the optical diaphragm (denoted as Fno), and is represented by the following equation (6). It is a linear characteristic.
FIG. 7B is a control diagram of the correction order switching signal S112 according to the focal length of the zoom lens (denoted as Zoom) and the focus lens position (denoted as Focus), and is a linear characteristic represented by the following equation (7). It is.

Zoom = K ₁ Fno + C ₁ (6)
K ₁ and C ₁ are fixed coefficients and constant terms
Zoom = K ₂ Fno + C ₂ (7)
K ₂ and C ₂ are fixed coefficients and constant terms

The imaging optical system drive setting data S115 is Zoom, Fno, and Focus shown in FIGS. 7A and 7B. Therefore, when the imaging optical system drive setting data S115 is given from the microcomputer 117, the correction order switching signal S112 is obtained by applying the linear characteristics shown in FIGS. 7A and 7B and comparing the values. decide. This determination process will be described with reference to the flowchart of the correction order switching control in FIG.
Processing starts at the timing when the imaging optical system drive setting data S115 is updated. In step S1, the first determination according to the equation (6) is performed, and in steps S2 and S3, the correction order switching signal S112 is determined. That is, the correction order switching signal S112 is set to 1 in a region where the focal length of the zoom lens is at the tele end or near the tele end, or in a region where the aperture amount of the optical aperture is small from F5.6 or less, and otherwise. The correction order switching signal S112 is set to 0.
Next, in step S4, the second determination according to the equation (7) is performed, and in step S5, the correction order switching signal S112 is updated. That is, when the focal length of the zoom lens is a macro photography region close to the wide end and the focus lens position is a close range region of 10 cm or less, the correction order switching signal S112 is updated to 1, and in other cases Is the value of the correction order switching signal S112 determined in steps S2 and S3. Thereafter, the process ends.

As a result of the operation of the correction order switching determination unit 114 described above, the focal length and the focus lens position are in the vicinity of the macro imaging region, the focal length is closer to the tele end, or the aperture amount of the optical aperture is closer to the open side than F5.6. In this case, the correction order switching signal S112 becomes 1, and in other cases, the correction order switching signal S112 becomes 0.
Accordingly, when the focal length and the focus lens position are close to the macro imaging region, or when the focal length is close to the tele end and the aperture amount of the optical aperture is closer to the open side than F5.6 due to the operation of the correction order switching determination unit 114. The first switching unit 104, the second switching unit 106, and the third switching unit 108 are connected to one side. As a result, the first switching unit output image signal S102 is corrected for chromatic aberration of magnification S105, the second switching unit output image signal S104 is a RAW image signal S101, and the third switching unit output image signal S106 is corrected for light loss. The image signal S103 is obtained. Therefore, the camera signal processing unit 109 receives an image signal that has been corrected for chromatic aberration of magnification first and then corrected for light loss.
In this way, in the vicinity of the macro shooting region where the lateral chromatic aberration is remarkable and in the small aperture to the open position from the telephoto end where the light drop is gentle with respect to the image height, the lateral chromatic aberration is corrected by performing the lateral chromatic aberration correction first. Optical correction can be realized with emphasis on correction accuracy.

On the other hand, in other cases, the first switching unit 104, the second switching unit 106, and the third switching unit 108 are connected to the 0 side. Accordingly, the first switching unit output image signal S102 is the RAW image signal S101, the second switching unit output image signal S104 is the light amount drop corrected image signal S103, and the light amount drop corrected image signal S103 is the magnification chromatic aberration corrected image signal. S105 is performed. Therefore, the camera signal processing unit 109 receives an image signal that has been subjected to light amount drop correction first and then subjected to magnification chromatic aberration correction.
In this way, under conditions where the chromatic aberration of magnification is not so noticeable and the light quantity drop occurs steeply with respect to the image height, an optical system that emphasizes the correction accuracy of the light quantity drop correction by performing the light quantity drop correction first. Correction can be realized.

  As a result of the operations described above, it is possible to achieve a light amount drop correction and a magnification chromatic aberration correction with a small correction error comprehensively from the light amount drop characteristics and magnification chromatic aberration characteristics with respect to the zoom / focus / aperture position, and make the final correction image quality appropriate. be able to. It should be noted that the same effect can be obtained by replacing the lateral chromatic aberration correction unit of this embodiment with a distortion correction unit.

(Second Embodiment)
FIG. 9 is a diagram showing a configuration of a video camera according to the second embodiment, which is an example of an image processing apparatus to which the present invention is applied. In addition, the same code | symbol is attached | subjected to the component, signal, and data similar to 1st Embodiment, and the description is abbreviate | omitted.
A correction order history database 922 holds a correction order history. S928 is correction order switching determination data. The second embodiment differs from the first embodiment in that a correction order switching determination unit 114 that generates a correction order switching signal S112 using the correction order history database 922 and the correction order switching determination data S928. Is the action. This will be described below with reference to the flowchart of FIG.

Processing starts at the timing when the imaging optical system drive setting data S115 is updated. In step S11, the first determination according to the equation (6) is performed, and in steps S12 to S15, the correction order switching determination data S928 is referenced to determine whether or not the correction order switching signal S112 is updated. That is, when it is desired to set the correction order switching signal S112 to 0, if the correction order switching determination data S928 is also 0 (step S12), the correction order switching signal S112 is set to 0 (step S14). If the correction order switching signal S112 is to be set to 1, if the correction order switching determination data S928 is also 1 (step S13), the correction order switching signal S112 is set to 1 (step S15). Otherwise, the correction order switching signal S112 is not updated, and the process proceeds to step S19.
Next, in step S16, a second determination according to the equation (7) is performed. Next, in steps S17 and S18, whether or not the correction order switching signal S112 is updated with reference to the correction order switching determination data S928. Determine. When the correction order switching signal S112 is to be set to 1, if the correction order switching determination data S928 is also 1 (step S17), the correction order switching signal S112 is set to 1 (step S18). Otherwise, the correction order switching signal S112 is not updated, and the process proceeds to step S19.
Next, in step S19, the value of the correction order switching signal S112 is updated as correction order switching determination data S928. As a result, the correction order switching signal S112 is stored in the correction order history database 922 as the past correction order history. Thereafter, the process ends.

Through the above operation, not only the first and second determinations according to the equations (6) and (7) but also the past determination history can be reflected from the correction order switching determination data S928. Thereby, it is possible to prevent the light quantity drop correction and the magnification chromatic aberration correction described in the first embodiment from being frequently switched, and to realize a more stable and high quality light quantity drop correction and magnification chromatic aberration correction. . In the present embodiment, the same effect can be obtained by replacing the magnification chromatic aberration correction unit with a distortion correction unit.
In addition, the reference to the correction order switching determination data S928 in the present embodiment is a reference to the previous result, but the number of times is not limited to the previous one.
In the first and second embodiments, a video camera has been described as an example. However, a function for correcting aberration and peripheral light amount drop for an image to which variable optical parameter information is given. The present invention can be applied to any image processing apparatus provided.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

101: Imaging optical system, 102: RGB three-plate image area sensor, 103: AFE, 104: First switching unit, 105: Light quantity drop correction unit, 106: Second switching unit, 107: Chromatic chromatic aberration correction unit, 108 : Third switching unit, 109: camera signal processing unit, 110: optical system driving unit, 111: sensor driving unit, 112: light quantity drop correction value generation unit, 113: magnification chromatic aberration correction value generation unit, 114: correction order switching Determination unit, 115: synchronization signal generation unit, 116: image height calculation unit, 117: microcomputer, 118: optical correction database, 119: AE evaluation value generation unit, 120: AF evaluation value generation unit, 121: zoom operation input unit, 122: Correction order history database

Claims (6)

First optical correction means for performing light quantity drop correction on the image signal;
A second optical correction unit that performs aberration correction of at least one of lateral chromatic aberration correction and distortion correction on the image signal;
When the optical parameter of the imaging optical system used when the image signal is generated by the image sensor is set to the first value, after the aberration correction by the second optical correction unit, the first optical correction unit performs the first correction. When the optical parameter is set to the second value and the optical parameter is set to the second value, the light aberration correction by the second optical correction unit is performed after the light amount correction is performed by the first optical correction unit. Control means for controlling to perform correction ,
When the optical parameter is set to the first value, the correction amount of the aberration correction by the second optical correction unit is larger than when the optical parameter is set to the second value. An image processing apparatus.   The image processing apparatus according to claim 1, wherein the optical parameter is at least one of a focal length of a zoom lens, a focus lens position, and an optical aperture amount. Storage means for holding a history of correction orders of aberration correction by the second optical correction unit and light quantity drop correction by the first optical correction unit ;
The control means also refers to the history of the correction order , and determines which of the aberration correction by the second optical correction means and the light quantity drop correction by the first optical correction means is performed first. the image processing apparatus according to claim 1 or 2, characterized. And an imaging element,
An imaging optical system ;
First optical correction means for performing light quantity drop correction on the image signal generated by the image sensor;
A second optical correction unit that performs aberration correction of at least one of lateral chromatic aberration correction and distortion correction on the image signal;
When the optical parameter of the imaging optical system used when generating the image signal is set to the first value, after the aberration correction by the second optical correction unit, the first optical When the light quantity drop correction is performed by the correction means and the optical parameter is set to the second value, the light quantity drop correction is performed by the first optical correction means, and then the aberration correction is performed by the second optical correction means. Control means to control to perform,
When the optical parameter is set to the first value, the correction amount of the aberration correction by the second optical correction unit is larger than when the optical parameter is set to the second value. An imaging device. With respect to the image signal and the first optical compensation means for performing light amount drop correction, and a second optical compensation means for performing at least one of the aberration correction before Symbol magnification chromatic aberration correction and distortion correction on the image signal An image processing apparatus control method comprising:
When the optical parameter of the imaging optical system used when the image signal is generated by the image sensor is set to the first value, after the aberration correction by the second optical correction unit, the first optical correction unit performs the first correction. When the optical parameter is set to the second value and the optical parameter is set to the second value, the light aberration correction by the second optical correction unit is performed after the light amount correction is performed by the first optical correction unit. have a step of controlling to perform the correction,
When the optical parameter is set to the first value, the correction amount of the aberration correction by the second optical correction unit is larger than when the optical parameter is set to the second value. For controlling an image processing apparatus. With respect to the image signal and the first optical compensation means for performing light amount drop correction, and a second optical compensation means for performing at least one of the aberration correction before Symbol magnification chromatic aberration correction and distortion correction on the image signal A program for controlling the image processing apparatus,
When the optical parameter of the imaging optical system used when the image signal is generated by the image sensor is set to the first value, after the aberration correction by the second optical correction unit, the first optical correction unit performs the first correction. When the optical parameter is set to the second value and the optical parameter is set to the second value, the light aberration correction by the second optical correction unit is performed after the light amount correction is performed by the first optical correction unit. Let the computer execute a process that controls to perform the correction ,
When the optical parameter is set to the first value, the correction amount of the aberration correction by the second optical correction unit is larger than when the optical parameter is set to the second value. program to be.

Images