JP6344909B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method therefor, and in particular, a pixel structure capable of phase difference focus detection is provided on an image sensor, and autofocus (hereinafter referred to as AF) control is performed using the phase difference focus detection result. It is related with the imaging device which implements.

  2. Description of the Related Art In recent years, there has been an imaging apparatus configured to read out accumulated charges by dividing some or all of pixels of an imaging image sensor into a plurality of photodiodes. For example, when the division is divided into two in the horizontal direction, an image generated only by the left side pixel (hereinafter referred to as A image) and an image generated only by the right side pixel (hereinafter referred to as B image) are read out independently. . A technique for calculating distance information to a subject by detecting an image shift amount corresponding to the distance to the subject formed by the A image and the B image in an image region targeted for AF is disclosed. . Hereinafter, it is referred to as imaging plane phase difference AF.

  In the imaging plane phase difference AF, an image shift amount corresponding to the distance to the subject formed by the A image and the B image is detected. As a method for this, a method of searching for a shift amount that maximizes the correlation between the A and B image data while shifting the A image and the B image by a predetermined shift amount, usually for each pixel, and using this as the image shift amount. Is taken. Here, the highest correlation may be defined as the highest correlation coefficient between the A and B image data itself, the sum of the absolute differences of the A and B image data, or the sum of squares of the differences, etc. It may be a definition that minimizes.

  In any case, it is important that a significant contrast is obtained for each of the A and B image data, and it is also known that the accuracy of the image shift amount is lowered unless a significant contrast is obtained.

  However, a shooting situation in which a significant contrast cannot be obtained can easily occur. For example, the focus lens position is greatly deviated with respect to the distance to the subject, and the entire subject is greatly blurred (hereinafter referred to as a large defocus state).

JP 2011-257444 A

  A method for obtaining the correlation between the A and B image data even in such a large defocus state is disclosed in Patent Document 1. According to this, the accuracy of the obtained correlation is improved even if the contrast of each of the A and B image data is somewhat low by widening the image area to be AF and increasing the amount of A and B image data.

  However, the conventional technique has a problem that the AF time is extended. That is, simply increasing the A / B image data amount requires time for calculating the correlation coefficient or calculating the sum of absolute differences, etc., and the AF time is extended.

  In addition, since the two-stage correlation coefficient is calculated or the sum of absolute differences is calculated, the AF time is excessively extended.

  Thus, with only the technique disclosed in Patent Document 1, it is difficult to achieve both AF time reduction and AF accuracy. Further, at the time of moving image shooting, if the above two-stage AF is repeated every time a large defocus state is entered, the quality of the AF operation deteriorates.

  The present invention has been made in view of the above problems, and provides an imaging apparatus that can achieve both AF time reduction and AF accuracy and can focus on a subject stably, and a control method thereof. One of the purposes.

In order to solve such a problem, as a technical feature of the present invention, there is provided an imaging means having a plurality of photoelectric conversion elements for one microlens, and the microlenses are arranged two-dimensionally. A control method for an imaging apparatus, wherein a setting step of setting a first focus detection area and a second focus detection area wider than the first focus detection area in an imaging screen, and the first and first Based on the output signals from the photoelectric conversion elements in the two focus detection areas, a pair of image signals for performing a phase difference type focus detection process is generated for each focus detection area, and the pair of image signals A detection step for detecting an image shift amount; and an acquisition step for acquiring an image shift amount used when performing focus adjustment from the image shift amount for each focus detection area detected by the detection step. A first image signal having a first resolution is generated from a pair of signals output from a photoelectric conversion element included in the first focus detection region, and a photoelectric signal included in the second focus detection region is generated. A second image signal having a second resolution lower than the first resolution is generated from a pair of signals output from the conversion element, and a first image shift amount is detected using the first image signal. Then, a second image shift amount is detected using the second image signal. In the obtaining step, the first image shift amount and the second image shift amount are weighted at a predetermined ratio. Thus, an image shift amount used for focus adjustment is obtained.

  According to the first feature of the present invention described above, it is possible to provide an imaging apparatus that can achieve both AF time reduction and AF accuracy and can focus on a subject stably, and a control method therefor.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. It is explanatory drawing of 1 pixel structure of the image sensor 102 of FIG. FIG. 2 is an explanatory diagram of a circuit configuration for one pixel of the image sensor 102 in FIG. 1. (A) The trimming pixel range of the trimming processing circuit 107 in FIG. (B) The pixel thinning range of the thinning processing circuit 109 in FIG. (C) It is another example of the trimming pixel range of the trimming processing circuit 107 of FIG. (D) Another example of the pixel thinning range of the thinning processing circuit 109 in FIG. 2 is an operation flowchart of a defocus amount calculation circuit 112 in FIG. 1. This is a target image area of AF corresponding to defocusing. It is the block diagram which showed the structure of the imaging device which concerns on the 2nd Embodiment of this invention. FIG. 8 is an explanatory diagram of a circuit configuration 702 for one pixel of the image sensor 102 in FIG. 7.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.

(First embodiment)
The first embodiment will be described with reference to FIGS. 1 to 5.

<Configuration of Imaging Device in First Embodiment>
FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to the first embodiment of the present invention. In FIG. 1, reference numeral 101 denotes an imaging optical system including a focus lens. Reference numeral 102 denotes an image sensor (also referred to as an image sensor). Reference numeral 103 denotes an A / B image addition circuit. Reference numeral 104 denotes a camera signal processing circuit. Reference numeral 105 denotes a display device such as a liquid crystal panel. Reference numeral 106 denotes a recording unit. Reference numeral 107 denotes a trimming processing circuit. Reference numeral 108 denotes a detailed area focus detection processing circuit. Reference numeral 109 denotes a thinning processing circuit. Reference numeral 110 denotes a wide area focus detection processing circuit. Reference numeral 111 denotes an image shift amount comparison circuit. Reference numeral 112 denotes a defocus amount calculation circuit. Reference numeral 113 denotes a focus lens driving circuit. Reference numeral 114 denotes a threshold setting circuit. In FIG. 1, S101 is a sensor A image signal, and S102 is a sensor B image signal. S103 is an imaging signal. S104 is a display / recording image signal. S105 is a detailed area A image signal, and S106 is a detailed area B image signal. S107 is a thinned A image signal, and S108 is a thinned B image signal. S109 is a detailed area focus detection reliability flag, and S110 is a wide area focus detection reliability flag. S111 is a detailed area image shift amount, and S112 is a wide area image shift amount. S113 is a detailed / wide area image shift amount comparison result. S114 is defocus amount data. S115 is a lens driving signal. S116 is a detailed area focus detection reliability determination threshold value. S117 is a wide area focus detection reliability determination threshold value. S118 is an image shift amount comparison threshold value. In the present embodiment, the trimming processing circuit 107 and the thinning-out processing circuit 109 are combined to serve as a setting unit 115 that sets a focus detection area in the imaging screen.

<One pixel configuration of image sensor>
FIG. 2 is an explanatory diagram of a one-pixel configuration of the image sensor 102 of FIG. In FIG. 2, 201 is a left side portion (subpixel) of the left and right divided pixels, 202 is a right side portion (subpixel) of the left and right divided pixels, and 203 is a micro lens.

<Circuit configuration for one pixel of image sensor>
FIG. 3 is an explanatory diagram of a circuit configuration for one pixel of the image sensor 102 of FIG. In FIG. 3, reference numeral 301 denotes a circuit showing the left-side portion 201 divided into left and right in FIG. 302 is a circuit showing the right-side portion 202 divided into right and left in FIG. Reference numeral 3011 denotes a photodiode (hereinafter referred to as PD). Reference numeral 3012 denotes a floating diffusion (hereinafter referred to as FD). Reference numeral 3013 denotes a read transistor. Reference numeral 3014 denotes a reset transistor. Reference numeral 3015 denotes a source follower amplifier. Reference numeral 3016 denotes a row selection transistor. Note that the photodiode 3011 is also referred to as a photoelectric conversion element. In the present embodiment, a single microlens has a plurality of photoelectric conversion elements, and the microlenses are arranged two-dimensionally.

  Each of the constituent elements 3011 to 3016 is the configuration of the circuit 301 of the left and right divided part 201 in FIG. 2, and similarly shows the right and left divided part 202 in FIG. 2 by the same end number in 3021 to 3026. It is a component of the circuit 302.

  Reference numeral 303 denotes a left image (sensor A image) column ADC, 304 denotes a right image (sensor B image) column ADC, 305 denotes a sensor A image horizontal drive circuit, and 306 denotes a sensor B image horizontal drive circuit.

  S3011 is a power supply, S3012 is a reset control signal, S3013 is a read control signal, S3014 is a row selection signal, and is an internal signal of the circuit 301. Similarly, the internal signal of the circuit 302 is determined by the same end number in S3021 to S3024.

  S303 is a sensor A image vertical signal line, S304 is a sensor B image vertical signal line, S305 is a sensor A image signal, and S306 is a sensor B image signal.

  The subject image is projected onto the image sensor 102 through the imaging optical system including the focus lens 101 and subjected to photoelectric conversion.

  As shown in FIG. 2, the image sensor 102 is a sensor having a structure in which a left side portion 201 of a pixel obtained by dividing each pixel into left and right portions and a right side portion 202 of the divided pixels are shared by a microlens 203. For this reason, a pair of subject images formed by passing light fluxes of exit pupils in different ranges are projected by the left side portion 201 of the left and right divided pixels and the right side portion 202 of the left and right divided pixels.

  As shown in FIG. 3, the image sensor 102 includes a left side portion 201 of the left and right divided pixels and a right side portion 202 of the left and right divided pixels, respectively, and a circuit 301 indicating the left and right divided left portions and a right side divided right and left. The circuit 302 is shown as a part. A pair of subject images formed by light fluxes passing through exit pupils in different ranges are independently photoelectrically converted and output as sensor A image signal S305 and sensor B image signal S306.

  The outline of the operation of the image sensor is as follows. That is, the PD 3011 and the FD 3012 are reset to the power supply level of the power supply S3011 obtained through the reset transistor 3014 by the reset control signal S3012. Thereafter, charges corresponding to the amount of incident light obtained by the PD 3011 are transferred to the FD 3012. The transfer is controlled by a read control signal S3013, and voltage conversion is performed by the source follower amplifier 3015. Then, in response to a row selection signal S3014, the sensor A image vertical signal line S303 is further passed through the row selection transistor 3016 and sent to the sensor A image column ADC 303, where it becomes digital A image data. Similarly, digital B image data is generated by the column ADC 304 of the sensor B image. This operation is controlled in units of horizontal lines by row selection signals S3014 and S3024. In addition, the left and right divided pixel signals of the left and right parts are sequentially generated as the sensor A image signal S305 and the sensor B image signal S306 by the sensor A image horizontal drive circuit 305 and the sensor B image horizontal drive circuit 306, respectively. Is done.

  Note that the reset control signal S3012, the read control signal S3013, and the row selection signal S3014 are controlled in accordance with so-called rolling driving. As a result, all the A and B images of the two-dimensional image sensor are generated and output by sequentially driving the control in units of horizontal lines by vertical lines.

  Since the above is also the operation control of the CMOS image sensor disclosed in the related art, further detailed explanation is omitted.

  The sensor A image signal S305 and the sensor B image signal S306 are respectively the sensor A image signal S101 and the sensor B image signal S102 of FIG. The sensor A image signal S101 and the sensor B image signal S102 are first added in the A / B image addition circuit 103. Thereby, it is formed as an imaging signal S103 for display / recording. The camera signal processing circuit 104 performs luminance and color signal processing to generate a display / recording image signal S104, which is externally displayed by a display device such as the liquid crystal panel 105, and recorded on various recording media by the recording unit 106. The

  On the other hand, the sensor A image signal S101 and the sensor B image signal S102 are also sent to the trimming processing circuit 107 and the thinning processing circuit 109.

  4A shows the trimming pixel range of the trimming processing circuit 107, and FIG. 4B shows the pixel thinning range of the thinning processing circuit 109. FIG.

  4A and 4B, 401 is an effective pixel area of the sensor A image or sensor B image, 402 is a trimming pixel range of the trimming processing circuit 107, and 403 is a pixel thinning range of the thinning processing circuit 109.

  The effective pixel area 401 of the sensor A image or the sensor B image is an area having 4096 pixels in the horizontal direction and 2160 pixels in the vertical direction. In the trimming pixel range of the trimming processing circuit 107, a region of 256 pixels in the horizontal direction and 135 pixels in the vertical direction in the center of the effective pixel region 401 of the sensor A image or the sensor B image is trimmed without thinning. And it outputs as the detailed area A image signal S105 and the detailed area B image signal S106. The detailed area A image signal S105 and the detailed area B image signal S106 include a center-weighted focus detection area for the effective pixel area.

  Further, the thinning-out processing circuit 109 has a thinning-out range of 3072 pixels in the horizontal direction and 1620 pixels in the vertical direction by thinning out the horizontal and vertical by 1/12 thinning, and the thinning-out A image signal S107 and the thinning-out B image. Output as signal S108. Note that the pixel thinning range of the thinning processing circuit 109 includes the trimming pixel range at the center. Since both the thinned-out A image signal S107 and the thinned-out B image signal S108 are thinned by 1/12, the image size is the same as the detailed area A image signal S105 and the detailed area B image signal S106, and the horizontal size is 256 pixels and the vertical direction is 135 pixels. This is a pixel signal.

  4A and 4B are examples of focus detection with an emphasis (center) at the center. However, as shown in FIGS. 4C and 4D, the trimming pixel range of the trimming processing circuit 107 does not necessarily need to be at the center of the effective pixel region. In addition, the pixel thinning range of the thinning processing circuit 109 does not necessarily include the trimming pixel range evenly in the vertical and horizontal directions.

  The detailed area A image signal S105 and the detailed area B image signal S106 are sent to the detailed area focus detection processing circuit 108. Further, the thinned-out A image signal S107 and the thinned-out B image signal S108 are sent to the wide area focus detection processing circuit 110. Then, as shown in Equation 1, the difference absolute value sum is calculated while shifting the A and B images within a predetermined range, and the shift amount that minimizes the difference absolute value sum is calculated as the image shift amount.

  In Equation 1, A (x) represents the xth pixel of the A image signal, and B (x + i) represents the x + ith pixel of the B image signal.

  Since the detailed area A image signal S105, the detailed area B image signal S106, the thinned A image signal S107, and the thinned B image signal S108 are signals for 256 pixels in the horizontal direction, x and i are in the range of 0 to 127. To change Equation (1).

  C (i) is a sum of absolute differences obtained by changing x by 1 in the range of 0 to 127 for any i of 0 to 127. Accordingly, the image shift amount to be obtained is determined from i with which C (i) obtained by further shaking i from 0 to 127 is minimized. In this way, the detailed area image shift amount S111 and the wide area image shift amount S112 are generated.

  For the detailed area focus detection processing circuit 108 and the wide area focus detection processing circuit 110, the threshold value setting circuit 114 causes the detailed area focus detection reliability determination threshold S116 and the wide area focus detection reliability determination threshold S117, respectively. Is set. The two reliability determination threshold values determine whether each of the detailed area A image signal S105, the detailed area B image signal S106, the thinned A image signal S107, and the thinned B image signal S108 has a significant contrast. Use for. In other words, the reliability of the image shift amount is detected according to the contrast of the subject obtained based on the output signal from the photoelectric conversion element.

  If the Peak-to-Peak value of the detailed area A image signal S105 and the Peak-to-Peak value of the detailed area B image signal S106 are both greater than or equal to the detailed area focus detection reliability determination threshold S116, the detailed area A image signal S105, The contrast of the detailed area B image signal S106 is high. Therefore, it is determined that the reliability of the detailed area image shift amount S111 is high, and the detailed area focus detection reliability flag S109 is activated.

  Similarly, if the Peak-to-Peak value of the thinned-out A image signal S107 and the Peak-to-Peak value of the thinned-out B image signal S108 are both greater than or equal to the wide-range focus detection reliability determination threshold value S117, the thinned-out A image signal S107, thinned-out B The contrast of the image signal S108 is high. Accordingly, it is determined that the reliability of the wide area image shift amount S112 is high, and the wide area focus detection reliability flag S110 is activated.

  In the above description, the contrast of the subject is taken as an example, but the reliability may be determined based on the degree of coincidence between the A image signal and the B image signal. In this case, if the degree of coincidence between the detailed area A image signal S105 and the detailed area B image signal S106 is equal to or greater than the detailed area focus detection reliability determination threshold, it is determined that the reliability of the detailed area image deviation amount S111 is high. The detailed area focus detection reliability flag S109 is set active. Similarly, if the degree of coincidence between the thinned-out A image signal S107 and the thinned-out B image signal S108 is equal to or larger than the wide-area focus detection reliability determination threshold, it is determined that the reliability of the wide-area image shift amount S112 is high, and wide-area focus detection is performed. The reliability flag S110 is activated.

  Consider a situation where both the detailed area focus detection reliability flag S109 and the wide area focus detection reliability flag S110 are active. In this case, the difference between the detailed area image displacement amount S111 and the wide area image displacement amount S112 is further obtained in the image displacement amount comparison circuit 111 by the difference between the detailed area image displacement amount S111 and the wide area image displacement amount S112. Further, the image displacement amount comparison circuit 111 receives an image displacement amount comparison threshold value S118 from the threshold setting circuit 114, and is compared with the difference between the detailed region image displacement amount S111 and the wide area image displacement amount S112 to obtain the detailed / wide area image displacement. An amount comparison result S113 is output.

  The detailed area focus detection reliability flag S109, the wide area focus detection reliability flag S110, the detailed area image deviation amount S111, the wide area image deviation amount S112, and the detailed / wide area image deviation amount comparison result S113 thus obtained are calculated as defocus amounts. Input to the circuit 112. Then, according to the operation flowchart of FIG. 5, the defocus amount is calculated and output as defocus amount data S114. The focus lens drive circuit 113 outputs the lens drive signal S115, and the focus lens of the imaging optical system including the focus lens 101 is moved to execute the AF operation.

<Operation Flowchart of Defocus Amount Calculation Circuit>
FIG. 5 is an operation flowchart of the defocus amount calculation circuit 112. In FIG. 5, when the AF operation of step 501 is started, first, the reference of the detailed area image shift amount S111 of step 502 and the reference of the wide area image shift amount S112 of step 503 are executed. Next, the first determination based on the detailed area focus detection reliability flag S109 and the wide area focus detection reliability flag S110 in step 504 is executed.

  When only the wide area focus detection reliability flag S110 is active, the detailed area image shift amount S111 is not obtained because of the large defocus state. Therefore, in step 508, the defocus amount data S114 is calculated using the wide-area image shift amount S112, the lens movement using the defocus amount data S114 in step 511 is executed, and the AF operation ends.

  When only the wide area focus detection reliability flag S110 is not active, the second determination is performed by the detailed area focus detection reliability flag S109 and the wide area focus detection reliability flag S110 in step 505.

  When both the wide area focus detection reliability flag S110 and the detailed area focus detection reliability flag S109 are active, it is possible to refer to both the detailed area image deviation amount S111 and the wide area image deviation amount S112. This state is not at least a large defocus state. Then, the processing branch is executed according to the detailed / wide-area image shift amount comparison result S113 in step 506.

  If the detailed / wide area image shift amount comparison result S113 is equal to or greater than the image shift amount comparison threshold S118, the defocus amount data S114 is calculated using the detailed area image shift amount S111 of step 510. Then, the lens movement using the defocus amount data S114 of step 511 is executed, and the AF operation ends. Here, when the detailed / wide area image displacement amount comparison result S113 is equal to or larger than the image displacement amount comparison threshold S118, that is, when the difference between the detailed area image displacement amount S111 and the wide area image displacement amount S112 is large.

  The reason is that the image displacement amount obtained from the image region originally intended for AF is the detailed region image displacement amount S111, while the wide-area image displacement amount S112 includes the region outside the image region originally intended for AF. This is because it is obtained from the thinned A / B images. Therefore, the accuracy of the image shift amount is low. Therefore, the fact that there is a difference between the detailed area image deviation amount S111 and the wide area image deviation amount S112 means that the detailed area image deviation amount S111 is calculated with high accuracy, and the wide area image deviation amount S112 is accurate due to the above-mentioned reason. This is because the situation has been reduced.

  On the other hand, when the detailed / wide area image displacement amount comparison result S113 is less than the image displacement amount comparison threshold S118, that is, when the difference between the detailed area image displacement amount S111 and the wide area image displacement amount S112 is small, in the large defocus state. Although there is no, a situation close to a large defocus can be considered. This is a phenomenon in which the accuracy of the detailed area image shift amount S111 is lowered because the image is blurred, and the wide area image shift amount S112 is also lowered because a coarse signal is seen. Alternatively, since the wide-area image shift amount S112 includes the area outside the image area that is originally intended for AF, whether the reduction in the accuracy of the wide-area image shift amount S112 due to the above-described reason can no longer be seen due to the influence of the subject outside the area. It is.

  In the present embodiment, when the detailed / wide-area image shift amount comparison result S113 is less than the threshold value, the influence of the subject outside the area is a rare case, so that it is not a large defocus state but a situation close to large defocus. Define. In this case, defocus amount data S114 is calculated using a weighted average result at a predetermined ratio between the detailed region image shift amount S111 and the wide area image shift amount S112 in step 509. Then, the lens movement using the defocus amount data S114 of step 511 is executed, and the AF operation ends.

  This weighting is performed when the detailed / wide area image displacement amount comparison result S113 is less than the image displacement amount comparison threshold value S118, and as the difference between the detailed area image displacement amount S111 and the wide area image displacement amount S112 increases, the detailed area image displacement amount increases. It sets so that the reference degree of quantity S111 may become large. Conversely, the smaller the difference between the detailed area image deviation amount S111 and the wide area image deviation amount S112, the greater the degree of reference to the wide area image deviation amount S112, which is inherently less accurate. In this case, since the difference between the detailed area image deviation amount S111 and the wide area image deviation amount S112 is small, there is no influence on the majority.

  By incorporating weighting in this way, particularly in moving image shooting, even when the state change from the in-focus state to the large defocus state frequently occurs, the detailed area image displacement amount S111 and the wide area image displacement amount S112 are referred to. It is possible to prevent a situation where the state is frequently switched.

  Next, as a result of the second determination by the detailed area focus detection reliability flag S109 and the wide area focus detection reliability flag S110 in step 505, both the wide area focus detection reliability flag S110 and the detailed area focus detection reliability flag S109 are both active. Think of no case. In this case, the third determination based on the detailed area focus detection reliability flag S109 and the wide area focus detection reliability flag S110 in step 507 is executed.

  When only the detailed area focus detection reliability flag S109 is active, the defocus amount is calculated using the detailed area image deviation amount S111 of step 510. Then, the lens movement using the defocus amount data S114 of step 511 is executed, and the AF operation ends.

  The case where only the detailed area focus detection reliability flag S109 is active is only a fine picture as a subject. Therefore, there is almost no contrast when viewed with the thinned-out A image signal S107 and the thinned-out B image signal S108, which are the basis for calculating the wide-area image shift amount S112.

  Finally, when neither the detailed area focus detection reliability flag S109 nor the wide area focus detection reliability flag S110 is active, it is a situation in which a subject contrast at a level capable of AF control cannot be obtained in the first place. For this reason, the AF operation ends without the lens movement of step 512.

  For continuous AF control during moving image shooting, the above-described flowchart of FIG. 5 is repeated at a predetermined frame rate. As a result, stable AF is possible even in a large defocus state, in an intermediate state between the in-focus state and the large defocus state, and even when a state change from the near-focus state to the large defocus state frequently occurs. AF control is possible with time and AF quality.

  In the one-shot AF control at the time of still image shooting, the above-described flowchart of FIG. 5 is re-executed during AF from the large defocus state. Thereby, it is possible to smoothly shift from the reference state of the wide area image deviation amount S112 to the reference state of the detailed area image deviation amount S111, and when viewed from the user, an effect that looks like a single AF operation is obtained. It is done.

(Second Embodiment)
The second embodiment will be described with reference to FIGS.

<Configuration of Imaging Device in Second Embodiment>
FIG. 7 is a block diagram showing a configuration of an imaging apparatus according to the second embodiment of the present invention. In FIG. 7, reference numeral 701 denotes an imaging optical system including a focus lens. Reference numeral 702 denotes a stacked image sensor. Reference numeral 703 denotes an A / B image addition circuit. Reference numeral 704 denotes a camera signal processing circuit. Reference numeral 705 denotes a display device such as a liquid crystal panel. Reference numeral 706 denotes a recording unit. Reference numeral 708 denotes a detailed area focus detection processing circuit. Reference numeral 710 denotes a wide area focus detection processing circuit. Reference numeral 711 denotes an image shift amount comparison circuit. Reference numeral 712 denotes a defocus amount calculation circuit. Reference numeral 713 denotes a focus lens driving circuit. Reference numeral 714 denotes a threshold setting circuit. In FIG. 7, S701 is a sensor A image signal, and S702 is a sensor B image signal. S703 is an imaging signal. S704 is a display / recording image signal. S705 is a detailed area A image signal, and S706 is a detailed area B image signal. S707 is a thinned A image signal, and S708 is a thinned B image signal. S709 is a detailed area focus detection reliability flag, and S710 is a wide area focus detection reliability flag. S711 is a detailed area image shift amount, and S712 is a wide area image shift amount. S713 is the detailed / wide area image shift amount comparison result. S714 is defocus amount data. S715 is a lens drive signal. S716 is a detailed area focus detection reliability determination threshold value. S717 is a wide area focus detection reliability determination threshold value. S718 is an image shift amount comparison threshold value.

  FIG. 8 is an explanatory diagram of a circuit configuration for one pixel of the image sensor 702 of FIG. In FIG. 8, reference numeral 801 denotes a circuit showing the left-side portion 201 divided into left and right in FIG. Reference numeral 802 denotes a circuit showing the right-side portion 202 divided into right and left in FIG. Reference numeral 8011 denotes a PD. Reference numeral 8012 denotes an FD. Reference numeral 8013 denotes a read transistor. Reference numeral 8014 denotes a reset transistor. Reference numeral 8015 denotes a source follower amplifier. Reference numeral 8016 denotes a first row selection transistor. Reference numeral 8017 denotes a second row selection transistor. Reference numeral 8018 denotes a third row selection transistor.

  Each of the components 8011 to 8018 is a configuration of the circuit 801 showing the left and right divided left part 201 in FIG. 2. Similarly, the right and left divided parts 202 in FIG. The constituent elements of the circuit 802 shown in FIG.

  Reference numeral 803 denotes a first column ADC of the sensor A image. Reference numeral 804 denotes a second column ADC of the sensor A image. Reference numeral 805 denotes a third column ADC of the sensor A image. Reference numeral 806 denotes a first column ADC of the sensor B image. Reference numeral 807 denotes a second column ADC of the sensor B image. Reference numeral 808 denotes a third column ADC of the sensor B image. Reference numeral 809 denotes a first horizontal drive circuit for the sensor A image, 810 denotes a second horizontal drive circuit for the sensor A image, and 811 denotes a third horizontal drive circuit for the sensor A image. Reference numeral 812 denotes a first horizontal drive circuit for the sensor B image, 813 denotes a second horizontal drive circuit for the sensor B image, and 814 denotes a third horizontal drive circuit for the sensor B image.

  S8011 is a power source. S8012 is a reset control signal. S8013 is a read control signal. S8014 is a first row selection signal. S8015 is the second row selection signal. Further, S8016 is a third row selection signal, which is an internal signal of the circuit 801 showing the left and right divided parts 201 in FIG. Similarly, the same end number in S8021 to S8026 is used as an internal signal of the circuit 802 indicating the right-side portion 202 divided in the left-right direction in FIG.

  S803 is a first vertical signal line of the sensor A image. S804 is a second vertical signal line of the sensor A image. S805 is a third vertical signal line of the sensor A image. S806 is a first vertical signal line of the sensor B image. S807 is the second vertical signal line of the sensor B image. S808 is a third vertical signal line of the sensor B image. S809 is a first sensor A image signal. S810 is a second sensor A image signal. S811 is a third sensor A image signal. S812 is a first sensor B image signal. S813 is the second sensor B image signal. S814 is a third sensor B image signal.

  In FIG. 7, the components other than the stacked image sensor 702 are the same as those of the imaging apparatus according to the first embodiment of the present invention shown in FIG. To do.

  The components that are different in each component of the imaging apparatus according to the second embodiment of the present invention shown in FIG. 7 are related to the above-described laminated image sensor 702 and its output signal, so that part will be described below.

  The laminated image sensor 702 has recently been established and put into practical use, and an image sensor in which a plurality of silicon chips are stacked and configured in an image sensor package and signals are connected between the plurality of silicon chips. It is. The number of signal connections between multiple silicon chips is dramatically greater than the number of signal inputs and outputs to the outside of conventional image sensor packages, greatly increasing the degree of freedom in physical design of image sensors. It is an improvement.

  An example of the internal configuration of the stacked image sensor 702 is as shown in FIG. The functions of the PD 8011, the FD 8012, the reset control signal S8012, the reset transistor 8014, the power supply S8011, and the readout control signal S8013 are the same as those of the circuit for one pixel of the image sensor 102 in FIG. The functions of the read transistor 8013 and the source follower amplifier 8015 are the same as those of the circuit for one pixel of the image sensor 102 in FIG.

  The difference is that the first row selection transistor 8016, the second row selection transistor 8017, and the third row selection transistor 8018 are commonly connected to the source follower amplifier 8015 in order to improve the physical design flexibility of the stacked image sensor. It is a point that has been. The first row selection signal S8014, the second row selection signal S8015, and the third row selection signal S8016 can control each row selection transistor (8016, 8017, and 8018, respectively).

  The first sensor A image signal S809 is output through the first vertical signal line S803 for the sensor A image, the first column ADC 803 for the sensor A image, and the first horizontal drive circuit 809 for the sensor A image. .

  The second sensor A image signal S810 is output through the second vertical signal line S804 for the sensor A image, the second column ADC 804 for the sensor A image, and the second horizontal drive circuit 810 for the sensor A image. .

  The third sensor A image signal S811 is output through the third vertical signal line S805 for the sensor A image, the third column ADC 805 for the sensor A image, and the third horizontal drive circuit 811 for the sensor A image. It has become a thing.

  Therefore, the output voltage of the source follower amplifier 8015 obtained according to the photocharge amount of the PD 8011 by each row selection signal (respectively S8014, S8015, S8016) is output as three independent sensor A image signals. .

  Similarly, each component of PD8021, FD8022, reset control signal S8022, reset transistor 8024, power source S8021, read control signal S8023, read transistor 8023, and source follower amplifier 8025 operates. A signal is output to the first vertical signal line S806 of the sensor B image by each row selection transistor (each of 8026, 8027, and 8028) and each row selection signal (each of S8024, S8025, and S8026). The first sensor B image signal S812 is output through the first column ADC 806 of the sensor B image and the first horizontal drive circuit 812 of the sensor B image.

  The second sensor B image signal S813 is output through the second vertical signal line S807 for the sensor B image, the second column ADC 807 for the sensor B image, and the second horizontal drive circuit 813 for the sensor B image. .

  The third sensor B image signal S814 is output through the third vertical signal line S808 for the sensor B image, the third column ADC 808 for the sensor B image, and the third horizontal drive circuit 814 for the sensor B image. . Accordingly, the output voltage of the source follower amplifier 8025 obtained according to the photocharge amount of the PD 8021 by each row selection signal (S8024, S8025, S8026, respectively) is output as three independent sensor B image signals.

  The three independent systems of sensor A image and B image signal can be read out at the timing when the sensor image signal is required at each use destination.

  In FIG. 7, a sensor A image signal S701 is a first sensor A image signal S809. The detailed area A image signal S705 is the second sensor A image signal S810. The thinned A image signal S707 is the third sensor A image signal S811. The sensor B image signal S702 is the first sensor B image signal S812. The detailed area B image signal S706 is the second sensor B image signal S813. The thinned B image signal S708 is a third sensor B image signal S814. Then, as shown in FIGS. 4A, 4B, 4C, and 4D, the drive control of the stacked image sensor 702 is performed so that a sensor A / B image image similar to trimming or pixel thinning is obtained. Do. As a result, the same operation as in the first embodiment of the present invention can be performed. Using the detailed area focus detection processing circuit 708, the wide area focus detection processing circuit 710, the image shift amount comparison circuit 711, the defocus amount calculation circuit 712, the focus lens drive circuit 713, and the threshold setting circuit 714, the operation flowchart of FIG. AF control is performed.

  According to the second embodiment of the present invention, in addition to the effects obtained in the first embodiment, the AF target area is read out independently of the path for reading out the image for recording / recording from the image sensor. And it becomes possible to read in the thinned-out state. For this reason, the AF time can be further shortened.

<Examples of pupil division of other image sensors>
Here, the sub-pixels 201 in FIG. 2 are regularly arranged in the x direction as shown in the figure, and the first image signal acquired by these sub-pixel groups is converted into an image signal A (a different emission of the imaging optical system). One of the image signals obtained from a pair of luminous fluxes that have passed through the pupil region. Further, the subpixels 202 are also regularly arranged in the x direction as shown in the figure, and the second image signal acquired by these subpixel groups passes through the image signal B (a different exit pupil region of the imaging optical system). The other of the image signals obtained from the paired luminous fluxes). Then, the relative image shift amount between the image signal A and the image signal B can be calculated. Based on this, the defocus amount of the focus lens is adjusted. This is called phase difference focus adjustment. Although the configuration corresponding to a subject having a luminance distribution in the x direction has been described here, a configuration corresponding to a subject having a luminance distribution in the y direction can be obtained by developing a similar configuration in the y direction. It is possible.

  In the above-described embodiment, an example is shown in which two subpixels that are eccentric in one pixel are divided only in a one-dimensional direction in order to divide the pupil. Sub-pixels may be formed by dividing in the two-dimensional direction.

  Further, in the present embodiment, an example in which a plurality of subpixels are arranged per microlens for pupil division is shown, but with respect to the pupil division method, one eccentric pixel is arranged per microlens, Focus detection may be performed by dividing the pupil using pixels having different eccentricity.

<Other examples of the above embodiment, effects of the above embodiment, etc.>
In the above-described embodiment, the trimming processing circuit 107 in FIG. 2 performs signal trimming, and the thinning-out processing circuit 109 performs signal thinning.

  However, instead of the trimming processing circuit 107 and the thinning processing circuit 109, a first thinning processing circuit and a second thinning processing circuit having different thinning rates may be provided. Further, instead of the trimming processing circuit 107 and the thinning processing circuit 109, a first addition processing circuit and a second addition processing circuit having different addition rates may be provided. The resolution of the image signal obtained in this case is lower in the thinning processing circuit 109 than in the trimming processing circuit 107. Further, the higher the thinning rate and the addition rate, the lower the resolution of the obtained image signal.

  In the above-described embodiment, for example, a thinned A / B image in a wide AF target area and a non-thinned A / B image in the original AF target area are generated simultaneously, and two images obtained from each A / B image are generated. An AF operation can be performed based on the focus amount. For this reason, the AF time can be shortened even in the large defocus state or in the vicinity of the in-focus state.

  In addition, as shown in FIG. 6, in the case where the subject has a relatively high spatial frequency component in the image region that is originally intended for AF, and the pattern is a fine line, the method for expanding the image region that is the subject of AF is described. For example, there are many cases where the effect is low only by spreading only in the horizontal direction. In this case, the image area targeted for AF should be extended also in the vertical direction so as to include the image area desired to be originally targeted for AF. According to the above-described embodiment, for example, when a wide AF target area is extended also in the vertical direction, a wide AF target area is read out independently of a path for reading a shooting / recording image from the image sensor, and , It is possible to read in the thinned-out state. For this reason, it is possible to prevent a delay in the timing at which the A / B image data is acquired from the image sensor, and it is possible to shorten the AF time.

  Further, for example, the reliability of the image shift amount obtained from the thinned A / B image in the wide AF target region and the image shift amount obtained from the non-thinned A / B image in the original AF target region is evaluated. It can be determined whether AF control based on the amount of deviation is appropriate. In addition, when the reliability of any image displacement amount is obtained, by comparing the image displacement amounts, it is possible to control which image displacement amount is used in more detail or with a predetermined weight. it can. Thereby, the transfer from the AF control based on the thinned A / B image to the AF control based on the non-thinned A / B image can be smoothly shifted. For this reason, the user can obtain an effect that looks like a single AF operation. Furthermore, during moving image shooting, even when a large defocus state is occasionally obtained, it is possible to smoothly connect the two-stage AF control, and there is an effect of improving the quality of the AF operation.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined. Also, when a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication, and the program is executed Are also included in the present invention. Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS. As a recording medium for supplying the program, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a nonvolatile semiconductor memory may be used. As a program supply method, a computer program that forms the present invention is stored in a server on a computer network, and a connected client computer downloads and programs the computer program.

DESCRIPTION OF SYMBOLS 101 Imaging optical system including a focus lens 102 Image sensor 103 A / B image addition circuit 104 Camera signal processing circuit 105 Display device, such as a liquid crystal panel 106 Recording part 107 Trimming processing circuit 108 Detailed area focus detection processing circuit 109 Thinning-out processing circuit 110 Wide area Focus detection processing circuit 111 Image shift amount comparison circuit 112 Defocus amount calculation circuit 113 Focus lens drive circuit 114 Threshold setting circuit

Claims (11)

An imaging means having a plurality of photoelectric conversion elements for one microlens, and the microlenses are arranged two-dimensionally;
Setting means for setting a first focus detection area and a second focus detection area wider than the first focus detection area in the imaging screen;
Based on the output signals from the photoelectric conversion elements in the first and second focus detection areas, a pair of image signals for performing phase difference focus detection processing is generated for each focus detection area, Detection means for detecting an image shift amount of a pair of image signals; and an acquisition means for acquiring an image shift amount used when performing focus adjustment from an image shift amount for each focus detection region detected by the detection means,
The detection means generates a first image signal having a first resolution from a pair of signals output from a photoelectric conversion element included in the first focus detection region, and is included in the second focus detection region. A second image signal having a second resolution lower than the first resolution is generated from a pair of signals output from the photoelectric conversion element, and the first image shift is performed using the first image signal. A second image shift amount is detected using the second image signal,
The acquisition unit obtains an image shift amount used when performing focus adjustment by weighting the first image shift amount and the second image shift amount at a predetermined ratio. apparatus.   The detection means further detects the reliability of the first image deviation amount and the second image deviation amount, and the acquisition means comprises the first image deviation amount, the second image deviation amount, The image shift amount used when performing the focus adjustment is acquired based on the reliability of the first image shift amount and the reliability of the second image shift amount. Imaging device.   3. The imaging according to claim 2, wherein the reliability of the first and second image shift amounts is detected according to a contrast of a subject obtained based on an output signal from the photoelectric conversion element. apparatus.   The imaging apparatus according to claim 2, wherein the reliability of the first and second image shift amounts is detected according to the degree of coincidence of the images of the pair of image signals.   The acquisition unit is configured such that when the reliability of the first image shift amount is higher than a predetermined threshold value and the reliability of the second image shift amount is higher than a predetermined threshold value, the first image shift amount is higher than the predetermined threshold value. The difference between the amount of deviation and the second amount of image deviation is obtained, and the amount of image deviation used when the focus adjustment is performed is obtained based on the difference. The imaging device according to item.   In the case where the difference is less than a predetermined threshold, the acquisition means includes the first difference between the case where the difference is a first difference and the case where the difference is a second difference larger than the first difference. The imaging apparatus according to claim 5, wherein a weighting ratio of the first image shift amount is increased when the difference is two. The imaging means is an imaging device, and reads out signals with different thinning rates or addition rates for the first and second focus detection areas,
The detection unit generates a first image signal having a first resolution and a second image signal having a second resolution with respect to a plurality of signals having different thinning rates or addition rates. The imaging device according to any one of claims 1 to 6.   The imaging apparatus according to claim 1, wherein a center of the first focus detection region and a center of the second focus detection region are the same.   The imaging apparatus according to claim 1, further comprising a control unit configured to control to display an image on a display device using a signal output from the imaging unit.   The image pickup apparatus according to claim 1, further comprising a control unit that controls to record an image on a recording unit using a signal output from the image pickup unit. A method for controlling an imaging apparatus including an imaging unit having a plurality of photoelectric conversion elements for one microlens, and the microlenses are arranged two-dimensionally,
A setting step for setting a first focus detection region and a second focus detection region wider than the first focus detection region in the imaging screen;
Based on the output signals from the photoelectric conversion elements in the first and second focus detection areas, a pair of image signals for performing phase difference focus detection processing is generated for each focus detection area, A detection step of detecting an image shift amount of a pair of image signals; and an acquisition step of acquiring an image shift amount used when performing focus adjustment from an image shift amount for each focus detection area detected by the detection step. ,
In the detection step, a first image signal having a first resolution is generated from a pair of signals output from a photoelectric conversion element included in the first focus detection region, and is included in the second focus detection region. A second image signal having a second resolution lower than the first resolution is generated from a pair of signals output from the photoelectric conversion element, and a first image shift amount is generated using the first image signal. And detecting a second image shift amount using the second image signal,
In the acquisition step, the first image shift amount and the second image shift amount are weighted at a predetermined ratio to acquire an image shift amount used when focus adjustment is performed. Method.

Images