JP6632582B2 - Imaging device and control method thereof - Google Patents

Description

  The present invention relates to an imaging device capable of controlling focus adjustment.

  2. Description of the Related Art In recent years, an imaging device that acquires a recording or display image and that performs phase difference type focus detection on an imaging surface, that is, an imaging device including a plurality of imaging devices having a so-called imaging surface phase difference AF function has been proposed. .

  Patent Literature 1 discloses an imaging apparatus having two imaging elements having different pixel pitches for capturing a moving image and capturing a still image. Since the two image sensors have a plurality of focus detection pixel portions having different base line lengths, the focus detection accuracy is different. When an image is generated using one image sensor, the focus detection is performed using the other image sensor. Is disclosed.

  In Patent Document 2, one image sensor includes a focus detection pixel group that divides a pupil in a horizontal direction in a predetermined area on a photographing screen, and the other image sensor performs pupil division in a vertical direction in an area different from the predetermined area. An imaging device including a focus detection pixel group is disclosed. By selecting an appropriate focus detection pixel group according to the position (coordinates) of the subject to be focus-detected and the directionality of the light / dark pattern of the subject, focus detection performance can be efficiently exhibited for various subjects. It is possible.

  Patent Literature 3 discloses an imaging apparatus that calculates an optimal correction value used for focus adjustment based on information on characteristics of a subject signal, characteristics of an imaging signal, and aberration of an imaging optical system.

  Patent Literature 4 discloses an imaging apparatus that calculates a correction value from information on focus detection obtained from one image sensor and corrects a focus detection result obtained by the other image sensor.

JP 2015-34917 A JP 2008-177903 A JP 2015-138200 A JP-A-2005-045804

  In the case where the focus detection result is corrected from the viewpoint disclosed in Patent Literature 3 by the imaging device of Patent Literature 1, the two imaging elements acquire recorded images (still images and moving images) having different imaging characteristics, so that an optimal image is obtained. The correction value differs for each image sensor. Further, when the focus detection result is corrected by the imaging device of Patent Literature 2 from the viewpoint disclosed in Patent Literature 3, the focus detection signals (vertical direction and horizontal direction) having different characteristics are acquired by the two image sensors. The optimal correction value differs for each focus detection result. However, none of the patent documents discloses a method of correcting a focus detection result according to different imaging characteristics or characteristics of a focus detection signal.

  Also, in the imaging device of Patent Document 4, there is no disclosure of a method of simultaneously acquiring a recorded image (still image, moving image) with two imaging elements. Further, in each of the patent documents, the focus detection result is corrected according to the change in the aberration state when the light is received by the first image sensor and the change in the aberration state when the light is received by the second image sensor. No method is disclosed.

  An object of the present invention is to provide an imaging device capable of performing highly accurate focus control and a control method thereof.

An imaging apparatus according to one aspect of the present invention includes a light beam splitting unit that splits a light beam that has passed through an exit pupil of a photographing lens having a focusing mechanism into a first light beam and a second light beam, and a plurality of first photoelectric conversion units. A first imaging element including a plurality of first pixels including : a first microlens that guides a light beam of the first light beam that has passed through a different region of the exit pupil to the plurality of first photoelectric conversion units; A plurality of second pixels each including: a plurality of second photoelectric conversion units; and one microlens that guides a light flux of the second light flux that has passed through a different region of the exit pupil to the plurality of second photoelectric conversion units. A second imaging element provided, aberration information of the imaging lens, first aberration information that is aberration information of the first light beam , and spatial frequency characteristics of a first imaging signal acquired from the first imaging element. and weighting values, before Symbol first imaging element And weighting values for the spatial frequency characteristics of the focus detection signal of the pair is obtained from, based on the first focus position obtained by using the focus detection signal of the pair is acquired from the first image sensor A first focus adjustment correction amount for correction is calculated, and aberration information of the photographing lens, second aberration information that is aberration information of the second light flux, and the second focus information are acquired from the second image sensor. and weighting values for the spatial frequency characteristics of the second image pickup signal, and a weighting value for the spatial frequency characteristics of a pair of focus detection signals obtained from the previous SL second imaging element, based on, obtained from the second image sensor Correction amount calculating means for calculating a second focus adjustment correction amount for correcting a second focus position obtained using a pair of focus detection signals to be obtained, and the first and second focus adjustment correction amounts On the basis of the And having a control means for controlling the focusing mechanism.

Further, an imaging apparatus according to another aspect of the present invention includes a luminous flux movable between a first position located in an optical path of a photographing lens having a focus adjustment mechanism and a second position retracted from the optical path. A splitting unit, a plurality of photoelectric conversion units, and a light beam that has passed through a different area of the exit pupil of the photographing lens among light beams transmitted through the light beam splitting unit when the light beam splitting unit is located at the first position. An imaging element including a plurality of pixels each including one microlens leading to the plurality of photoelectric conversion units; and when the light beam splitting unit is located at the first position, aberration information of the photographing lens; a first aberration information is an aberration information means, the spatial frequency characteristics of the weighting values and a pair of focus detection signal obtained from the previous SL imaging element with respect to the spatial frequency characteristic of the image pickup signal obtained from the imaging device Weighting value against, on the basis of a correction amount calculating means for calculating the focusing correction amount for correcting the focal position obtained by using the focus detection signal of the pair is acquired from the imaging device, the focusing Control means for controlling the focus adjustment mechanism based on the correction amount.

Further, an imaging apparatus according to another aspect of the present invention includes a light beam splitting unit that splits a light beam that has passed through an exit pupil of a photographing lens having a focusing mechanism into a first light beam and a second light beam, and a plurality of first light beams . A first image pickup unit including a plurality of first pixels each including: a photoelectric conversion unit; and one microlens that guides a light beam of the first light beam that has passed through a region having a different exit pupil to the plurality of first photoelectric conversion units. An element, a plurality of second photoelectric conversion units, and one microlens that guides a light beam of the second light beam that has passed through a different region of the exit pupil to the plurality of second photoelectric conversion units. A pair of focus detection signals obtained from the first image sensor based on a second image sensor having a plurality of pixels, shape parameters of the light beam splitting means, and correction information on the imaging lens and the shape parameters; Using For correcting the second focus position obtained using the first focusing correction amount and the second pair of focus detection signal obtained from the imaging device to correct the first focal position is because a correction amount calculating means for calculating a second focusing correction amount, based on the first and second focusing correction amount, and having a control means for controlling the focus adjustment mechanism .

A control method of an imaging apparatus as another aspect of the present invention includes a beam splitter for splitting the light flux passing through the exit pupil of the taking lens having a focus adjusting mechanism to the first light flux and second light flux, a plurality A first pixel having a plurality of first pixels each including : a first photoelectric conversion unit; and a single microlens that guides a light beam of the first light beam that has passed through a different region of the exit pupil to the first photoelectric conversion unit. A second pixel, comprising: an imaging element; a plurality of second photoelectric conversion units; and one microlens that guides a light beam of the second light beam that has passed through a region having a different exit pupil to the second photoelectric conversion unit. A control method of an image pickup apparatus including a plurality of second image pickup elements, wherein aberration information of the imaging lens, first aberration information that is aberration information of the first light beam, and the first image pickup element are provided. Spatial frequency of the first imaging signal obtained from And weighting values for characteristics, before Symbol weighting values for the spatial frequency characteristics of the focus detection signal of the pair is acquired from the first image sensor, based on the focus detection of the pair is obtained from the first image sensor A first focus adjustment correction amount for correcting a first focus position obtained using a signal is calculated, and aberration information of the photographing lens and second aberration, which is aberration information of the second light flux, are calculated. information, the weighting value for the spatial frequency characteristics of the weighting values for the spatial frequency characteristic of the second imaging signal obtained from the second image sensor, a pair of focus detection signals that are acquired before the previous SL second image sensor , the correction amount calculating step which calculates a second focusing correction amount for correcting the second focus position obtained by using the focus detection signal of the pair is obtained from the second image sensor based on Based on said first and second focusing correction amount, and having a control step of controlling the focusing mechanism.

According to another aspect of the present invention, there is provided a method of controlling an imaging apparatus , comprising: moving between a first position located in an optical path of a photographing lens having a focus adjustment mechanism and a second position retracted from the optical path. A light beam splitting unit, a plurality of photoelectric conversion units, and a light beam transmitted through the light beam splitting unit, when the light beam splitting unit is located at the first position, passes through a region having a different exit pupil of the photographing lens. A microlens that guides the divided light beam to the plurality of photoelectric conversion units ; and an image pickup device that includes a plurality of pixels that include the plurality of pixels, wherein the light beam dividing unit is positioned at the first position. to case, the aberration information of the photographing lens, the first aberration information is an aberration information of said beam splitting means, the weighting value for the spatial frequency characteristic of the image pickup signal obtained from the imaging device, either before Symbol imaging element And weighting values for the spatial frequency characteristics of the focus detection signal of the pair to be acquired, based on, focusing correction for correcting the focal position obtained by using the focus detection signal of the pair is acquired from the imaging device A correction amount calculating step of calculating an amount; and a control step of controlling the focus adjustment mechanism based on the focus adjustment correction amount.

A control method of an imaging apparatus as another aspect of the present invention includes a beam splitter for splitting the light flux passing through the exit pupil of the taking lens having a focus adjusting mechanism to the first light flux and second light flux, a plurality A plurality of first pixels each including: a first photoelectric conversion unit; and a single microlens that guides a light beam of the first light beam that has passed through a different region of the exit pupil to the plurality of first photoelectric conversion units. A first imaging element, a plurality of second photoelectric conversion units, and one microlens that guides a light beam of the second light beam that has passed through a region having a different exit pupil to the plurality of second photoelectric conversion units. A second imaging element having a plurality of second pixels, wherein the second imaging element is provided with a plurality of second pixels . Shooting 1 First focusing correction amount and a pair of focus detection signal obtained from the second imaging device for correcting the first focus position obtained by using the focus detection signal of the pair is acquired from the device A correction amount calculating step of calculating a second focus adjustment correction amount for correcting a second focus position obtained by using the first and second focus adjustment correction amounts, and the focus adjustment mechanism based on the first and second focus adjustment correction amounts. And a control step of controlling

  According to the present invention, it is possible to provide an imaging device capable of performing highly accurate focus control and a control method thereof.

FIG. 2 is a configuration diagram of an imaging device (Examples 1 and 2). FIG. 2 is a configuration diagram of a first image sensor. It is a block diagram of a 2nd image sensor. FIG. 4 is a diagram illustrating a relative position between an exit pupil and a focus detection pupil. FIG. 3 is a diagram illustrating a focus detection area. It is a main flowchart which shows a photography process. It is a flowchart which shows a focus detection process. It is a flowchart which shows a subject information extraction process. FIG. 3 is a diagram illustrating an example of subject information obtained by an imaging element. FIG. 9 is an explanatory diagram of calculation of an AF evaluation band and a captured image evaluation band. 9 is a flowchart illustrating a BP amount calculation process (first embodiment). FIG. 4 is an explanatory diagram of photographing lens aberration information (Example 1). FIG. 5 is an explanatory diagram of the angle dependence of the spectral transmittance of a half mirror. It is a figure explaining change of an incidence angle to a half mirror. FIG. 6 is an explanatory diagram of a timing of extracting subject information. 9 is a flowchart illustrating a BP amount calculation process (second embodiment). FIG. 9 is an explanatory diagram of photographing lens aberration information (Example 2). FIG. 9 is an explanatory diagram relating to a data sequence of wavefront aberration information (Example 2). FIG. 9 is a configuration diagram of an imaging device (Example 3). 11 is a flowchart illustrating a photographing process (third embodiment). 14 is a flowchart illustrating focus detection processing (Example 3). 11 is a flowchart illustrating a BP amount calculation process (Example 3). 14 is a flowchart illustrating a BP amount calculation process (Example 4). FIG. 3 is a diagram illustrating an optical path of a photographing optical system and half mirror information. FIG. 4 is a relationship diagram between a thickness of a half mirror and a BP amount. It is a figure showing half mirror information to thickness. It is a figure which shows the half mirror from which a bending state differs. FIG. 7 is a diagram illustrating an optical path of a photographing optical system to a second image sensor. FIG. 9 is a diagram illustrating an example of a measurement result of the number of interference fringes. It is a figure which shows a bending direction. It is a correlation diagram of the amount of bending, and the amount of BP generated by the amount of bending. FIG. 9 is a diagram illustrating half mirror information with respect to a deflection amount.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each of the drawings, the same members are denoted by the same reference numerals, and redundant description will be omitted.

(Configuration of imaging device)
FIG. 1 is a configuration diagram of an interchangeable lens type digital camera main body (hereinafter, referred to as a camera main body) 100 and a photographing lens 500 detachably attached to the camera main body 100 as an example of an imaging apparatus embodying the present invention. is there.

  The photographing lens 500 has a photographing optical system, and in this embodiment, is a zoom lens having a variable focal length. The luminous flux from the subject passes through the first lens group 501, the second lens group 502, and the third lens group 503, and a subject image is formed on an imaging surface in the camera body 100. The second lens group 502 functions as a variator that moves forward and backward along the optical axis to perform zooming. The third lens group 503 functions as a focus lens that moves forward and backward along the optical axis to perform focus adjustment. The third lens group 503 is driven by a focus driving unit 504 using a stepping motor or the like. The iris diaphragm 505 includes a plurality of diaphragm blades for adjusting the amount of light incident on the photographing lens 500. The aperture driving unit 506 drives the aperture blade down until it reaches a predetermined F number. The lens CPU 507 communicates with the camera CPU 104 via the lens-side communication terminal 508 and the camera-side communication terminal 113, transmits and receives various information, and drives the focus drive unit 504 and the aperture drive unit 506 based on instructions from the camera CPU 104. Control.

  Although the zoom range and the open F number of the photographing lens 500 are designed according to the photographing intention, in the embodiment of the present invention, the open F number is set to a constant value of F2 regardless of the zoom state or the focus state. It is configured. On the other hand, the distance between the exit pupil and the imaging plane, the so-called exit pupil distance, changes according to the zoom state and the focus state.

  The luminous flux transmitted through each lens group in the taking lens 500 enters a beam splitter 103 as a luminous flux splitting unit provided in the camera body 100. The beam splitter 103 is fixed in the camera, and is a half mirror in the present embodiment. One of the light beams split by the beam splitter 103 passes through the beam splitter 103 and is guided to the first image sensor 101 in which the subject image is arranged on the first image surface. The other light beam is reflected by the beam splitter 103 and guided to the second image sensor 102 arranged on the second image plane. Note that the beam splitter 103 does not necessarily have to be a half mirror as long as it can split an incident light beam like a half mirror.

  The first and second imaging surfaces are at optically equivalent positions when viewed from the photographing lens 500. In other words, the first image sensor 101 arranged on the first image plane and the second image sensor 102 arranged on the second image plane are each optically conjugate to the subject via the photographing lens 500. On the image plane.

  On the first and second imaging surfaces, subject images having brightness according to the transmittance and the reflectance of the beam splitter 103 are formed. It is desirable that the half mirror be an ideal plane and have a uniform refractive index in a region through which the light beam passes. However, in reality, a certain amount of undulation or refractive index distribution is generated, so that the half mirror transmits through the beam splitter 103 and has a uniform refractive index. The image formed by the reflected light beam may have a reduced image quality. When the half mirror is made of thin glass, the degree of image quality deterioration is relatively larger in an image formed by reflected light beams than in an image formed by transmitted light beams. Therefore, in the present embodiment, the first image sensor 101 on the transmission side is an image sensor for high-resolution recording, that is, an image sensor for mainly photographing a still image, and the second image sensor 102 on the reflection side records more than a still image. The imaging device is used for capturing a moving image having a small number of pixels. However, the present invention is not limited to this mode, and the positions of the first image sensor 101 and the second image sensor 102 may be switched according to the characteristics of the beam splitter 103 and other conditions.

  The first image sensor 101 and the second image sensor 102 composed of a CMOS area sensor are configured by pixel units arranged in a matrix for converting a subject image into an electric signal. The pixel information converted into the electric signal is subjected to various correction processes for obtaining an image signal and a focus detection signal by the camera CPU 104, a process for converting the obtained image signal into a live view image and a recorded image, and the like. In the present embodiment, these processes and the like are performed by the camera CPU 104. However, these processes and the like may be performed by providing a dedicated circuit and performing such processing. The image sensor driving unit 114 is a unit for moving the second image sensor 102 in the optical axis direction. The first image sensor 101 and the second image sensor 102 are arranged on the optically conjugate image plane as described above. However, due to assembly errors and the difference between the best image planes due to the influence of aberrations described later. , Is difficult to dispose on a conjugate plane, and has a predetermined error. When simultaneously obtaining images with the two image sensors, the image sensor drive unit 114 is provided because the focus cannot be adjusted with respect to the two image sensors only by the focus adjustment unit provided in the photographing lens 500. . The image sensor driving unit 114 may drive the second image sensor 102 or move the beam splitter 103. In addition, an infrared cut filter, an optical low-pass filter, and the like are integrally arranged in each image sensor.

  The operation member 105 is various members for setting a photographing mode and photographing conditions of the camera. The storage medium 106 is a flash memory, and is a medium for recording captured still images and moving images. The display 107 in the finder includes a display 108 as a small and high-definition display means such as an organic EL display or a liquid crystal display, and an eyepiece 109. As the external display 110, an organic EL display or a liquid crystal display having a screen size suitable for naked eyes is used. Various information such as the setting state of the camera body 100, the live view image, and the photographed image are displayed on the in-finder display 107 and the external display 110.

  The focal plane shutter 111 is arranged on the front surface of the first image sensor 101. The shutter driving unit 112 is, for example, a motor, and controls the exposure time when capturing a still image by controlling the driving of the blades of the shutter. The camera-side communication terminal 113 is provided on a camera mount for mounting the photographing lens 500. The camera communication terminal 113 transmits and receives information exchanged between the camera CPU 104 and the lens CPU 507 together with the lens communication terminal 508 provided on the lens mount unit.

  FIG. 2 is a configuration diagram of the first image sensor 101. In the present embodiment, the first and second imaging elements 101 and 102 differ in pixel pitch and the number of photoelectric conversion units corresponding to one microlens. Other functions and configurations are similar.

  FIG. 2A is a plan view of a part of the pixel portion near the center (near an image height 0) of the first imaging surface when viewed from the photographing lens 500 side. Each of the plurality of pixel units included in the first imaging element 101 is a square pixel unit having a size of 4 μm in both the horizontal direction (x) and the vertical direction (y) on the imaging surface, and the structure of the pixel unit is substantially the same. All the same. The first image sensor 101 is an image sensor having 24 million effective pixels, in which a pixel portion is arranged in 6000 pixels in the horizontal direction and 4000 pixels in the vertical direction. The size of the imaging area can be obtained by multiplying the size of the pixel portion, that is, the pixel pitch by the number of pixels. In this case, the size is 24 mm in the horizontal direction and 16 mm in the vertical direction. In each pixel portion, RGB color filters are arranged in a mosaic pattern.

  FIG. 2B is a cross-sectional view of the pixel portion. Photoelectric conversion units 101a and 101b are provided in a silicon substrate 101d serving as a base of the CMOS image sensor. In the silicon substrate 101d, a switching transistor (not shown) that converts photoelectrons generated in each photoelectric conversion unit into a voltage and reads the voltage to the outside is formed, and an output signal after the photoelectric conversion is read by the wiring layer 101e.

  The wiring layer 101e is insulated by a transparent interlayer film 101f. A color filter 101g for color separation is provided below the on-chip micro lens 101c. The shape of the on-chip micro lens 101c is determined so that the focal position thereof substantially matches the upper surfaces of the photoelectric conversion units 101a and 101b. Therefore, the photoelectric conversion units 101a and 101b are back-projected to the vicinity of the exit pupil of the photographing optical system via the on-chip micro lens 101c, and function as a focus detection pupil when the back-projected image performs focus detection by the phase difference detection method. . When performing focus detection by the phase difference detection method, output signals of the photoelectric conversion units 101a and 101b are individually processed to generate a phase difference image of one to two images. The camera CPU 104 as a focus detection unit calculates the defocus amount of the subject image on the imaging surface from the relative image shift amount of the two images. Further, the camera CPU 104 as an addition control unit adds the signals of the photoelectric conversion units 101a and 101b to obtain a still image or moving image recording image signal or a live view (display) image signal. Note that the addition process may be performed by providing a dedicated circuit, or the signals individually output from the photoelectric conversion units 101a and 101b may be added in the first imaging element 101 and output.

  FIG. 3 is a configuration diagram of the second imaging element 102, and is a plan view of a part of the pixel portion near the center (near an image height 0) of the second imaging surface when viewed from the imaging lens 500 side. Each of the plurality of pixel units included in the second image sensor 102 is a square pixel unit having a size of 12 μm in both the horizontal direction (x) and the vertical direction (y) on the imaging surface, and the structure of the pixel unit is substantially All the same. The second image sensor 102 is an image sensor having a pixel section of 2000 pixels in the horizontal direction and 1333 pixels in the vertical direction and an effective pixel number of about 2.67 million pixels. The size of the imaging area can be obtained by multiplying the size of the pixel portion, that is, the pixel pitch by the number of pixels. In this case, the size is 24 mm in the horizontal direction and 16 mm in the vertical direction. In each pixel portion, RGB color filters are arranged in a mosaic pattern. Since the pixel pitch is three times that of the first image sensor 101, the total number of pixels is ３.

  Each pixel portion of the second image sensor 102 has the same configuration as the first image sensor 101, such as each wiring layer, on-chip microlens, color filter for color separation, and the like. The difference from the first image pickup device 101 is that nine photoelectric conversion units are provided below the on-chip micro lens 102j. Thereby, the angular resolution of the light beam received with respect to the first image sensor 101 can be increased. In the first image sensor 101, the back-projected image generated near the exit pupil of the imaging optical system corresponds to the two photoelectric conversion units 101a and 101b. On the other hand, in the second image sensor 102, the back-projected images generated near the exit pupil of the imaging optical system correspond to the nine photoelectric conversion units 102a to 102i. Accordingly, a plurality of methods of configuring a focus detection pupil when performing focus detection using the phase difference detection method, such as division in the horizontal direction and division in the vertical direction, can be considered. The camera CPU 104 as a focus detection unit calculates the defocus amount of the subject image on the imaging surface from the relative image shift amount of the two images. Further, the camera CPU 104 as addition control means adds all or the signals of the photoelectric conversion units 102a to 101i or selectively adds a still image or moving image recording image signal or a live view (display) image signal. obtain. Note that the addition process may be performed by providing a dedicated circuit, or the signals individually output from the photoelectric conversion units 102a to 102i may be added and output in the second image sensor 102.

  The first and second imaging elements 101 and 102 of the present embodiment have two types of readout modes. The first readout mode is referred to as an all-pixels readout mode, and is a mode for capturing a still image or moving image for recording. In this mode, signals of all the pixel units are read. The second reading mode is referred to as a thinning-out reading mode, and is a mode for displaying a live view image having a smaller number of pixels than a recording still image. The live view image is an image for displaying an image obtained by each image sensor on the display 107 in the finder or the external display 110 in real time. Since the number of pixels required for the live view is smaller than the total number of pixels, by reading signals only from the pixel portions thinned out at a predetermined ratio in both the x and y directions, the processing load on the signal processing circuit is reduced and the power consumption is reduced. It also contributes to the reduction of In each of the first and second readout modes, the signals of a predetermined number of photoelectric conversion units included in each pixel unit are read out independently, so that it is possible to generate a signal for focus detection. . The number of pixels and the pixel pitch of the read signal change according to the read mode.

  In the present embodiment, the first image sensor 101 is mainly used for still image shooting, but may be used for moving image shooting. For example, the first imaging element 101 can record a thinned-out read image as a low-resolution moving image while the second imaging element 102 is capturing a moving image. Similarly, the second image sensor 102 is mainly used for moving image shooting, but may be used for still image shooting. For example, it is possible to record a desired one frame as a still image during moving image recording.

  Next, the structure of the pixel unit and the focus detection pupil will be described. FIG. 4 is a diagram illustrating the correspondence between the photoelectric conversion unit of each image sensor and the focus detection pupil.

  FIGS. 4A and 4B show the conjugate relationship between the exit pupil plane of the photographing optical system and the photoelectric conversion units 101a and 101b of the first image sensor 101 arranged near the image height of zero, that is, near the center of the image plane. FIG. The photoelectric conversion unit and the exit pupil plane of the imaging optical system are designed to have a conjugate relationship by the on-chip microlens 101c. The exit pupil of the photographing optical system generally coincides with the surface on which the iris diaphragm for adjusting the light amount is placed.

  The photographing lens 500 of this embodiment is a zoom lens having a zooming function. However, depending on the optical type, when a zooming operation is performed, the distance and the size of the exit pupil from the image plane change. The photographing optical system in FIG. 4A shows a state in which the focal length is between the wide-angle end and the telephoto end, that is, the state of Middle. Assuming this as the standard exit pupil distance Zep, the optimal design of the eccentricity parameter according to the shape of the on-chip micro lens 101c and the image height (X, Y coordinates) is performed.

  In FIG. 4A, lens barrel members 501r and 503r hold a first lens group 501 and a third lens group 503, respectively. The aperture plate 505a defines the aperture diameter when the aperture is opened, and the aperture blade 505b is an aperture blade for adjusting the aperture diameter when the aperture is stopped down. Note that the lens barrel member 501r, the aperture plate 505a, the aperture blade 505b, and the lens barrel member 503r that function as a restricting member for the light flux passing through the imaging lens 500 show an optical virtual image when observed from the image plane. . The synthetic aperture near the stop 505 is defined as the exit pupil of the lens, and the distance from the image plane to the exit pupil is Zep as described above.

  The two photoelectric conversion units 101a and 101b are projected on the exit pupil plane of the imaging optical system by the on-chip micro lens 101c. In other words, the exit pupils (projected images EP1a, EP1b) of the photographing optical system are projected on the surfaces of the photoelectric conversion units 101a, 101b via the on-chip micro lens 101c.

  FIG. 4B shows a projected image of the photoelectric conversion unit on the exit pupil plane of the imaging optical system, and the projected images on the photoelectric conversion units 101a and 101b are EP1a and EP1b, respectively. In the present embodiment, the first imaging element 101 has a pixel unit that can obtain an output of one of the two photoelectric conversion units 101a and 101b and an output of the sum of both. The output of the sum of the two is a signal output by photoelectrically converting a light beam passing through both areas of the projected images EP1a and EP1b, which are substantially the entire pupil area of the photographing optical system.

  In FIG. 4A, L (outermost part) of the light beam passing through the photographing optical system is regulated by the aperture plate 505a, and the projected images EP1a and EP1b are vignetted by the photographing optical system. Almost has not occurred. In FIG. 4B, vignetting occurs because most of the projected images EP1a and EP1b of the photoelectric conversion units 101a and 101b are included in a circle TL formed by the light beam L in FIG. 4A. It can be seen that almost no occurrence occurs. Since the light beam L is limited only by the aperture plate 505a, the circle TL can be rephrased as the aperture plate 505a. At this time, the vignetting state of the projected images EP1a and EP1b is symmetric with respect to the optical axis at the center of the image plane, and the amounts of light received by the photoelectric conversion units 101a and 101b are equal to each other.

  FIG. 4C is different from FIG. 4A only in a portion related to the photoelectric conversion units 102 a to 102 i of the second image sensor 102. The three photoelectric conversion units 102d, 102e, and 102f are projected on the exit pupil plane of the imaging optical system by the on-chip micro lens 102n. In other words, the exit pupils (projected images EP2d, EP2e, EP2f) of the photographing optical system are projected onto the surfaces of the photoelectric conversion units 102d, 102e, 102f via the on-chip micro lens 102n.

  FIG. 4D shows a projected image of the photoelectric conversion unit on the exit pupil plane of the imaging optical system, and the projected images of the photoelectric conversion units 102a to 102i are EP2a to EP2i, respectively. Further, in the present embodiment, the second imaging element 102 has a pixel unit that can obtain the output of each of the nine photoelectric conversion units 102a to 102i and the sum of all outputs. The output of all the sums is a signal output by photoelectrically converting a light beam that has passed through all areas of the projected images EP2a to EP2i, which are substantially the entire pupil area of the photographing optical system.

  In FIG. 4C, when a light beam (outermost part) passing through the photographing optical system is indicated by L, the light beam L is regulated by the aperture plate 505a, and the projected images EP2d, EP2e, and EP2f are formed by the photographing optical system. Vignetting hardly occurred in. In FIG. 4D, vignetting occurs because most of the projected images EP2a to EP2i of the photoelectric conversion units 102a to 102i are included in a circle TL formed by the light beam L in FIG. 4C. It can be seen that almost no occurrence occurs. Since the light beam L is limited only by the aperture plate 505a, the circle TL can be rephrased as the aperture plate 505a. At this time, the vignetting state of each projected image is symmetric with respect to the optical axis at the center of the image plane.

  Here, a pixel signal in the case of performing focus detection by the phase difference detection method will be described. In the present embodiment, the on-chip microlens 101c and the divided photoelectric conversion units 101a and 101b pupil-divide the emitted light beam of the imaging optical system. Then, in a plurality of pixels within a predetermined range arranged on the same row, an image formed by connecting the outputs of the photoelectric conversion units 101a to form an A image for AF is used. Similarly, a knitted image obtained by connecting the outputs of the photoelectric conversion units 101b is referred to as an AF B image. The outputs of the photoelectric conversion units 101a and 101b are obtained by subjecting the outputs of green, red, blue, and green in a Bayer array to signal addition processing, and use what is artificially calculated as a luminance (Y) signal. However, the A and B images for AF may be organized for each of the colors red, blue, and green. By detecting the relative image shift amount between the A image and the B image for AF generated by the correlation calculation, the defocus amount of a predetermined area, that is, the defocus amount can be detected.

The second image sensor 102 also acquires a pair of image signals. Since the second image sensor 102 has a larger number of pupil divisions than the first image sensor 101, a plurality of image signal configuration methods can be considered. For example, the outputs of the photoelectric conversion units 102a, 102d, and 102g are added to form a single signal, and the combined signal is used as an AF C image. Similarly, the outputs of the photoelectric conversion units 102c, 102f, and 102i are added to form a single signal, and the combined signal is used as an AF D image. The output of the photoelectric conversion unit 102d may be used for the C image for AF, and the output of the photoelectric conversion unit 102f may be used for the D image for AF. The outputs of the left six photoelectric conversion units (102a, 102b, 102d, 102e, 102g, 102h) for the C image for AF, and the six photoelectric conversion units (102b, 102c) for the right for the D image for AF. , 102e, 102f, 102h, 102i) may be used. Further, the number of photoelectric conversion units used for the C image and the D image for AF need not be uniform. At locations where the image height is high on the image sensor, vignetting of the photographing light beam occurs. Therefore, the photoelectric conversion unit used for the AF C image and the D image may be selected based on the vignetting situation. Even if there is a difference in the amount of light after the selection, that is, a difference in the magnitude of the signal output, there is no problem because the magnitude of the signal output is made substantially equal by shading correction described later. The signals of the A and B images for AF of the first image sensor 101 and the C and D images for AF of the second image sensor 102 are configured such that image shifts occur in the horizontal direction of each image sensor. However, the second image sensor 102 can also acquire a pair of image signals in which an image shift occurs in the vertical direction. In the present embodiment, an AF E image and an F image are acquired as a pair of signals in which an image shift occurs in the vertical direction. As described above, various types of photoelectric conversion outputs used for the E and F images for AF can be considered.
(Explanation of conjugate relation)
As described with reference to FIGS. 2 to 4, the first and second imaging elements 101 and 102 have not only a function of imaging only but also a function as a focus detection device. Further, as the focus detection method, since a focus detection pixel that receives a light beam obtained by dividing the exit pupil is provided, focus detection using a phase difference detection method can be performed.

  Next, a focus detection area according to the present embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating a focus detection area in an imaging range 217 indicated by a dotted line in which pixels of the first and second imaging elements 101 and 102 are formed. In the present embodiment, a total of three focus detection areas are provided at the center of the imaging range 217 and at two places on the left and right. In the present embodiment, the imaging surface phase difference AF (phase difference detection AF) is performed based on signals obtained from the first and second imaging elements 101 and 102 in the focus detection area. The focus detection area in FIG. 5 includes a focus detection unit including pixels that perform pupil division in the horizontal direction (lateral direction) of the first image sensor 101. Further, a focus detection unit including pixels for performing pupil division in the horizontal direction (horizontal direction) and the vertical direction (vertical direction) of the second image sensor 102 is provided.

  In the present embodiment, the photographing range 217 is common, but may be divided separately. For example, when still image shooting and moving image shooting are performed simultaneously by the first and second image sensors 101 and 102, different image capturing ranges such as the aspect ratio of a still image and a moving image may be provided. In such a case, when displaying the photographed image, the photographing range of each image sensor can be clearly indicated by a frame or the like, so that the photographer can be notified.

  Within the imaging range 217, three horizontal focus detection areas 218ah, 218bh, and 218ch for performing the imaging surface phase difference AF, and vertical focus detection areas 218av, 218bv, and 218cv are provided. The first imaging element 101 obtains a focus detection signal from the areas corresponding to the focus detection areas 218ah, 218bh, and 218ch to perform focus detection only in the horizontal direction with respect to the direction having the contrast of the subject. The second imaging element 102 obtains a focus detection signal from the areas corresponding to the focus detection areas 218ah, 218bh, and 218ch, and performs focus detection in the horizontal direction with respect to the direction having the contrast of the subject. In addition, the second imaging element 102 obtains a focus detection signal from an area corresponding to the focus detection areas 218av, 218bv, and 218cv, and performs focus detection in the vertical direction with respect to the direction having the contrast of the subject.

The display frames 219a, 219b, and 219c indicate the range of the focus detection area on the external display 110. By setting the display frame to be approximately the same size as the focus detection area, it is possible to appropriately perform focus detection on a subject arranged by the photographer in the display frame.
(Shooting flow)
With reference to FIG. 6, the photographing processing of the present embodiment will be described. FIG. 6 is a main flowchart showing the photographing processing in this embodiment. When the photographer turns on the power switch of the camera body 100, the camera CPU 104 checks the operation of each actuator and the first and second image sensors 101 and 102 in the camera body 100, and initializes the memory contents and the execution program. I do.

  In step S101, the camera CPU 104 communicates with the lens CPU 507 to store information such as the open F number of the photographing lens, the focal length, the exit pupil distance PL, and the focus sensitivity which is a proportional constant between the focus lens extension amount and the focus change amount. Receive.

  In step S102, the camera CPU 104 determines whether or not the camera body 100 is set to a still image shooting mode (a mode in which only a still image is shot, or a still image / moving image simultaneous shooting mode). When the still image shooting mode is set, the process proceeds to step S103. When the still image shooting mode is not set, that is, when only the moving image is set, the process proceeds to step S104.

  In step S103, the camera CPU 104 drives the first image sensor 101 for still image shooting in the subject information acquisition mode, and drives the second image sensor 102 for moving image shooting in the live view mode.

  The live view mode is a mode in which an image acquired by the image sensor is displayed on the display 107 in the finder or the external display 110 in real time. Since the number of pixels of each display is smaller in the horizontal direction and the vertical direction than the number of pixels of the image for recording, in the live view mode, when reading out from the image sensor, the pixels are thinned out in both the horizontal and vertical directions, and the image sensor And the power consumption of the signal processing circuit is kept low. Further, the phase difference detection is also performed using the image signal read in the live view mode. However, in order to maintain the resolution of the focus detection signal, information of all pixels may be read without thinning out only the focus detection area.

  The subject information acquisition mode is a mode for acquiring subject information such as the spatial frequency characteristics, spectral, and contrast directions of the pattern of the subject, and focus detection information. When there are a plurality of focus detection ranges and the area is wide, driving is performed in the second read mode. On the other hand, when the focus detection area is limited due to a photographer's instruction, subject detection, or the like, only the vicinity of the focus detection area in the first readout mode is output without thinning out the horizontal and vertical directions. Get. Further, as necessary, more frame information is obtained by increasing the frame rate and shortening the time interval of information acquisition. By obtaining signals in a limited area, the power consumption of the image sensor and the signal processing circuit is kept low.

  In step S104, the camera CPU 104 drives the second image sensor 102 in order to support shooting of only a moving image. In the present embodiment, the first image sensor 101 for photographing a still image is not driven in photographing only a moving image. This is because the resolution of an image obtained by shooting a moving image is limited by the pixel pitch of the second image sensor 102, and therefore, even if higher frequency information is obtained, it may not be possible to effectively use the information. As a result, power consumption can be reduced. However, the first image sensor 101 for still image shooting may be configured to obtain more subject information by increasing the frame rate than the second image sensor 102 for moving image shooting.

  In step S105, the camera CPU 104 converts the signal acquired by the second image sensor 102 into a display signal, transmits the signal to the in-finder display 107 or the external display 110, and starts live view display.

  In step S106, the camera CPU 104 determines the brightness of the image signal obtained by driving the second image sensor 102, and performs aperture control during live view. The storage time and aperture control of the second image sensor 102 may be determined depending on which recording image has priority. When the frame rate of a moving image is determined, the storage time is restricted in view of the continuity of a moving subject in a recorded image. Therefore, in order to adjust an appropriate exposure state, the only parameters that can be adjusted are aperture and signal gain adjustment. For a still image, the photographer has set the photographed image, but in a standby state before photographing the still image, there is a degree of freedom in setting the aperture, accumulation time, and gain. Aperture control is performed based on these requirements.

  In step S107, the camera CPU 104 executes a focus detection process. In this embodiment, focus detection by two image sensors and acquisition of subject information are performed. Further, the focus detection result is corrected using the subject information. Further, based on the focus detection result, lens driving, focusing display, and the like are also performed.

  In step S108, the camera CPU 104 determines whether the moving image shooting trigger button has been turned on. If an ON operation has been performed, the process proceeds to step S109. If an ON operation has not been performed, the process proceeds to step S110.

  In step S109, the camera CPU 104 performs image processing for a moving image, and starts generating a moving image. The generated moving image is recorded.

  In step S110, the camera CPU 104 determines whether or not the still image start trigger button has been turned on. In the present embodiment, when a still image shooting is instructed during moving image live view or moving image recording, a still image can be recorded by the first image sensor 101. If an ON operation has been performed, the process proceeds to step S111. If an ON operation has not been performed, the process proceeds to step S112. Note that when the camera body 100 is set to a mode for capturing only a moving image, the process of step S110 may be omitted.

  In step S111, the camera CPU 104 executes still image shooting. First, the camera CPU 104 transmits the F number for photographing a still image to the lens CPU 507 via the camera-side communication terminal 113 and the lens-side communication terminal 508. Next, the photographing lens 500 controls the drive of the diaphragm drive unit 506 to control the aperture diameter of the iris diaphragm to a value corresponding to the F number for photographing a still image. This process is skipped in the case of priority for moving image shooting. After that, the focal plane shutter that has been opened for the live view is driven to be reset to the closed state, and the first image sensor 101 starts a charge accumulation operation for photographing a still image. Next, the first and second curtains of the focal plane shutter are drive-controlled based on the shutter speed for photographing a still image calculated by a predetermined exposure calculation program, and a predetermined exposure amount is given to the first image sensor 101. When the travel of the focal plane shutter is completed, the accumulation operation of the first image sensor 101 ends, and the charge transfer is performed.

  The still image recording performed in step S111 may be a single shooting mode for recording one image or a continuous shooting mode for recording a plurality of images. In the present embodiment, the following description will be made assuming a continuous shooting mode.

  In step S112, the camera CPU 104 determines whether or not the moving image shooting trigger button has been turned off. If the off operation has not been performed, that is, if the on state has been continued, the processing from step S105 to step S111 is repeatedly executed to continue the AF control and the moving image recording for the moving image, and also permit the interruption of the still image. . If it is turned off, the photographing process ends.

  In this embodiment, focus detection and imaging are performed using signals obtained from two image sensors for still image shooting and moving image shooting. Instructions for focus detection and imaging are provided by preparing different operation members for a still image and a moving image. As a result, not only can the recording timing of the still image and the moving image be controlled independently, but also at the time of the focus detection, the focus detection of the still image can be performed at a high speed, and the focus detection of the moving image can be performed at a low speed. , Different controls can be performed.

  Next, a focus detection process (AF process) according to the present embodiment will be described with reference to FIG. FIG. 6 is a flowchart of the seven focus detection process. Each step in FIG. 7 is mainly executed by the camera CPU 104.

  In step S901, the camera CPU 104 sets a focus detection area. In this embodiment, the camera CPU 104 selects the focus detection areas 219a, 219b, and 219c. The selection may be based on a photographer's instruction, or may be based on a prior focus detection result, a subject recognition result, or the like.

  In step S902, the camera CPU 104 acquires an image signal for the selected focus detection area. The camera CPU 104 acquires an A image and a B image for AF from the first image sensor 101 and a C image, a D image, an E image, and an F image from the second image sensor 102.

  In step S <b> 903, the camera CPU 104 performs various kinds of signal correction and filter processing. First, the camera CPU 104 adjusts the offset and the gain according to the signal output characteristics. Thereafter, so-called shading correction for correcting a light amount difference due to the vignetting of the photographing lens 500 of each AF signal is performed. The light amount ratio of a pair of AF signals is obtained from the position of the exit pupil on the optical axis of the photographing optical system, the aperture value, the range of the light flux obtained from the frame information that blocks the light beam, and the intensity information for each angle of the photoelectric conversion unit. . The correction value used for the shading correction is the reciprocal of the light amount ratio, and the correction is performed by multiplying a pair of signals. The look-up table may be stored for the coefficients used for the shading correction for each position of the exit pupil on the optical axis, the aperture value, frame information for blocking the light flux, and the like. After the signal correction, digital filter processing is performed in accordance with the evaluation band used for focus detection. Generally, a detectable defocus area becomes narrower when a high frequency band is evaluated. Therefore, a plurality of filter processes are performed to evaluate a plurality of types of bands. The A and B images for AF obtained from the first image sensor 101 can be evaluated in a higher frequency range because the pixel pitch of the first image sensor 101 is small. Therefore, filter processing corresponding to the high band and the middle band is performed. Further, when the C image, the D image, the E image, and the F image for AF obtained from the second image sensor 102 are used, low-pass and ultra-low-pass filter processing is performed.

  In step S904, the camera CPU 104 calculates the defocus amount from the phase difference between the two images by performing a correlation operation using the paired signal S1 and signal S2.

  The camera CPU 104 as a correlation amount calculating means calculates a correlation amount between the pair of focus detection signals. When performing the correlation calculation, the number of data in the field of view and the number of shift data are set. The visual field data corresponds to a window when performing a correlation operation, and determines the size of a region for performing focus detection. When the data in the visual field is increased, a more reliable correlation calculation result is obtained, but the frequency of occurrence of so-called perspective conflict in which objects at different distances exist in the same focus detection area increases. Therefore, based on information such as the size of the subject and the focal length of the photographing optical system, the in-field data corresponding to the focus detection area having an appropriate size is set. The number of shift data corresponds to the maximum shift amount when evaluating the correlation amount while shifting the positional relationship between the pair of images. By increasing the amount of shift in the positional relationship between the pair of images, it is possible to detect the focus of a subject in a larger defocus state. On the other hand, increasing the shift amount of the positional relationship between the A image and the B image leads to an increase in the amount of calculation, and the focus detection calculation takes a long time. Therefore, the setting is appropriately made in consideration of the defocus amount to be detected and the accuracy. The correlation amount COR (h) used for the correlation operation can be calculated, for example, by the following equation (1).

  In equation (1), the pair of focus detection signals are S1 (k) and S2 (k) (1 ≦ k ≦ P), respectively. NW1 corresponds to the data in the visual field, and hmax corresponds to the number of shift data. After obtaining the correlation amount COR (h) for the shift amount h, the shift amount h at which the correlation between the paired images is the highest, that is, the value of the shift amount h at which the correlation amount COR is the minimum is obtained. Note that the shift amount h at the time of calculating the correlation amount COR (h) is an integer, but when calculating the shift amount h at which the correlation amount COR (h) is minimized, in order to improve the accuracy of the defocus amount, Interpolation processing is performed as appropriate to determine a value (real value) in subpixel units. As a pair of signals, the above-described phase difference is calculated for a combination of the A and B images for AF, the C and D images for AF, and the E and F images for AF.

  The camera CPU 104 calculates the reliability evaluation value simultaneously with the calculation of the correlation amount. The reliability evaluation value is an evaluation value for determining whether or not the minimum value of the correlation amount can be accurately calculated. For example, the evaluation value is whether the minimum value of the correlation amount COR is sufficiently small and the degree of coincidence between the focus detection signals S1 and S2 is high, and whether the change in the correlation amount COR near the minimum value of the correlation amount COR is large. . Further, the peak bottom of the focus detection signals S1 and S2 may be used.

  The camera CPU 104 detects a minimum value of the correlation amount waveform. A well-known sub-pixel operation or the like is performed to accurately detect. The difference value DCOR of the correlation amount is calculated according to the following equation (2).

DCOR (h) = COR (h) -COR (h-1) (2)
The camera CPU 104 calculates the shift amount dh at which the sign of the difference amount of the correlation amount COR changes as the shift amount h at which the correlation amount COR (h) becomes minimum, using the difference value DCOR of the correlation amount. Assuming that the value of h immediately before the sign of the difference amount changes is h1 and the value of h whose sign changes is h2 (h2 = h1 + 1), the shift amount dh is calculated according to the following equation (3).

dh = h1 + | DCOR (h1) | / | DCOR (h1) −DCOR (h2) | (3)
As described above, the camera CPU 104 as the correlation calculation means calculates the shift amount dh at which the correlation between the paired images is maximum in sub-pixel units, and multiplies the obtained shift amount dh by the sensitivity. , The amount of defocus. The method for calculating the phase difference between the two one-dimensional image signals is not limited to the method described here, and any known method can be used.

  In step S905, the camera CPU 104 extracts subject information. The subject information is information used for correcting a focus detection result described later, and is information on a spatial frequency characteristic, a spectral (color), a direction of contrast, and the like of the subject.

  In step S906, the camera CPU 104 calculates a correction amount (BP amount) corresponding to the subject information.

  In step S907, the camera CPU 104 corrects the detected defocus amount.

  In step S908, the camera CPU 104 selects a highly reliable detection result from the obtained defocus amounts. In the present embodiment, the first focus detection result is obtained from the A and B images for AF obtained from the first image sensor 101, the C image for AF obtained from the second image sensor 102, and the second focus is obtained from the D image. As a detection result, a third focus detection result is obtained from the AF E image and the F image obtained from the second image sensor 102. The camera CPU 104 selects a highly reliable result from the reliability evaluation values, and selects an appropriate detection result in view of the subject information. For example, when the subject information includes many high-frequency components, the first focus detection result obtained from the first image sensor 101 is prioritized. When the contrast direction includes many components in the vertical direction, the second focus obtained from the second image sensor 102 is used. The focus detection result of No. 3 is given priority. Further, when the detected defocus amount is large, the lower the evaluation band is, the higher the reliability is. Therefore, the focus detection result to be used may be determined from the detected defocus amount.

  In step S909, the camera CPU 104 drives the third lens group 503 and the second image sensor 102 based on the selected defocus amount. In this embodiment, the focus can be adjusted not only by moving the third lens group 503 but also by driving the second image sensor 102. This corresponds to the fact that the positions of the two image pickup devices are not optically conjugate due to the BP amount or an error during assembly. In this step, the third lens group 503 is driven based on the focus detection result obtained from the first image sensor 101. Further, based on the focus detection result obtained from the second image sensor 102, the excess or deficiency of the focus adjustment by driving the third lens group 503 is supplemented by driving the second image sensor 102. The drive amount of the second image sensor 102 may be determined from a focus detection result obtained from the first image sensor 101 and a difference between an assembly error amount and a BP amount of the two image sensors. Conversely, the driving amount of the third lens group 503 may be determined using the second focus detection result.

In step S910, the camera CPU 104 displays a frame or the like corresponding to the focus detection area on the display 108 or the display 107 in the viewfinder in order to notify the photographer that focus has been achieved.
(Method of extracting subject information)
The subject information extraction processing in step S905 in FIG. 7 will be described with reference to FIGS. FIG. 8 is a flowchart showing the subject information extraction processing. Each step in FIG. 8 is mainly executed by the camera CPU 104. The camera CPU 104 functions as subject information extracting means for extracting subject information from signals obtained from the first and second imaging elements 101 and 102.

  In step S9051, the camera CPU 104 obtains subject information using the signal obtained from the first image sensor 101. An imaging signal (a signal obtained from a light beam that has passed through substantially the entire pupil region of the photographing optical system) obtained from a region corresponding to the set focus detection region is used. The signal obtained in the Bayer state is separated for each of RGB colors, and spatial frequency characteristics for each direction are obtained. As a method of acquiring the spatial frequency characteristics for each direction, a two-dimensional FFT may be performed, or a signal amount (power) of a signal that has been subjected to digital filtering in several different bands in the vertical direction and the signal amount (power) of the signal may be horizontally applied. The signal amount of the signal may be obtained. Thereby, it is possible to obtain the color, frequency component, and contrast direction of the subject for which focus detection is performed. In the first image sensor 101, the focus detection direction is only the horizontal direction. Therefore, when performing focus detection or imaging using only the first image sensor 101, the extraction of subject information in the vertical direction may be omitted.

  In step S9052, the camera CPU 104 obtains subject information using the signal obtained from the second image sensor 102.

  Here, with reference to FIG. 9, characteristics of subject information obtained by the first and second imaging elements 101 and 102 will be described.

  Since the first image sensor 101 has a fine pixel pitch, it can be driven in the subject information acquisition mode as necessary to obtain a wide range of information on the spatial frequency band from high to low. Further, by specializing in the acquisition of the subject information, the accumulation time can be set short, and the information acquisition can be performed while suppressing the loss of the high-frequency component of the image signal due to the influence of camera shake or subject shake. Further, regarding the direction of the contrast, both the horizontal direction and the vertical direction can be obtained by the above-described method. However, when considering the common processing of the focus detection signal, signal correction processing, digital filter processing, and the like, calculation of a low-frequency signal not used for focus detection and filtering in the vertical direction are separately required. The amount of calculation can be reduced by extracting the subject information using the sum of the pair of focus detection signals subjected to various processes. In the present embodiment, the calculation is omitted for the items described as "necessary calculation" in FIG.

  Since the second image sensor 102 has a relatively large pixel pitch, it is impossible to obtain a high-frequency signal. On the other hand, using the signal processed for focus detection, low- and ultra-low-range subject information can be obtained without increasing the amount of calculation. Similarly, regarding the direction of the contrast, since the focus detection signal in the vertical direction is obtained, the subject information in the vertical direction can be obtained without increasing the calculation amount.

  In the present embodiment, as described above, by sharing the acquired subject information in accordance with the characteristics of the image sensor, necessary and sufficient information can be obtained without increasing the amount of calculation.

  In step S9053, the camera CPU 104 integrates subject information obtained from the two image sensors. In the present embodiment, it is possible to obtain a magnitude relationship between the information amounts of 24 categories obtained by multiplying three types of colors, two types of contrast directions, and four types of frequencies. In the present embodiment, the information obtained by the two image sensors is different. However, for example, it is possible to calibrate the signal amount of each other by obtaining a signal in the horizontal middle region from both the image sensors. it can.

  In step S9054, the camera CPU 104 calculates the weight of the band of the signal used for focus detection (AF evaluation band). The AF evaluation band is calculated in consideration of the subject information, the imaging optical system, the sampling of the image sensor, and the effects of digital filters used for evaluation. Further, the camera CPU 104 calculates the weight of the band of the signal used for the captured image (captured image evaluation band). The captured image evaluation band is calculated in consideration of the influence of the subject, the imaging optical system, the sampling of the image sensor, and the evaluation band of the viewer of the captured image.

  The calculation of the AF evaluation band and the captured image evaluation band will be described with reference to FIG. The AF evaluation band is calculated for each pair of focus detection signals. Further, the captured image evaluation band is calculated for each of the two imaging elements. 10A to 10F all show the intensity for each spatial frequency, with the horizontal axis representing the spatial frequency and the vertical axis representing the intensity.

  FIG. 10A shows the spatial frequency characteristic (I) of the subject. F1, F2, F3, and F4 on the abscissa are spatial frequency bands to be evaluated, and become higher as going from F1 to F4. In the subject information acquired in step S9053, the high range corresponds to F4, the middle range corresponds to F3, the low range corresponds to F2, and the very low range corresponds to F1. Similar information exists for each of the RGB and the contrast directions, but for simplicity, green object information in the horizontal direction is shown. Nq is a Nyquist frequency determined according to the pixel pitch of the first image sensor 101. Similarly, Nq2 is a Nyquist frequency determined according to the pixel pitch of the second image sensor 102. The spatial frequencies F1 to F4 and the Nyquist frequency Nq are also shown in FIGS. 10B to 10F.

  In the present embodiment, the subject information acquired in step S9053 is used as the spatial frequency characteristic (I) of the subject. In FIG. 10A, the spatial frequency characteristic (I) of the subject is drawn as a curve, but has discrete values corresponding to the spatial frequencies F1, F2, F3, and F4, which are represented by I (n). (1 ≦ n ≦ 4).

  FIG. 10B shows a spatial frequency characteristic (O) of the photographing optical system at the time of focusing. This information may be obtained from the lens CPU 507, or may be stored in a RAM or the like in the camera CPU 104. In FIG. 10B, the spatial frequency characteristic (O) of the photographing optical system is drawn by a curve, but discretely has values corresponding to the spatial frequencies F1, F2, F3, and F4, which are represented by O ( n) (1 ≦ n ≦ 4).

  FIG. 10C shows a spatial frequency characteristic (L) of the optical low-pass filter. This information is stored in the RAM in the camera CPU 104. In FIG. 10C, the spatial frequency characteristic (L) of the optical low-pass filter is drawn as a curve, but has discrete values corresponding to the spatial frequencies F1, F2, F3, and F4. (N) (1 ≦ n ≦ 4). When different optical low-pass filters are used between the two image sensors, the spatial frequency characteristics are stored separately.

  FIG. 10D shows the spatial frequency characteristics (M1, M2) by signal generation. The image sensor of the present embodiment has a plurality of read modes. In the first read mode, that is, the all-pixel read mode, the spatial frequency characteristics do not change when a signal is generated. The spatial frequency characteristic (M1) in FIG. 10D is the spatial frequency characteristic in the first read mode. On the other hand, in the second read mode, that is, the thinning read mode, the spatial frequency characteristics change at the time of signal generation. Since the signals are added during the thinning in the X direction to improve the S / N, a low-pass effect due to the addition occurs. A spatial frequency characteristic (M2) in FIG. 10D indicates a spatial frequency characteristic when a signal is generated in the second read mode. Here, the effect of thinning is not taken into account, and a low-pass effect by addition is shown. In FIG. 10D, the spatial frequency characteristics (M1, M2) due to signal generation are drawn as curves, but have discrete values corresponding to the spatial frequencies F1, F2, F3, F4, which are represented by M1. (N) and M2 (n) (1 ≦ n ≦ 4).

  FIG. 10E shows a spatial frequency characteristic (D1) indicating the sensitivity for each spatial frequency when viewing a captured image, and a spatial frequency characteristic (D2) of a digital filter used when processing the AF evaluation signal. Sensitivity for each spatial frequency when viewing a captured image is affected by individual differences among viewers, viewing environment such as image size, viewing distance, brightness, and the like. In this embodiment, the sensitivity for each spatial frequency at the time of viewing is set and stored as a representative value. The viewing distance means the distance from the user to the display on which the recorded image is displayed, and the distance from the user to the paper on which the recorded image is printed. On the other hand, in the second read mode, aliasing noise of the frequency component of the signal occurs due to the effect of thinning. The spatial frequency characteristic (D2) is a spatial frequency characteristic of the digital filter in consideration of the effect. In FIG. 10E, the spatial frequency characteristic (D1) at the time of viewing and the spatial frequency characteristic (D2) of the digital filter are drawn by curves, but discretely correspond to the spatial frequencies F1, F2, F3, and F4. Values, which are denoted as D1 (n) and D2 (n) (1 ≦ n ≦ 4).

  As described above, by storing various information in either the camera or the lens, the evaluation band W1 and the AF evaluation band W2 of the captured image are calculated using the following equations (4) and (5). Is done.

W1 (n) = I (n) × O (n) × L (n) × M1 (n) × D1 (n) (1 ≦ n ≦ 4) (4)
W2 (n) = I (n) × O (n) × L (n) × M2 (n) × D2 (n) (1 ≦ n ≦ 4) (5)
FIG. 10F illustrates an evaluation band W1 (imaging characteristic information) and an AF evaluation band W2 (characteristic information of a focus detection signal) of a captured image. By performing the calculations represented by equations (4) and (5), quantifying the degree of influence of a factor that determines the focus state of a captured image for each spatial frequency. Can be. Similarly, it is possible to quantify how much the error of the focus detection result has for each spatial frequency. In the present embodiment, six types of evaluation bands W1 and AF evaluation bands W2 of a captured image corresponding to the horizontal and vertical contrast directions are calculated for each of the RGB colors.

In FIG. 10, four spatial frequencies (F1 to F4) are used for the sake of simplicity. However, as the number of spatial frequencies having data increases, the spatial frequency characteristics of the captured image and the AF evaluation band increase. Can be accurately reproduced, and a highly accurate correction value can be calculated.
(BP subroutine for calculating BP amount)
The BP amount calculation processing of step S906 in FIG. 7 will be described with reference to FIGS. FIG. 11 is a flowchart showing the BP amount calculation processing. Each step in FIG. 11 is mainly executed by the camera CPU 104.

  In step S9061, the camera CPU 104 acquires the aperture information determined in step S106 in FIG.

  In step S9062, the camera CPU 104 obtains information on the focus detection area set in step S901 in FIG. 7, and calculates the image height on the image sensor.

  In step S9063, the camera CPU 104 acquires photographing lens aberration information. The photographing lens aberration information is information obtained from the lens CPU 507 in response to a request from the camera CPU 104, and is information relating to the imaging position of the photographing optical system for each color, contrast direction, and spatial frequency of the subject. The information on the imaging position for each color of the subject is information mainly related to chromatic aberration. The information on the imaging position of the subject in each contrast direction is mainly information relating to astigmatism. The information on the imaging position of the subject for each spatial frequency is mainly information relating to spherical aberration.

  An example of photographing lens aberration information stored in a memory in a lens (not shown) will be described with reference to FIG. FIG. 12A shows the focus lens position indicating the maximum value of the defocus MTF for each spatial frequency which is a characteristic of the imaging optical system. The horizontal axis represents the spatial frequency, which corresponds to the evaluation band (F1, F2, F3, F4) described in FIG. The vertical axis indicates the focus lens position corresponding to the maximum value of the defocus MTF. FIG. 12A shows information (MTF_GH) regarding a green color and a horizontal contrast direction (MTF_GH), and information about a green color and a vertical contrast direction (MTF_GV). Similarly, it has information corresponding to red and blue, and has a total of six types of defocus MTF peak information. The information MTF_GH has focus lens positions of PGH1, PGH2, PGH3, and PGH4 as information corresponding to the four frequencies. The information MTF_GV has focus lens positions of PGV1, PGV2, PGV3, and PGV4 as information corresponding to the four frequencies. The same applies to red (R) and blue (B). In this embodiment, the imaging lens aberration information in the case of green and horizontal directions is expressed as MTF_GH (n) (1 ≦ n ≦ 4), and the imaging lens aberration information in the case of green and vertical directions is expressed as MTF_GV (n). The photographing lens aberration information in the case of red and horizontal directions is represented by MTF_RH (n), and the photographing lens aberration information in the case of red and vertical directions is represented by MTF_RH (n). The imaging lens aberration information in the case of blue and horizontal directions is expressed as MTF_RV (n), and the imaging lens aberration information in the case of blue and vertical directions is expressed as MTF_BH (n) and MTF_BV (n).

  The imaging lens aberration information stores different values depending on the zoom state, focus state, aperture value, and image height of the focus detection area. In this step, the aperture value information and the image height information obtained in advance are transmitted to the lens, and the lens side selects the photographing lens aberration information along with the current zoom state and focus state, and communicates with the camera. I do. For example, the zoom position, the focus position, and the aperture value of the photographing optical system are divided into eight zones, and each of the divided zones is divided into three with respect to the image height to have photographing lens aberration information. However, the present invention is not limited to these.

  In step S9064, the camera CPU 104 processes photographic lens aberration information using the beam splitter 103 (half mirror) information. The processing of the photographing lens aberration information will be described with reference to FIG. The second image sensor 102 receives a light beam transmitted through the imaging lens 500 and reflected by the half mirror. Therefore, when the half mirror has no distortion or the like and no new aberration occurs due to reflection, the photographing lens aberration information may be used without processing. On the other hand, the light flux transmitted through the photographing lens 500 and the half mirror and received by the first image sensor 101 is converted by a half mirror made of a material having a predetermined refractive index and dispersion according to the angle and wavelength of light incident on the half mirror. Different refractions occur. Since the imaging lens aberration information has focus lens position information corresponding to the maximum value of the defocus MTF, the transmission of the half mirror causes the focus lens position having the maximum value to be offset. This offset amount (first aberration information) can be calculated from the angle of incidence of the light beam on the half mirror determined by the color wavelength, the exit pupil distance, and the image height. FIG. 12B shows the imaging lens aberration information MTF_GH2 offset by transmission of the half mirror with respect to the imaging lens aberration information MTF_GH (wavy line) before the offset. Similarly, the offset amount is calculated for the other photographic lens aberration information, and the offset photographic lens aberration information MTF_GV2, MTF_RH2, MTF_RV2, MTF_BH2, and MTF_BV2 are calculated.

  In step S9065, the camera CPU 104 acquires the AF evaluation band and the imaging evaluation band calculated in step S9054 of FIG. 8 from the memory in the camera CPU 104. As the AF evaluation band, W1_a (n) for the A and B images for AF, W1_b (n) for the C and D images for AF, and W1_c for the E and F images for AF (N) is obtained. As the imaging evaluation band, W2_a (n) corresponding to the image recorded by the first image sensor 101 and W2_b (n) corresponding to the image recorded by the second image sensor 102 are acquired. Each of these evaluation bands has a total of six types of three types of colors (RGB) × two types of contrast directions (horizontal and vertical). For example, the AF evaluation band W1_a (n) includes six types of W1_a_RH (n), W1_a_RV (n), W1_a_GH (n), W1_a_GV (n), W1_a_BH (n), and W1_a_BV (n). These six coefficients each have four frequency components and are composed of a total of 24 coefficients. These 24 types of coefficients show the magnitude relationship of the information amount for each color, spatial frequency, and contrast direction. In the present embodiment, these 24 coefficients are normalized so that the sum of the 24 coefficients becomes 1, and used as weighting of the subject information.

  In step S9066, the camera CPU 104 acquires a spectral distribution based on the spectral reflectance and the spectral transmittance of the half mirror.

  Here, with reference to FIG. 13 and FIG. 14, the angle dependence of the spectral reflectance and the spectral transmittance of the half mirror will be described. FIG. 13 shows the spectral transmittance of the half mirror for each incident angle of the light beam. The horizontal axis shows the wavelength, and the vertical axis shows the transmittance. The spectral reflectance is a value obtained by subtracting the transmittance from 100%. When a half mirror is formed by a dielectric multilayer film, the transmittance changes depending on the incident angle and wavelength of a light beam. FIG. 13 shows the spectral transmittance when the incident angles are 30, 45, and 60 degrees. In the present embodiment, three types of RGB spectral transmittances corresponding to the main wavelengths of the color filters of the image sensor are stored in a memory in the camera CPU 104 for 30 degrees, 45 degrees, and 60 degrees.

  FIG. 14 is an explanatory diagram of a change in the angle of incidence on the half mirror (beam splitter 103) depending on the exit pupil distance and the image height of the imaging optical system. When the taking lens 500 is an interchangeable lens or a zoom lens as in this embodiment, the exit pupil distance changes. FIG. 14 illustrates LPO1 and LPO2 as examples of different exit pupil distances, and illustrates light rays that pass through the center of the exit pupil and reach image heights IH1 and IH2 on the image sensor. Although two rays reaching the image height IH1 are drawn from the positions of the exit pupil distances LPO1 and LPO2, the angles of incidence on the half mirror differ from each other. In addition, light rays that are emitted from the position of the exit pupil distance LPO1 and reach the image heights IH1 and IH2 are depicted, but the angles of incidence on the half mirror are different from each other.

  In step S9066, the camera CPU 104 calculates the angle of incidence on the half mirror based on the exit pupil distance of the photographing optical system and the image height for which the BP amount is calculated. Then, the reflectance / transmittance of each wavelength is calculated from the obtained incident angle by interpolation or the like. As spectral distribution information, red, green, and blue reflectances are obtained as Rr, Gr, and Br, and red, green, and blue transmittances are obtained as Rt, Gt, and Bt, respectively. Note that one of them may be calculated from the relationship between the reflectance and the transmittance described above.

  In step S9067, the camera CPU 104 calculates a focus position assuming an AF signal using the weighting using the spectral transmittance distribution and the spectral reflectance distribution of the half mirror, the AF evaluation band, and the imaging lens aberration information. Since the correlation calculation (first correlation calculation) of the AF A-image and the B-image is performed by the first image sensor 101, it is performed by the light beam transmitted through the half mirror, and the signal to be detected is only in the horizontal direction. The focus position P_AF1 assumed by the focus detection signal in the first correlation calculation is calculated by the following equation (6).

  By calculating the equation (6), the aberration information of the photographing lens 500 is weighted and added in consideration of the spectral distribution of the subject and the spectral transmittance of the half mirror, thereby assuming a focus detection signal in the first correlation operation. The calculated focal position P_AF1 can be calculated. The focus position P_AF1 corresponds to a defocus MTF peak position estimated from a color distribution and a spatial frequency distribution evaluated by the focus detection signal in the aberration information of the imaging lens 500.

  Since the correlation calculation (second correlation calculation) of the C image and the D image for AF is performed by the second image sensor 102, it is performed by the light beam reflected by the half mirror, and the signal to be detected is only in the horizontal direction. The focus position P_AF2 assumed by the focus detection signal in the second correlation calculation is calculated by the following equation (7).

  Since the correlation calculation (third correlation calculation) of the AF E image and the F image is performed by the second image sensor 102, the correlation calculation is performed by the light beam reflected by the half mirror, and the signal to be detected is only in the vertical direction. The focus position P_AF3 assumed by the focus detection signal in the third correlation calculation is calculated by the following equation (8).

  In step S9068, the camera CPU 104 calculates a focal position assuming an imaging signal using the weighting using the spectral transmittance distribution and the spectral reflectance distribution of the half mirror, the imaging evaluation band, and the imaging lens aberration information.

  A still image (first recorded image) obtained from the first image sensor 101 is formed by a light beam transmitted through a half mirror. The evaluation of the focus adjustment state also evaluates the contrast in the horizontal direction and the vertical direction. Therefore, the focus position P_IMG1 assumed in the evaluation of the focus adjustment state in the first recorded image is calculated by the following equation (9).

  The focal position P_IMG1 corresponds to the defocus MTF peak position estimated from the color distribution, spatial frequency distribution, and distribution in the contrast direction evaluated in the first recorded image in the aberration information of the imaging lens 500.

  A moving image (second recorded image) obtained from the second image sensor 102 is formed by a light beam reflected by the half mirror. The evaluation of the focus adjustment state also evaluates the contrast in the horizontal direction and the vertical direction. Therefore, the focus position P_IMG2 assumed in the evaluation of the focus adjustment state in the second recorded image is calculated by the following equation (10).

  As described above, in steps S9067 and S9068, the assumed defocus MTF peak position is determined using the imaging lens aberration information from the focus detection signal and the characteristics of the imaging signal (spectral, spatial frequency distribution, and distribution in the contrast direction). Is calculated.

  In step S9069, the camera CPU 104 calculates the BP amount. The BP amount differs depending on which focus detection result is used for focus adjustment of which recorded image. The BP amount BP1 when the first focus detection result is used for the first recorded image and the BP amount BP2 when the first focus detection result is used for the second recorded image are calculated by the following equations (11) and (12), respectively. You.

BP1 = P_IMG1-P_AF1 (11)
BP2 = P_IMG2-P_AF1 (12)
The BP amount BP3 when the second focus detection result is used for the first recording image and the BP amount BP4 when the second focus detection result is used for the second recording image are calculated by the following equations (13) and (14). .

BP3 = P_IMG1-P_AF2 (13)
BP4 = P_IMG2-P_AF2 (14)
The BP amount BP5 when the third focus detection result is used for the first recorded image and the BP amount BP6 when the third focus detection result is used for the second recorded image are calculated by the following equations (15) and (16). .

BP5 = P_IMG1-P_AF3 (15)
BP6 = P_IMG2-P_AF3 (16)
The calculated BP amount is switched and used according to the shooting state. For example, when the contrast component in the vertical direction is large and the reliability of the focus detection result obtained from the third correlation operation is high, the BP amount (first focus adjustment correction amount) for the first image sensor 101 is set as the BP amount. The focus detection result is corrected using BP5, and the focus lens driving amount is set. On the other hand, the focus detection result is corrected using the BP amount BP6 as the BP amount (second focus adjustment correction amount) for the second image sensor 102, and the drive amount of the second image sensor 102 is set.

As described above, the focus detection position (P_AF1, P_AF2, P_AF3) of the assumed focus detection signal is calculated from the signal acquired for each imaging element in view of the characteristics of the focus detection signal.
On the other hand, in consideration of the characteristics of the imaging signal, the assumed focus detection positions (P_IMG1, P_IMG2) of the imaging signal are calculated. By calculating the BP amount using these, it is possible to adjust the focus in accordance with the recorded image while employing a highly reliable focus detection result.

  In the present embodiment, the position of the second image sensor 102 may be adjusted along the optical axis in preparation for calculating different values for the BP amount for recording a still image and the BP correction amount for recording a moving image. it can. That is, it is possible to perform focus adjustment by driving a focus lens as a focus adjustment mechanism and driving an image sensor as an optical path length changing unit. However, the second image sensor 102 may not be driven. In such a case, for example, the average value (third focus adjustment correction amount) of the two calculated BP amounts (first focus adjustment correction amount, second focus adjustment correction amount) is used to calculate the focus by driving the focus lens. Adjustments may be made. Thereby, the cost can be reduced although the focus adjustment accuracy is inferior.

  If the second image sensor 102 cannot be driven, the BP amount may be calculated according to the priority of the recorded images obtained from the first and second image sensors 101 and 102. For example, in a case where priority is given to a still image (recorded image of the first image sensor 101), a correction amount obtained by weighting the BP amount for the first image sensor 101 and the BP amount for the second image sensor 102 by 4: 1 is set as a third amount. The focus adjustment may be performed by calculating the focus adjustment correction amount. Further, the magnitude of the BP amount is determined based on the aberration information of the photographing lens 500. Therefore, the magnitude of the aberration amount is determined from the photographing lens aberration information, and if the aberration amount is small, the calculation of the BP amount may be omitted.

  Next, the timing of subject information extraction performed in steps S9051 and S9052 in FIG. 8 by the first and imaging elements 101 and 102 will be described using FIG. FIG. 15 shows an example in which the vertical synchronization signals (V1, V2, V3, V4) of the first and second imaging elements 101, 102 are synchronized. The first and second imaging elements 101 and 102 repeat accumulation and readout at the vertical synchronization timing V1. The storage time of the second image sensor 102 is set to be longer than the storage time of the first image sensor 101 for capturing a moving image. However, since the number of recording pixels is small, the reading time is short. Since the first imaging element 101 functions as a subject information extracting unit, it is driven with a short accumulation time in order to reduce the influence of subject shake and camera shake. When the reading of the signals of the two image sensors is completed, subject information is detected using the acquired signals of the image sensors. When the subject information detection is completed, the BP amount is calculated using the obtained subject information. With this configuration, the subject information at approximately the same timing can be obtained by the two image sensors, so that the information is less affected by the change in the subject information due to the time lag, and highly reliable information can be obtained. In driving the first image sensor 101, the frame rate may be doubled, for example, by narrowing the extraction area of the subject information to obtain more information. At this time, half of the data is not synchronized with the information obtained from the second image sensor 102, but more reliable object information is obtained by considering continuity from the information obtained from the first image sensor 101. Can be extracted. Further, when the frame rate of the moving image recording is high, the driving of the first imaging element 101 may be thinned out to make the extraction interval of the subject information longer. Although less subject information is obtained, power consumption can be reduced.

  The configuration of the present embodiment will be described with reference to FIGS. The main difference from the first embodiment is the BP amount processing of step S906 in FIG. 7 by the camera CPU 104 as the correction amount calculation means. In the first embodiment, photographing lens aberration information is acquired as focus lens position information indicating the maximum value of the defocus MTF in step S9063, and photographing lens aberration information is processed from the half mirror aberration information in step S9064. In the present embodiment, by acquiring the photographing lens aberration information as the wavefront aberration information, it is possible to cope with a case where distortion occurs in the half mirror and the aberration state changes complicatedly.

  FIG. 16 is a flowchart illustrating the BP amount calculation processing according to the present embodiment. Each step in FIG. 16 is mainly executed by the camera CPU 104.

  In steps S9061 and S9062, the aperture information and the image height on the image sensor are calculated as in the first embodiment.

  In step S9101, the camera CPU 104 acquires photographic lens aberration information. With reference to FIG. 17, an example of photographing lens aberration information stored in a memory in a lens (not shown) will be described. Here, the imaging lens aberration information is information on the wavefront aberration of the imaging optical system. FIG. 17A is a diagram illustrating an imaging state of a light source in an ideal photographing lens id. The solid line represents the propagation of light when the light is captured as a wave. In the ideal lens state, light propagates and forms an image at one point as shown in FIG. However, in the photographing lens re as shown in FIG. 17B in which aberration actually occurs, the light passing through the lens is not symmetric with the incident wave. In FIG. 17C, the exit pupil position of the lens shown in FIGS. 17A and 17B is set to En_plane, the wavefront at the exit pupil position En_plane of the photographing lens id is indicated by a dotted line, and the exit pupil position En_plane of the photographing lens re is shown. FIG. 5 is a diagram in which the wavefronts at are overlapped as solid lines. The wavefront aberration information WA (x, y) acquired in this step is optical path difference information with respect to an ideal wavefront on the XY plane as shown in FIG. In FIG. 17D, the wavefront aberration information WA (x, y) is stored as 6 × 6 discrete data, but the number of divisions is not limited to this. In general, the greater the number of divisions, the better the accuracy. Alternatively, wavefront aberration information WA (x, y) may be represented as a function of pupil position coordinates (x, y), such as by approximation using a Zernike polynomial, and coefficients of the function may be stored. The wavefront aberration information WA (x, y) is stored as a data sequence for each color wavelength and exit pupil distance PL.

  In step S9102, the camera CPU 104 acquires the wavefront aberration information of the half mirror (beam splitter 103) as the imaging lens aberration information. In the present embodiment, as the wavefront aberration information unique to the half mirror, the wavefront shown in FIG. 17A in the same manner as the imaging lens wavefront aberration information WA (X, Y) in step S9101 for each of the transmission side and the reflection side. Aberration information is obtained. The wavefront aberration information (first aberration information) on the transmission side of the half mirror is WA_HT (X, Y), and the wavefront aberration information (second aberration information) on the reflection side is WA_HR (X, Y). Since the wavefront aberration information WA_HT (X, Y) and WA_HR (X, Y) are values that change depending on the exit pupil distance PL, they are stored according to the exit pupil distance PL acquired in step S101 in FIG. It is obtained by selecting from wavefront aberration information. The independent wavefront aberration information WA_HT (X, Y) and WA_HR (X, Y) of the half mirror can be obtained by a general measuring instrument. Therefore, in the manufacturing process, when the wavefront aberration information WA_HT (X, Y) and WA_HR (X, Y) are obtained and written into the camera memory, the aberration information in consideration of the bending and distortion of the half mirror in the manufacturing process is obtained. Can be obtained. At this time, the half mirror may also have frame vignetting due to the light shielding member. In the half mirror, a frame may be provided for reasons such as prevention of reflection on a wall surface. In the XY plane, it is assumed that a frame as shown in FIG. 18B (white is a transmissive part, black is a light-shielding part) is installed on the half mirror. In this case, as shown in FIG. 18C, the transmitted light amount may be converted into a weight corresponding to the number of divisions, and each of the wavefront aberration information may be weighted.

  In step S9103, the camera CPU 104 adds the half mirror aberration information. The transmitted wavefront aberration information WA_T (X, Y) transmitted through the half mirror is WA (X, Y) + WA_HT (X, Y), and the reflected wavefront aberration information WA_R (X, Y) reflected on the half mirror is WA (X, Y). , Y) + WA_HR (X, Y). As described above, by adding the transmission-side or reflection-side wavefront aberration information to the wavefront aberration information WA (X, Y) of the imaging optical system, the transmission wavefront aberration information WA_T (X, Y) and the reflection wavefront aberration information WA_R are obtained. (X, Y) can be calculated.

  In step S9104, the camera CPU 104 processes aberration information. In the present embodiment, from the transmitted wavefront aberration information WA_T (X, Y) and the reflected wavefront aberration information WA_R (X, Y), focus lens position information corresponding to the maximum value of the defocus MTF similar to the first embodiment is processed. The method will be described. Since the processing method is the same for any wavefront aberration information, this embodiment describes a case where the transmitted wavefront aberration information WA_T (X, Y) is used. When the focal length of the imaging optical system is sufficiently long with respect to the resolution in XY of the transmitted wavefront aberration information WA_T (X, Y), the transmitted wavefront aberration information WA_T (X, Y) at the entrance pupil position En_Plane is obtained on the imaging surface by Fourier transform. Is the amplitude of the point spread function (hereinafter, PSF). The point spread function is a function representing the response of an optical system to a point light source, and the PSF of an ideal lens is an impulse response. The transmitted wavefront aberration information WA_T (x, y) is converted into PSF (x, y) by the following equation (7).

  FIG. 17E shows the PSF as two-dimensional information, with the vertical axis representing the x coordinate and the horizontal axis representing the intensity of the PSF. PSF1 and PSF2 indicate the difference between the defocused states. The closer the PSF is to the focused state, the closer the PSF is to the impulse response. In FIG. 17E, the state of the PSF1 in the defocus state 1 is closer to the in-focus state than the state of the PSF2 in the defocus state 2. When this PSF is further subjected to Fourier transform using the following equation (18), an MTF characteristic having a spatial frequency on the horizontal axis and an intensity on the vertical axis shown in FIG.

  Generally, the intensity of the vertical axis (response of MTF) increases as the focus state approaches. Therefore, as shown in FIG. 17F, at the spatial frequencies F1 to Nq, the MTF1 corresponding to the PSF1 in the defocus state 1 and the MTF2 corresponding to the PSF2 in the defocus state 2 show changes as illustrated. FIG. 17G illustrates the defocus state on the horizontal axis and the MTF intensity on the vertical axis at the spatial frequencies F1 to F4. This corresponds to the defocus MTF, and the aberration information of the first embodiment indicates the focus lens position at which the defocus MTF takes a maximum value. The horizontal axis in FIG. 17G indicates defocus, which indicates a one-to-one relationship with the focus lens position. Assuming that the color of the defocus MTF in FIG. 17G is G (Green) and the direction is H (horizontal), PGH1, PGH2, PGH3, and PGH4 that are focus lens position information that takes the maximum value of the defocus MTF are shown in FIG. The focus lens position information is 17 (g). This is the same information as the focus lens position where the maximum value of the defocus MTF, which is the aberration information described above in the first embodiment, takes a maximum value. By obtaining the aberration information in the same way for the reflection aberration, other colors, and directions, the processing after step S9065 is the same as that of the first embodiment.

  As described above, by acquiring the imaging lens aberration information as the wavefront aberration information, it is possible to cope with a case where distortion or the like occurs in the half mirror and the aberration state changes complicatedly. Further, by storing the wavefront aberration information of the half mirror independently and adding it to the wavefront aberration information of the photographing optical system, it becomes possible to calculate the aberration state of each of the transmission side and the reflection side. Therefore, even when only the wavefront aberration information of the photographing optical system is stored in the lens or the memory of the camera, highly accurate correction can be performed. In the interchangeable lens system, in a plurality of types of lenses, it is not necessary to store aberration information of the photographing lens in the transmission light path and aberration information of the photographing lens in the reflection light path, so that the memory capacity can be significantly reduced.

Hereinafter, the configuration of the present embodiment will be described with reference to FIGS. In the present embodiment, a description will be given of a configuration of an imaging apparatus in which only a first imaging element 101 that receives a transmitted light beam is present and a half mirror can be retracted during exposure by the first imaging element 101.
(Configuration of imaging device)
Regarding the configuration of the imaging apparatus according to the present embodiment, only the parts that are different from the first embodiment will be described with reference to FIG. The beam splitter 103 is movable between a first position located in the optical path of the taking lens 500 and a second position retracted from the optical path. When the imaging device is in the state shown in FIG. 19A, the actuator 150 retracts the beam splitter 103 from the optical path of the imaging optical system when receiving an instruction from the camera CPU 104. At this time, the light blocking member 151 operates following the beam splitter 103 to prevent the reflected light in the optical path when the beam splitter 103 is retracted, so that the light blocking member 151 covers the beam splitter 103. FIG. 19B shows the configuration of the imaging device after the beam splitter 103 has retreated from the optical path. In FIG. 19, the retracting direction of the beam splitter 103 is illustrated on the upper surface of the camera, but the retracting method and direction are not limited thereto.

In this embodiment, an optical finder is assumed instead of the electronic finder as in the first embodiment. The prism 152 guides the light beam to the finder 153. With this configuration, the user can detect a subject with the optical viewfinder in real time, and at the same time, can perform focus detection based on the speed of light received by the first image sensor 101.
(Shooting flow)
FIG. 20 is a flowchart illustrating the photographing processing according to the present embodiment. In this embodiment, since there is only one sensor and simultaneous recording of a still image and a moving image cannot be performed, the photographing flow for recording a still image is used. Each step in FIG. 20 is mainly executed by the camera CPU 104.

  The processing in step S301 is similar to the processing in step S101 in FIG.

  In step S302, the camera CPU 104 drives the first image sensor 101.

  In step S303, the camera CPU 104 converts the signal acquired by the first image sensor 101 into a signal for display, transmits the signal to the in-finder display 107 or the external display 110, and starts live view display.

  In step S304, the camera CPU 104 determines the brightness of the image signal obtained by driving the first image sensor 101, and performs aperture control during live view.

  In step S307, the camera CPU 104 executes a focus detection subroutine. In this embodiment, focus detection in the first image sensor 101 and correction of the focus detection result are performed.

  In step S308, the camera CPU 104 performs mirror up. In this embodiment, the beam splitter 103 is retracted from the optical path of the imaging optical system by using the actuator 150. By retracting the beam splitter 103 from the optical path, it is possible to suppress the occurrence of aberration due to transmission, and to record a higher definition captured image.

In step S307, the camera CPU 104 executes still image shooting.
(Focus detection processing)
The focus detection processing (AF processing) of the present embodiment will be described with reference to FIG. FIG. 21 is a flowchart illustrating the focus detection processing according to the present embodiment. Each step in FIG. 21 is mainly executed by the camera CPU 104.

  The processes in steps S931-S934 and S936-S939 are the same as the processes in steps S901-S904 and S907-S910 in FIG.

The details of the subroutine for calculating the BP amount in step S935 will be described later.
(BP subroutine for calculating BP amount)
With reference to FIG. 22, the BP amount calculation processing of the present embodiment will be described. FIG. 22 is a flowchart illustrating the BP amount calculation processing in the present embodiment. Each step in FIG. 22 is mainly executed by the camera CPU 104.

  The processing in steps S9361 and S9362 is the same as the processing in steps S9061 and S9062 in FIG. 11, respectively, and a description thereof will be omitted.

  In step S9363, the camera CPU 104 acquires photographing lens aberration information. The photographing lens aberration information may be focus lens position information corresponding to the maximum value of the defocus MTF obtained in step S9063 in FIG. 11 or wavefront aberration information obtained in step S9101 in FIG.

  In step S9364, the camera CPU 104 creates photographing lens aberration information based on the half mirror information. The processing in this step may be processing of photographing lens aberration information using half mirror information performed in step S9064 of FIG. 11 or processing of photographing lens aberration information performed in steps S9102 to S9104 of FIG. Good. However, in this embodiment, since processing on the reflection side is not necessary, only processing on aberration information on the transmission side may be performed.

  The process in step S9365 is the same as the process in step S9054 in FIG.

  In step S9366, the camera CPU 104 acquires a spectral distribution based on the spectral transmittance of the half mirror. In addition, red, green, and blue transmittances stored in advance as normal spectral distribution information in a state where the half mirror is not arranged are acquired as R, G, and B, respectively.

  In step S9367, the camera CPU 104 assumes an AF signal using the weighting using the spectral transmittance distribution of the half mirror, the AF evaluation band, and the imaging lens aberration information transmitted through the half mirror calculated in step S9364. Is calculated. Since the correlation calculation (first correlation calculation) of the AF A-image and the B-image is performed by the first image sensor 101, it is performed by the light beam transmitted through the half mirror, and the signal to be detected is only in the horizontal direction. Here, the focus position P_AF1 assumed by the focus detection signal in the first correlation calculation is calculated by Expression (6). In this embodiment, since processing on the reflection side is not necessary, only the focus position assuming the AF signal on the transmission side may be calculated.

  In step S9368, the camera CPU 104 calculates a focal position assuming an imaging signal using the spectral distribution information, the imaging evaluation band, and the imaging lens aberration information acquired in step S9366. The focus position P_IMG assumed in the evaluation of the focus adjustment state in the recorded image in which the half mirror is not arranged is calculated by the following equation (19).

  In step S3509, the camera CPU 104 calculates the BP amount. The BP amount is calculated in accordance with the present embodiment in which the focus is detected in a state of being transmitted through the half mirror and a recorded image is captured without the half mirror. Assuming that the calculated BP amount is BP4, the BP amount can be obtained by the following equation (20).

BP4 = P_IMG-P_AF3 (20)
As described above, appropriate focus detection and BP correction can be performed even in a configuration of an imaging device capable of detecting a subject in real time with the optical viewfinder and at the same time detecting a focus by a light beam received by the first imaging element 101.

The configuration of the present embodiment will be described with reference to FIGS. The main difference from the first embodiment is the method of calculating the BP amount in step S907 in FIG. 7 by the camera CPU 104 as the correction amount calculating means. In the first embodiment, photographing lens aberration information is acquired as focus lens position information corresponding to the maximum value of the defocus MTF in step S9063 of FIG. 11, and photographing lens aberration information is processed from aberration information of the half mirror in step S9064. Was. In the present embodiment, as half mirror information, correction information on the shape parameters of the imaging lens 500 and the half mirror is acquired. By acquiring the shape parameters of the half mirror, it is possible to cope with a case where distortion or the like occurs in the half mirror and the aberration state changes complicatedly.
(BP subroutine for calculating BP amount)
FIG. 23 is a flowchart illustrating the BP amount calculation processing of the present embodiment. Each step in FIG. 23 is mainly executed by the camera CPU 104.

  In step S9461, the camera CPU 104 acquires zoom information of the photographing lens determined as the photographing information, information of the third lens group, and aperture information.

  In step S9462, the camera CPU 104 obtains information on the focus detection area set in step S901 in FIG. 7, and calculates the image height on the image sensor.

  In step S9463, the camera CPU 104 acquires half mirror information. The half mirror information here is, for example, information on the thickness and bending of the half mirror, and in the present embodiment, is correction information on the shape parameters of the photographing lens 500 and the half mirror.

  FIG. 24 shows the optical path of the photographing optical system and half mirror information (thickness D). When the beam splitter 103 does not exist, the light beam from the photographing lens 500 takes an optical path as indicated by a dotted line and forms an image on the first image sensor 101. However, when the beam splitter 103 has a thickness, since the refractive index of air and the refractive index of the beam splitter 103 are different, the light beam takes an optical path as shown by a solid line, and as a result, aberration occurs. The BP amount caused by the aberration generated at this time is correlated with the thickness D, and shows a relationship as shown in FIG. Generally, as the thickness D of the half mirror is larger, the BP amount BP_T generated on the first image sensor 101 is increased, and has a correlation represented by Expression (21).

BP_T = a_T × D + BP_T (0) (21)
BP_T (0) represents the y-intercept in FIG. 25, and is the BP amount generated when the thickness D is 0. Even when the thickness D of the half mirror is 0, the photographing lens 500 generally does not have no aberration. Therefore, the imaging lens 500 has the amount of aberration of the intercept BP_T (0) even when the thickness D is 0. a_T is half mirror information indicating a unit increase of the BP amount BP_T with respect to the thickness D.

  In this step, the camera CPU 104 has, for example, half-mirror information for the thickness D in FIG. 26 and a half-mirror corresponding to the shooting information (zoom position, focus position, F value, image height) acquired in steps S9461 and S9462. Obtain information a_T. Even in the state of the photographing optical system where the half mirror does not exist, the aberration state changes depending on the photographing information and the manner of changing the aberration state by the half mirror also differs. Therefore, it is desirable to change the half mirror information a_T according to the photographing information. For example, when the zoom is Zoom3, the focus is Focus2, the F value is F1, and the image height is h1, half mirror information a_T6 is acquired as correction information on the thickness of the half mirror. Here, the change in the BP amount BP_T with respect to the thickness D is a linear function, but the function shape is not limited to this.

  Next, half mirror information relating to the deflection C will be described. FIG. 27 shows half mirrors having different bending states. FIG. 27A shows a state where there is no bending when three axes (mx, mr, mz) are set on the beam splitter 103. The mx and mz cross sections and the my and mz cross sections are as shown in the right diagram of FIG. 27A, and have a planar shape having a constant value. FIG. 27B to FIG. 27E show a bent shape. FIGS. 27 (b) and 27 (c) show a case where bending occurs in a biconvex or biconcave bowl shape in which bending occurs in the same direction at mx and my. 27 (e) shows a saddle point shape in which bending occurs in different directions at mx and my.

  FIG. 28 shows the optical path of the photographing optical system to the second image sensor 102 when the beam splitter 103 is bent as shown in FIG. When there is no deflection in the beam splitter 103 as shown in FIG. 27A, the light flux from the taking lens 500 takes an optical path as indicated by a dotted line and forms an image on the second image sensor 102. However, when the bending as shown in FIG. 27E occurs in the beam splitter 103, the optical path changes depending on whether the bending shape of the half mirror is positive or negative (the right diagram in FIG. 27E). Therefore, as shown in the straight optical path in FIG. 28, the image forming position of the light beam changes depending on which coordinate point of the half mirror passes through the half mirror, and aberration occurs. The BP amount caused by the aberration generated at this time is correlated with the deflection amount C. The amount of deflection can be quantified, for example, as the number of interference fringes (also called the number of Newton rings).

  FIG. 29 shows an example of the measurement result of the number of interference fringes when the half mirror has a bent shape as shown in FIG. The method of measuring interference fringes is one of the methods of measuring the flatness of a half mirror, and the more fringes caused by light wave interference, the lower the flatness. Therefore, if the beam splitter 103 is completely flat as shown in FIG. 27A, no interference fringes are generated on the beam splitter 103 as shown in FIG. 29A, and the number of interference fringes is zero. . The direction in which the interference fringes are generated differs depending on the bending direction of the half mirror, and in the case of a bowl shape in which the bending directions are the same as in FIGS. 27B and 27C, as shown in FIG. 29B. It becomes a concentric interference fringe. In the case of a saddle point shape in which the bending directions are opposite to each other as shown in FIGS. 27D and 27E, radial interference fringes as shown in FIG. 29C are generated. The center of the bending is called a saddle point, and the greater the number of stripes from the saddle point, the stronger the bending. For example, in FIGS. 29B and 29C, the saddle point is at the center of the half mirror. In FIG. 29B, when the black line is counted from the saddle point in a direction perpendicular to the interference fringes, seven interference fringes are generated. In FIG. 29C, when the black lines are counted from the saddle point in a direction perpendicular to the interference fringes, three interference fringes are generated. Here, the bending direction and the number of interference fringes are used as an example of the parameter of the bending amount C. The bending direction indicates the direction of bending with respect to the optical axis direction. For example, the state in FIG. 30A is positive, and the state in FIG. 30B is negative. That is, in FIGS. 27 (b) and 27 (c) and FIGS. 27 (d) and 27 (e), the number of generated interference fringes is the same, but the sign of the parameter of the amount of deflection C is reversed. Become. Since the state of occurrence of the aberration changes depending on the direction in which the bending occurs, it is desirable that the amount of bending C be defined with different signs. FIG. 31 is a correlation diagram between the amount of deflection C and the BP amount BP_H generated by the amount of deflection C. Generally, as the deflection amount C of the half mirror increases, the BP amount BP_H generated on the second image sensor 102 increases, and has a correlation represented by the following equation (22).

BP_H = a_H × C + BP_H (0) (22)
BP_H (0) represents the y-intercept in FIG. 31 and is the BP amount generated when the amount of deflection C is 0. Even when the deflection amount of the half mirror is zero, the photographing lens 500 generally has no aberration. Therefore, even if the deflection amount C is zero, it has a BP amount due to the aberration of the intercept BP_H (0). Further, when the sign of the intercept BP_H (0) and the BP amount due to the aberration generated by the deflection amount C of the half mirror cancel each other out, the total BP amount BP_H may decrease. The half mirror information a_H in FIG. 31 is a unit increase amount of the BP amount BP_H with respect to the bending amount C. In this step, the camera CPU 104 has, for example, half-mirror information a_H corresponding to the deflection amount C in FIG. 32 and responds to the shooting information (zoom position, focus position, F value, image height) acquired in steps S9061 and S9062. Obtain half mirror information a_H. Even in the state of the photographing optical system where the half mirror does not exist, the aberration state changes depending on the photographing information and the manner of changing the aberration state by the half mirror also differs. Therefore, it is desirable to change the half mirror information a_H according to the photographing information. For example, when the zoom is Zoom3, the focus is Focus2, the F value is F1, and the image height is h1, the half mirror information a_H6 is acquired as correction information on the amount of deflection of the half mirror. Here, the change of the BP amount BP_H with respect to the deflection amount C is shown as a linear function, but the function shape is not limited to this. Since the BP amount BP_H may change depending on the saddle point, the function in FIG. 31 and the correction information in FIG. 32 may be changed for each half mirror position coordinate of the saddle point.

  In step S9464, the camera CPU 104 acquires the shape parameters of the half mirror. As described above, the thickness and the amount of bending of the half mirror change from the design state due to a manufacturing error or the like. Here, for example, the characteristics of the half mirror are measured, and the shape parameters such as the thickness D1 and the bending amount C1 (the bending direction and the number of interference fringes) stored in the memory in the camera are acquired.

  In step S9465, the camera CPU 104 calculates the BP amount. The BP amounts BP_T and BP_H are calculated using the correction information a_T6 and a_H6 for the half mirror acquired in step S9463 and the half mirror shape parameters D1 and C1 acquired in step S9464, respectively, using the following equations (23) and (23). 24). In the present embodiment, the BP amount BP_T is a first focus adjustment correction amount that is a BP amount for the first image sensor 101, and the BP amount BP_H is a second BP amount that is a BP amount for the second image sensor 102. This is a focus adjustment correction amount.

BP_T = a_T6 × D1 + BP_T (0) (23)
BP_H = a_H6 × C1 + BP_H (0) (24)
The BP amount calculated in this step is switched and used according to the shooting state. For example, when the reliability of the first focus detection result is high, the focus detection result is corrected using the BP amount BP_T as the BP amount for the first image sensor 101, and the focus lens driving amount is set. When the reliability of the second focus detection result is high, the focus detection result is corrected using the BP amount BP_H as the BP amount for the second image sensor 102, and the focus lens drive amount or the drive amount of the image sensor drive unit is corrected. Set.

  As described above, in this embodiment, the shape parameter of the half mirror is acquired, and the BP amount is calculated using the correlation between the shape parameter and the BP amount. By measuring and storing only the shape parameter of a typical half mirror and using it for calculating the BP amount, it is possible to reduce the storage capacity of the memory and the calculation load, and to perform highly accurate focus detection. In addition, BP correction according to the shape of the half mirror when a manufacturing error occurs can be performed.

As described above, the preferred embodiments of the present invention have been described, but the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist.
[Other Examples]
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

Reference Signs List 100 imaging device 101 first imaging device 102 second imaging device 103 beam splitter 104 camera CPU
500 shooting lens

Claims (14)

Light beam splitting means for splitting a light beam passing through an exit pupil of a photographing lens having a focus adjustment mechanism into a first light beam and a second light beam;
A plurality of first pixels each including: a plurality of first photoelectric conversion units; and one microlens that guides a light beam of the first light beam that has passed through a different region of the exit pupil to the plurality of first photoelectric conversion units. A first imaging element provided;
A plurality of second pixels each including: a plurality of second photoelectric conversion units; and one microlens that guides a light flux of the second light flux that has passed through a different region of the exit pupil to the plurality of second photoelectric conversion units. A second imaging element provided;
Aberration information of the imaging lens, first aberration information that is aberration information of the first light flux, a weighting value for a spatial frequency characteristic of a first imaging signal acquired from the first imaging device, A first focus obtained using the pair of focus detection signals acquired from the first image sensor, based on a weight value for a spatial frequency characteristic of the pair of focus detection signals acquired from the image sensor. A first focus adjustment correction amount for correcting the position is calculated, and the aberration information of the photographing lens, the second aberration information that is the aberration information of the second light beam, and the second focus information are acquired from the second image sensor. The weighting value for the spatial frequency characteristic of the second imaging signal to be obtained and the weighting value for the spatial frequency characteristic of the pair of focus detection signals acquired from the second imaging element, A correction amount calculating means for calculating a second focusing correction amount for correcting the second focus position obtained by using the focus detection signal of the pair to be acquired,
An imaging device comprising: a control unit that controls the focus adjustment mechanism based on the first and second focus adjustment correction amounts. Weighting value with respect to the spatial frequency characteristic of the first image pickup signal is calculated based on the read mode before Symbol first pixel,
The weighting value for the spatial frequency characteristics of the second image pickup signal, before SL imaging apparatus according to claim 1, characterized in that calculated on the basis of the reading mode of the second pixel. Light beam splitting means movable between a first position located in an optical path of a taking lens having a focus adjustment mechanism and a second position retracted from the optical path;
A plurality of photoelectric conversion units, and when the light beam splitting unit is located at the first position, among the light beams transmitted through the light beam splitting unit, light beams that have passed through different areas of the exit pupil of the photographing lens, An imaging element including a plurality of pixels including one microlens leading to the conversion unit;
When the light beam splitting means is located at the first position, aberration information of the photographing lens, first aberration information which is aberration information of the light beam splitting means, and a space of an imaging signal acquired from the image sensor. Based on a weighting value for the frequency characteristic and a weighting value for the spatial frequency characteristic of the pair of focus detection signals acquired from the image sensor, a value is obtained using a pair of focus detection signals acquired from the image sensor. Correction amount calculating means for calculating a focus adjustment correction amount for correcting the focus position to be corrected,
An imaging device comprising: a control unit configured to control the focus adjustment mechanism based on the focus adjustment correction amount. Weighting value with respect to the spatial frequency characteristic of the imaging signal read mode before Symbol pixel imaging apparatus according to claim 3, characterized in that calculated on the basis of.   When the light beam splitting unit is located at the second position, the correction amount calculating unit includes: an aberration information of the photographing lens; a weight value for a spatial frequency characteristic of the imaging signal; The imaging apparatus according to claim 3, wherein the focus adjustment correction amount is calculated based on a weight value for a spatial frequency characteristic. Wherein the first and second aberration information imaging apparatus according to claim 1 or 2, characterized in that a value based on the amount of transmitted light in the light-shielding member of the beam splitting means.   The imaging apparatus according to claim 1, wherein the aberration information of the imaging lens is wavefront aberration information. Light beam splitting means for splitting a light beam passing through an exit pupil of a photographing lens having a focus adjustment mechanism into a first light beam and a second light beam;
A plurality of first pixels each including: a plurality of first photoelectric conversion units; and one microlens that guides a light beam of the first light beam that has passed through a different region of the exit pupil to the plurality of first photoelectric conversion units. A first imaging element provided;
A plurality of second pixels each including: a plurality of second photoelectric conversion units; and one microlens that guides a light flux of the second light flux that has passed through a different region of the exit pupil to the plurality of second photoelectric conversion units. A second imaging element provided;
A first focus position obtained by using a pair of focus detection signals acquired from the first imaging device based on the shape parameter of the light beam splitting unit and correction information on the imaging lens and the shape parameter; Focus adjustment correction for correcting a second focus position obtained by using a first focus adjustment correction amount for correcting the first focus detection signal and a pair of focus detection signals acquired from the second image sensor. Correction amount calculating means for calculating the amount,
An imaging device comprising: a control unit that controls the focus adjustment mechanism based on the first and second focus adjustment correction amounts.   The imaging apparatus according to claim 8, wherein the shape parameter is information on a thickness of the light beam splitting unit.   9. The imaging apparatus according to claim 8, wherein the shape parameter is information on deflection of the light beam splitting unit.   The imaging apparatus according to claim 10, wherein the information on the bending is the number of interference fringes of the light beam splitting unit. A light beam splitting unit that splits a light beam that has passed through an exit pupil of a photographing lens having a focus adjusting mechanism into a first light beam and a second light beam, a plurality of first photoelectric conversion units, and the emission light beam of the first light beam A first imaging device including a plurality of first pixels including a single microlens that guides a light beam that has passed through regions having different pupils to the first photoelectric conversion unit; a plurality of second photoelectric conversion units; A second imaging element including a plurality of second pixels including a single microlens that guides a light beam of the light beam that has passed through a different area of the exit pupil to the second photoelectric conversion unit. So,
Aberration information of the imaging lens, first aberration information that is aberration information of the first light flux, a weighting value for a spatial frequency characteristic of a first imaging signal acquired from the first imaging device, A first focus obtained using the pair of focus detection signals acquired from the first image sensor, based on a weight value for a spatial frequency characteristic of the pair of focus detection signals acquired from the image sensor. A first focus adjustment correction amount for correcting the position is calculated, and the aberration information of the photographing lens, the second aberration information that is the aberration information of the second light beam, and the second focus information are acquired from the second image sensor. The weighting value for the spatial frequency characteristic of the second imaging signal to be obtained and the weighting value for the spatial frequency characteristic of the pair of focus detection signals acquired from the second imaging element, A correction amount calculating step of calculating a second focus adjustment correction amount for correcting a second focus position obtained by using the acquired pair of focus detection signals; and the first and second focus adjustment corrections A control step of controlling the focus adjustment mechanism based on the amount. A light beam splitting means movable between a first position located in an optical path of a photographing lens having a focus adjustment mechanism and a second position retracted from the optical path; a plurality of photoelectric conversion units; One microlens that guides a light beam that has passed through a different region of the exit pupil of the photographing lens to the plurality of photoelectric conversion units among light beams transmitted through the light beam splitting device when the unit is located at the first position; An imaging device including a plurality of pixels including:
When the light beam splitting means is located at the first position, aberration information of the photographing lens, first aberration information which is aberration information of the light beam splitting means, and a space of an imaging signal acquired from the image sensor. Based on a weighting value for the frequency characteristic and a weighting value for the spatial frequency characteristic of the pair of focus detection signals acquired from the image sensor, a value is obtained using a pair of focus detection signals acquired from the image sensor. A correction amount calculating step of calculating a focus adjustment correction amount for correcting the focal position to be corrected;
A control step of controlling the focus adjustment mechanism based on the focus adjustment correction amount. A light beam splitting unit that splits a light beam that has passed through an exit pupil of a photographing lens having a focus adjusting mechanism into a first light beam and a second light beam, a plurality of first photoelectric conversion units, and the emission light beam of the first light beam A first imaging element including a plurality of first pixels including a single microlens that guides a light beam that has passed through regions having different pupils to the plurality of first photoelectric conversion units; a plurality of second photoelectric conversion units; An imaging device including a plurality of second pixels including: a microlens that guides a light flux of the two light fluxes that have passed through different regions of the exit pupil to the plurality of second photoelectric conversion units; Control method,
A first focus position obtained by using a pair of focus detection signals acquired from the first imaging device based on the shape parameter of the light beam splitting unit and correction information on the imaging lens and the shape parameter; Focus adjustment correction for correcting a second focus position obtained by using a first focus adjustment correction amount for correcting the first focus detection signal and a pair of focus detection signals acquired from the second image sensor. A correction amount calculating step of calculating an amount,
A control step of controlling the focus adjustment mechanism based on the first and second focus adjustment correction amounts.