JP7807429B2 - Imaging device and readout control method - Google Patents

Description

The present invention relates to an imaging device and a readout control method for reading out pixel signals from an imaging element.

Image capture devices are known that perform autofocus control using a phase difference signal obtained on the image plane of an image sensor. Image sensors capable of outputting such phase difference signals have two photoelectric conversion units, each with a pupil-divided pixel for one microlens. If the pixel signals output from the two photoelectric conversion units are used as separate phase difference signals, they can be used to calculate the amount of blur in autofocus (AF). If the pixel signals output from the two photoelectric conversion units are added to generate an image signal, the image signals of effective pixels can be aggregated to generate a frame image representing the subject (see, for example, Patent Document 1). Recent image capture devices employ such image sensors to perform high-precision AF even during video capture, enabling them to maintain focus even on moving subjects. Furthermore, the total number of pixels in image sensors has gradually increased, leading to corresponding improvements in focusing accuracy.

Japanese Patent Application Laid-Open No. 2017-134154

High frame mode (slow motion mode) is gaining popularity. By capturing images at a faster-than-normal frame rate and playing them back at the normal frame rate, you can enjoy footage of subjects moving smoothly, even though it appears slower than in real life. In high frame mode, pixel signals are read out at a faster-than-normal frame rate, which consumes a lot of power and, in some cases, can cause the pixel signal readout process to be unable to keep up. When the pixel signal readout process is unable to keep up, the AF process breaks down and the subject cannot be kept in focus.

The present invention was made to solve these problems by providing an imaging device that achieves highly accurate subject tracking when capturing images at a normal frame rate, and that can continue AF processing while reducing power consumption when capturing images at a frame rate faster than the normal frame rate.

An imaging device according to a first aspect of the present invention comprises an imaging element having a plurality of pixels arranged two-dimensionally, each pixel including a first photoelectric conversion unit that receives a light beam passing through a first partial region of an imaging optical system and a second photoelectric conversion unit that receives a light beam passing through a second partial region different from the first partial region; and a readout control unit that, when a first mode is selected, performs first readout control by separately AD converting the charges of the first photoelectric conversion unit and the second photoelectric conversion unit to generate pixel signals; and, when a second mode, in which a frame rate faster than the frame rate set in the first mode, performs second readout control by separately AD converting the charges of the first photoelectric conversion unit and the second photoelectric conversion unit to generate first pixel signals in discretely set first pixels, and adding up the charges of the first photoelectric conversion unit and the second photoelectric conversion unit and then AD converting them in second pixels that are not the first pixels, to generate second pixel signals.

In addition, a readout control method according to a second aspect of the present invention includes a step of reading out pixel signals from an image sensor in which a plurality of pixels are two-dimensionally arranged, each pixel including a first photoelectric conversion unit that receives a light beam passing through a first partial region of an imaging optical system and a second photoelectric conversion unit that receives a light beam passing through a second partial region different from the first partial region; a receiving step of accepting a selection from a first mode and a second mode in which a frame rate faster than the frame rate set in the first mode is set; and, when the first mode is selected, a step of accepting a selection from the first photoelectric conversion unit and the second photoelectric conversion unit. The computer executes a readout control step in which first readout control is performed to separately AD convert the charges of the second photoelectric conversion units to generate pixel signals, and when the second mode is selected, second readout control is performed to separately AD convert the charges of the first and second photoelectric conversion units to generate first pixel signals for discretely set first pixels among the pixels, and to add up the charges of the first and second photoelectric conversion units and then AD convert them to generate second pixel signals for second pixels that are not the first pixels among the pixels.

This invention makes it possible to provide an imaging device that can achieve highly accurate subject tracking when capturing images at a normal frame rate, and that can continue AF processing while reducing power consumption when capturing images at a frame rate faster than the normal frame rate.

1 is a diagram showing the appearance of an imaging device according to an embodiment of the present invention. FIG. 1 is a diagram illustrating a main hardware configuration of an imaging apparatus. FIG. 2 is a diagram illustrating a pixel arrangement of an imaging element. FIG. 2 is a diagram illustrating a configuration of a pixel. 10A and 10B are diagrams illustrating pixel signal readout control by a readout control unit. FIG. 10 is a diagram illustrating the setting of a first pixel and a second pixel. FIG. 10 is a diagram illustrating a window width for generating a phase waveform. FIG. 10 is a flowchart showing a processing procedure of a system control unit.

The present invention will be described below through embodiments of the invention, but the claimed invention is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems. In addition, when there are multiple structures with the same or similar configurations in each figure, to avoid complexity, some may be given reference numerals and others may not be given the same reference numerals.

FIG. 1 is a diagram showing the appearance of an imaging device 100 according to this embodiment. In particular, FIG. 1(A) is a diagram mainly showing the first surface side of imaging device 100, and FIG. 1(B) is a diagram mainly showing the second surface side opposite the first surface side. Imaging device 100 according to this embodiment is a so-called smartphone; in other words, a smartphone that also functions as an imaging device. In the following, we will explain the imaging function related to the present invention among the functions of a smartphone, and will omit explanation of other smartphone functions, such as the use of image data generated by imaging. Note that in this embodiment, imaging device 100 will be explained using a smartphone as an example, but it may of course also be an imaging device as a standalone camera, or a device with imaging functions incorporated into a tablet terminal or the like.

The imaging device 100 includes a first camera unit 110 and a second camera unit 120 arranged facing the same direction on the first surface side. The first camera unit 110 is a camera unit for generating wide-angle images. The second camera unit 120 is a camera unit for generating telephoto images. A user designates the first camera unit to capture images when a wide-angle image is desired, and designates the second camera unit to capture images when a telephoto image is desired. In the figure, the first camera unit 110 and the second camera unit 120 are arranged parallel to the long side of the imaging device 100, but the arrangement of the two camera units is not limited to this; for example, they may be arranged along a straight line obliquely intersecting the long side. Furthermore, the arrangement of the first camera unit 110 and the second camera unit 120 may be reversed from the positions shown in the figure.

The imaging device 100 is equipped with a display 130 on the second surface side. The display 130 is a display device that uses, for example, an organic EL (Electro Luminescence) panel, and displays a real-time image of the subject before capture (live view display), a real-time image during video capture (rec view display), and an image after capture. Note that a camera unit for selfies that is independent of the first camera unit 110 and the second camera unit 120 may also be provided on the second surface side.

A touch panel 162 is provided superimposed on the display 130. By tapping the shutter button displayed on the display 130, the user can instruct the imaging device 100 to capture a still image, and by tapping the video capture button, the user can instruct the imaging device 100 to start or stop recording a video. The user can also tap any point on the subject image displayed by live view or rec view to specify a certain area including that point as the focus area. In addition, the user can switch between the first camera unit 110 and the second camera unit 120 or select displayed menu items by performing a contact operation such as tapping.

Figure 2 is a diagram showing the main hardware configuration of the imaging device 100. In addition to the first camera unit 110, second camera unit 120, and display 130 described above, the imaging device 100 is composed of a system control unit 150 that controls these units, and peripheral elements that work in conjunction with the system control unit 150.

The first camera unit 110 is a camera unit for generating wide-angle images as described above, and mainly comprises a first optical system 111, a first drive mechanism 112, a first image sensor 113, and a first analog front end (AFE) 114. The first optical system 111 is an optical system for forming an image of incident subject light beams on the imaging surface of the first image sensor 113. Although shown as a single lens in the figure, it is generally composed of multiple lenses, at least some of which are focus lenses that can be advanced and retreated along the optical axis direction. The first drive mechanism 112 is a drive mechanism for moving the focus lens of the first optical system 111 along the optical axis direction, and includes an actuator that operates according to instructions from the system control unit 150.

The first image sensor 113 is, for example, a CMOS image sensor. The first image sensor 113 will be described in more detail later. The first image sensor 113 passes its output signal to the first AFE 114 under the control of the system control unit 150 and the first AFE 114. In addition to performing the AD conversion control described below, the first AFE 114 also adjusts the readout timing of the output signal and adjusts the level according to a specified gain. The pixel signal adjusted by the first AFE 114 is passed to the work memory 151.

The second camera unit 120 is a camera unit for generating telephoto images as described above, and mainly comprises a second optical system 121, a second drive mechanism 122, a second image sensor 123, and a second analog front end (AFE) 124. The second optical system 121 is an optical system for forming an image of incident subject light beams on the imaging surface of the second image sensor 123. Although shown as a single lens in the figure, the second optical system 121, like the first optical system 111, is generally composed of multiple lenses, at least some of which are focus lenses that can be advanced and retreated along the optical axis direction. The second drive mechanism 122 is a drive mechanism for moving the focus lens of the second optical system 121 along the optical axis direction, and includes an actuator that operates according to instructions from the system control unit 150.

The second image sensor 123 is, for example, a CMOS image sensor. The second image sensor 123 will be described in detail later, along with the first image sensor 113. The second image sensor 123 passes its output signal to the second AFE 114 under the control of the system control unit 150 and the second AFE 124. In addition to performing the AD conversion control described below, the second AFE 124 adjusts the read timing of the output signal and adjusts the level according to a specified gain. The pixel signal adjusted by the second AFE 124 is passed to the work memory 151. Note that in this embodiment, both the first optical system 111 and the second optical system 121 are assumed to be single-focus optical systems with fixed focal lengths, but at least one of them may be a variable-focus optical system (zoom lens) with a variable focal length.

The system control unit 150 is a processor (CPU: Central Processing Unit) that directly or indirectly controls each element that makes up the imaging device 100. The system control unit 150 plays the role of various functional control units depending on the control method being executed. For example, when executing autofocus control of the first camera unit 110 and the second camera unit 120, it functions as a detection unit that performs phase difference detection for the subject image, and when displaying the captured image on the display 130, it functions as a display control unit. In particular, in this embodiment, it functions as a readout control unit that reads out output signals from the first imaging element 113 and the second imaging element 123 and generates pixel signals. Specific control by the readout control unit will be described later.

The imaging device 100 mainly includes a work memory 151, an image processing unit 152, an operation unit 160, a recording unit 170, and a communication interface 180 as peripheral elements that work in conjunction with the system control unit 150. The work memory 151 is a volatile, high-speed memory, such as a static random access memory (SRAM). The work memory 151 receives sequentially converted pixel signals from the first AFE 114 and the second AFE 124, and stores them as frame data for one frame. It also extracts specific pixel signals as phase difference signals, and generates and stores phase waveform data used to calculate the amount of blurring in AF. The work memory 151 passes the frame data to the image processing unit 153 and passes the phase waveform data to the system control unit 150. The work memory 151 is also used as a temporary storage area as appropriate during the image processing stage performed by the image processing unit 153 and the focusing stage performed by the system control unit 150.

The image processing unit 153 is configured, for example, by an ASIC (Application Specific Integrated Circuit) that exclusively performs image processing, and performs various image processing such as interpolation on the received frame data to generate image data in a predetermined format. If the generated image data is for recording, it is recorded in the recording unit 170, and if it is for display, it is displayed on the display 130.

The operation unit 160 is an input device that includes a touch panel 162, and is a member that the user operates to give instructions to the imaging device 100. If the imaging device 100 accepts audio input, the operation unit 160 may also include a microphone. The recording unit 170 is non-volatile memory, and is configured, for example, by an SSD (Solid State Drive). In addition to recording image data generated by imaging, the recording unit 170 also holds constants, variables, setting values, control programs, etc. required for operation of the imaging device 100. The communication interface 180 may include a communication unit for a 5G line or wireless LAN. The communication interface 180 is used, for example, when transferring generated image data to an external device.

Next, we will explain the structure of the image sensor and readout control of pixel signals. In this embodiment, the first image sensor 113 and the second image sensor 123 employ image sensors with similar structures, and the first AFE 114 and the second AFE 124 perform similar readout control on these image sensors. Therefore, assuming that the first camera unit 110 captures moving images, we will explain the structure of the first image sensor 113 (hereinafter simply referred to as "image sensor 113") and the readout control by the system control unit 150 (readout control unit) via the first AFE 114 (hereinafter simply referred to as "AFE 114").

3 is a diagram illustrating the pixel array of the image sensor 113. In the image sensor 113, each pixel 200 includes a first photoelectric conversion unit 201 and a second photoelectric conversion unit 202, and a plurality of such pixels 200 are two-dimensionally arranged to form a pixel plane. The first photoelectric conversion unit 201 and the second photoelectric conversion unit 202 each form a substantially rectangular shape obtained by approximately dividing a square pixel area into two equal parts. With respect to the center of the pixel 200, the first photoelectric conversion unit 201 is offset in a first direction (to the left in the figure), and the second photoelectric conversion unit 202 is offset in a second direction opposite to the first direction (to the right in the figure).
In one possible embodiment, each of the pixels 200 may include two or more photoelectric conversion units, for example, four, and it should be understood that the four photoelectric conversion units may be arranged in a 2x2 array, for example.

Figure 4 is a diagram explaining the configuration of the pixel 200, and is a schematic diagram showing a cross section of the first optical system 111 (hereinafter simply referred to as the "optical system 111"), a cross section of the pixel 200, and part of the circuitry of the AFE 114.

The first photoelectric conversion unit 201 receives a light beam that passes through a first partial area PA1 of the optical system 111, which is an imaging optical system. The second photoelectric conversion unit 202 receives a light beam that passes through a second partial area PA2, which is different from the first partial area PA1 of the optical system 111. The first partial area PA1 and the second partial area PA2 may have an overlapping area. Both light beams pass through a microlens 211 and a color filter 212 arranged above the pixel and reach the respective photoelectric conversion unit.

The charge accumulated in the first photoelectric conversion unit 201 is converted into a digital signal by the AD converter 230 when the first switch 221 is closed, and is output as a pixel signal. The charge accumulated in the second photoelectric conversion unit 202 is converted into a digital signal by the AD converter 230 when the second switch 222 is closed, and is output as a pixel signal.

Figure 5 is a diagram illustrating pixel signal readout control by the readout control unit. The readout control unit can generate pixel signals by performing either individual readout control or batch readout control on pixels 200 in a charge accumulation state.

Figure 5(A) is a diagram illustrating individual readout control. In individual readout control, first, as shown in the upper diagram of Figure 5(A), the readout control unit closes the first switch 221 and leaves the second switch 222 open, thereby causing the AD converter 230 to AD convert only the charge accumulated in the first photoelectric conversion unit 201 and output pixel signal A. At this time, there is no change in the charge accumulated in the second photoelectric conversion unit 202. Next, as shown in the lower diagram of Figure 5(A), the readout control unit closes the second switch 222 and opens the first switch 221, thereby causing the AD converter 230 to AD convert only the charge accumulated in the second photoelectric conversion unit 202 and output pixel signal B. In this way, by repeating AD conversion twice, pixel signal A corresponding to the charge of the first photoelectric conversion unit 201 and pixel signal B corresponding to the charge of the second photoelectric conversion unit 202 can be output sequentially. Because pixel signal A and pixel signal B are both signals generated from the same pixel, they are collectively referred to here as the first pixel signal. Note that the order in which pixel signal A and pixel signal B are generated may be reversed.

Figure 5 (B) is a diagram illustrating batch readout control. In batch readout control, the readout control unit closes both the first switch 221 and the second switch 222, adds the charge accumulated in the first photoelectric conversion unit 201 and the charge accumulated in the second photoelectric conversion unit 202, performs AD conversion using the AD converter 230, and outputs a second pixel signal. In this way, a single AD conversion can output a second pixel signal corresponding to the charge obtained by adding the charge in the first photoelectric conversion unit 201 and the charge in the second photoelectric conversion unit 202.

When a normal imaging mode with a normal frame rate (e.g., 30 fps or 60 fps) is selected, the readout control unit executes first readout control, which applies individual readout control to all pixels. In this case, the pixel value of the pixel in the frame data is determined by adding the pixel values of pixel signal A, which is the first pixel signal, and pixel signal B. Furthermore, by separately collecting pixel signals A and B of a pixel group corresponding to a specific subject image designated by the user or selected by an automatic algorithm, phase waveform data for the specific subject image can be generated. In other words, if first pixel signals are generated from all pixels, dense phase waveform data can be generated for any subject, making it possible to maintain a highly accurate focus state even for a moving subject during the imaging period.

On the other hand, when attempting to perform first readout control during image capture at a frame rate faster than the normal frame rate (for example, 120 fps or 240 fps), two AD conversions are required for each pixel, resulting in a significant increase in power consumption. This increase in power consumption not only shortens the usage time of the image capture device, but can also cause the image capture device to become unstable due to the heat generated. Furthermore, the processing time required for the two AD conversions and the processing time required to process the enormous amount of generated first pixel signals may not fit within the frame rate cycle. In other words, the readout process and data processing may not be able to keep up with the high-speed frame rate. When this happens, not only will it be impossible to maintain focus on the subject, but it may also be impossible to generate frame images.

In this embodiment, when an imaging mode with a high frame rate is selected, a second readout control that combines individual readout control and batch readout control is executed. An example of an imaging mode with a high frame rate is high frame mode (slow motion mode), which plays back images at a normal frame rate, allowing you to enjoy smooth, yet slower-moving, subject footage.

When performing the second readout control, all effective pixels of the image sensor 111 are set in advance to be divided into first pixels to which individual readout control is applied and second pixels to which collective readout control is applied. Figure 6 is a diagram explaining the setting of first and second pixels when performing the second readout control. In the figure, the pixels surrounded by a thick frame are set as first pixels 200a, and the other pixels are set as second pixels 200b.

In the example shown in the figure, the first pixels 200a are periodically arranged at a ratio of one for every five pixels vertically and five pixels horizontally. By arranging the first pixels 200a discretely in this manner, surrounded by the second pixels 200b, it is possible to significantly reduce the number of first pixels 200a that perform individual readout control. By significantly reducing the number of first pixels 200a, it is possible to reduce overall power consumption. Furthermore, since the amount of pixel signals generated can also be reduced, the time required for calculation processing can be shortened, and AF processing and frame image generation can continue appropriately even at high frame rates.

However, because the phase waveform data for AF processing can only be obtained from the discretely arranged first pixels 200a, it is necessary to consider the balance with the reduction in focusing accuracy. In other words, the ratio of the first pixels 200a to the second pixels 200b is determined by considering the balance between the advantages of power saving and high-speed processing and the disadvantage of reduced focusing accuracy. For example, if the user can select multiple frame rates, an adjustment can be made such that the ratio of the first pixels 200a to the second pixels 200b is reduced for a faster frame rate.
The discretely arranged first pixels 200a for performing AF processing may be arranged in a variety of ways. For example, the first pixels 200a may be arranged evenly at a fixed ratio, i.e., the first pixels 200a may be distributed across all pixels at a fixed ratio. This arrangement can maintain uniform overall focusing performance, or a large number of first pixels 200a may be arranged in the center of the image and fewer first pixels 200a may be arranged on the edges. It should be understood that such a center-prioritized arrangement method is used for portrait mode or a mode that emphasizes a specific part, and provides better focusing performance in the center of the image.

Furthermore, to improve focusing accuracy even with a limited number of first pixels 200a, the window width for generating the phase waveform may be adjusted. Figure 7 is a diagram explaining the window width for generating the phase waveform.

7A shows the first window width set when performing the first readout control that applies individual readout control to all pixels 200. The first window width is set to W ₁ pixel, and when performing the first readout control, the detection unit generates a phase waveform using pixel signals A and B generated from these pixels 200, and performs phase difference detection for the subject image to achieve focusing.

7B shows the second window width set when performing second readout control, which combines individual readout control and batch readout control. The second window width is set to W2 pixels, which is greater than _W1 pixel _. When performing second readout control, the detection unit generates a phase waveform using pixel signal A and pixel signal B generated from the first pixel 200a that is the target of individual readout control among the pixels 200 included in this second window width, and performs phase difference detection for the subject image.

In other words, when performing second readout control, if the same first window width as when performing first readout control is set, the phase waveform that can be generated will be short. However, by setting a second window width larger than the first window width and lengthening the length of the phase waveform that is generated, this reduces the accuracy of phase difference detection. However, if the window width is increased, the possibility of capturing multiple subjects across the image increases, making perspective conflicts more likely to occur. Therefore, it is best to set the window width appropriately according to the ratio of first pixels 200a to second pixels 200b.

Next, the processing steps of the system control unit 150 in the video capture process will be described. Figure 8 is a flow diagram showing the processing steps of the system control unit 150.

In step S101, the system control unit 150 accepts from the user a selection between a first mode in which a normal frame rate is set and a second mode in which a frame rate faster than the normal frame rate is set. The system control unit 150 switches to the selected imaging mode and waits for an instruction to start imaging from the user in step S102. If an instruction to start imaging is accepted, the system control unit 150 proceeds to step S103, where it checks the current imaging mode. If the current imaging mode is the first mode (normal frame rate), it proceeds to step S104, and if the current mode is the second mode (higher frame rate), it proceeds to step S108.

Proceeding from step S103 to step S104, the system control unit 150, functioning as a readout control unit, executes first readout control. In the following step S105, the system control unit 150, functioning as a detection unit, generates a phase waveform from the first window width set for the pixel group corresponding to the specific subject image, performs phase difference detection, and executes AF processing. In the following step S106, the system control unit 150 generates one frame of frame data in accordance with the set normal frame rate. In step S107, the system control unit 150 checks whether an instruction to stop image capture has been received from the user. If an instruction to stop image capture has not been received, the system control unit 150 returns to step S104 and continues video capture. If an instruction to stop image capture has been received, the system control unit 150 performs the set formatting process on the frame data accumulated up to that point to generate a video file, and the series of processes ends.

When the process proceeds from step S103 to step S108, the system control unit 150, functioning as a readout control unit, executes second readout control. In the following step S109, the system control unit 150, functioning as a detection unit, generates a phase waveform from the second window width set for the pixel group corresponding to the specific subject image, performs phase difference detection, and executes AF processing. In the following step S110, the system control unit 150 generates one frame's worth of frame data in accordance with the set high-speed frame rate. In step S111, the system control unit 150 checks whether an instruction to stop image capture has been received from the user. If an instruction to stop image capture has not been received, the process returns to step S108 to continue video capture; if an instruction to stop image capture has been received, the system control unit 150 performs the set formatting process on the frame data accumulated up to that point to generate a video file, and the process ends.

In the embodiment described above, it is assumed that a moving image is captured, but similar focus control can be performed even when a still image is captured. That is, when a still image is captured each time the shutter button is tapped or when an imaging mode for capturing continuous images at a low speed is selected, a first readout control that applies individual readout control to all of the pixels 200 can be applied, and when an imaging mode for capturing continuous images at a high speed is selected, a second readout control that combines individual readout control and batch readout control can be applied.
In one possible embodiment, two or more imaging modes may be provided based on different imaging frame rates or continuous shooting speeds, each corresponding to a different focusing accuracy. For example, the higher the frame rate/faster the continuous shooting speed, the fewer pixels are used for focusing, providing a flexible balance between power consumption and performance.

100...imaging device, 110...first camera unit, 111...first optical system (optical system), 112...first drive mechanism (drive mechanism), 113...first image sensor (image sensor), 114...first analog front end (AFE), 120...second camera unit, 121...second optical system, 122...second drive mechanism, 123...second image sensor, 124...second analog front end, 130...display, 150...system System control unit, 151...work memory, 152...image processing unit, 160...operation unit, 162...touch panel, 170...recording unit, 180...communication interface, 200...pixel, 200a...first pixel, 200b...second pixel, 201...first photoelectric conversion unit, 202...second photoelectric conversion unit, 210...microlens, 212...color filter, 221...first switch, 222...second switch, 230...AD converter

Claims (4)

An imaging device,
an image sensor in which a plurality of pixels are two-dimensionally arranged, each pixel including at least a first photoelectric conversion unit that receives a light beam passing through a first partial region of an imaging optical system and a second photoelectric conversion unit that receives a light beam passing through a second partial region different from the first partial region;
a readout control unit that, when a first mode is selected, performs first readout control to separately AD convert the charges of the first photoelectric conversion unit and the second photoelectric conversion unit to generate pixel signals, and when a second mode in which a frame rate faster than the frame rate set in the first mode is selected, performs second readout control to separately AD convert the charges of the first photoelectric conversion unit and the second photoelectric conversion unit to generate first pixel signals in first pixels that are discretely set among the pixels, and adds together the charges of the first photoelectric conversion unit and the second photoelectric conversion unit and then AD converts them to generate second pixel signals in second pixels that are not the first pixels among the pixels ,
the imaging device includes a detection unit,
When the readout control unit performs the second readout control, the detection unit increases a window width for generating a phase waveform compared to when the readout control unit performs the first readout control . The imaging device according to claim 1 , wherein the detection unit performs phase difference detection autofocus using pixel signals obtained by separately AD-converting the electric charges of the first photoelectric conversion unit and the second photoelectric conversion unit. A read control method, comprising:
reading out pixel signals from an image sensor in which a plurality of pixels are two-dimensionally arranged, each pixel including a first photoelectric conversion unit that receives a light beam passing through a first partial region of an imaging optical system and a second photoelectric conversion unit that receives a light beam passing through a second partial region different from the first partial region;
determining one mode from a first mode and a second mode in which a frame rate higher than that set in the first mode is set;
a step of performing a first readout control in the first mode to separately AD convert the charges of the first photoelectric conversion unit and the second photoelectric conversion unit to generate a pixel signal, and performing a second readout control in the second mode to separately AD convert the charges of the first photoelectric conversion unit and the second photoelectric conversion unit to generate a first pixel signal in first pixels that are discretely set among the pixels, and to add up the charges of the first photoelectric conversion unit and the second photoelectric conversion unit and then AD convert the sum to generate a second pixel signal in second pixels that are not the first pixels among the pixels,
A readout control method , wherein, when the second readout control is performed, a window width for generating a phase waveform is made larger than when the first readout control is performed . performing phase difference detection autofocus using pixel signals obtained by separately AD-converting the charges of the first photoelectric conversion unit and the second photoelectric conversion unit;
The read control method according to claim 3 .