JP7614737B2 - Photographing control device, imaging device, photographing control method, and program - Google Patents

Description

The present invention relates to an imaging control device, an imaging device, an imaging control method, and a program.

Conventionally, a method of adjusting focus by detecting a subject and performing distance measurement calculations at the detected location is known. The technology in Patent Document 1 determines the detection direction of the focus detection signal according to the part of the detected subject, making it possible to perform optimal distance measurement according to the subject.

JP 2009-192774 A

In general subject detection processing, there is no strict distinction between areas where the subject to be detected exists and areas where it does not exist, and areas including areas where the subject to be detected does not exist (e.g. background areas) are detected as the subject area. For this reason, even if focus adjustment (focus control) is performed based on the subject area, there is a possibility that the focus will be set on an area where the subject to be detected does not exist (e.g. background areas). This is also true when performing shooting control other than focus control (e.g. exposure control), and even if shooting control is performed based on the subject area, it is not necessarily the case that a control result suitable for the subject to be detected is obtained.

The present invention was made in consideration of these circumstances, and aims to provide a technology that improves the accuracy of shooting control based on a subject detection area that partially contains the subject to be detected.

In order to solve the above problem, the present invention provides an imaging control device comprising: a detection means for detecting a subject area in a shooting range that partially includes a subject to be detected; an acquisition means for acquiring a plurality of defocus amounts corresponding to a plurality of ranging points in a ranging area including the subject area; a classification means for classifying the ranging area into a plurality of partial areas based on the plurality of defocus amounts, each of the plurality of partial areas corresponding to a different partial range in the range of the plurality of defocus amounts; a determination means for determining a subject degree indicating the possibility that the subject to be detected is present for each of the plurality of partial areas based on a geometric relationship between each of the plurality of partial areas and the subject area; and a control means for performing imaging control based on the plurality of partial areas, the control means performing the imaging control such that a contribution of a partial area having a first subject degree is greater than a contribution of a partial area having a second subject degree that is smaller than the first subject degree, the imaging control including a focus control, and the control means performing the focus control based on a representative defocus amount obtained from a defocus amount corresponding to the partial area having the first subject degree among the plurality of defocus amounts.

The present invention makes it possible to improve the accuracy of shooting control based on a subject detection area that partially includes the subject to be detected.

Other features and advantages of the present invention will become apparent from the accompanying drawings and the following detailed description of the invention.

1 is a block diagram showing a configuration of an imaging apparatus 10 including an imaging control device. 3 is a schematic diagram of an array of imaging pixels (and focus detection pixels) of an image sensor 122. 3A is a plan view of one pixel 200G of the image sensor 122 shown in FIG. 2, as viewed from the light receiving surface side (+z side) of the image sensor 122; FIG. 3B is a cross-sectional view of the aa cross section of FIG. 3A, as viewed from the -y side. 4 is a schematic explanatory diagram showing the correspondence between the pixel structure and pupil division according to the embodiment shown in FIG. 3; 5 is a schematic diagram showing the correspondence between the image sensor 122 and pupil division. 11 is a schematic diagram showing the relationship between a defocus amount based on a first focus detection signal and a second focus detection signal and an image shift amount between the first focus detection signal and the second focus detection signal. 4 is a flowchart of a photographing process. 3 is a schematic diagram showing the relationship between a subject (detection target) and a detection area (subject area). 8 is a flowchart showing details of the focus adjustment process in S703 of FIG. 7; FIG. 8 is a schematic explanatory diagram of distance measurement areas set for each of the subject areas 801 to 806 based on the position and size of the subject area. FIG. 11 is a schematic explanatory diagram showing a state in which defocus amounts in the distance measurement area in FIG. 10 are classified into classes. FIG. 13 is a schematic explanatory diagram of class regions detected in S905. 8 is a flowchart showing details of the white balance adjustment process in S705 of FIG. 7. 8 is a flowchart showing details of the exposure adjustment process in S706 of FIG. 7; 10 is a flowchart showing details of the subject degree determination process in S906 of FIG. 9;

The following embodiments are described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe multiple features, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicate explanations are omitted.

[First embodiment]
(Configuration of imaging device 10)
FIG. 1 is a block diagram showing the configuration of an imaging device 10 equipped with an imaging control device. In the example of FIG. 1, the imaging device 10 is a single-lens reflex type digital camera with interchangeable lenses. The imaging device 10 has the form of a camera system having a lens unit 100 (interchangeable lens) and a camera body 120. The lens unit 100 is detachably attached to the camera body 120 via a mount M shown by a dotted line in FIG. 1. However, this embodiment is not limited to the configuration shown in FIG. 1, and can also be applied to an imaging device (digital camera) in which a lens unit (imaging optical system) and a camera body are integrally configured. In addition, this embodiment is not limited to a digital camera, and can also be applied to other imaging devices such as a video camera.

The lens unit 100 has a first lens group 101, an aperture 102, a second lens group 103, a focus lens group (hereinafter simply referred to as "focus lens") 104, and a drive/control system as an optical system. In this way, the lens unit 100 is a photographing lens (imaging optical system) that includes the focus lens 104 and forms a subject image.

The first lens group 101 is disposed at the tip of the lens unit 100 and is held so that it can move forward and backward in the optical axis direction OA. The diaphragm 102 adjusts the amount of light during shooting by adjusting its aperture diameter, and also functions as a shutter for adjusting the exposure time when shooting still images. The diaphragm 102 and the second lens group 103 can move together in the optical axis direction OA, and a zoom function is achieved by linking with the forward and backward movement of the first lens group 101. The focus lens 104 can move in the optical axis direction OA, and the subject distance (focus distance) at which the lens unit 100 focuses changes depending on its position. By controlling the position of the focus lens 104 in the optical axis direction OA, focus adjustment (focus control) that adjusts the focus distance of the lens unit 100 is possible.

The drive/control system includes a zoom actuator 111, an aperture actuator 112, a focus actuator 113, a zoom drive circuit 114, an aperture drive circuit 115, a focus drive circuit 116, a lens MPU 117, and a lens memory 118. The zoom drive circuit 114 drives the first lens group 101 and the second lens group 103 in the optical axis direction OA using the zoom actuator 111 to control the angle of view of the optical system of the lens unit 100 (performs a zoom operation). The aperture drive circuit 115 drives the aperture 102 using the aperture actuator 112 to control the aperture diameter and opening/closing operation of the aperture 102. The focus drive circuit 116 drives the focus lens 104 in the optical axis direction OA using the focus actuator 113 to control the focal distance of the optical system of the lens unit 100 (performs focus control). The focus drive circuit 116 also functions as a position detector that detects the current position (lens position) of the focus lens 104 using the focus actuator 113.

The lens MPU 117 (processor) performs all calculations and controls related to the lens unit 100, and controls the zoom drive circuit 114, the aperture drive circuit 115, and the focus drive circuit 116. The lens MPU 117 is also connected to the camera MPU 125 through the mount M to communicate commands and data. For example, the lens MPU 117 detects the position of the focus lens 104 and notifies the lens position information in response to a request from the camera MPU 125. This lens position information includes information such as the position of the focus lens 104 in the optical axis direction OA, the position and diameter of the exit pupil in the optical axis direction OA when the optical system is not moving, and the position and diameter of the lens frame that limits the light beam of the exit pupil in the optical axis direction OA. The lens MPU 117 also controls the zoom drive circuit 114, the aperture drive circuit 115, and the focus drive circuit 116 in response to a request from the camera MPU 125. The lens memory 118 stores optical information necessary for automatic focus adjustment (AF control). The camera MPU 125 controls the operation of the lens unit 100 by executing programs stored, for example, in the built-in non-volatile memory or the lens memory 118.

The camera body 120 has an optical low-pass filter 121, an image sensor 122, and a drive/control system. The optical low-pass filter 121 and the image sensor 122 function as an image sensor (imaging section) that photoelectrically converts the subject image (optical image) formed via the lens unit 100 and outputs image data. In this embodiment, the image sensor 122 photoelectrically converts the subject image formed via the imaging optical system and outputs an imaging signal and a focus detection signal as image data. In this embodiment, the first lens group 101, the aperture 102, the second lens group 103, the focus lens 104, and the optical low-pass filter 121 constitute an imaging optical system.

The optical low-pass filter 121 reduces false colors and moiré in the captured image. The image sensor 122 is composed of a CMOS image sensor and its peripheral circuits, and is arranged with m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers of 2 or more). In this embodiment, the image sensor 122 also serves as a focus detection element, has a pupil division function, and has pupil division pixels that are capable of phase difference detection focus detection (phase difference AF) using image data (image signal). The image processing circuit 124 generates data for phase difference AF and image data for display, recording, and subject detection based on the image data output from the image sensor 122.

The drive/control system includes an image sensor drive circuit 123, an image processing circuit 124, a camera MPU 125, a display 126, a group of operation switches (operation SW) 127, a memory 128, a phase difference AF section 129 (image plane phase difference focus detection section, control section), an AE section 130 (control section), a white balance adjustment section 131 (control section), and a subject detection section 132 (detection section). The image sensor drive circuit 123 controls the operation of the image sensor 122, and performs A/D conversion on the image signal (image data) output from the image sensor 122 and transmits it to the camera MPU 125. The image processing circuit 124 performs general image processing performed in digital cameras, such as gamma conversion, color interpolation processing, and compression encoding processing, on the image signal output from the image sensor 122. The image processing circuit 124 also generates a signal for phase difference AF, a signal for AE, a signal for white balance adjustment, and a signal for subject detection. In this embodiment, a signal for phase difference AF, a signal for AE, a signal for white balance adjustment, and a signal for subject detection are generated, but for example, a signal for AE, a signal for white balance adjustment, and a signal for subject detection may be generated as a common signal. Also, the combination of common signals is not limited to this.

The camera MPU 125 (processor, control device) performs all calculations and control related to the camera body 120. That is, the camera MPU 125 controls the image sensor drive circuit 123, image processing circuit 124, display 126, operation switch group 127, memory 128, phase difference AF section 129, AE section 130, white balance adjustment section 131, and subject detection section 132. The camera MPU 125 is connected to the lens MPU 117 via the signal line of the mount M, and communicates commands and data with the lens MPU 117. The camera MPU 125 issues requests to the lens MPU 117 to acquire the lens position and drive the lens at a specified drive amount, and also issues requests to acquire optical information specific to the lens unit 100 from the lens MPU 117.

The camera MPU 125 incorporates a ROM 125a that stores a program for controlling the operation of the camera body 120, a RAM 125b (camera memory) that stores variables, and an EEPROM 125c that stores various parameters. The camera MPU 125 also executes focus detection processing based on the program stored in the ROM 125a. In the focus detection processing, a known correlation calculation process is executed using a pair of image signals obtained by photoelectrically converting optical images formed by light beams that have passed through different pupil regions (pupil partial regions) of the imaging optical system.

The display 126 is comprised of an LCD and the like, and displays information regarding the shooting mode of the imaging device 10, a preview image before shooting and a confirmation image after shooting, and an in-focus state display image at the time of focus detection. The operation switch group 127 is comprised of a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The memory 128 (recording unit) is a removable flash memory, and records captured images.

The phase difference AF unit 129 performs focus detection processing using a phase difference detection method based on the image signal (signal for phase difference AF) of the focus detection image data obtained from the image sensor 122 and the image processing circuit 124. More specifically, the image processing circuit 124 generates a pair of image data formed by light beams passing through a pair of pupil regions of the image pickup optical system as focus detection data, and the phase difference AF unit 129 detects the amount of focus deviation based on the amount of deviation of the pair of image data. In this manner, the phase difference AF unit 129 of this embodiment performs phase difference AF (image plane phase difference AF) based on the output of the image sensor 122 without using a dedicated AF sensor. In this embodiment, the phase difference AF unit 129 has an acquisition unit 129a and a calculation unit 129b. The operation of each of these units will be described later.

At least a part of the phase difference AF unit 129 (a part of the acquisition unit 129a or the calculation unit 129b) may be provided in the camera MPU 125. Details of the operation of the phase difference AF unit 129 will be described later. The phase difference AF unit 129 functions as a focus control unit that controls the position of the focus lens 104 using the focus detection result.

The AE unit 130 performs exposure adjustment processing to set appropriate shooting conditions by performing photometry based on the AE signal obtained from the image sensor 122 and the image processing circuit 124. Specifically, it performs photometry based on the AE signal and calculates the exposure amount at the currently set aperture value, shutter speed, and ISO sensitivity. From the difference between the calculated exposure amount and a predetermined appropriate exposure amount, it performs exposure adjustment processing by calculating the appropriate aperture value, shutter speed, and ISO sensitivity to be set at the time of shooting and setting these as the shooting conditions. The AE unit 130 functions as an exposure adjustment unit that calculates the exposure conditions at the time of shooting using the photometry results and controls the aperture value, shutter speed, and ISO sensitivity of the aperture 102.

The white balance adjustment unit 131 performs white balance adjustment processing based on a signal for white balance adjustment obtained from the image sensor 122 and the image processing circuit 124. Specifically, the white balance adjustment processing is performed by calculating the white balance of the signal for white balance adjustment and adjusting the color weight based on the difference from a predetermined appropriate white balance.

The subject detection unit 132 performs subject detection processing based on a subject detection signal generated by the image processing circuit 124. The subject detection processing detects the type and state of the subject (detection attributes) and the position and size of the subject (detection area). The operation of the subject detection unit 132 will be described in detail later.

In this way, the imaging device 10 of this embodiment can perform a combination of phase difference AF, photometry (exposure adjustment), white balance adjustment, and subject detection, and can select the position (image height range) for performing phase difference AF, photometry, and white balance adjustment depending on the result of subject detection.

(Configuration of the image sensor 122)
2 is a schematic diagram of the array of imaging pixels (and focus detection pixels) of the image sensor 122. FIG. 2 shows the pixel (imaging pixel) array of the two-dimensional CMOS sensor (image sensor 122) of this embodiment in a range of 4 columns x 4 rows, and the focus detection pixel array in a range of 8 columns x 4 rows. In the first embodiment, the 2 columns x 2 rows pixel group 200 shown in FIG. 2 has a pixel 200R having a spectral sensitivity of R (red) arranged at the upper left, a pixel 200G having a spectral sensitivity of G (green) arranged at the upper right and lower left, and a pixel 200B having a spectral sensitivity of B (blue) arranged at the lower right. Furthermore, each pixel is composed of a first focus detection pixel 201 and a second focus detection pixel 202 arranged in 2 columns x 1 row.

A large number of 4 columns x 4 rows of pixels (8 columns x 4 rows of focus detection pixels) as shown in Figure 2 are arranged on a surface, making it possible to acquire a captured image (focus detection signal). In this embodiment, the pixel period P is 4 μm, the number of pixels N is 5,575 columns horizontally x 3,725 rows vertically = approximately 20.75 million pixels, the column direction period PAF of the focus detection pixels is 2 μm, and the number of focus detection pixels NAF is 11,150 columns horizontally x 3,725 rows vertically = approximately 41.5 million pixels.

Figure 3(a) is a plan view of one pixel 200G of the image sensor 122 shown in Figure 2, as viewed from the light receiving surface side (+z side) of the image sensor 122, and Figure 3(b) is a cross-sectional view of the a-a cross section of Figure 3(a) as viewed from the -y side.

As shown in FIG. 3, in the pixel 200G of this embodiment, a microlens 305 for focusing incident light is formed on the light receiving side of each pixel, and a photoelectric conversion unit 301 and a photoelectric conversion unit 302 are formed that are NH-divided (2-divided) in the x direction and NV-divided (1-divided) in the y direction. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 correspond to the first focus detection pixel 201 and the second focus detection pixel 202, respectively.

The photoelectric conversion units 301 and 302 may be pin structure photodiodes with an intrinsic layer sandwiched between a p-type layer and an n-type layer, or may be pn junction photodiodes with the intrinsic layer omitted as necessary. In each pixel, a color filter 306 is formed between the microlens 305 and the photoelectric conversion units 301 and 302. Furthermore, as necessary, the spectral transmittance of the color filter 306 may be changed for each subpixel, or the color filter 306 may be omitted.

The light incident on pixel 200G shown in FIG. 3 is collected by microlens 305, dispersed by color filter 306, and then received by photoelectric conversion unit 301 and photoelectric conversion unit 302. In photoelectric conversion unit 301 and photoelectric conversion unit 302, pairs of electrons and holes are generated according to the amount of received light, and after separation by a depletion layer, negatively charged electrons are accumulated in an n-type layer (not shown). Meanwhile, holes are discharged to the outside of image sensor 122 through a p-type layer connected to a constant voltage source (not shown). The electrons accumulated in the n-type layers (not shown) of photoelectric conversion unit 301 and photoelectric conversion unit 302 are transferred to a capacitance unit (FD) via a transfer gate and converted into a voltage signal.

Figure 4 is a schematic diagram showing the correspondence between the pixel structure of this embodiment shown in Figure 3 and pupil division. Figure 4 shows a cross-sectional view of the a-a cross section of the pixel structure of this embodiment shown in Figure 3(a) viewed from the +y side, and the pupil plane (pupil distance Ds) of the image sensor 122. In Figure 4, the x-axis and y-axis of the cross-sectional view are inverted with respect to Figure 3 in order to correspond to the coordinate axes of the pupil plane of the image sensor 122.

In FIG. 4, the first pupil partial region 501 of the first focus detection pixel 201 is in a roughly conjugate relationship with the light receiving surface of the photoelectric conversion unit 301, whose center of gravity is decentered in the -x direction, due to the microlens, and represents the pupil region that can receive light at the first focus detection pixel 201. The first pupil partial region 501 of the first focus detection pixel 201 has its center of gravity decentered on the +X side on the pupil plane. In FIG. 4, the second pupil partial region 502 of the second focus detection pixel 202 is in a roughly conjugate relationship with the light receiving surface of the photoelectric conversion unit 302, whose center of gravity is decentered in the +x direction, due to the microlens, and represents the pupil region that can receive light at the second focus detection pixel 202. The second pupil partial region 502 of the second focus detection pixel 202 has its center of gravity decentered on the -X side on the pupil plane. Also, in FIG. 4, pupil region 500 is the pupil region that can receive light in the entire pixel 200G when the photoelectric conversion unit 301 and the photoelectric conversion unit 302 (the first focus detection pixel 201 and the second focus detection pixel 202) are all combined.

In image plane phase difference AF, the pupil is divided using the microlenses of the image sensor, and is therefore subject to the effects of diffraction. In Figure 4, the pupil distance to the pupil plane of the image sensor is several tens of mm, while the diameter of the microlens is several μm. This results in an aperture value of several tens of thousands for the microlens, resulting in diffraction blurring on the level of several tens of mm. As a result, the image on the light receiving surface of the photoelectric conversion unit does not become a clear pupil region or pupil partial region, but becomes a light receiving sensitivity characteristic (incident angle distribution of light receiving rate).

Figure 5 is a schematic diagram showing the correspondence between the image sensor 122 and pupil division. The image sensor 122 is disposed on the imaging surface 600. Light beams that pass through different pupil partial regions, the first pupil partial region 501 and the second pupil partial region 502, are incident on each pixel of the image sensor 122 at different angles, and are received by the first focus detection pixel 201 and the second focus detection pixel 202, which are divided into 2 x 1 regions. This embodiment is an example in which the pupil region is divided into two in the horizontal direction. If necessary, pupil division may also be performed in the vertical direction.

The image sensor 122 of this embodiment has an array of imaging pixels each having a first focus detection pixel 201 and a second focus detection pixel 202. The first focus detection pixel 201 receives a light beam that passes through a first pupil partial region 501 of the imaging optical system. The second focus detection pixel 202 receives a light beam that passes through a second pupil partial region 502 of the imaging optical system that is different from the first pupil partial region 501. The imaging pixel receives a light beam that passes through a pupil region that is the combined first pupil partial region 501 and second pupil partial region 502 of the imaging optical system.

In the image sensor 122 of this embodiment, each imaging pixel is composed of a first focus detection pixel 201 and a second focus detection pixel 202. If necessary, the imaging pixel, the first focus detection pixel 201, and the second focus detection pixel 202 may be configured as separate pixels, and the first focus detection pixel 201 and the second focus detection pixel 202 may be partially arranged in a portion of the imaging pixel array.

In this embodiment, focus detection is performed by collecting the light receiving signals of the first focus detection pixels 201 of each pixel of the image sensor 122 to generate a first focus signal, and collecting the light receiving signals of the second focus detection pixels 202 of each pixel to generate a second focus signal. In addition, an image signal (captured image) with a resolution of N effective pixels is generated by adding the signals of the first focus detection pixels 201 and the second focus detection pixels 202 for each pixel of the image sensor 122. The method of generating each signal is not limited to the above-mentioned method, and for example, the second focus detection signal may be generated from the difference between the image signal and the first focus signal.

(Relationship between defocus amount and image shift amount)
The relationship between the defocus amount and the image shift amount based on the first focus detection signal and the second focus detection signal acquired by the image sensor 122 of this embodiment will be described below.

Figure 6 is a schematic diagram showing the relationship between the defocus amount based on the first focus detection signal and the second focus detection signal and the image shift amount between the first focus detection signal and the second focus detection signal. The image sensor 122 is disposed on the imaging plane 600. As in Figures 4 and 5, the pupil plane of the image sensor 122 is divided into a first pupil partial region 501 and a second pupil partial region 502.

The defocus amount d has a magnitude |d|, which is the distance from the subject's imaging position to the imaging plane, and is defined as a negative sign (d<0) in a front-focus state where the subject's imaging position is on the subject side of the imaging plane. Also, a back-focus state where the subject's imaging position is on the opposite side of the subject from the imaging plane is defined as a positive sign (d>0). In a focused state where the subject's imaging position is on the imaging plane (focused position), d=0. In FIG. 6, subject 601 shows an example of a focused state (d=0), and subject 602 shows an example of a front-focus state (d<0). The front-focus state (d<0) and the back-focus state (d>0) are combined to form a defocused state (|d|>0).

In the front focus state (d<0), the light beam from the subject 602 that passes through the first pupil partial region 501 (second pupil partial region 502) is once focused, and then spreads to a width Γ1 (Γ2) centered on the center of gravity position G1 (G2) of the light beam, forming a blurred image on the imaging surface 600. The blurred image is received by the first focus detection pixel 201 (second focus detection pixel 202) constituting each pixel arranged on the image sensor 122, and a first focus detection signal (second focus detection signal) is generated. Therefore, the first focus detection signal (second focus detection signal) is recorded as a subject image in which the subject 602 is blurred to a width Γ1 (Γ2) at the center of gravity position G1 (G2) on the imaging surface 600. The blur width Γ1 (Γ2) of the subject image increases roughly in proportion to the increase in the magnitude |d| of the defocus amount d. Similarly, the magnitude |p| of the image shift amount p (= the difference G1-G2 in the center of gravity position of the light beam) of the subject image between the first focus detection signal and the second focus detection signal also increases roughly in proportion to the increase in the magnitude |d| of the defocus amount d. In the back focus state (d>0), the direction of the image shift of the subject image between the first focus detection signal and the second focus detection signal is the opposite to that in the front focus state, but the same is true.

As the magnitude of the defocus amount of the imaging signal obtained by adding the first focus detection signal and the second focus detection signal, or the first focus detection signal and the second focus detection signal, increases, the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases. Therefore, in this embodiment, the phase difference AF unit 129 utilizes the relationship in which the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases as the magnitude of the defocus amount of the imaging signal increases, and converts the image shift amount into a detected defocus amount using a conversion coefficient calculated based on the baseline length.

(Flow of the photography process)
7 is a flowchart of the photographing process. This photographing process is executed by the camera MPU 125. Also, the lens MPU 117 shares part of the process as necessary.

In S701, the camera MPU 125 controls the subject detection unit 132 to perform subject detection based on the imaging data (e.g., data obtained by live view shooting) generated by the image sensor 122. The position and size of the subject (subject area) detected by the subject detection unit 132 in this embodiment will be described below.

Figure 8 is a schematic diagram showing the relationship between a subject (detection target) and a detection area (subject area). In Figure 8, subject areas 801 to 806 are subject areas detected by the subject detection unit 132. As can be seen from Figure 8, a subject area is an area that partially includes the subject to be detected (a dog in the example of Figure 8). In other words, a subject area also includes an area that does not include the subject to be detected (for example, an area such as a pole or the background). That is, when referring to "subject detection" or "detecting a subject" in this embodiment, it does not mean that an area that strictly includes only a specific subject is detected, but rather that an area that partially includes the subject to be detected is detected.

The position and size of each subject area are shown in Table 1. The subject detection unit 132 detects subject areas within the shooting range based on the subject detection signal generated by the image processing circuit 124. The subject area includes both subjects to be detected and subjects not to be detected.

In FIG. 8, reference numerals 810, 820, 830, 840, 850, and 860 indicate the background (subject not to be detected). Reference numerals 811, 821, 831, 841, 851, and 861 indicate the dog (subject to be detected). Reference numerals 812 to 817 indicate poles (subjects not to be detected), reference numerals 822, 823, and 852 indicate triangular cones (subjects not to be detected), and reference numerals 832, 833, and 853 indicate loops (subjects not to be detected) through which the dog passes. Reference numeral 842 indicates a cage (subject not to be detected) that is larger than the dog (subject to be detected), and reference numeral 862 indicates a cage (subject not to be detected) that is about the same size as the dog (subject to be detected).

In the example of FIG. 8, multiple subject regions related to the subject to be detected are detected from one subject detection signal. However, the subject detection unit 132 may detect only the subject region that is most likely to be the main subject. In addition, although a dog is detected as the subject here, other animals such as cats and birds, or vehicles such as cars and airplanes may also be detected. In addition, although the entire body of a dog is detected as the subject here, specific parts such as the face or eyes may also be detected. Also, specific parts such as the driver of a vehicle or the driver's head may also be detected.

Returning to FIG. 7, in S702, the camera MPU 125 selects a main subject from among the subjects detected in S701. The main subject is selected (determined) according to a predetermined priority order of the subject area (position, size). For example, the closer the subject area is to the central image height, the higher the priority order is set, and when the positions are the same (when the distance from the central image height is the same), the larger the subject area is, the higher the priority order is set.

In S703, the camera MPU 125 controls the phase difference AF unit 129 to perform focus adjustment (focus control). Details of focus adjustment will be described later.

In S704, the camera MPU 125 performs a focus determination. If it is determined that the focus is achieved, the process proceeds to S705, and if it is determined that the focus is not achieved, the process returns to S701.

In S705, the camera MPU 125 controls the white balance adjustment unit 131 to perform white balance adjustment (white balance control). Details of the white balance adjustment will be described later.

In S706, the camera MPU 125 controls the AE unit 130 to perform exposure adjustment (exposure control). Details of exposure adjustment will be described later.

In S707, the camera MPU 125 performs the actual shooting (taking an image for recording).

(Details of the focus adjustment process in S703)
Fig. 9 is a flowchart showing details of the focus adjustment process in S703 of Fig. 7. The process in Fig. 9 is executed by the camera MPU 125 and the phase difference AF unit 129.

In S901, the phase difference AF unit 129 obtains the subject detection result (subject area) of the main subject detected in S701 to S702.

In S902, the phase difference AF unit 129 sets a ranging area (defocus amount calculation area) based on the position and size of the subject area acquired in S901.

Figure 10 is a schematic diagram of the distance measurement areas set for each of the subject areas 801 to 806 based on the position and size of the subject area. Figure 10(a) corresponds to subject area 801, Figure 10(b) corresponds to subject area 802, Figure 10(c) corresponds to subject area 803, Figure 10(d) corresponds to subject area 804, Figure 10(e) corresponds to subject area 805, and Figure 10(f) corresponds to subject area 806. Reference numerals 1010, 1020, 1030, 1040, 1050, and 1060 indicate distance measurement areas.

In the example of FIG. 10, the phase difference AF unit 129 sets the center of the ranging area for each of the subject areas 801 to 806 based on the position of the subject area, and sets the range of the ranging area to a size three times the vertical and horizontal size of the subject area. In this embodiment, the range of the ranging area is set to a size three times the size of the subject area, but this is not limited to this, and the range of the ranging area may be set according to the size of the subject area at another magnification. In the example of FIG. 10, the ranging area is larger than the subject area and includes the entire subject area, but the ranging area may include only a part of the subject area, or the ranging area may be smaller than the subject area. Reference symbols 1001_n to 1006_n are one of the ranging points of each ranging area.

In this way, by setting the ranging area based on the position of the subject area, it is possible to obtain a high density of ranging results near the main subject, even when the number of ranging points that can be calculated simultaneously is limited. Then, by selecting the optimal ranging result from the obtained ranging results, it becomes possible to properly focus on the main subject. In addition, by setting the ranging area range with a magnification relative to the size of the subject area, it is possible to always use the results of the ranging area range with the same magnification for the subject area when determining the subject degree, improving the accuracy of determining the subject degree.

Returning to FIG. 9, in S903, the phase difference AF unit 129 calculates the defocus amount for each ranging point within the ranging area set in S902.

In S904, the camera MPU 125 classifies the defocus amount calculated in S903.

Figure 11 is a schematic diagram showing the state in which the defocus amount in the ranging area of Figure 10 has been classified. In Figure 11, the horizontal axis indicates the defocus amount, and the vertical axis indicates the frequency of the defocus amount. Figure 11(a) corresponds to ranging area 1010, Figure 11(b) corresponds to ranging area 1020, Figure 11(c) corresponds to ranging area 1030, Figure 11(d) corresponds to ranging area 1040, Figure 11(e) corresponds to ranging area 1050, and Figure 11(f) corresponds to ranging area 1060.

Reference numerals 1110, 1120, 1130, 1140, 1150, and 1160 indicate the class of the defocus amount of the backgrounds 810, 820, 830, 840, 850, and 860. Reference numerals 1111, 1121, 1131, 1141, 1151, and 1161 indicate the class of the defocus amount of the subjects 811, 821, 831, 841, 851, and 861. Reference numerals 1112 to 1117 indicate the class of the defocus amount of the poles 812 to 817. Reference numerals 1122, 1123, and 1152 indicate the class of the defocus amount of the triangular cones 822, 823, and 852. Reference numerals 1132, 1133, and 1153 indicate the class of the defocus amount of the rings 832, 833, and 853 through which the dog passes. Reference numeral 1142 indicates the class of the defocus amount for a cage 842 that is larger than a dog (the subject to be detected). Reference numeral 1162 indicates the class of the defocus amount for a cage 862 that is about the same size as a dog (the subject to be detected).

In this way, in S904, the defocus amount is classified (the multiple defocus amount ranges corresponding to the multiple distance measurement points are classified into multiple partial ranges) to divide the subject area in the optical axis direction (defocus direction).

Returning to FIG. 9, in S905, the camera MPU 125 detects class areas corresponding to each class obtained in S904 in the ranging area set in S902. A class area is a collection of pixels having defocus amounts that belong to the corresponding class. The positions of the pixels in the class area (partial area) do not need to be continuous and may be discrete.

Figure 12 is a schematic diagram of the class areas detected in S905. Figure 12(a) corresponds to the ranging area 1010, Figure 12(b) corresponds to the ranging area 1020, Figure 12(c) corresponds to the ranging area 1030, Figure 12(d) corresponds to the ranging area 1040, Figure 12(e) corresponds to the ranging area 1050, and Figure 12(f) corresponds to the ranging area 1060. The gray areas indicate the class areas corresponding to each class. In this way, the ranging area is classified into multiple class areas (partial areas), and each of the multiple class areas (partial areas) corresponds to a different defocus amount class (a different partial range in the multiple defocus amount ranges corresponding to multiple ranging points).

Reference numerals 1011, 1021, 1031, 1041, 1051, and 1061 indicate the range for normalizing the evaluation value calculated when determining the subject degree, and are set within a range β times the vertical and horizontal sizes of the subject region based on the center position of the subject region. In the example of FIG. 12, β=2.

Reference numerals 1210, 1220, 1230, 1240, 1250, and 1260 indicate class regions corresponding to classes 1110, 1120, 1130, 1140, 1150, and 1160, which correspond to backgrounds 810, 820, 830, 840, 850, and 860. Reference numerals 1211, 1221, 1231, 1241, 1251, and 1261 indicate class regions corresponding to classes 1111, 1121, 1131, 1141, 1151, and 1161, which correspond to subjects 811, 821, 831, 841, 851, and 861. Reference numerals 1212 to 1217 indicate class regions corresponding to classes 1112 to 1117, which correspond to poles 812 to 817. Reference numerals 1222, 1223, and 1252 indicate class regions corresponding to classes 1122, 1123, and 1152, which correspond to triangular cones 822, 823, and 852. Reference numerals 1232, 1233, and 1253 indicate class regions corresponding to classes 1132, 1133, and 1153, which correspond to rings 832, 833, and 853 through which the dog passes. Reference numeral 1242 indicates a class region corresponding to class 1142, which corresponds to a cage 842 that is larger than a dog (subject to be detected). Reference numeral 1262 indicates a class region corresponding to class 1162, which corresponds to a cage 862 that is about the same size as a dog (subject to be detected).

In this way, by detecting the class area in S905 (classifying the ranging area into multiple class areas), it is possible to understand how each defocus amount class is distributed within the ranging area.

Returning to FIG. 9, in S906, the camera MPU 125 determines the subject degree, which indicates the possibility that a subject to be detected exists, for each class area based on the subject area acquired in S901 and each class area detected in S905. The subject degree of each class area is determined, for example, based on the geometric relationship between each class area and the subject area (in the following example, the relationship between positional relationship and size). Alternatively, the subject degree of each class area may be determined based on the shape of each class area in addition to the geometric relationship.

The subject degree determination process will be described with reference to FIG. 15. Note that the process in FIG. 15 is executed by the camera MPU 125.

In S1501, the camera MPU 125 calculates an evaluation value for each item of position, size, and shape for each class area. The evaluation value for each item (position: E1, size: E2, shape: E3) is calculated by the following formula. In the following formula, the position of the center of gravity of the class area is represented by ( _xc , _yc ), the size of the class area normalized by the size of the subject area (maximum x coordinate, maximum y coordinate) is represented by ( _wc , _hc ), and the area of the class area is represented by _Sc . Note that the position of the center of gravity of the class area ( _xc , _yc ) is a relative position with respect to the center position of the subject area, and is normalized so that the upper right corner of the subject area is (1, 1) and the lower left corner is (-1, -1).

The position evaluation value E1 approaches 1 (100%) as the center of gravity position ( _xc , _yc ) of the class area is closer to the center of the subject area, and approaches 0 (0%) as the center of gravity position (xc, _yc ) of the class area is farther away. If the distance between the center of the subject area and the center of gravity position ( _xc , yc) of the class area exceeds β, which normalizes the evaluation value, the evaluation value E1 becomes 0 (0%). In the example of Fig. 12(a), if the center of gravity position ( _xc , _yc ) of the class area is not within the inscribed circle of the normalization range 1011, the evaluation value E1 becomes 0 (0%).

The size evaluation value E2 approaches 1 (100%) as the class area size ( _wc , _hc ) is closer to the size of the subject area, approaches 0 (0%) as it is farther away, and becomes 0 (0%) when it exceeds β, which normalizes the evaluation value. Moreover, the evaluation value E2 is calculated using the one of ( _wc , _hc ) that has the largest difference from the size of the subject area.

The shape evaluation value E3 represents the density of the class area within a rectangular area that circumscribes the class area; the denser it is, the closer it is to 1 (100%), and the sparser it is, the closer it is to 0 (0%).

Next, in S1502, the camera MPU 125 refers to the evaluation values E1 to E3 of each class area calculated in S1501, and determines the subject likelihood to be 0% for class areas having items with an evaluation value = 0%. Therefore, if even one item has an evaluation value of 0%, for example (E1 = 93%, E2 = 0%, E3 = 88%), it is determined that there is virtually no possibility that the object is the subject of the detection target, regardless of the evaluation values of the other items.

At S1503, the camera MPU 125 determines the number of class areas with an evaluation value of 50% or more for each item (excluding class areas that have been determined to have a subject degree of 0%).

In S1504, the camera MPU 125 determines whether there is only one item in which the number of class regions with an evaluation value of 50% or more is 1. If there is only one such item, processing proceeds to S1505; if not, processing proceeds to S1506. The evaluation value threshold in the determination in S1504 is not limited to 50%, and another value (e.g., 60%) may be used as the threshold.

In S1505, the camera MPU 125 determines the subject degree of each class area (excluding class areas that have been determined to have a subject degree of 0%) based on the evaluation value of the item that has one class area with an evaluation value of 50% or more. For example, the camera MPU 125 uses the evaluation value of such an item as is as the subject degree. If there is only one item that has one class area with an evaluation value of 50% or more, the class area with that item's evaluation value of 50% or more is considered to have a high probability of being the subject to be detected, and the subject degree is determined without using items other than that item.

In S1506, the camera MPU 125 determines the subject degree of each class area (excluding class areas that have already been determined to have a subject degree of 0%) based on the evaluation values of all items calculated in S1501. For example, the camera MPU 125 uses the average of the evaluation values of all items as the subject degree. Alternatively, the camera MPU 125 may determine the subject degree by taking a weighted average of the evaluation values of all items. The weighting may be set based on the predetermined importance of each item, or a larger weight may be set for an item whose evaluation value differs greatly from other items, or a larger weight may be set for an item whose evaluation value is large.

Now, with reference to Figures 12 and 15, we will explain a specific example of the subject degree determination process for the class areas corresponding to each of the subject areas 801 to 806.

First, referring to FIG. 12(a), the process of determining the subject degree for the class areas of the subject area 801 will be described. Table 2 shows the position and size of each class area normalized by the position and size of the subject area 801, the evaluation value of each item calculated for each class area in S1501, and the subject degree of each class area.

In Table 2, the evaluation value of the size of class area 1210 is 0%, and the evaluation value of the position of class area 1217 is 0%. Therefore, in S1502, the subject degree of class area 1210 and class area 1217 is determined to be 0%.

The processing from S1503 onwards is carried out on class areas 1211 to 1216 whose subject degree has not yet been determined. For the position and shape evaluation values, there are two or more class areas with evaluation values of 50% or more. For the size evaluation value, there is one class area with an evaluation value of 50% or more (class area 1211). Therefore, there is only one item in which the number of class areas with an evaluation value of 50% or more is one (size evaluation value). Therefore, for class areas 1211 to 1216, the size evaluation value is determined as the subject degree in S1505.

In this way, the subject likelihood of the class area 1211 corresponding to the subject being detected can be correctly determined to be high.

Next, the process of determining the subject degree for the class areas of the subject area 802 will be described with reference to FIG. 12(b). Table 3 shows the position and size of each class area normalized by the position and size of the subject area 802, the evaluation value of each item calculated for each class area in S1501, and the subject degree of each class area.

In Table 3, the evaluation value of the size of class area 1220 is 0%. Therefore, in S1502, the subject degree of class area 1220 is determined to be 0%.

The processing from S1503 onwards is carried out for class areas 1221 to 1223 whose subject degree has not yet been determined. For the size evaluation value, there are two or more class areas with an evaluation value of 50% or more. For the shape evaluation value, there are no class areas with an evaluation value of 50% or more. For the position evaluation value, there is one class area with an evaluation value of 50% or more (class area 1221). Therefore, there is only one item in which the number of class areas with an evaluation value of 50% or more exists (position evaluation value). Therefore, for class areas 1221 to 1223, the position evaluation value is determined as the subject degree in S1505.

In this way, the subject likelihood of the class area 1221 corresponding to the subject being detected can be correctly determined to be high.

Next, the process of determining the subject degree for the class areas of the subject area 803 will be described with reference to FIG. 12(c). Table 4 shows the position and size of each class area normalized by the position and size of the subject area 803, the evaluation value of each item calculated for each class area in S1501, and the subject degree of each class area.

In Table 4, the evaluation value of the size of class area 1230 is 0%. Therefore, in S1502, the subject degree of class area 1230 is determined to be 0%.

The processing from S1503 onwards is carried out on class areas 1231 to 1233 whose subject degree has not yet been determined. For the position and size evaluation values, there are two or more class areas with evaluation values of 50% or more. For the shape evaluation value, there is one class area with an evaluation value of 50% or more (class area 1231). Therefore, there is only one item in which the number of class areas with an evaluation value of 50% or more is one (shape evaluation value). Therefore, for class areas 1231 to 1233, the shape evaluation value is determined as the subject degree in S1505.

In this way, the subject likelihood of the class area 1231 corresponding to the subject being detected can be correctly determined to be high.

Next, the process of determining the subject degree for the class area of the subject area 804 will be described with reference to FIG. 12(d). Table 5 shows the position and size of each class area normalized by the position and size of the subject area 804, the evaluation value of each item calculated for each class area in S1501, and the subject degree of each class area.

In Table 5, the evaluation value of the size of class area 1240 and class area 1242 is 0%. Therefore, in S1502, the subject degree of class area 1240 and class area 1242 is determined to be 0%.

The processing from S1503 onwards is performed on class areas 1241 whose subject degree has not been determined. For the shape evaluation value, there are no class areas with an evaluation value of 50% or more. For the position and size evaluation values, there is one class area with an evaluation value of 50% or more (class area 1241). Therefore, there are two items for which there is one class area with an evaluation value of 50% or more (position and size evaluation values). Therefore, for class area 1241, the average evaluation value of all items is determined as the subject degree in S1505.

In this way, the subject likelihood of the class area 1241 corresponding to the subject being detected can be correctly determined to be high.

Next, the process of determining the subject degree for the class areas of the subject area 805 will be described with reference to FIG. 12(e). Table 6 shows the position and size of each class area normalized by the position and size of the subject area 805, the evaluation value of each item calculated for each class area in S1501, and the subject degree of each class area.

In Table 6, the evaluation value of the size of class area 1250 is 0%. Therefore, in S1502, the subject degree of class area 1250 is determined to be 0%.

The processing from S1503 onwards is carried out on class areas 1251 to 1253 whose subject degree has not yet been determined. For the evaluation values of position and size, there are two or more class areas with evaluation values of 50% or more. For the evaluation value of shape, there are no class areas with evaluation values of 50% or more. Therefore, there is no item in which the number of class areas with an evaluation value of 50% or more is one. Therefore, for class areas 1251 to 1253, the average evaluation value of all items is determined as the subject degree in S1505.

In this way, the subject likelihood of the class area 1251 corresponding to the subject being detected can be correctly determined to be high.

Next, the process of determining the subject degree for the class areas of the subject area 806 will be described with reference to FIG. 12(f). Table 7 shows the position and size of each class area normalized by the position and size of the subject area 806, the evaluation value of each item calculated for each class area in S1501, and the subject degree of each class area.

In Table 7, the evaluation value of the size of class area 1260 is 0%. Therefore, in S1502, the subject degree of class area 1260 is determined to be 0%.

The processing from S1503 onwards is carried out on class areas 1261 to 1262 whose subject degree has not yet been determined. For the evaluation values of all items, the number of class areas with evaluation values of 50% or more is two or more. Therefore, there is no item for which the number of class areas with evaluation values of 50% or more is one. Therefore, for class areas 1261 to 1262, the average evaluation value of all items is determined as the subject degree in S1505.

In this way, the subject likelihood of the class area 1261 corresponding to the subject being detected can be correctly determined to be high.

By determining the subject degree in the above manner, it becomes possible to identify with high accuracy the class area corresponding to the subject to be detected, even if a subject other than the subject to be detected is present within the subject area. Note that in the above explanation, the subject degree of the class area was determined, but it is also possible to determine the subject degree of the class to which the class area belongs.

Returning to FIG. 9, in S907, the phase difference AF unit 129 selects a ranging point based on the subject degree determined in S906. If there is a ranging point in the class area with the highest subject degree whose reliability of the ranging result (reliability of the defocus amount) is higher than a predetermined threshold, the phase difference AF unit 129 selects the ranging point closest to the center of the subject area. In other words, the phase difference AF unit 129 selects a first defocus amount (e.g., the defocus amount of a ranging point close to the center of the subject area) having a reliability equal to or higher than the reliability threshold from among multiple defocus amounts (first multiple defocus amounts) corresponding to multiple ranging points in the class area with the highest subject degree.

If there is no ranging point with reliability higher than a predetermined threshold in the class area with the highest subject degree, the phase difference AF unit 129 selects in the class area with the second highest subject degree. That is, if there is a ranging point with reliability higher than a predetermined threshold in the class area with the second highest subject degree, the phase difference AF unit 129 selects the ranging point closest to the center of the subject area. In other words, the phase difference AF unit 129 selects a second defocus amount (e.g., the defocus amount of a ranging point close to the center of the subject area) with reliability equal to or higher than the reliability threshold from among multiple defocus amounts (second multiple defocus amounts) corresponding to multiple ranging points in the class area with the next highest subject degree after the highest subject degree.

If there is no ranging point with a reliability higher than the predetermined threshold in the class area with the second highest subject degree, then similarly, a ranging point (defocus amount) is selected in the class area with the third or higher subject degree.

When there are multiple class areas with the same subject degree, the phase difference AF unit 129 selects a ranging point from a class area that is not the closest. In a situation where the subject degree is equal, there is a high possibility that an obstacle is in front of the subject to be detected (on the close side), and since it is considered that the closest class area is likely to correspond to an obstacle, the priority of the ranging point selection is lowered. Also, when there are multiple class areas with the same subject degree in class areas that are not the closest, the phase difference AF unit 129 searches for a selectable ranging point (ranging point whose reliability is higher than a predetermined threshold) in order from the class area whose representative defocus amount for the class is closest to the close side.

In this way, by preferentially selecting ranging points from areas with a high degree of subjectivity, it is possible to select ranging points in areas that are highly likely to be the subject of the detection target. Therefore, even if the subject area includes a subject that is not the subject of the detection target, it is possible to perform appropriate focus control on the subject of the detection target.

In S908, the phase difference AF unit 129 calculates the focus lens drive amount based on the defocus amount detected at the ranging point selected in S908. In the explanation of S907 and S908, it was assumed that in a class area with a high degree of subjectivity, a ranging point close to the center of the subject area is selected from among ranging points with reliability higher than a predetermined threshold, and the defocus amount of the selected ranging point is used. However, this embodiment is not limited to this configuration, and a representative defocus amount of the class to which the class area with a high degree of subjectivity belongs may be used. The representative defocus amount is, for example, the average defocus amount or central defocus amount of the class.

In S909, the phase difference AF unit 129 performs control to drive the focus lens 104 based on the focus lens drive amount calculated in S908.

(Details of the white balance adjustment process in S705)
Fig. 13 is a flowchart showing details of the white balance adjustment process in S705 of Fig. 7. Note that the process in Fig. 13 is executed by the white balance adjustment unit 131 controlled by the camera MPU 125.

In S1301, the white balance adjustment unit 131 obtains the subject detection result (subject area) of the main subject detected in S701 to S702.

In S1302, the white balance adjustment unit 131 obtains the subject degree of each class area determined in S906.

In S1303, the white balance adjustment unit 131 calculates the white balance based on the subject degree of each class area acquired in S1302. The white balance adjustment unit 131 calculates the white balance for each class area, and calculates the current white balance by weighting the white balance of each class area with the subject degree acquired in S1302. The weight used in the white balance calculation may be the subject degree value as it is. Therefore, the white balance adjustment (white balance control) here includes a calculation in which each of the multiple class areas is treated as a target class area and the pixel value of the target class area is weighted by the subject degree of the target class area. Shooting control using such weighting calculation can be similarly applied to the focus adjustment (focus control) described above and the exposure adjustment (exposure control) described below.

The method of calculating the white balance is not particularly limited, and for example, the white balance adjustment unit 131 may calculate the white balance of the class area with the highest subject degree as the current white balance. In other words, the white balance adjustment unit 131 may perform white balance control based only on the class area with the highest subject degree among multiple class areas. Such shooting control based only on the class area with the highest subject degree can be similarly applied to the focus adjustment (focus control) described above and the exposure adjustment (exposure control) described below.

In addition, if there are multiple class areas with the same subject degree, if there is a closest class area among them, the weight of the closest class area may be set to the smallest, and for the remaining class areas, the weight may be set to a larger value the closer they are to the closest side.

In S1304, the white balance adjustment unit 131 obtains a predetermined appropriate white balance.

In S1305, the white balance adjustment unit 131 calculates the difference between the current white balance calculated in S1303 and the appropriate white balance obtained in S1304, and adjusts the white balance for the imaging data (imaging signal).

(Details of Exposure Adjustment Processing in S706)
Fig. 14 is a flowchart showing details of the exposure adjustment process in S706 of Fig. 7. Note that the process in Fig. 14 is executed by the AE unit 130 controlled by the camera MPU 125.

In S1401, the AE unit 130 obtains the subject detection result (subject area) of the main subject detected in S701 and S702.

In S1402, the AE unit 130 obtains the subject degree of each class area determined in S906.

In S1403, the AE unit 130 calculates the exposure by weighting the photometry results of each class area based on the subject degree of each class area acquired in S1402. The weight used in the exposure calculation may be 1 for the class area with the highest subject degree and 0 for the other class areas. In addition, if there are multiple class areas with the same subject degree, and if there is a closest class area among them, the weight of the closest class area may be set to the smallest, and for the remaining class areas, the weight may be set to a larger value the closer they are to the closest side. In addition, the same weight may be used for the white balance calculation (S1303) and the exposure calculation (S1403), or different weights may be used.

In S1404, the AE unit 130 obtains a predetermined appropriate exposure amount.

In S1405, the AE unit 130 calculates the difference between the exposure amount calculated in S1403 and the exposure amount obtained in S1404, and adjusts the exposure conditions during shooting.

As described above, according to the first embodiment, the imaging device 10 detects a subject area that partially includes a subject to be detected in the shooting range, and acquires multiple defocus amounts corresponding to multiple distance measurement points in the distance measurement area that includes the subject area. Then, the imaging device 10 classifies the distance measurement area into multiple class areas based on the multiple defocus amounts. Each of the multiple class areas corresponds to a different defocus amount class (a different partial range in the range of the multiple defocus amounts). The imaging device 10 determines the subject degree for each of the multiple class areas based on a geometric relationship (e.g., at least one of a positional relationship and a size relationship) between each of the multiple class areas and the subject area. Then, the imaging device 10 performs shooting control based on the multiple class areas so that the contribution of a class area having a first subject degree is greater than the contribution of a class area having a second subject degree that is smaller than the first subject degree. Specific examples of such shooting control include focus adjustment (focus control) shown in FIG. 9, white balance adjustment (white balance control) shown in FIG. 12, and exposure adjustment (exposure control) shown in FIG. 14, but the shooting control of this embodiment is not limited to these specific examples.

With the above-described configuration, this embodiment makes it possible to improve the accuracy of shooting control based on a subject detection area that partially includes the subject to be detected.

[Other embodiments]
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

The invention is not limited to the above-described embodiment, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to disclose the scope of the invention.

10...imaging device, 100...lens unit, 120...camera body, 122...imaging element, 125...camera MPU, 129...phase difference AF section, 130...AE section, 131...white balance adjustment section, 132...subject detection section

Claims (13)

a detection means for detecting a subject area that partially includes a subject to be detected within a shooting range;
an acquisition means for acquiring a plurality of defocus amounts corresponding to a plurality of distance measuring points within a distance measuring area including the subject area;
a classification means for classifying the distance measurement area into a plurality of partial areas based on the plurality of defocus amounts, each of the plurality of partial areas corresponding to a different partial range in a range of the plurality of defocus amounts;
a determination means for determining a subject degree indicating a possibility that the detection target subject exists for each of the plurality of partial regions based on a geometric relationship between each of the plurality of partial regions and the subject region;
a control means for controlling photography based on the plurality of partial regions, the control means performing the photography control such that a contribution of a partial region having a first subject degree is greater than a contribution of a partial region having a second subject degree smaller than the first subject degree;
Equipped with
The photographing control includes a focus control,
the control means performs the focus control based on a representative defocus amount obtained from among the plurality of defocus amounts, the representative defocus amount being obtained from a defocus amount corresponding to a partial region having the first subject degree. The imaging control device according to claim 1 , wherein the geometric relationship between each of the plurality of partial regions and the subject region includes at least one of a positional relationship and a size relationship. 3. The imaging control device according to claim 1, wherein the determining means determines the subject degree of each of the plurality of partial regions based on a shape of each of the plurality of partial regions in addition to a geometric relationship between each of the plurality of partial regions and the subject region. The determination means is
calculating a first evaluation value for each of the plurality of partial regions based on a positional relationship between each of the plurality of partial regions and the subject region;
calculating a second evaluation value for each of the plurality of partial regions based on a size relationship between each of the plurality of partial regions and the subject region;
calculating a third evaluation value for each of the plurality of partial regions based on a shape of each of the plurality of partial regions;
The imaging control device according to claim 1 , further comprising: determining a subject degree of each of the plurality of partial regions based on the first evaluation value, the second evaluation value, and the third evaluation value of each of the plurality of partial regions. The photographing control device according to claim 1 , wherein the distance measurement area is larger than the subject area and includes the entire subject area. 6. The photographing control device according to claim 1, wherein the control means performs the photographing control based only on the partial region having the highest subject degree among the plurality of partial regions. The photographing control device according to any one of claims 1 to 5, characterized in that the photographing control includes a calculation for weighting the pixel values or photometry results of each of the plurality of partial regions as a target partial region by a degree of subjectivity of the target partial region. The photographing control device according to claim 1 , wherein the photographing control further includes at least one of white balance control and exposure control. a detection means for detecting a subject area that partially includes a subject to be detected within a shooting range;
an acquisition means for acquiring a plurality of defocus amounts corresponding to a plurality of distance measuring points within a distance measuring area including the subject area;
a classification means for classifying the distance measurement area into a plurality of partial areas based on the plurality of defocus amounts, each of the plurality of partial areas corresponding to a different partial range in a range of the plurality of defocus amounts;
a determination means for determining a subject degree indicating a possibility that the detection target subject exists for each of the plurality of partial regions based on a geometric relationship between each of the plurality of partial regions and the subject region;
a control means for controlling photography based on the plurality of partial regions, the control means performing the photography control such that a contribution of a partial region having a first subject degree is greater than a contribution of a partial region having a second subject degree smaller than the first subject degree;
Equipped with
The photographing control includes a focus control,
the control means selects one of the plurality of defocus amounts based on the subject degree of each of the plurality of partial regions determined by the determination means, and performs the focus control based on the selected defocus amount;
The control means
selecting a first defocus amount having a reliability equal to or greater than a reliability threshold from among a plurality of first defocus amounts corresponding to a plurality of distance measuring points in a partial area having a highest subject degree among the plurality of partial areas;
The imaging control device performs the focus control based on the first defocus amount. When the first plurality of defocus amounts does not include a defocus amount having a reliability equal to or greater than the reliability threshold, the control unit
selecting a second defocus amount having a reliability equal to or greater than the reliability threshold from among a plurality of second defocus amounts corresponding to a plurality of ranging points in a partial region having a subject degree next to the highest subject degree among the plurality of partial regions;
The imaging control device according to claim 9 , wherein the focus control is performed based on the second defocus amount. The imaging control device according to any one of claims 1 to 10,
An imaging means;
An imaging device comprising: A photographing control method executed by a photographing control device,
a detection step of detecting a subject area that partially includes a subject to be detected within a shooting range;
acquiring a plurality of defocus amounts corresponding to a plurality of distance measuring points within a distance measuring area including the subject area;
a classification step of classifying the ranging area into a plurality of partial areas based on the plurality of defocus amounts, each of the plurality of partial areas corresponding to a different partial range in the range of the plurality of defocus amounts;
a determination step of determining a subject degree indicating a possibility that the detection target subject exists for each of the plurality of partial regions based on a geometric relationship between each of the plurality of partial regions and the subject region;
a control step of performing photography control based on the plurality of partial regions, the photography control being performed such that a contribution of a partial region having a first subject degree is greater than a contribution of a partial region having a second subject degree smaller than the first subject degree;
Equipped with
The photographing control includes a focus control,
the control step performs the focus control based on a representative defocus amount obtained from a defocus amount corresponding to a partial region having the first subject degree among the plurality of defocus amounts. A program for causing a computer to function as each of the means of the imaging control device according to any one of claims 1 to 10.

Images