JP7765200B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

There is technology that changes exposure conditions for each region within the image sensor of an imaging device. Patent Document 1 discloses an imaging device that allows exposure conditions (exposure time and analog gain settings) to be changed for each region of the image sensor.

In addition, contrast AF is often used in autofocus (hereinafter referred to as AF) for imaging devices. In contrast AF, the focus lens, which is part of the imaging optical system and affects focus adjustment, is moved throughout the focus adjustment range while capturing an image, and high-frequency components are extracted from the output image signal within a partial area, allowing a contrast evaluation value for focusing to be calculated sequentially. The position of the focus lens where this contrast evaluation value is maximized is then set as the in-focus position.

JP 2010-136205 A

However, with the technology described in Patent Document 1, exposure conditions are set for each exposure area, which can result in areas within an image that are set to high gain (high gain areas) and areas that are set to low gain (low gain areas). In this case, an exposure step occurs at the boundary between the high gain and low gain areas, where the gain changes sharply. If this exposure step is included in the area referenced by AF, the edge of the exposure step will unintentionally increase the contrast evaluation value, making it difficult to accurately focus on the plane.

The present invention aims to improve the autofocus accuracy in an image sensor that allows exposure conditions to be set for each area.

To achieve the object of the present invention, for example, an information processing apparatus according to one embodiment has the following configuration: That is, an information processing apparatus that performs autofocusing on an imaging device capable of setting exposure conditions for each partial region of a light-receiving surface of an image sensor, comprising: first setting means for setting, on the light-receiving surface, a first partial region for capturing an image under first exposure conditions and a second partial region adjacent to the first partial region for capturing an image under second exposure conditions different from the first exposure conditions, second setting means for setting, as an area to be used for the autofocusing, a third partial region that does not include an exposure step that is a boundary between the first partial region and the second partial region, control means for controlling the autofocusing of the imaging device in accordance with image contrast based on an image signal of the third partial region, and display control means for displaying the area to be used for the autofocusing that has been set by the second setting means so as not to include the exposure step .

Improves autofocus accuracy in image sensors that allow exposure conditions to be set for each area.

FIG. 1 is a diagram showing an example of the functional configuration of an imaging system including an imaging apparatus according to a first embodiment. 5A and 5B are diagrams showing examples of images in which exposure differs for each region of the imaging device according to the first embodiment. 6 is a flowchart showing an example of information processing performed by the imaging device according to the first embodiment. 5A and 5B are diagrams showing an example of a reset display of an AF area of the imaging device according to the first embodiment. 5A and 5B are diagrams showing an example in which an AF area of the imaging device according to the first embodiment is set without any problems. 10 is a flowchart showing an example of information processing performed by an imaging apparatus according to a second embodiment. 10A and 10B are diagrams for explaining correction of an AF area by an imaging apparatus according to a second embodiment. 10A and 10B are diagrams showing an example of an AF area before and after correction by an imaging apparatus according to a second embodiment. 10A and 10B are diagrams showing another example of an AF area before and after correction by the imaging apparatus according to the second embodiment. 10A and 10B are diagrams showing another example of an AF area before and after correction by the imaging apparatus according to the second embodiment. 11 is a flowchart showing an example of information processing by an imaging device according to a third embodiment. 10A and 10B are diagrams showing an example of setting a plurality of AF areas by an imaging apparatus according to a third embodiment. 5A and 5B are diagrams illustrating how an AF area is set based on a subject position by the imaging apparatus according to the first embodiment. 10A and 10B are diagrams illustrating how an AF area is set based on a subject position by an imaging apparatus according to a third embodiment. FIG. 10 is a diagram showing the functional configuration of a computer according to a fourth embodiment.

The following describes the embodiments in detail with reference to the attached drawings. Note that the following embodiments do not limit the scope of the claimed invention. While the embodiments describe multiple features, not all of these features are necessarily essential to the invention, and multiple features may be combined in any desired manner. Furthermore, in the attached drawings, the same reference numbers are used to designate identical or similar components, and redundant explanations will be omitted.

[Embodiment 1]
FIG. 1 is a block diagram showing an example of an imaging system including an imaging device, which is an information processing device according to this embodiment. The imaging device 100 is an imaging device that captures images, such as a digital camera, or a smartphone or tablet device equipped with a camera. In this embodiment, each process performed by the imaging device 100 is performed by a processor built into the imaging device 100. The client device 110 is a device that allows a user to check images captured by the imaging device 100 and operate the imaging device, such as a personal computer, smartphone, or tablet device. In this embodiment, the imaging device 100 and the client device 110 are described as separate devices, but these devices may also be implemented as the same device.

The client device 110 according to this embodiment includes an input unit 111, a display unit 112, an operation unit 113, and an output unit 114. The display unit 112 is, for example, an LCD display, and displays images captured by the imaging device 100 and UIs for performing processing by each functional unit. The operation unit 113 is composed of, for example, a keyboard and mouse, or a touch panel, and accepts input from the user. The user can perform various inputs to the imaging device 100, such as setting the AF area described below, via the operation unit 113. The input unit 111 acquires various information from the imaging device 100, such as captured images, and the output unit 114 transmits various information, such as the AF area setting, to the imaging device 100.

The imaging device 100 according to this embodiment includes an image sensor unit 101, a lens 102, an image processing unit 103, an output unit 104, a calculation unit 105, a lens control unit 106, a condition setting unit 107, and an input unit 108. The imaging device 100 focuses image light incident on the lens 102 on the light receiving surface of the image sensor unit 101, which is an image sensor, and outputs image data. The image processing unit 103 can convert the image data output by the image sensor unit 101 into a format readable by the client device 110. The output unit 104 transmits each piece of data generated by the imaging device 100 to the client device 110. In this embodiment, the output unit 104 can output the image data converted by the image processing unit 103 to the client device 110.

The condition setting unit 107 sets exposure conditions for the image sensor unit 101 for each partial region of the light-receiving surface. Here, exposure conditions (hereinafter also referred to as exposure conditions) refer to setting values related to the brightness of image data, such as the exposure time (shutter speed) or gain (analog gain or digital gain) value of the image sensor for each pixel. The image sensor unit 101 according to this embodiment forms an image of image light based on the exposure conditions set by the condition setting unit 107. The condition setting unit 107 can set exposure conditions for each pixel of the light-receiving surface, but it may also set exposure conditions for each unit region 203 obtained by equally dividing the light-receiving surface of the image sensor unit 101 (here, divided into 16 equal parts, 4 x 4), corresponding to, for example, the screen 201 in Figure 2 described below. The following description will be given assuming that the condition setting unit 107 sets exposure conditions for each partial region of the light-receiving surface.

The condition setting unit 107 also sets an exposure step between areas with different exposure conditions. An exposure step in this embodiment refers to the boundary between adjacent areas on the light-receiving surface that have different exposure conditions. Edges are likely to appear in exposure steps due to differences in brightness based on differences in exposure conditions. For this reason, the condition setting unit 107 sets a partial area on the light-receiving surface that does not include an exposure step as an area where pixel values are referenced in contrast AF (AF area). Note that the exposure step refers to the boundary between pixel areas with different exposure conditions (i.e., not a pixel area), but to more reliably exclude edge areas from the AF area, a pixel area, such as an area several pixels wide centered on the boundary, may be set as the exposure step.

The imaging device 100 performs contrast AF according to the contrast of the image based on the pixel values of the AF area. Specifically, the imaging device 100 captures an image while adjusting the position of the lens 102 over the focus adjustment range, extracts high-frequency components from the image signal within the AF area, and sequentially calculates a contrast evaluation value for focusing (hereinafter referred to as the contrast evaluation value). The imaging device 100 then sets the position of the lens 102 where the sequentially calculated contrast evaluation value is maximized as the focus position.

The lens control unit 106 controls the position of the lens 102 for contrast AF. The image processing unit 103 acquires image data from the image sensor unit 101 for each position of the lens 102 and transmits the output image signal within the AF area to the calculation unit 105. The calculation unit 105 calculates a contrast evaluation value for the chorus from the output image signal of the AF area for each position of the lens 102. The calculation unit 105 then sets the position of the lens 102 where the calculated contrast evaluation value is maximized as the focus position, and the lens control unit 106 controls the lens position to that focus position.

In this embodiment, autofocus is performed based on the contrast evaluation value of the output image signal of the AF area. The contrast evaluation value is an intra-image parameter used to evaluate the level of contrast within the AF area. Any parameter used in general contrast evaluation methods may be used as the contrast evaluation value, but in the following explanation of this embodiment, the contrast evaluation value will be described as the sum of the high-frequency components of the output image signal of the AF area.

Figure 2 shows an example of image data displayed on the UI of the client device 110 based on exposure conditions set by the condition setting unit 107, which is transmitted to the client device 110 by the output unit 104 according to this embodiment and displayed to the user. In this embodiment, an analog gain value (hereinafter simply referred to as gain) is set as the exposure condition, but as described above, different conditions such as exposure time may also be used. In the example of Figure 2, the condition setting unit 107 sets exposure conditions for each of the unit areas 203 obtained by dividing the screen 201 into 16 parts, as described above, and the screen 201 is displayed in a color-coded manner according to the exposure conditions, with the color becoming darker as the gain value decreases. Each of areas 204 to 207 is an area in which the same gain is set (hereinafter referred to as a gain area).

The condition setting unit 107 sets exposure steps 210 at the boundaries of each of the gain areas 204-207. In the example of Figure 2, the condition setting unit 107 sets area 202 on screen 201 as the AF area, and this area 202 includes exposure steps 210 within it. When performing contrast AF by referencing an area that includes an exposure step, the calculated contrast evaluation value often becomes unintentionally high due to the edges of the exposure step. Therefore, by performing contrast AF by setting an AF area so that it does not include the exposure step, it is possible to improve AF focusing accuracy. Note that in the screens shown in Figures 4-5, 7-10, and 12-14, which will be described later, the same exposure conditions as in Figure 2 are set for each area on the screen.

Here, the condition setting unit 107 may set all boundaries between regions with different exposure steps as exposure steps, or may set a condition for only those boundaries that satisfy a condition to be set as exposure steps. For example, the condition setting unit 107 may take into account cases where the gain difference is slight and the resulting edges are negligible, and set the boundary between those regions as an exposure step only when the difference in the set gain values is equal to or greater than a predetermined threshold. This setting makes it possible to omit setting unnecessary exposure steps and reduce the processing load. In the following explanation, when two regions are described as having "the same exposure conditions," it is understood to include cases where the exposure condition settings between the regions are the same, as well as cases where the difference in those settings does not satisfy the condition for setting an exposure step.

In this example, the user can set an AF area on the screen 201 via the UI 208 displayed on the display unit 112 of the client device 110. The condition setting unit 107 may set the AF area, for example, in response to a user touch operation or a mouse click operation. Here, if the user input causes an AF area that includes an exposure step within the area, the condition setting unit 107 may notify the user of this and prompt the user to reset the AF area, or may set the AF area so that it does not include an exposure step while following the user input. Furthermore, an AF setting frame 109 is displayed on the screen 201, and the user can select whether to set the AF area manually or automatically (auto) by operating the AF setting frame 109. The AF area may be displayed on the screen 201 with a frame (e.g., a dotted line, a solid line, or a blinking line) for a predetermined period of time after the user input, depending on the user input. Alternatively, the AF area may be displayed with a border until the AF area is set, or only AF may be performed without displaying the AF area.

When the AF area is set automatically, the condition setting unit 107 sets a partial area on the light-receiving surface that is automatically acquired so as not to include exposure gaps as the AF area. In this case, the condition setting unit 107 may set the AF area from a region of interest, or may set a partial area randomly acquired from the entire light-receiving surface as the AF area. In this embodiment, the region of interest may be a region specified by the user, or may be a region in which a subject to be tracked is located if the imaging device 100 has a function for tracking a subject in a captured image, and is not particularly limited as long as it is a region in which focusing is expected to occur.

When the imaging device 100 has a subject tracking function and the subject being tracked is located on an exposure step, the condition setting unit 107 can set an AF area based on the tracking position of the subject. For example, the condition setting unit 107 may set the tracking position of the subject as the AF area, or may set all or part of the gain area in which the subject was located immediately before moving onto the exposure step as the AF area. The condition setting unit 107 may also estimate the gain area to which a subject residing on an exposure step will next move, and set all or part of the estimated gain area as the AF area. Here, the condition setting unit 107 can estimate the gain area to which the subject will next move from among multiple gain areas in which the subject is located across the exposure step, using known tracking technology. The process for manually setting the AF area will be described later.

Figure 13 is a diagram illustrating an AF area determined by the imaging device 100 tracking a subject according to the tracking position of the subject. In this figure, a pull-down list for selecting whether or not to perform tracking is displayed on the UI 208, and the user selects a tracking mode. In Figure 13, a subject 1301 moving in the direction from gain area 206 to gain area 204 is located within a unit area that makes up gain area 206. Therefore, the condition setting unit 107 sets an AF area 1302 within the unit area of gain area 206 where subject 1301 is located.

The following describes, with reference to Figures 3 to 5, the AF area setting process performed by the imaging device 100 according to this embodiment when capturing an image including at least two partial areas with different exposure conditions. Note that in this example, as described above, at least two partial areas with different exposure conditions are set, but if the exposure conditions for the entire light-receiving surface are set to the same, normal contrast AF is performed (i.e., no exposure step is set).

Figure 3 is a flowchart showing an example of processing by the imaging device 100 according to this embodiment. In S301, the condition setting unit 107 sets the exposure conditions of the imaging element unit 101 for each partial area of the light receiving surface.

In S302, the condition setting unit 107 determines whether exposure conditions have been set for the entire area of the light-receiving surface of the image sensor. If so, processing proceeds to S303; if not, processing returns to S301.

In S303, the condition setting unit 107 sets areas on the light receiving surface that have the same exposure conditions. In the example of Figure 2, the condition setting unit 107 sets areas 204 to 207 as having the same exposure conditions.

In S304, the condition setting unit 107 sets exposure steps based on each region with the same exposure conditions. In the example of Figure 2, the condition setting unit 107 sets the boundaries between regions 204 to 207 as exposure steps 210. As described above, if the exposure conditions between regions are smaller than a predetermined threshold, the condition setting unit 107 does not need to set exposure steps at those boundaries.

In S305, the condition setting unit 107 determines whether the AF area is to be set automatically or manually. In this example, the condition setting unit 107 makes this determination by referring to the user input into the AF setting frame 209 in Figure 2, but for example, the AF area may be set manually (or automatically) as an initial setting. If the AF area is to be set automatically, processing proceeds to S306, and if it is to be set manually, processing proceeds to S307.

In S306, the condition setting unit 107 automatically sets the AF area so that it does not include the exposure step, and the process proceeds to S311. In step S307, the condition setting unit 107 presents the exposure step to the user, and the process proceeds to S308. The condition setting unit 107 may present the exposure step 210 by displaying it as a dotted line, as shown in FIG. 2, for example, but the method of presentation is not particularly limited as long as it is a display format that is recognizable to the user, such as displaying it as a line of a predetermined color (e.g., red). Note that S307 may be omitted, and the process may proceed to the AF area setting process without displaying the exposure step.

In S308, the condition setting unit 107 acquires user input from the client device 110 and sets the AF area based on the acquired input. In S309, the condition setting unit 107 determines whether the AF area set in S308 includes an exposure step within the area. If an exposure step is included, processing proceeds to S310; if not, processing proceeds to S311.

In S310, the condition setting unit 107 notifies the user that the AF area to be set contains an exposure gap, and prompts the user to reset the AF area. Figure 4 is a diagram showing an example of a message displayed in S310. In the example of Figure 4, a message prompting the user to reset the AF area, stating "Please reset the AF area," is displayed on the UI 208. Note that the method by which the condition setting unit 107 prompts the user to reset the AF area is not limited to this; for example, a sound notification may be used, or a display indicating that the setting failed (e.g., an X display) may be displayed, returning the user to the setting screen. Next, processing returns to S308, and the processing of S308 to S310 is repeated until the AF area is set within an area with the same exposure conditions, as shown in Figure 5. Figure 5 is a diagram showing a case in which the AF area 501 has been set within the area 206 with the same exposure conditions on the screen of Figure 2.

In S311, the calculation unit 105 calculates a contrast evaluation value from the output image signal of the AF area, and the lens control unit 106 drives the lens 102 to a position where the calculated contrast evaluation value is highest.

This type of processing allows the AF area to be set while avoiding the position of the exposure step. Therefore, it is possible to avoid high-frequency components due to the edges of the exposure step position, which can cause noise in the calculation of the AF contrast evaluation value, thereby improving focusing accuracy.

[Embodiment 2]
The imaging device according to the first embodiment sets an AF area so as not to include an exposure step, and performs AF by calculating a contrast evaluation value from the output image signal of the AF area. The imaging device according to this embodiment acquires a user's designation of an AF area and sets the AF area by correcting the designated area. Here, if the designated area includes an exposure step, the condition setting unit 107 sets the corrected area so as not to include the exposure step as the AF area. In this embodiment, the imaging device 100 and the client device 110 have the same configuration as those shown in FIG. 1. Furthermore, the exposure step setting process and the processes subsequent to the AF area setting are performed in the same manner as in the first embodiment, and therefore redundant description will be omitted.

In this embodiment, the user inputs a designation for the AF area via the operation unit 113 of the client device 110. Next, the condition setting unit 107 sets an AF area that does not include an exposure step based on the AF area (designated area) designated by the user input. Here, if the designated area does not include an exposure step, the condition setting unit 107 may set the designated area as the AF area as is. On the other hand, if the designated area includes an exposure step, the condition setting unit 107 sets the AF area by correcting the designated area so that an exposure step is not included within the area.

The following description will basically assume that the specified area includes an exposure step. The condition setting unit 107 can extract an area on the light receiving surface of the image sensor where the specified area and a gain area overlap, and set the extracted area as an AF area. Furthermore, if the specified area has an area that overlaps with two or more gain areas, the condition setting unit 107 may set the area that overlaps with one of those gain areas as the AF area, or may set the areas that overlap with each of multiple gain areas as AF areas. The case where multiple AF areas are set will be described in detail in embodiment 3. Furthermore, here, the condition setting unit 107 may set the entire area where the specified area and the gain area overlap as the AF area, or may set an area that excludes a predetermined number of pixels from the boundary that constitutes the exposure step as the AF area to reduce the possibility of being affected by the exposure step.

Figure 7 is a diagram illustrating an example in which the condition setting unit 107 according to this embodiment sets an AF area based on a specified area. In Figure 7, AF area 701 is set as the AF area. Here, area 202 in Figure 2 is specified as the specified area, and area 202 includes exposure step 210. Therefore, the condition setting unit 107 sets area 702, where area 202 and gain area 204 overlap, as the AF area.

Furthermore, the condition setting unit 107 may set an overlapping area between a specified area and a gain area as an AF area if that area satisfies a predetermined condition. An example of setting an overlapping area as an AF area that satisfies a condition will be described below. For example, the condition setting unit 107 may set an overlapping area between a specified area and a gain area as an AF area if the overlapping area has an area equal to or greater than a threshold (note that the area here refers to the area on the light receiving surface). The area threshold set here is not particularly limited. For example, the area threshold may be set as desired by the user, or may be equal to or greater than half the area of the gain area, or may be the average area of the overlapping areas between each gain area and the specified area (area of the specified area / number of gain areas that overlap with the specified area). In this case, if the area of any of the overlapping areas between the specified area and the gain area is less than the threshold, the condition setting unit 107 notifies the user to urge them to reset the AF area. By ensuring sufficient area for the AF area, it is possible to improve the reliability of the contrast evaluation value calculated from the AF area.

Furthermore, for example, when there are multiple areas where the designated area and the gain area overlap, the condition setting unit 107 may set the area with the largest area as the AF area. Figure 8 is a diagram illustrating an example in which the area with the largest area is set as the AF area. Figure 8(a) is a diagram showing the designated area 801 indicated by a dotted line, and Figure 8(b) is a diagram showing the AF area 802 set based on the designated area 801. The designated area 801 overlaps with three gain areas 204 to 206. Here, the condition setting unit 107 selects gain area 204 from the three gain areas as the area with the largest overlap with the designated area, and sets the area where the designated area 801 and the gain area 204 overlap as the AF area 802.

For example, if the exposure condition (here, the amount of gain) of a gain area that overlaps with the specified area is smaller than a threshold, the condition setting unit 107 may set the area where the gain area overlaps with the specified area as an AF area. The gain threshold set here may be set as a value desired by the user, may be the average value of the gain amounts of each gain area, or may be the average value of the gain within the specified area, and is not particularly limited. In this case, if a gain amount equal to or greater than the threshold is set in any of the gain areas that overlap with the specified area, the condition setting unit 107 notifies the user to prompt them to reset the AF area. By setting the gain value of the AF area sufficiently small, it is possible to reduce the possibility that contrast will not be evaluated correctly due to excessive gain.

For another example, when there are multiple areas where the specified area and the gain area overlap, the condition setting unit 107 may set the area where the specified area overlaps with the area with the smallest gain among the multiple gain areas as the AF area. For another example, the condition setting unit 107 may set the area where the gain area that is the area of interest among each gain area overlaps with the specified area as the AF area. Furthermore, the condition setting unit 107 may set the area where the specified area overlaps with the area selected by the user among the multiple gain areas as the AF area.

The condition setting unit 107 may set an area that satisfies one of the above-mentioned conditions as the AF area, or may set an area that satisfies multiple conditions as the AF area. Figure 9 shows an example in which, among the areas where the designated area and gain area overlap, an area whose area is equal to or greater than the threshold and has the smallest gain value is set as the AF area. Figure 9(a) shows the designated area 901 indicated by a dotted line, and Figure 9(b) shows the AF area 902 set based on the designated area 901. The designated area 901 overlaps with three gain areas 204 to 206. Of these three gain areas, area 204 and area 206 are the only two areas where the area that overlaps with the designated area is equal to or greater than the threshold, and the area where area 206, which has the smallest gain value, overlaps with the designated area is set as the AF area 902.

Here, we have explained an example in which an area that overlaps with a gain area from a specified area is extracted as an AF area, that is, an example in which the specified area is corrected as an AF area by shrinking it. However, the condition setting unit 107 may also set an AF area by enlarging the specified area as a correction. This may be a process in which the specified area is enlarged as is, or a process in which the area extracted from the specified area where the specified area and gain area overlap is enlarged. Hereinafter, the process of enlarging a corrected specified area in this way will be referred to as the process of "enlarging a specified area."

The condition setting unit 107 may enlarge the specified area, for example, when it receives an instruction from the user to enlarge the AF area, or when the specified area satisfies the conditions for enlarging the area. Here, the conditions for enlarging the specified area are not particularly limited, but may be, for example, when the area of the specified area is equal to or less than a predetermined value, or when it is expected that the AF accuracy in the specified area will be improved. Furthermore, the instruction from the user to enlarge the AF area may be obtained by, for example, pressing an enlargement execution button (not shown) provided on the UI 208. Furthermore, for example, the instruction to enlarge the AF area may be obtained by user input on the screen 201 (for example, specifying an area by dragging with the cursor or swiping via a touch panel).

The condition setting unit 107 expands the specified area so that it does not include exposure gaps within the area, and sets it as an AF area. Here, the condition setting unit 107 may expand the specified area according to user input, or may expand the specified area automatically (for example, so that the entire gain area where the specified area overlaps becomes the AF area). Figure 10 is a diagram illustrating an example of a specified area expanded in this manner. In Figure 10(a), the user inputs an instruction to expand AF area 1001 to area 1002. In the example of Figure 10(a), the exposure conditions are set uniformly across the entire area, so the exposure conditions are also uniform across the entire area of area 1002. Therefore, the condition setting unit 107 sets the AF area to area 1002. On the other hand, in Figure 10(b), area 1002 spans multiple gain areas, so the AF area is not expanded to area 1002. Therefore, in response to an instruction to expand the AF area 1001, the condition setting unit 107 limits the expansion range to within the gain area 204 and sets area 1003 as the AF area. This is an example showing an AF area that has been expanded to the maximum extent possible from area 1001; corrections may be made to satisfy desired conditions, such as setting an AF area obtained by excluding a predetermined number of pixels from the boundary that forms the exposure step 210 from AF area 1003.

Figure 6 is a flowchart showing an example of processing by the imaging device 100 according to this embodiment. The processing in Figure 6 is performed in the same manner as the processing shown in Figure 3 of embodiment 1, except that S601 is added as processing subsequent to S307, and processing of S602 and S603 is performed instead of S308 and S310, respectively, and therefore a duplicated description will be omitted.

In S601, the imaging device 100 acquires a specified area input by the user via the client device 110. In the following S602, the condition setting unit 107 sets the AF area by referencing the acquired specified area, and the process proceeds to S309. Here, the AF area is set by correcting the specified area, for example, as described above.

If it is determined in S309 that the AF area includes an exposure step, then in S603 the condition setting unit 107 reduces (corrects) the AF area so that it does not include the exposure step. For example, as described above, the condition setting unit 107 can correct the AF area by extracting an area that overlaps with the gain area. After S603, processing proceeds to S311.

This type of processing allows the area specified by the user to be corrected and an AF area that does not include exposure gaps to be set. This makes it possible to avoid high-frequency components due to the edges at the exposure gap positions that become noise in the calculation of the AF contrast evaluation value, thereby improving focusing accuracy. Furthermore, by correcting the specified area and setting the AF area so that the area or exposure conditions meet specified conditions, AF can be performed under more suitable conditions.

Note that if the specified area includes an exposure step within the area, the condition setting unit 107 may set the specified area as the AF area by uniformly changing the exposure conditions across the entire specified area only when calculating the AF contrast evaluation value. In other words, the condition setting unit 107 may set an AF area that does not include an exposure step within the area by temporarily erasing the exposure step when calculating the contrast evaluation value. By performing processing in this manner, it becomes possible to set an AF area that does not include an exposure step using the area specified by the user and perform AF processing.

[Embodiment 3]
In the first and second embodiments, AF is performed by setting one AF area and then controlling the lens to a lens position where the contrast evaluation value of that AF area is the highest. The imaging device 100 according to this embodiment sets multiple AF areas and performs AF based on the contrast evaluation value of each of the multiple AF areas that have been set.

The condition setting unit 107 according to this embodiment acquires a designated region in the same manner as in embodiment 2. Next, if the designated region includes an exposure step within the region, it sets multiple AF regions that do not include an exposure step within the region based on the designated region. Here, for example, if the designated region has areas that overlap with two or more gain regions, the condition setting unit 107 extracts the areas that overlap with the designated region for each of the gain regions in the same manner as in embodiment 2. Next, the condition setting unit 107 selects multiple of the extracted regions as AF regions and controls AF based on the contrast evaluation value calculated from each of the selected regions. Here, the condition setting unit 107 may set AF regions from all gain regions that overlap with the designated region, or may set thresholds for the gain value or the area of the region that overlaps with the designated region, and set only regions that satisfy the conditions for the thresholds as AF regions. The gain value threshold and area threshold here are set in the same manner as in embodiment 2, but may also be set to values different from those in embodiment 2.

The calculation unit 105 according to this embodiment calculates a contrast evaluation value from the output image signals of the multiple AF areas, and the lens control unit 106 drives the lens 102 to a position where the calculated contrast evaluation value is highest. The calculation unit 105 may, for example, calculate a contrast evaluation value from each of the multiple AF areas in the same manner as in embodiment 1, and calculate the contrast evaluation value according to this embodiment based on each calculated value. For example, the calculation unit 105 may use, as the contrast evaluation value to be used for AF, the average value, median value, or weighted average value of the contrast evaluation values calculated from each AF area, weighted according to the area of the AF area on the light receiving surface.

Figure 12 is a diagram illustrating an example of setting multiple AF areas. Here, area 202 in Figure 2 is specified as the specified area, and area 202 includes an exposure step. Therefore, the condition setting unit 107 sets the area where area 202 and gain area 204 overlap as AF area 1201, and the area where area 202 and gain area 206 overlap as AF area 1202. In the example shown in this figure, area 202 is divided and two AF areas that do not include an exposure step are set, but the specified area may be used as the AF area as is, and the exposure conditions within the AF area may be changed uniformly only when the AF contrast evaluation value is calculated.

Furthermore, if the imaging device 100 has a subject tracking function and the subject being tracked is located on an exposure step, the condition setting unit 107 may set multiple AF areas from multiple gain areas in which the subject is located across the exposure step, based on the tracking position of the subject. For example, the condition setting unit 107 can set AF areas from the gain area in which the subject was located immediately before moving onto the exposure step and the gain area to which the subject on the exposure step is estimated to move next. When AF is performed by referencing the gain area in which the subject was immediately located, it is possible to maintain the focus position even while the subject is moving on the exposure step. When AF is performed by referencing the gain area to which the subject is estimated to move, it is possible to adjust the focus position to a position where the subject will be in focus after moving off the exposure step. When AF areas are performed using both of these areas, it is possible to perform a chorus taking into account both the subject's movements before and after they move onto the exposure step.

Figure 14 is a diagram illustrating an AF area determined by the imaging device 100 tracking a subject according to the tracking position of the subject. In Figure 14, the same subject 1301 as in Figure 13 is located at the boundary (exposure step 210) between the gain area 206 and the gain area 204. In this case, the condition setting unit 107 may set an AF area 1401 in the gain area 206 where the subject 1301 was located immediately before it was positioned on the exposure step 210. The condition setting unit 107 may also set an AF area 1402 in the gain area 204 where the subject 1301 is estimated to move next based on the tracking results, or may perform AF using both the AF area 1401 and the AF area 1402.

The processing performed by the imaging device 100 according to this embodiment will be described below with reference to FIG. 11. FIG. 11 is a flowchart showing an example of processing performed by the imaging device 100 according to this embodiment. The processing in FIG. 11 is performed in the same way as the processing shown in FIG. 6 of the second embodiment, except that steps S1101 to S1103 are performed as processing subsequent to S309 instead of S603, and therefore a duplicated description will be omitted.

In S309, the condition setting unit 107 determines whether the AF area set in S602 includes an exposure step within the area. If an exposure step is included, processing proceeds to S1101; if not, processing proceeds to S311.

In S1101, the condition setting unit 107 sets multiple AF areas based on the AF area set in S602. The condition setting unit 107 divides the AF area (specified area) and sets multiple AF areas, for example, as shown in FIG. 12.

In S1102, the condition setting unit 107 calculates a contrast evaluation value for each of the multiple AF areas that have been set. In this process, the same calculation as in S311 is performed for each AF area.

In S1103, the condition setting unit 107 calculates a contrast evaluation value to be used for AF based on each of the contrast evaluation values calculated in S1102. As described above, the condition setting unit 107 can calculate, for example, the average value of each evaluation value as the final contrast evaluation value. The lens control unit 106 drives the lens 102 to a position where the contrast evaluation value calculated in S1103 is highest.

This type of processing allows multiple AF areas to be set so as to avoid the positions of exposure gaps. This means that high-frequency components caused by the edges of exposure gap positions, which can become noise in the calculation of AF contrast evaluation values, can be avoided, improving focusing accuracy.

[Embodiment 4]
In the above-described embodiments, each processing unit shown in, for example, FIG. 1 is realized by dedicated hardware, such as a camera serving as an information processing device, as described above. However, some or all of the processing units of the imaging device 100 (and the client device 110) may be realized by a computer. In this embodiment, at least some of the processing according to each of the above-described embodiments is executed by a computer.

Figure 15 is a diagram showing the basic configuration of a computer. In Figure 15, processor 1501 is, for example, a CPU, and controls the operation of the entire computer. Memory 1502 is, for example, RAM, and temporarily stores programs, data, etc. Computer-readable storage medium 1503 is, for example, a hard disk or CD-ROM, and stores programs, data, etc. long-term. In this embodiment, programs that realize the functions of each unit, stored in storage medium 1503, are read into memory 1502. Then, the functions of each unit are realized by processor 1501 operating in accordance with the programs on memory 1502.

In FIG. 15, input interface 1504 is an interface for acquiring information from an external device. If the client device 110 is an external device to the imaging device 100, the input interface 1504 may acquire information input by a user from the client device 110. Furthermore, output interface 1505 is an interface for outputting information to an external device. Bus 1506 connects the above-mentioned components and enables data exchange.

(Other Examples)
The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to clarify the scope of the invention.

100: Imaging device, 101: Imaging element, 102: Lens, 103: Image processing unit, 104: Output unit, 105: Calculation unit, 106: Lens control unit, 107: Condition setting unit, 108: Input unit, 110: Client device, 111: Input unit, 112: Display unit, 113: Operation unit, 114: Input unit

Claims (18)

An information processing device that performs autofocusing of an imaging device that can set exposure conditions for each partial area of a light receiving surface of an image sensor,
a first setting means for setting, on the light receiving surface, a first partial area for which an image is captured under a first exposure condition, and a second partial area adjacent to the first partial area for which an image is captured under a second exposure condition different from the first exposure condition;
a second setting means for setting a third partial area that does not include an exposure step that is a boundary between the first partial area and the second partial area as an area to be used for autofocus;
a control unit that controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region;
a tracking means for tracking a subject in an image captured by the imaging device,
the second setting means sets the third partial area from an area of the subject on the light receiving surface;
The information processing device is characterized in that, when the subject is located on the exposure step, the second setting means sets the third partial area from the partial area, of the first partial area and the second partial area, in which the subject was located immediately before moving onto the exposure step. the first exposure condition and the second exposure condition are exposure time or gain value,
The information processing apparatus according to claim 1 , wherein a difference between the first exposure condition and the second exposure condition is equal to or greater than a predetermined threshold value. a first acquisition means for acquiring a user input specifying an area to be used for the autofocus;
3. The information processing apparatus according to claim 1, wherein the second setting means sets the third partial region that does not include the exposure step based on the user input. The information processing device described in claim 3, characterized in that, when a fourth partial area including the exposure step on the light receiving surface is specified by the user input, the second setting means corrects the fourth partial area so that the exposure step is not included within the area and sets the area as the third partial area. The information processing device of claim 4, wherein the second setting means, when there is an area between the overlapping area of the fourth partial area and the first partial area and the overlapping area of the fourth partial area and the second partial area that exceeds a predetermined area, sets the area that exceeds the predetermined area as the third partial area. The information processing device described in claim 4 or 5, characterized in that the second setting means sets the larger area of the overlapping area between the fourth partial area and the first partial area and the overlapping area between the fourth partial area and the second partial area as the third partial area. The information processing device of claim 4, wherein the second setting means, when there is an area in which the gain amount is smaller than a predetermined threshold value among the overlapping area between the fourth partial area and the first partial area and the overlapping area between the fourth partial area and the second partial area, sets the area in which the gain amount is smaller than the predetermined threshold value as the third partial area. The information processing device described in claim 4 or 7, characterized in that the second setting means sets, as the third partial region, the region where the amount of gain is smaller between the overlapping region between the fourth partial region and the first partial region and the overlapping region between the fourth partial region and the second partial region. a second acquisition means for acquiring a user's instruction to enlarge the area used for the autofocus from the third partial area;
9. The information processing device according to claim 1, wherein the control means controls the autofocus of the imaging device in accordance with the contrast of an image based on an image signal of a fifth partial area of the light receiving surface expanded from the third partial area based on a user instruction acquired by the second acquisition means. a third acquisition means for acquiring a user input specifying an area to be used for the autofocus;
3. The information processing device according to claim 1, wherein the control means controls the autofocus of the imaging device based on the contrast of an image based on an image signal of the sixth partial area when exposure conditions are set uniformly across the entire sixth partial area when a sixth partial area including the exposure step is specified on the light receiving surface by user input acquired by the third acquisition means. the second setting means further sets a seventh partial area of the light receiving surface, which is different from the third partial area and does not include the exposure step, as the area to be used for the autofocus;
11. The information processing device according to claim 1, wherein the control means controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region and the contrast of an image based on the image signal of the seventh partial region. The information processing device described in claim 11, characterized in that the control means controls the autofocus of the imaging device by adjusting the position of the lens of the imaging device so as to maximize the average value of the evaluation value of the contrast of the image based on the image signal of the third partial region and the evaluation value of the contrast of the image based on the image signal of the seventh partial region. a tracking unit configured to track a subject in an image captured by the imaging device;
3. The information processing apparatus according to claim 1, wherein the second setting means sets the third partial area from an area of the subject on the light receiving surface. An information processing device that performs autofocusing of an imaging device that can set exposure conditions for each partial area of a light receiving surface of an image sensor,
a first setting means for setting, on the light receiving surface, a first partial area for which an image is captured under a first exposure condition, and a second partial area adjacent to the first partial area for which an image is captured under a second exposure condition different from the first exposure condition;
a second setting means for setting a third partial area that does not include an exposure step that is a boundary between the first partial area and the second partial area as an area to be used for autofocus;
a control unit that controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region;
a tracking means for tracking a subject in an image captured by the imaging device;
an estimation means for estimating a partial region to which the subject will next move from the first partial region and the second partial region when the subject is located on the exposure step,
an information processing device, characterized in that the second setting means sets the third partial area from the area of the subject on the light receiving surface, and sets the third partial area from the partial area to which the subject will next move, estimated by the estimation means. the second setting means further sets an eighth partial area of the light receiving surface, which is different from the third partial area and does not include the exposure step, from a partial area of the first partial area and the second partial area where the subject was located immediately before moving onto the exposure step;
15. The information processing device according to claim 14, wherein the control means controls autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region and the contrast of an image based on the image signal of the eighth partial region. The information processing device according to claim 1 , wherein the information processing device is built into the imaging device. An information processing method for performing autofocusing on an imaging device capable of setting exposure conditions for each partial area of a light receiving surface of an image sensor, comprising:
setting, on the light receiving surface, a first partial area in which an image is captured under a first exposure condition, and a second partial area adjacent to the first partial area in which an image is captured under a second exposure condition different from the first exposure condition;
setting a third partial area that does not include an exposure step that is a boundary between the first partial area and the second partial area as an area to be used for autofocus;
controlling autofocus of the imaging device in accordance with the contrast of an image based on the image signal of the third partial region;
and tracking a subject in an image captured by the imaging device,
the third partial area is set from an area of the subject on the light receiving surface,
An information processing method characterized in that, when the subject is located on the exposure step, the third partial area is set from the partial area between the first partial area and the second partial area in which the subject was located immediately before moving onto the exposure step. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 16 .