JP7754101B2 - Imaging device, imaging method, and program - Google Patents

Description

This technology relates to an imaging device, an imaging method, and a program, and in particular to an imaging device, an imaging method, and a program that enable a user to obtain images of the quality desired.

Patent document 1 proposes an imaging device that detects a face area, which is the area of a person's face, and if an pupil area, which is the area of the pupil, can be detected from the detected face area, focuses on the pupil area, and if an pupil area cannot be detected, focuses on the face area.

International Publication No. 2015/045911

However, when capturing images of animals, imaging devices detect the animal's pupil area without detecting the animal's face area. Therefore, if this imaging device cannot detect the animal's pupil area, there is no object to focus on, making it difficult for users to obtain images of the quality they desire.

This technology was developed in light of these circumstances, allowing users to obtain images of the quality they desire.

An imaging device according to one aspect of the present technology includes a setting unit that sets a first region including a first specific part of a subject as a focus region to be focused on, and if the first specific part is not detected, the setting unit sets a second region including a second specific part of the subject, which is a specific part of the same type as the first specific part, as the focus region.

In one aspect of the present technology, a first region including a first specific part of a subject is set as a focus region to be focused on. Furthermore, if the first specific part is not detected, a second region including a second specific part of the subject that is the same type of specific part as the first specific part is set as the focus region.

1 is a block diagram illustrating an example of the configuration of an imaging device according to an embodiment of the present technology. FIG. 2 is a block diagram showing an example of the configuration of a digital signal processing unit. 10A and 10B are diagrams showing display examples in an pupil detection mode for each subject; FIG. 10 is a diagram showing some of the settings related to the pupil detection mode of the imaging device 11. FIG. 10 is a diagram illustrating an example of internal processing in an animal pupil detection mode. 10A and 10B are diagrams illustrating an example of internal processing when a setting for selecting left or right pupil in the person's pupil detection mode is applied. 10A and 10B are diagrams illustrating an example of internal processing when the left/right pupil selection setting of the present technology is applied in the animal pupil detection mode. 10 is a flowchart illustrating an image capturing process in a person's pupil detection mode. 9 is a flowchart illustrating the image capturing process in the person's pupil detection mode, following FIG. 8 . 10 is a flowchart illustrating the image capturing process in the person's pupil detection mode, following FIG. 9 . 10 is a flowchart illustrating an imaging process in an animal pupil detection mode. 12 is a flowchart illustrating the imaging process in the animal pupil detection mode, following FIG. 11 . 13 is a flowchart illustrating the imaging process in the animal pupil detection mode, following FIG. 12 . 10A and 10B are diagrams illustrating an example of selection of pupil regions when left/right pupil selection is set to auto. 10 is a flowchart illustrating a bird's pupil region selection process. 16 is a flowchart illustrating the bird's pupil region selection process, following FIG. 15 . 17 is a flowchart illustrating the bird's pupil region selection process, following FIG. 16 . FIG. 10 is a diagram showing an example of a detection target switching setting screen. FIG. 10 is a diagram showing an example of a display screen when the detection target is switched. FIG. 10 is a diagram showing a hierarchical structure of object types.

Hereinafter, embodiments of the present technology will be described in the following order.
1. Image capture device 2. Subject pupil detection mode 3. Animal pupil detection mode 4. Operation of image capture device 5. Other

<1. Imaging device>
<Configuration example of imaging device>
FIG. 1 is a block diagram showing an example of the configuration of an imaging device according to an embodiment of the present technology.

The imaging device 100 in Figure 1 has a specific part detection mode that detects an area that focuses on a specific part of a subject, such as a person, animal, or bird. Specific parts include a face or an eye. Examples of detection modes for specific parts of a subject include a human eye detection mode, an animal eye detection mode, and a bird eye detection mode.

In addition, the imaging device 100 can provide detection modes for specific parts of the body, not just the eyes of a person or an animal. The user can select and set the desired detection mode from among the detection modes for the subject or specific parts of the subject.

As shown in Figure 1, the imaging device 100 is configured to include a lens 101, an aperture 102, an image sensor 103, an analog signal processing unit 104, an A/D conversion unit 105, and a digital signal processing unit 106. The imaging device 100 is configured to include a lens driver 121, a TG (Timing Generator) 122, a gyro (sensor) 123, and a system controller 131.

The imaging device 100 is also configured to include a display unit 141, a memory unit 142, an input unit 143, an output unit 144, a communication unit 145, an operation unit 146, and a drive 147.

The lens 101 adjusts the focus on the subject and collects light from the subject. The aperture 102 adjusts the exposure.

The image sensor 103 captures an image of a subject to obtain a captured image. That is, the image sensor 103 photoelectrically converts light from the subject and outputs the converted image signal to the analog signal processing unit 104. Through this photoelectric conversion, the image sensor 103 can capture both still images and moving images.

The analog signal processing unit 104 performs analog signal processing on the image signal obtained by the image sensor 103. The A/D conversion unit 105 A/D converts the analog signal processed image signal to obtain image data as a digital signal.

The digital signal processing unit 106 performs digital signal processing on the image data obtained by the A/D conversion unit 105. As part of its digital signal processing, the digital signal processing unit 106 performs at least processing to detect the area of the subject or a specific part of the subject from the moving image supplied as image data, and to set a focus area, which is the area on which to focus. Hereinafter, the specific part of the subject will be simply referred to as the "specific part."

In addition, the digital signal processing unit 106 also performs processing to set one of the areas as the focus area based on the detection results of the area of the subject or specific part, and to control the display of a focus preview frame that notifies the area that will be set as the focus area before focusing begins, and a focus frame that indicates the focus area after focusing begins.

The content of the digital signal processing is arbitrary, and processing other than that described above may also be performed. For example, the digital signal processing unit 106 may perform color mixing correction, black level correction, white balance adjustment, demosaic processing, matrix processing, gamma correction, and YC conversion as digital signal processing. Furthermore, the digital signal processing unit 106 may perform codec processing, which is processing related to the encoding and decoding of image data, as digital signal processing.

The lens driver 121 drives the lens 101 and aperture 102 to control the focal length, exposure, etc. The TG 122 generates a synchronization signal and supplies it to the image sensor 103 to drive the image sensor 103 and control imaging. The gyro 123 is a sensor that detects the position and orientation of the image sensor 100. The gyro 123 outputs information indicating the position and orientation of the image sensor 100 to the A/D conversion unit 105.

The system controller 131 is composed of, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory), and executes programs and processes data to control each processing unit of the imaging device 100. The system controller 131 also receives operation inputs from the user based on signals supplied from the operation unit 146, and performs processing or control corresponding to the operation inputs.

For example, the system controller 131 can control the focal length or exposure, etc. based on information indicating the focus area supplied from the digital signal processing unit 106.

The display unit 141 is configured, for example, by a liquid crystal display, and displays an image corresponding to image data stored in the memory of the digital signal processing unit 106. For example, the display unit 141 can display captured images obtained by the image sensor 103, stored captured images, etc.

The storage unit 142 stores image data stored in the memory of the digital signal processing unit 106. In doing so, the storage unit 142 stores encoded data encoded by the digital signal processing unit 106 in order to reduce the amount of data. The encoded data stored in the storage unit 142 is read and decoded by the digital signal processing unit 106 and displayed, for example, on the display unit 141.

The input unit 143 has an external input interface such as an external input terminal, and outputs various data (e.g., image data and encoded data) supplied from outside the imaging device 100 via the external input interface to the digital signal processing unit 106.

The output unit 144 has an external output interface such as an external output terminal, and outputs various data supplied via the digital signal processing unit 106 to the outside of the imaging device 100 via the external output interface.

The communication unit 145 performs predetermined communication with other devices, which may be wired communication or wireless communication, and transmits and receives data to and from other devices via the predetermined communication. For example, the communication unit 145 transmits various data (e.g., image data and encoded data) supplied from the digital signal processing unit 106 to other devices via the predetermined communication. The communication unit 145 also receives various data from other devices via the predetermined communication and outputs the received data to the digital signal processing unit 106.

The operation unit 146 is composed of any input device, such as a key, button, or touch panel. The operation unit 146 includes, for example, a shutter button. The operation unit 146 receives operation input from the user and outputs a signal corresponding to the operation input to the system controller 131.

Drive 147 reads information (programs, data, etc.) stored on a removable recording medium 148, such as a semiconductor memory, attached to the drive 147. Drive 147 supplies the information read from the removable recording medium 148 to system controller 131. Furthermore, when a writable removable recording medium 148 is attached, drive 147 can store information (image data, encoded data, etc.) supplied via system controller 131 on the removable recording medium 148.

In addition, the lens 101, aperture 102, and lens driver 121 described above may be formed as an interchangeable lens 151 that is detachable (interchangeable) from the imaging device 100 in a separate housing from the imaging device 100.

<Configuration example of digital signal processing unit>
FIG. 2 is a block diagram showing an example of the configuration of the digital signal processing unit 106.

The digital signal processing unit 106 has a memory 211, a subject detection unit 212, a subject tracking unit 213, an area setting unit 214, a display control unit 215, and a codec processing unit 216.

The memory 211 stores image data supplied from the A/D conversion unit 105. The image data is, for example, image data of each frame of a moving image or image data of a still image.

The subject detection unit 212 detects the area of the subject or specific body part from the image data stored in the memory 211 based on a signal corresponding to a user's operation input supplied from the system controller 131. The subject detection unit 212 outputs the detection result of the area of the subject or specific body part to the area setting unit 214 and the display control unit 215.

The subject detection unit 212 consists of a person detection unit 212-1, an animal detection unit 212-2, and a bird detection unit 212-3.

When the detection mode for each specific part of the subject is set to the person's pupil detection mode, the person detection unit 212-1 detects the person's face area and outputs the detection result of the detected face area to the area setting unit 214 and the display control unit 215. When the user performs an operation to instruct the start of focusing, the person detection unit 212-1 detects the person's pupil area based on the detection result of the face area and outputs the detection result of the pupil area to the area setting unit 214 and the display control unit 215.

When the detection mode for each specific part of the subject is set to the animal pupil detection mode, the animal detection unit 212-2 detects the animal pupil area and outputs the detection result of the detected animal pupil area to the area setting unit 214 and the display control unit 215.

To detect animal pupils, techniques such as deep learning using an animal dictionary are used. There are two types of animal dictionaries: one for front-facing images and one for side-facing images. When the user performs an operation to instruct the start of focusing, the animal detection unit 212-2 detects the animal's pupil area according to the focus frame and outputs the detection results of the detected animal's pupil area to the area setting unit 214 and the display control unit 215.

A focus frame is a frame used by the user to set the approximate focus position. When a focus frame is set, area detection processing is performed preferentially from inside the focus frame.

Examples of animal types include, but are not limited to, dogs, cats, lions, etc. It is also possible to provide a different animal detection unit for each type of animal that has similar characteristics to be detected.

When the detection mode for each specific part of the subject is set to bird's pupil detection mode, the bird detection unit 212-3 detects the bird's pupil area and outputs the detection result of the detected bird's pupil area to the area setting unit 214 and the display control unit 215.

To detect the bird's pupil, for example, a method such as deep learning using a bird dictionary is used. When the user performs an operation to instruct the start of focusing, the bird detection unit 212-3 detects the bird's pupil area according to the focus frame and outputs the detection result of the detected bird's pupil area to the area setting unit 214 and the display control unit 215.

The subject tracking unit 213 tracks the focus area set by the area setting unit 214. For example, tracking is performed on each area set as a focus area by the area setting unit 214 (a detected or tracked face area, pupil area, or an area specified by a normal AF (autofocus) frame, described below). The subject tracking unit 213 outputs information indicating the tracked focus area as a tracking result to the area setting unit 214 and the display control unit 215.

The area setting unit 214 sets the area of the specific part of the subject detected by the subject detection unit 212 as the focus area according to the detection mode for each specific part of the subject. If the subject detection unit 212 does not detect the area of the specific part of the subject, the area setting unit 214 instead uses the tracking result of the specific part of the subject tracked by the subject tracking unit 213.

If the area of a specific part of the subject is not detected and cannot be tracked, the area setting unit 214 sets the area that would be set as the focus area in normal AF (autofocus) mode as the focus area, rather than in the detection mode for each specific part of the subject. This area can be moved by the user and can be specified using the AF frame in normal AF mode (hereinafter referred to as the normal AF frame). The area setting unit 214 supplies information about the set focus area to the system controller 131.

The display control unit 215 generates a focus frame in response to a signal corresponding to a user's operation input supplied from the system controller 131, and displays it on the display unit 141 by superimposing it on the image from the memory 211. Information about the focus frame is output to the subject detection unit 212.

The display control unit 215 also generates a focus preview frame, in-focus frame, focus frame, normal AF frame, etc. based on a signal corresponding to a user's operation input supplied from the system controller 131. The display control unit 215 superimposes the generated focus preview frame, in-focus frame, focus frame, and normal AF frame on the image from the memory 211 and displays them on the display unit 141. Information on the focus preview frame, in-focus frame, focus frame, and normal AF frame is output to the subject detection unit 212 as necessary.

In addition, the display control unit 215 generates GUI (Graphical User Interface) images such as menus, buttons, or cursors, and displays them together with captured images, photographed images, etc.

The codec processing unit 216 performs processing related to encoding and decoding of image data of moving images and still images stored in memory 211.

<2. Eye detection mode for each subject>
<Outline of eye detection modes for each subject>
FIG. 3 shows an example of a display when the pupil detection mode for each subject is set.

Figure 3 shows, from top to bottom, example screens displayed on the display unit 141 when the human eye detection mode, animal eye detection mode, and bird eye detection mode of the imaging device 11 are set.

Screens A1 to A3 in the center of Figure 3 are screens that are displayed before the user performs an operation to instruct the camera to start focusing.Screens B1 to B3 on the right are screens that are displayed after the user performs an operation to instruct the camera to start focusing.

The user can switch between human eye detection mode, animal eye detection mode, and bird eye detection mode.

When the person's eye detection mode is set, as shown on screen A1, the face area of the person's face that overlaps the focus frame F is detected, and the focus preview frame PF is displayed in the detected face area. In the example of screen A1, the focus preview frame PF is displayed so that it partially overlaps the focus frame F.

When the user performs an operation to start focusing in this state, as shown on screen B1, the pupil area is detected within the face area, and the detected pupil area is set as the in-focus area. An AF focusing frame is then displayed in place of the focus preview frame PF within the in-focus area, and focusing is performed on the in-focus area surrounded by the AF focusing frame. In human pupil detection mode, as will be described in more detail below, if no pupil area is detected, the detected face area is set as the hand-held area and a hand-held frame is displayed.

When animal pupil detection mode is set, as shown on screen A2, the animal's pupil area is detected both inside and outside the focus frame F, and the focus preview frame PF is displayed in the detected pupil area. Note that at this time, pupil area detection is performed with priority given to the area inside the focus frame F.

If the user performs an operation to start focusing in this state, the pupil area is set as the in-focus area, and an in-focus frame AF is displayed in place of the focus preview frame PF, as shown on screen B2. Focusing is also performed on the pupil area surrounded by the in-focus frame AF. In animal pupil detection mode, as will be described in more detail below, if one pupil area is not detected, the other detected pupil area is set as the hand-holding area, and a hand-holding frame is displayed.

When bird's eye detection mode is set, bird's eye areas are detected both inside and outside the focus frame F, as shown on screen A3, and a focus preview frame PF is displayed in the detected eye area. Note that eye area detection is performed with priority given to areas inside the focus frame F.

If the user performs an operation to start focusing in this state, the pupil area is set as the focus area, as shown in screen B3, and the focus frame AF is displayed in place of the focus preview frame PF. Furthermore, focusing is performed on the pupil area surrounded by the focus frame AF.

In bird's eye detection mode, if the left and right pupil areas are detected, a focus preview frame PF is displayed in the pupil area closest to the imaging device, which is set as the in-focus area, and the in-focus frame AF is displayed. Note that, as will be described in more detail later, if the pupil area is not detected in bird's eye detection mode, an area preset by the user is set as the prayer area, and a prayer frame is displayed.

In addition, the focus frame AF is displayed in a different way from the focus preview frame PF, for example, the focus preview frame PF is displayed in a white frame and the focus frame AF is displayed in a green frame.

As described above, when the human eye detection mode is set, the face area is detected before the user performs an operation to instruct the start of focusing, and the focus preview frame PF is displayed in the detected face area.

On the other hand, when the animal or bird eye detection mode is set, the eye area is detected before the user performs an operation to instruct the camera to start focusing, and a focus preview frame PF is displayed in the detected eye area.

By displaying the focus preview frame before the user performs an operation to start focusing, the user can know in advance whether the intended or unintended position has been detected, allowing the user to choose not to perform autofocus. This makes it easy to capture images that focus on specific parts of the subject.

<Settings related to eye detection mode>
FIG. 4 is a diagram showing some of the settings related to the pupil detection mode of the imaging device 11.

Figure 4 shows that the detection targets for the pupil detection mode can be set to people, animals, birds, and auto.

When a person is set as the detection target, the person's face or eyes will be detected in the eye detection mode. When an animal is set as the detection target, the animal's eyes will be detected in the eye detection mode. When a bird is set as the detection target, the bird's eyes will be detected in the eye detection mode.

When Auto is set as the detection target, the subject's eyes are detected in a separately set priority order. For example, if the priority order is set to people, animals, and birds, people's eyes are detected, and if people's eyes are not detected, animal eyes are detected, and if people's and animals' eyes are not detected, bird eyes are detected, and so on. Eyes are detected in the set priority order.

Figure 4 also shows that the left and right pupil selection in the pupil detection mode can be set to auto, right pupil, or left pupil.

When Auto is set as the left/right pupil selection in the pupil detection mode, the pupil to be focused on is left to the imaging device 111. When the right pupil is set as the left/right pupil selection in the pupil detection mode, the right pupil is the focus target. When the left pupil is set as the left/right pupil selection in the pupil detection mode, the left pupil is the focus target.

Even when left/right pupil selection is set for these pupil detection modes, the user can temporarily switch the pupil to be focused on by operating a specific button, etc. Note that, hereafter, the operation of the user to temporarily switch the pupil to be focused on will be referred to as "manual specification."

The above left/right pupil selection is performed in human eye detection mode. In animal eye detection mode, left/right pupil priority selection is performed. "Priority" here means "if both the right eye and the left eye can be selected, the specified eye will be the focus target."

Even when left/right pupil (priority) selection is set for these pupil detection modes, it is possible to temporarily switch the eye to be focused on according to manual user selection. In the case of animal pupil detection mode, if both the right and left pupils can be selected, the specified eye can be used as the focus target.

As described above, the operation for selecting the left or right pupil differs between the human pupil detection mode and the animal pupil detection mode.

The animal eye detection mode is explained in detail below.

<3. Animal Eye Detection Mode>
<Overview of internal processing>
FIG. 5 is a diagram showing an example of internal processing in the animal pupil detection mode.

In animal pupil detection mode, the detection and tracking processes of the animal's pupil (area) are performed in parallel.

In Figure 5, from top to bottom, the display screen displayed on the display unit 141, the results of animal eye detection, and the results of animal eye tracking are shown over time. Each display screen shows the animal (a fox) whose eye is being detected facing forward, and displays the focus frame AF when the focus area is set based on the results of detecting or tracking the fox's eye at each timing.

Here, "Detection OK" indicates that the pupil was detected by the detection process, while "Detection NG" indicates that the pupil could not be detected by the detection process due to various reasons such as the size of the subject's movement, blur, or insufficient detection performance. "Tracking OK" indicates that the pupil was tracked by the tracking process.

In addition, the frame P displayed for either the detection result or the tracking result indicates whether the focus area is set based on the detection result or the tracking result at that time, and the focus frame AF is displayed based on that result.

The following is an explanation in chronological order. At timing t1 in Figure 5, an AF focus frame is displayed on the fox's left eye on the display screen. At timing t1, the fox's left eye is successfully detected and tracked. In other words, the AF focus frame displayed at timing t1 is displayed with the focus area set based on the detection results of the fox's left eye.

At time t2, an AF focus frame is displayed on the fox's left eye on the display screen. At time t2, the fox's left eye was not detected, but was tracked successfully. In other words, the AF focus frame displayed at time t2 is displayed with the focus area set based on the tracking results of the fox's left eye.

At time t3, an AF focus frame is displayed on the fox's left eye on the display screen. At time t3, the fox's left eye was not detected, but was tracked successfully. In other words, the AF focus frame displayed at time t3 is displayed with the focus area set based on the tracking results of the fox's left eye.

At time t4, an AF focus frame is displayed on the display screen over the fox's left eye. At time t4, the fox's left eye is successfully detected and tracked. In other words, the AF focus frame displayed at time t4 is displayed with the focus area set based on the detection results of the fox's left eye.

As described above, in animal eye detection mode, if the animal's eye is detected successfully, the focus area is set based on the detection results and the AF focus frame is displayed.If the animal's eye cannot be detected but tracking is successful, the focus area is set based on the tracking results and the AF focus frame is displayed.

In other words, although the display screen remains largely unchanged, at times t2 and t3 the AF focus frame is displayed with the focus area set based on the tracking results, rather than the detection results of the animal's eye. Note that at these times, the detection accuracy of the eye on the opposite side (right) of the AF focus frame is not guaranteed.

However, whether the AF focus frame is displayed based on the detection results or the tracking results is an internal process, so the user does not know.

<Internal processing when the left/right pupil selection setting in the human eye detection mode is applied>
FIG. 6 shows an example of internal processing when the left/right pupil selection setting in the human pupil detection mode is applied in the animal pupil detection mode.

In Figure 6, similar to Figure 5, from top to bottom, the display screen displayed on the display unit 141, the results of detecting the animal's pupil, and the results of tracking the animal's pupil are shown over time. Furthermore, in Figure 6, a setting for selecting the left or right pupil has been added below the tracking results.

As described above, the imaging device 11 allows the pupil to be focused on to be set in advance by selecting the left or right pupil. It is also possible to switch the pupil to be focused on during imaging processing in response to manual specification by the user.

At timing t11 in Figure 6, an in-focus frame AF is displayed on the display screen over the fox's left eye. At timing t11, the left eye is selected as the focus target, and the fox's left eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t11 is displayed with the in-focus area set based on the detection results of the fox's left eye.

Here, the imaging device 11 switches the focus target to the right pupil in accordance with manual specification by the user.

At times t12 and t13, a focus frame E is displayed on the display screen near the fox's nose. At times t12 and t13, the focus target switches to the right eye. However, the right eye cannot be detected or tracked. However, the left eye, which continues to be tracked, can be tracked.

Although the fox's left eye can be tracked successfully, the switched right eye cannot be detected or tracked, so the in-focus area is set to the area that would be set as the in-focus area in normal AF mode, as described above. In other words, the in-focus frame E displayed at times t12 and t13 is the area that would be set as the in-focus area in normal AF (in Figure 6, the area located near the fox's nose).

In other words, if the timing of switching to the right eye means that the camera is unable to detect and track the right eye, it will operate in normal AF mode and the eye detection mode will not be performed.

Then, at timing t14, an in-focus frame AF is displayed on the display screen over the fox's right eye. At timing t14, the right eye is selected as the focus target, the detection and tracking processes for the right eye return, and the fox's right eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t14 is displayed with the in-focus area set based on the detection results of the right eye.

As mentioned above, there are many times when the tracking process in animal eye detection mode cannot respond to eye switching. Furthermore, in human eye detection mode, if an eye cannot be detected, the camera is set to focus on the detected face, but in animal eye detection mode, no face is detected, so normal AF operation occurs. Furthermore, when the depth of field is shallow, once the camera has focused on the nose, the pupil becomes blurred, making it difficult to return focus to the pupil.

For these reasons, in animal eye detection mode, it is difficult to determine the timing for switching the focus from the eye to the opposite eye.

Therefore, in this technology, when left or right pupil selection is performed in animal pupil detection mode, left or right pupil "priority" selection is performed.

<Internal processing for left/right pupil priority selection>
FIG. 7 shows an example of internal processing when a setting of left/right pupil priority selection is applied in the animal pupil detection mode of the present technology.

In Figure 7, as in Figure 6, from top to bottom, the display screen displayed on the display unit 141, the results of detecting the animal's pupil, the results of tracking the animal's pupil, and the settings for left/right pupil priority selection are shown over time.

At timing t21 in Figure 7, an in-focus frame AF is displayed on the display screen over the fox's left eye. At timing t21, the left eye is selected as the focus target, and the fox's left eye is successfully detected and tracked. In other words, the in-focus frame AF displayed at timing t21 is displayed with the in-focus area set based on the detection results of the left eye.

Here, the imaging device 11 switches the focus target to the right pupil in accordance with manual specification by the user.

At times t22 and t23, an AF focus frame is displayed on the fox's left eye on the display screen. At times t22 and t23, the focus target is switched to the right eye. However, because this is a priority selection, if the timing is not right for switching, for example, because the right eye has not been detected, or if it has been detected but the detection accuracy is poor, the results of detecting and tracking the left eye before the switch are used.

In other words, at times t22 and t23, the fox's right eye could not be detected or tracked, while the left eye could not be detected but could be tracked. Therefore, the in-focus frame AF displayed at times t22 and t23 is displayed with the in-focus area set based on the tracking results of the left eye.

At timing t24, an in-focus frame AF is displayed on the fox's right eye on the display screen. At timing t24, the focus target is switched to the right eye in response to manual selection by the user.

In other words, at time t24, the fox's right eye is detected and tracked successfully, and it is time to switch the focus target. Therefore, the focus frame AF displayed at time t24 is displayed with the focus area set based on the detection result of the right eye.

As described above, in animal pupil detection mode, when the timing is right to switch to the pupil on the side where priority selection is set or the side where manual selection is specified, the pupil on the side where priority selection is set or the side where manual selection is specified will be used for focusing.

On the other hand, if the timing is such that it is not possible to switch to the pupil on the side for which priority selection is set or the side for which manual selection is specified, the pupil on the side for which priority selection is not set or the side for which manual selection is not specified will be used for focusing.

This allows for high-quality images to be obtained while taking user operations into consideration.

4. Operation of the Imaging Device
<Image capture process in human eye detection mode>
8 to 10 are flowcharts illustrating the image capture process of the image capture device 100 in the person's pupil detection mode.

The imaging process in the human eye detection mode shown in Figures 8 to 10 begins when the power is turned on, for example, by operating the power button. The detection mode for each specific part of the subject is pre-set as the human eye detection mode from a settings screen, etc.

In step S11 of Figure 8, the imaging device 11 acquires and displays an image. Specifically, the imaging element 103 acquires an electrical signal for each pixel by photoelectrically converting light from the subject collected via the lens 101 and aperture 102 on a pixel-by-pixel basis. The image signal consisting of the electrical signals for each pixel is passed through the analog signal processing unit 104 and the A/D conversion unit 105 to generate digital image data, which is then stored in the memory 211 of the digital signal processing unit 106.

The display control unit 215 displays an image based on the image data stored in the memory 211 as a live view image on the display unit 141.

In step S12, the person detection unit 212-1 detects a face area from the image data stored in the memory 211. The person detection unit 212-1 supplies information about the detected face area to the area setting unit 214 and the display control unit 215.

In step S13, the person detection unit 212-1, under the control of the system controller 131, detects the pupil area from the face area within the focus frame and outputs the detection result (information on the pupil area) to the area setting unit 214 and the display control unit 215.

In step S14, the area setting unit 214 determines whether or not the pupil has been detected. If it is determined in step S14 that the pupil has been detected, the processing proceeds to step S15.

In step S15, the area setting unit 214 uses the pupil detection results to perform subsequent processing to set the focus area.

In step S16, the area setting unit 214 determines whether the left/right pupil selection setting is auto. If it is determined in step S16 that the left/right pupil selection setting is auto, processing proceeds to step S17.

In step S17, the area setting unit 214 sets the face area as the focus area. After step S17, processing proceeds to step S18 in Figure 9.

In step S18, the subject tracking unit 213 tracks the focus area set by the area setting unit 214 and outputs the tracking result to the area setting unit 214 and the display control unit 215. The tracking process in step S18 is performed on the face area set as the focus area.

The user issues a command to start focusing by pressing a specific button or half-pressing the shutter button. The command to start focusing is issued for each image capture unit (frame). The operation unit 146 receives operation input from the user and outputs a signal corresponding to the operation input to the system controller 131.

In step S19, the system controller 131 determines whether or not a command to start focusing has been issued. If it is determined in step S19 that a command to start focusing has been issued, processing proceeds to step S20.

In step S20, the system controller 131 controls the lens driver 121 to drive the optical system, such as the lens 101 and aperture 102, so that the focus is on the front pupil within the focus area. In other words, both the left and right pupils may be detected within the focus area. In this case, in the person pupil detection mode, the image capture device 100 is controlled to focus on the front pupil as seen from the perspective of the image capture device 100. The area setting unit 214 identifies the front pupil.

In step S21, the area setting unit 214 determines whether the foreground pupil has been identified. If it is determined in step S21 that the foreground pupil has been identified, processing proceeds to step S22.

In step S22, the area setting unit 214 changes the focus area to the front pupil. After step S22, processing proceeds to step S35.

If it is determined in step S21 that the foreground pupil could not be identified, processing proceeds to step S35. In other words, the focus area remains the face area and is not changed.

Also, if it is determined in step S19 that no instruction to start focusing has been given, processing proceeds to step S36.

Returning to Figure 8, if it is determined in step S16 that the left/right pupil selection setting is not auto, processing proceeds to step S23.

In step S23, the area setting unit 214 determines whether the pupil specified by the user is selectable. Note that, as described above, pupils specified by the user include pupils set by left/right pupil selection and pupils specified manually. For example, if the pupil specified by the user has been detected and the detection accuracy is high, it is determined in step S23 that the pupil specified by the user is selectable, and processing proceeds to step S24.

In step S24, the area setting unit 214 sets the pupil area specified by the user as the focus area.

If it is determined in step S23 that the pupil specified by the user is not selectable, processing proceeds to step S26.

If it is determined in step S14 that the pupil has not been detected, processing proceeds to step S25.

In step S25, the area setting unit 214 determines whether a face has been detected. If it is determined in step S25 that a face has been detected, processing proceeds to step S26.

In step S26, the area setting unit 214 sets the face area as the focus area.

If it is determined in step S25 that a face has not been detected, processing proceeds to step S27.

In step S27, the area setting unit 214 determines whether or not the in-focus area is being tracked. If it is determined in step S27 that the in-focus area is being tracked, the processing proceeds to step S28.

In step S28, the area setting unit 214 sets the focus area based on the tracking results.

After steps S24, S26, and S28, processing proceeds to step S32 in Figure 9.

On the other hand, if it is determined in step S27 that the focus area has not been tracked, processing proceeds to step S29 in Figure 9.

In step S29, the area setting unit 214 determines whether AF is set to be performed using a normal AF frame. If it is determined in step S29 that AF is set to be performed using a normal AF frame, processing proceeds to step S30.

In step S30, the area setting unit 214 sets the area specified by the normal AF frame as the focus area.

In step S31, the area setting unit 214 determines whether or not tracking is set to be performed using a normal AF frame. If it is determined in step S31 that tracking is set to be performed using an area specified by a normal AF frame, processing proceeds to step S32.

In step S32, the subject tracking unit 213 tracks the in-focus area and outputs the tracking results to the area setting unit 214 and the display control unit 215. Tracking here is performed on the areas set as in-focus areas in steps S24, S26, S28, and S30 (detected or tracked pupil area, face area, area specified by a normal AF frame). After step S32, processing proceeds to step S33.

If it is determined in step S31 that tracking is not set to be performed using the normal AF frame, processing proceeds to step S33.

In step S33, the system controller 131 determines whether or not a command to start focusing has been issued. If it is determined in step S33 that a command to start focusing has been issued, processing proceeds to step S34.

In step S34, the system controller 131 controls the lens driver 121 to drive the optical system, such as the lens 101 and the aperture 102, so that the focus is on the in-focus area. Processing then proceeds to step S35.

In step S35, the display control unit 215 causes the display unit 141 to display an image based on the image data stored in the memory 211 as a live view image. The display control unit 215 also causes the display unit 141 to display a focus frame indicating the set focus area superimposed on the live view image.

If it is determined in step S33 that no instruction to start focusing has been given, processing proceeds to step S36.

In step S36, the display control unit 215 generates a focus preview frame indicating the area that is to be set as the focus area, and displays the focus preview frame on the display unit 141 by superimposing it on the live view image.

After steps S35 and S36, processing proceeds to step S37 in Figure 10.

Also, if it is determined in step S29 that the setting is not to perform AF using the normal AF frame, processing proceeds to step S37 in Figure 10. In this case, a frame indicating the in-focus area is not displayed on the live view image.

In step S37, the system controller 131 determines whether the shutter button has been fully pressed based on a signal corresponding to an operation input from the operation unit 146. If it is determined in step S37 that the shutter button has been fully pressed, processing proceeds to step S38.

In step S38, the image sensor 103 obtains an electrical signal for each pixel of the image by photoelectrically converting, on a pixel-by-pixel basis, light from the subject that is collected through an optical system such as the lens 101 and the aperture 102. The image signal, which is the electrical signal for each pixel of the image, is converted into digital image data via the analog signal processing unit 104 and the A/D conversion unit 105, and stored as captured image data in the memory 211 of the digital signal processing unit 106.

In step S39, the display control unit 215 displays an image based on the captured image data stored in the memory 211 on the display unit 141 as a captured image.

In step S40, the codec processing unit 216 encodes the captured image data stored in the memory 211. The codec processing unit 216 supplies the encoded image data to the storage unit 142.

In step S41, the codec processing unit 216 stores the encoded captured image data in the memory unit 142.

In step S42, the system controller 131 determines whether to terminate the processing, for example, whether the power button has been operated.

If it is determined in step S42 that processing is to be terminated, the imaging process is terminated.

If it is determined in step S42 that the processing should not be terminated, the processing returns to step S11 in Figure 8 and the subsequent processing is repeated.

As described above, in the person's eye detection mode, if the eye area cannot be detected, the face area is set as the focus area.

<Image capture processing in animal pupil detection mode>
11 to 13 are flowcharts illustrating the imaging process of the imaging device 100 in the animal pupil detection mode.

The imaging process in the animal pupil detection mode shown in Figures 11 to 13 begins when the power is turned on, for example, by operating the power button. The detection mode for each specific part of the subject is set in advance as the animal pupil detection mode on a settings screen, etc. Figures 11 to 13 illustrate an example in which the animal pupil is detected by the animal detection unit 212-2, which detects the pupils of a cat or dog.

In step S61 of Figure 11, an image is acquired and displayed, similar to the processing in step S11 of Figure 8.

In step S62, the animal detection unit 212-2 uses the animal dictionary to detect the pupil area from the image data stored in the memory 211. The animal detection unit 212-2 supplies the detection result (information on the detected pupil area) to the area setting unit 214 and the display control unit 215.

In step S63, the region setting unit 214 determines whether the pupil region has been detected. If it is determined in step S63 that the pupil region has been detected, processing proceeds to step S64.

In step S64, the area setting unit 214 uses the pupil detection results to perform subsequent processing to set the focus area.

In step S65, the area setting unit 214 determines whether the left/right pupil selection setting is auto. If it is determined in step S65 that the left/right pupil selection setting is auto, processing proceeds to step S66.

In step S66, the area setting unit 214 automatically selects the pupil area and sets it as the focus area.

<Selecting the pupil area when set to auto>
FIG. 14 is a diagram showing an example of selection of pupil regions when the left/right pupil selection in step S66 of FIG. 11 is set to auto.

Figure 14 shows images P1 to P7 of a fox as the subject. The foxes in images P1 to P7 have different facial orientations (angles). The solid rectangles indicate the foreground pupil regions detected in each image, and the dashed rectangles indicate the background pupil regions detected in each image.

Image P1 shows a fox facing to the left. Image P2 shows a fox facing diagonally forward to the left. Image P3 shows a fox facing slightly diagonally forward to the left. Image P4 shows a fox facing straight ahead. Image P5 shows a fox facing slightly diagonally forward to the right. Image P6 shows a fox facing diagonally forward to the right. Image P7 shows a fox facing to the right.

Of these images, images P1, P2, P6, and P7, which show a fox facing diagonally forward or to the side (left or right), each show cases in which only the pupil area located in front of the imaging device 100 was detected.

In images P3 and P5, which show a fox facing slightly diagonally forward, two pupil areas are detected in each image, and it is easy to determine whether one pupil area is located closer to the image capture device 100 and the other pupil area is located further back.

In image P4, which shows a fox facing forward, it is difficult to determine which pupil area, the left or right, is in the foreground. In this case, as described above, the pupil area that is in the foreground as seen from the image capture device 100 and closest to the center of the focus frame is selected, and a focus preview frame or focus frame is displayed in the selected pupil area.

Returning to Figure 11, if it is determined in step S65 that the left/right pupil selection setting is not auto, processing proceeds to step S67.

In step S67, the area setting unit 214 determines whether the pupil specified by the user is selectable. For example, if the specified pupil area can be detected or the detection accuracy of the specified pupil area is higher than a predetermined threshold, it is determined in step S67 that the manually specified pupil is selectable, and processing proceeds to step S68.

In step S68, the area setting unit 214 sets one of the pupil areas specified by the user as the focus area.

If it is determined in step S67 that one of the pupil areas specified by the user is not selectable, processing proceeds to step S69.

In step S69, the area setting unit 214 sets the other pupil as the focus area.

If it is determined in step S63 that the pupil area has not been detected, processing proceeds to step S70.

In step S70, the area setting unit 214 determines whether the in-focus area is being tracked. If it is determined in step S70 that the in-focus area is being tracked, the processing proceeds to step S71.

In step S71, the area setting unit 214 sets the focus area based on the tracking results.

After steps S66, S68, S69, and S71, processing proceeds to step S75 in Figure 12.

If it is determined in step S70 that the focus area cannot be tracked, processing proceeds to step S72 in Figure 12.

In step S72, the area setting unit 214 determines whether AF is set to be performed using a normal AF frame. If it is determined in step S72 that AF is set to be performed using a normal AF frame, processing proceeds to step S73.

In step S73, the area setting unit 214 sets the area specified by the normal AF frame as the focus area.

In step S74, the area setting unit 214 determines whether or not tracking is set to be performed using a normal AF frame. If it is determined in step S74 that tracking is set to be performed using a normal AF frame, processing proceeds to step S75.

In step S75, the subject tracking unit 213 tracks the in-focus area. Tracking here is performed on the areas set as in-focus areas in steps S66, S68, S69, S71, and S73 (detected or tracked pupil area, face area, area specified by a normal AF frame). Processing then proceeds to step S76.

If it is determined in step S74 that tracking is not set to be performed using the normal AF frame, step S75 is skipped and processing proceeds to step S76.

If it is determined in step S72 that the setting is not to perform AF using the normal AF frame, processing proceeds to step S80 in Figure 13.

Note that steps S76 in Figure 12 to S85 in Figure 13 are basically the same processes as steps S33 in Figure 9 to S42 in Figure 10, so their explanation will be omitted to avoid repetition.

As described above, in animal pupil detection mode, if one of the selected pupil areas cannot be detected, the other pupil area is set as the focus area.

<Image capture processing in bird eye detection mode>
FIG. 15 is a flowchart illustrating the imaging process of the imaging device 100 in the bird's eye detection mode.

The imaging process in the bird's eye detection mode in Figure 15 begins, for example, when the power is turned on by operating the power button. The detection mode for each specific part of the subject is set in advance as the bird's eye detection mode on a settings screen, etc. Figure 15 illustrates an example in which the bird's eye is detected by the bird detection unit 212-3 that detects the bird's eye.

In step S101 of Figure 15, an image is acquired and displayed, similar to the processing in step S11 of Figure 8.

In step S102, the bird detection unit 212-3 uses a bird dictionary to detect the pupil area from the image data stored in the memory 211. The bird detection unit 212-3 supplies the detection results to the area setting unit 214 and the display control unit 215.

In step S103, the region setting unit 214 determines whether the pupil region has been detected. If it is determined in step S103 that the pupil region has been detected, processing proceeds to step S104.

In step S104, the area setting unit 214 uses the pupil detection results to perform subsequent processing to set the focus area.

In step S105, the area setting unit 214 automatically selects the pupil area and sets it as the focus area.

If it is determined in step S103 that the pupil area has not been detected, processing proceeds to step S106.

In step S106, the area setting unit 214 determines whether or not the in-focus area is being tracked. If it is determined in step S106 that the in-focus area is being tracked, the process proceeds to step S107.

In step S107, the area setting unit 214 sets the focus area based on the tracking results.

After steps S105 and S107, processing proceeds to step S111 in Figure 16.

If it is determined in step S106 that the focus area cannot be tracked, processing proceeds to step S108 in Figure 16.

In step S108, the area setting unit 214 determines whether AF is set to be performed using a normal AF frame. If it is determined in step S108 that AF is set to be performed using a normal AF frame, processing proceeds to step S109.

In step S109, the area setting unit 214 sets the area specified by the normal AF frame as the focus area.

In step S110, the area setting unit 214 determines whether or not tracking is set to be performed using a normal AF frame. If it is determined in step S110 that tracking is set to be performed using a normal AF frame, processing proceeds to step S111.

In step S111, the subject tracking unit 213 tracks the focus area. Tracking here is performed on the areas set as focus areas in steps S105, S107, and S109 (detected or tracked pupil area, face area, area specified by a normal AF frame). Processing then proceeds to step S112.

If it is determined in step S110 that tracking is not set to be performed using the normal AF frame, step S111 is skipped and processing proceeds to step S112.

If it is determined in step S108 that the setting is not to perform AF using the normal AF frame, processing proceeds to step S116 in Figure 17.

Note that steps S112 in Figure 16 to S121 in Figure 17 are basically the same processes as steps S33 in Figure 9 to S42 in Figure 10, so their explanation will be omitted to avoid duplication.

As described above, in bird eye detection mode, if the eye area cannot be detected, it will be set as the focus area in normal AF mode, i.e., the area specified by the normal AF frame will be set as the focus area.

<5. Other>
<Detection target switching setting screen>
FIG. 18 is a diagram showing an example of a detection target switching setting screen.

The detection target switching setting screen is a screen for setting detection modes that can be switched by performing a specified operation among detection modes for specific parts of the subject.

In Figure 18, from left to right, a person icon and a check box below the person icon, an animal icon and a check box below the animal icon, and a bird icon and a check box below the bird icon are displayed.

On the detection target switching screen, you can select the icon of each subject by moving cursor C. You can then check or uncheck the checkbox below the selected subject icon.

Then, when the execute button located at the bottom right of the screen is pressed, the detection mode that has been checked will be set as the detection mode that can be selected in response to user operation.

In addition, subject detection modes that the user does not need can be disabled by unchecking the checkbox. In other words, subject detection modes with unchecked checkboxes can be excluded from selection. This allows users to use modes according to their preferences.

In other words, if the checkbox below the person icon is checked, the person's eye detection mode is enabled and becomes selectable.

If the checkbox below the animal icon is checked, the Animal Eye Detection mode, which detects animal eyes, will be enabled and selectable.

If a check mark is entered in the checkbox below the bird icon, the detection mode of the imaging device 11 will be enabled and selectable as the bird eye detection mode, which detects the bird's eyes.

In addition, on the detection target switching screen, it is not limited to setting the detection of animal eyes or bird eyes, but it may also be possible to set the detection of specific parts of the subject, such as the eyes, face, part of the face, neck, head, or the entire body (subject), of any living creature such as fish, reptiles, and amphibians.

In addition, the subject to be detected is not limited to living things; it may be possible to set the detection of specific parts of the subject, such as a vehicle's headlights, front emblem, windshield, handlebars, driver or driver's seat, or a motorcycle's headlights or helmet.

<Display screen when detection target is switched>
In Fig. 18, the check boxes below the person icon, the animal icon, and the bird icon are checked. Therefore, as shown in Fig. 19, the detection mode of the imaging device 11 can be switched to any one of the person pupil detection mode, the animal pupil detection mode, and the bird pupil detection mode in response to a user operation.

Figure 19 shows an example of the display screen when the detection target is switched.

In Figure 19, the left side shows the display screen Mode_H when switched to human pupil detection mode. The center shows the display screen Mode_A when switched to animal pupil detection mode. The right side shows the display screen Mode_B when switched to bird pupil detection mode. Each display screen displays a person, an animal, and a bird.

In display screen Mode_H, a focus preview frame PF is displayed around the face of the person to be detected before the focus start command is issued.

In display screen Mode_A, a focus preview frame PF is displayed around the eyes of the animal to be detected before the focus start command is given.

In display screen Mode_B, the detection target is a bird, and a focus preview frame PF, which is displayed before the focus start command is issued, is displayed around the eye of the bird to be detected.

In the imaging device 11, when the detection mode is human pupil detection mode and the display screen Mode_H is displayed, if the user operates a specified button, the detection mode is switched to animal pupil detection mode and the display screen Mode_H changes to the display screen Mode_A.

In the imaging device 11, when the detection mode is animal pupil detection mode and the display screen Mode_A is displayed, if the user operates a specified button, the detection mode is switched to bird pupil detection mode and the display screen Mode_A changes to the display screen Mode_B.

In the imaging device 11, when the detection mode is bird pupil detection mode and the display screen Mode_B is displayed, if a user operates a predetermined button, the detection mode is switched to human pupil detection mode and the display screen Mode_B is changed to the display screen Mode_H.

<Example of subject type hierarchy>
In the above explanation, the human pupil detection mode, animal pupil detection mode, and bird detection mode are managed at the same hierarchical level, but for example, the bird detection mode may be managed at a hierarchical level below the animal pupil detection mode.

Figure 20 shows the hierarchical structure of subject types.

In the example of Figure 20, people and animals are managed at the same level. Dogs, cats, small animals, and birds are managed at a level below animals.

In this way, it is possible to manage each subject type in a hierarchical structure.

<Suboptimal means for each detection mode>
In each of the above processes, if the pupil area is not detected, the focusing process is performed on the face area, the focusing process is performed on the other pupil area, or the focusing process is performed under other conditions such as the normal AF frame.

The action taken when the pupil region is not detected is not limited to the above process, and may be differentiated by user settings. In this case, the action taken when the pupil region is not detected is changed to the "next best option" depending on the target subject or specific part that was not detected.

For example, the location on which a user wants to focus varies depending on the type of subject and the user's preferences. Also, the parts that can be recognized may differ depending on the subject.

Specifically, for people, the "face," "right pupil," and "left pupil" can be recognized. For animals, the "right pupil," "left pupil," and "nose" (internal) can be recognized. For birds, the "pupil (front pupil)" can be recognized. For cars, the "headlights," "windshield," and "driver" can be recognized.

Furthermore, there are "right pupils" and "left pupils." Even if the imaging device 11 is instructed to focus on the "right pupil," if the "right pupil" cannot be seen (detected), the "next best" method will differ depending on the type of subject.

For example, if the subject is an animal and the imaging device 11 is instructed to focus on the "right eye," if the "right eye" is not visible, the next best option would be to focus on the "face," but the focus will end up on the high "nose." However, focusing on the "left eye," which is visible, is preferable to focusing on the "nose."

When the subject is a person, even if the imaging device 11 is instructed to focus on the "right pupil," if the "right pupil" is not visible, it will feel more natural to the user to focus on the "face" because people do not have as high "noses" as animals. Also, when the subject is a person, if the imaging device 11 focuses on the visible "pupil" as with an animal, the user will feel that the imaging device 11 is ignoring the user's instructions. Therefore, when the subject is a person, the "face" is preferable as the next best option.

Also, there are subjects such as cars, where the components of the subject differ from car to car.

There may be limitations or differences in the characteristics of the recognition method, such as being able to recognize only the "nearest eye," as in the case of birds. In this case, the area preset by the user becomes the next best option.

As described above, by individually controlling the "next best means" depending on the subject, it is possible to automatically focus in line with the user's intentions. It is also possible to automatically select the "next best means" that can be provided to the user depending on different characteristics.

As described above, with this technology, the area of a first specific part of the subject is set as the focus area in response to a user's operation. Furthermore, if the area of the first specific part is not detected, the area of a second specific part of the subject, which is a specific part of the same type as the first specific part, is set as the focus area.

In addition, in this technology, if the area of the first specific part is not detected, an area according to the type of subject is set as the focus area.

This allows the focus to be automatically adjusted in line with the user's intentions.

Note that while the above explanation focuses on the process of detecting the eyes of animals such as dogs and cats, this technology can be applied to the eyes, face, parts of a face, neck, head, or other specific parts of a subject, or the entire body (subject), of all living creatures, including birds, fish, reptiles, and amphibians. This technology can also be applied to specific parts of these subjects or combinations of subjects.

Furthermore, this technology is not limited to living organisms; it can also be applied to specific parts of a subject, such as a vehicle's headlights, front emblem, windshield, or driver's seat, or a motorcycle's headlights or helmet.

In these cases, a detection mode for detecting specific parts of the subject is set in advance and used. This allows the user to communicate their intentions to the imaging device, such as which of multiple detection results or detection methods to prioritize, or which of multiple subjects to prioritize.

Unlike people, when focusing on the pupils of animals with long fur, the focus may end up on the hair that covers the pupils instead of the pupils. In this case, you can adjust the focal position further back or set the imaging device in advance to focus on hair as a subject that is easy to focus on, and then have the imaging device focus based on that setting, thereby obtaining imaging results that better reflect the user's intentions.

The above-described series of processes can be executed by hardware or software. When the above-described series of processes are executed by software, the programs that make up the software are installed from a network or recording medium.

This recording medium may be, for example, as shown in Figure 1, a removable recording medium 148 on which the program is recorded, which is distributed separately from the device itself for distribution to users. This removable recording medium 148 includes magnetic disks (including flexible disks) and optical disks (including CD-ROMs and DVDs). It also includes magneto-optical disks (including MDs (Mini Discs)) and semiconductor memories.

In this case, the program can be installed in the memory unit 142 by inserting the removable recording medium 148 into the drive 147.

The program can also be provided via a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received via the communication unit 145 and installed in the storage unit 142.

In addition, this program can also be pre-installed in the memory unit 142 or ROM (Read Only Memory) in the system controller 131.

The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or it may be a program that processes in parallel or at the required timing, such as when called.

In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device with multiple modules housed in a single housing, are both systems.

The embodiments of this technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of this technology.

For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

In addition, each step described in the above flowchart can be performed on a single device, or can be shared and executed by multiple devices.

Furthermore, if one step includes multiple processes, the multiple processes included in one step can be executed by one device, or can be shared and executed by multiple devices.

Please note that the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

<Configuration combination example>
The present technology can also be configured as follows.
(1)
a setting unit that sets a first area including a first specific portion of the subject as a focusing area;
When the first specific part is not detected, the setting unit sets a second area including a second specific part of the subject, which is a specific part of the same type as the first specific part, as the focusing area.
(2)
The imaging device according to (1), wherein the first specific portion is one of the left and right pupils of the subject, and the second specific portion is the other pupil.
(3)
The imaging device according to (2), wherein a user can set the first specific portion to the left pupil or the right pupil of the subject.
(4)
The imaging device according to any one of (1) to (3), wherein the subject is an animal.
(5)
The imaging device described in any one of (1) to (4), wherein the setting unit sets the second area as the focus area when the detection accuracy of the first specific area is lower than a predetermined threshold even when the first specific area is detected.
(6)
The imaging device according to (1), wherein the setting unit sets an area according to a type of the subject as the focus area when the first specific portion is not detected.
(7)
The imaging device described in (6), wherein, when the first specific part is not detected, the setting unit sets the second area or a third area including the first specific part and a third specific part of the subject that is larger than the first specific part as the focusing area depending on the type of the subject.
(8)
The imaging device according to (7), wherein the first specific part is one of the left and right pupils, the second specific part is the other pupil, and the third specific part is a face.
(9)
The imaging device described in (6), wherein, when the first specific portion is not detected, the setting unit sets the focusing area to the second area, a third area including the first specific portion and including a third specific portion of the subject that is larger than the first specific portion, or a fourth area specified by user operation, depending on the type of the subject.
(10)
The imaging device according to any one of (6) to (9), wherein the type of the subject is a person, an animal, or a bird.
(11)
The imaging device according to any one of (1) to (10), wherein the type of the subject to be detected can be set in advance.
(12)
The imaging device according to any one of (1) to (11), further comprising a region detection unit that detects at least the first region using a dictionary according to the type of the subject.
(13)
Further comprising a tracking unit that tracks at least the first area,
The imaging device according to any one of (1) to (12), wherein the setting unit sets the second area as the in-focus area when the first area cannot be tracked.
(14)
a focus instruction unit that instructs the start of focusing for each image capture unit; and
The imaging device according to any one of (1) to (13), further comprising: a display control unit that controls, when a command to start focusing is given, display of information indicating the focusing area.
(15)
an imaging instruction unit that instructs imaging;
and a focus control unit that controls the imaging unit so that, when the imaging is instructed, the imaging unit focuses on the focus area set by the setting unit and performs the imaging.
(16)
The imaging device
A first area including a first specific portion of the subject is set as a focus area to be focused on;
If the first specific portion is not detected, a second area including a second specific portion of the subject that is the same type as the first specific portion is set as the focused area.
(17)
causing the computer to function as a setting unit that sets a first region including a first specific portion of the subject as a focusing region;
The setting unit sets a second area including a second specific part of the subject, which is a specific part of the same type as the first specific part, as the focus area when the first specific part is not detected.

100 Imaging device, 101 Lens, 102 Aperture, 103 Imaging element, 104 Analog signal processing unit, 105 A/D conversion unit, 106 Digital signal processing unit, 121 Lens driver, 131 System controller, 141 Display unit, 142 Storage unit, 146 Operation unit, 211 Memory, 212 Subject detection unit, 212-1 Person detection unit, 212-2 Animal detection unit, 212-3 Bird detection unit, 213 Subject tracking unit, 214 Area setting unit, 215 Display control unit, 216 Codec processing unit

Claims (14)

a setting unit that sets a first pupil region including one of the left and right pupils of the subject as a focusing region;
When the setting unit cannot select the first pupil region as the focus region according to the detection result of the one pupil , in a first mode according to the type of subject, the setting unit sets a second pupil region including the other of the left and right pupils as the focus region. 2. The imaging device according to claim 1, wherein the setting unit sets a face area as the focus area when the first pupil area cannot be selected as the focus area based on the detection result of the one pupil in a second mode according to the type of the subject. 3. The imaging device according to claim 2, wherein, in a third mode according to the type of subject, when the first pupil region cannot be selected as the focus region according to the detection result of the one pupil, the setting unit sets the region of the focus frame as the focus region . The imaging device according to claim 3 , wherein the first mode, the second mode, and the third mode can be switched by a user operation . the first mode is a mode in which the type of subject is an animal;
the second mode is a mode in which the type of subject is a person,
The imaging device according to claim 3 , wherein the third mode is a mode in which the type of subject is a bird . The imaging device according to claim 1 , wherein the setting unit sets the second pupil region as the focus region when the one pupil is detected in the first mode with a detection accuracy lower than a predetermined threshold. In the first mode, when the second pupil region cannot be selected as the focusing region according to the detection result of the other pupil, the setting unit sets the region of the focusing frame as the focusing region.
The imaging device according to claim 1 . In the second mode, the setting unit first detects the face area and then detects the first pupil area for the detected face area.
The imaging device according to claim 2 . further comprising a tracking unit that tracks at least the first pupil region;
2. The imaging device according to claim 1, wherein when a user performs an operation to set the second pupil region as the focus region but the second pupil region cannot be selected as the focus region, the setting unit sets the first pupil region based on a tracking result as the focus region. 10. The imaging device according to claim 9, wherein the setting unit changes the in-focus area from the first pupil area to the second pupil area when the second pupil area changes from a state in which it cannot be selected to a state in which it can be selected while the first pupil area is being tracked . a focus instruction unit that instructs the start of focusing for each image capture unit;
The imaging device according to claim 1 , further comprising: a display control unit that controls display of information indicating the in-focus area when the start of the focusing is instructed. an imaging instruction unit that instructs imaging;
The imaging device according to claim 1 , further comprising: a focus control unit that controls the imaging unit so that, when the imaging is instructed, the imaging unit focuses on the focus area set by the setting unit and performs the imaging. The imaging device
setting a first pupil region including one of the left and right pupils of the subject as a focusing region;
When the first pupil region cannot be selected as the focus region according to the detection result of the one pupil , in a first mode according to the type of the subject, a second pupil region including the other of the left and right pupils is set as the focus region .
An imaging method comprising : On the computer,
setting a first pupil region including one of the left and right pupils of the subject as a focusing region;
When the first pupil region cannot be selected as the focus region according to the detection result of the one pupil , in a first mode according to the type of the subject, a second pupil region including the other of the left and right pupils is set as the focus region .
A program for executing a process including :