JP6060486B2 - Focus adjustment device, imaging device, and focus adjustment program - Google Patents

Description

The present invention relates to a focus adjustment device, an imaging device, and a focus adjustment program for focusing on a subject.

  Japanese Patent Application Laid-Open No. 2004-151820 discloses an imaging apparatus that aims to focus on a subject even when the subject moves in the direction of the imaging apparatus.

JP 2009-69748 A

  However, the imaging apparatus disclosed in Patent Document 1 detects a subject by performing image matching on each pixel. For this reason, the imaging device disclosed in Patent Document 1 has a problem that when the subject rotates, the images do not match and the subject cannot be focused with high accuracy.

SUMMARY An advantage of some aspects of the invention is that it provides a focus adjustment device, an imaging device, and a focus adjustment program that can accurately focus on a subject.

One embodiment of the present invention has been made to solve the above problem, and includes a first image of a photographed subject and a second image of a subject photographed after the first image. When the extraction unit that extracts the specific region, the evaluation value based on the contrast of the specific region extracted from the first image, and the evaluation value based on the contrast of the specific region extracted from the second image are different, A control unit that controls a lens used for focus adjustment so that the specific region extracted from the second image is in focus, and the control unit adjusts the contrast of the specific region extracted from the second image. The focus adjustment device limits the amount of movement of the lens based on the evaluation value based on it and the correspondence relationship between the contrast evaluation value and the position of the lens stored in advance in a storage unit .
In one embodiment of the present invention, an extraction unit that extracts a specific region from a first image of a photographed subject and a second image of a subject photographed after the first image, and the first When the evaluation value based on the contrast of the specific area extracted from the image is different from the evaluation value based on the contrast of the specific area extracted from the second image, the evaluation value for the specific area extracted from the second image A control unit that controls a lens used for focus adjustment so as to be in focus, and the control unit stores an evaluation value based on a contrast of a specific area extracted from the second image and a storage unit in advance. The image pickup apparatus restricts the amount of movement of the lens based on the correspondence between the contrast evaluation value and the position of the lens .

In one embodiment of the present invention, a specific region is extracted from a first image of a photographed subject and a second image of a subject photographed after the first image in a computer of the focus adjustment apparatus. If the procedure and the evaluation value based on the contrast of the specific area extracted from the first image are different from the evaluation value based on the contrast of the specific area extracted from the second image, it is extracted from the second image. A control program for controlling a lens used for focus adjustment so as to focus on a specific area , wherein the procedure for controlling the lens is extracted from the second image. The amount of movement of the lens is limited by the evaluation value based on the contrast of the specific area and the correspondence relationship between the contrast evaluation value and the position of the lens stored in advance in the storage unit. Is a focus adjustment program.

According to the present invention, the focus adjustment device and the imaging device can focus on the subject with high accuracy.

It is a block diagram showing the structure of the imaging device, the lens barrel, and the storage medium in the first embodiment of the present invention. It is a flowchart showing the operation | movement procedure of the focus adjustment apparatus in 1st Embodiment of this invention. It is a figure for demonstrating the timing made to focus when the position of a label area | region changes in the image in 1st Embodiment of this invention. It is a flowchart showing the operation | movement procedure of the focus adjustment apparatus in 2nd Embodiment of this invention. It is a figure for demonstrating the edge detection operator in 3rd Embodiment of this invention. It is a figure showing the outline of a label area | region and the window containing the whole outline in 3rd Embodiment of this invention. It is a figure showing frequency distribution of contrast evaluation value for every color component in a 3rd embodiment of the present invention. It is a figure showing frequency distribution of contrast evaluation value for every object distance in a 3rd embodiment of the present invention. It is a figure showing the relationship between the peak frequency of contrast evaluation value and a to-be-photographed object distance in 3rd Embodiment of this invention. It is a flowchart showing the operation | movement procedure of the focus adjustment apparatus in 3rd Embodiment of this invention. It is a flowchart showing the operation | movement procedure of the focus adjustment apparatus in 3rd Embodiment of this invention. It is a figure for demonstrating the outline of a label area | region in 3rd Embodiment of this invention. It is a figure showing the window containing a part of outline of a label area | region in 4th Embodiment of this invention. It is a figure showing the several window containing the whole outline of a label area | region in 4th Embodiment of this invention. It is a figure showing the several window which respectively contains the part of the outline of a label area | region in 4th Embodiment of this invention. It is a figure showing the change of the contrast evaluation value for every window in 4th Embodiment of this invention.

[First Embodiment]
A first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of the imaging device, the lens barrel, and the storage medium. The imaging apparatus 100 captures an optical image (subject image) via the lens barrel 111 and stores the obtained image in the storage medium 200 as a still image or a moving image.

First, the configuration of the lens barrel will be described.
The lens barrel 111 includes a focusing lens (hereinafter referred to as “AF (Auto Focus) lens”) 112, a lens driving unit 116, an AF encoder 117, and a lens barrel control unit 118. The lens barrel 111 may be detachably connected to the imaging device 100 or may be integrated with the imaging device 100.

  The AF lens 112 is driven by a lens driving unit 116 and guides an optical image of a subject to a light receiving surface (photoelectric conversion surface) of an imaging element 119 (described later) of the imaging unit 110 included in the imaging device 100.

  The AF encoder 117 detects the movement of the AF lens 112 and outputs a signal corresponding to the movement amount of the AF lens 112 to the lens barrel control unit 118. Here, the signal corresponding to the movement amount of the AF lens 112 is a sine wave signal whose phase changes according to the movement amount of the AF lens 112, for example.

  The lens barrel control unit 118 controls the lens driving unit 116 in accordance with a drive control signal input from the imaging device 100. Here, the drive control signal is a control signal for driving the AF lens 112 in the optical axis direction. The lens barrel control unit 118 changes the number of steps of the pulse voltage output to the lens driving unit 116 according to the drive control signal.

  Further, the lens barrel control unit 118 outputs the position (focus position) of the AF lens 112 in the lens barrel 111 to the imaging device 100 based on a signal corresponding to the movement amount of the AF lens 112. Here, the lens barrel control unit 118 calculates the movement amount (position) of the AF lens 112 in the lens barrel 111 by accumulating signals according to the movement amount of the AF lens 112 according to the movement direction.

  The lens driving unit 116 drives the AF lens 112 according to control by the lens barrel control unit 118 and moves the AF lens 112 in the optical axis direction within the lens barrel 111.

Next, the configuration of the imaging apparatus will be described.
The imaging apparatus 100 includes an imaging unit 110, an image processing unit 140, a display unit 150, a buffer memory unit 130, an operation unit 180, a storage unit 160, a CPU 190, a communication unit 170, and a focus adjustment device 191. Is provided.

  The imaging unit 110 includes an imaging element 119 and an A / D (Analog / Digital) conversion unit 120. Each unit of the imaging unit 110 is controlled by the CPU 190 based on the set imaging conditions (for example, an aperture value and an exposure value).

  The image sensor 119 includes a photoelectric conversion surface. The image sensor 119 converts an optical image formed on the photoelectric conversion surface by the lens barrel 111 (optical system) into an electric signal, and outputs the converted electric signal to the A / D conversion unit 120. The image sensor 119 is configured by, for example, a CMOS (Complementary Metal Oxide Semiconductor). Further, the image sensor 119 may convert an optical image formed on a part of the photoelectric conversion surface into an electric signal (cut out).

  Further, the image sensor 119 stores an image obtained when a shooting instruction is received via the operation unit 180 in the storage medium 200 via the A / D conversion unit 120. On the other hand, the imaging device 119 outputs a continuously obtained image as a through image to the focus adjustment device 191 and the display unit 150 via the A / D conversion unit 120 in a state where the imaging instruction is not received. .

The A / D conversion unit 120 digitizes the electrical signal converted by the image sensor 119 and outputs an image represented by the digitized electrical signal to the buffer memory unit 130.
The operation unit 180 accepts an operation input by the user and outputs a signal corresponding to the operation input to the CPU 190. The operation unit 180 includes, for example, a power switch, a shutter button, a multi selector (cross key), or other operation keys as a user interface (UI).

The image processing unit 140 refers to the image processing conditions stored in the storage unit 160 and performs image processing on the image temporarily stored in the buffer memory unit 130. The image processing unit 140 stores the image subjected to the image processing in the storage medium 200 via the communication unit 170.
The display unit 150 displays an image obtained by the imaging unit 110, an operation screen, and the like. The display unit 150 is, for example, a liquid crystal display.

The buffer memory unit 130 temporarily stores an image captured by the imaging unit 110.
The storage unit 160 stores the determination condition referred to by the CPU 190 when determining the scene to be imaged and the imaging condition (for example, aperture value, exposure value) associated with each determined imaging scene. Remember.

  The CPU 190 controls the imaging unit 110 according to the set imaging conditions. Further, the CPU 190 causes the image processing unit 140 to perform image processing as a still image or a moving image based on the “signal according to the operation input” input from the operation unit 180.

The communication unit 170 is connected to a removable storage medium 200 (for example, a memory card), and executes writing, reading, or erasing of information (image data or the like) on the storage medium 200.
The storage medium 200 is a storage unit that is detachably connected to the imaging apparatus 100 and stores various types of information (image data and the like). Note that the storage medium 200 may be integrated with the imaging device 100.

Next, the configuration of the focus adjustment device will be described.
The focus adjustment device 191 acquires a plurality of frame images (moving images, through images) with different captured times. Further, the focus adjustment device 191 focuses on an area in which a subject is captured in a frame image and an area to which a label (identification information) is assigned (hereinafter referred to as “label area”).

  The focus adjustment device 191 includes an acquisition unit 192 and a control unit 193. The acquisition unit 192 acquires a plurality of frame images captured at different times via the AF lens 112 from the image sensor 119 via the A / D conversion unit 120 as moving images or through images.

  A plurality of frame images acquired by the acquisition unit 192 are input to the control unit 193. The control unit 193 (subject region extraction unit) extracts a region (main subject region) in which the main subject is captured in the frame image from the frame image based on the image feature amount. Here, the image feature amount is, for example, a region distribution (position, shape (moment of inertia)), a region area, a number of regions, a defocus amount (subject distance), a motion vector (difference data), a hue (color). Component), saturation, brightness, and texture (pattern). The control unit 193 (calculation unit) detects (calculates) the contour line (edge) and contrast evaluation value of the extracted main subject region based on the image feature amount.

  In addition, the control unit 193 uses the image feature amount used for the frame image captured at the relatively previous time, with respect to the frame image (subsequent frame image) captured at the relatively later time. Is also used to extract a region where the subject is imaged in the frame image. Accordingly, the control unit 193 can easily compare with image matching based on the difference between frame images even if the subject's apparent size, brightness, area, or shape changes due to the subject's forward / backward movement or rotation. Regions can be extracted. Note that the controller 193 may relax the condition of the image feature amount when it is difficult to extract a region in a frame image (subsequent frame image) captured at a later time.

  In addition, the control unit 193 focuses on the area where the subject is imaged by executing hill-climbing autofocus (mountain-climbing AF) on the frame image. In addition, the control unit 193 causes the storage unit 160 to store the contrast evaluation value for each color component detected during hill-climbing autofocus as the initial value of the contrast evaluation value for each color component.

  When the initial value of the contrast evaluation value is stored in the storage unit 160, that is, when the subject is imaged at least once, the control unit 193 assigns a label to the extracted region, The extracted area is set as a label area (labeling). In addition, the control unit 193 determines a window (ranging frame) circumscribing the label area as an image as a range in which a contrast evaluation value is detected when executing contrast autofocus (contrast AF). The window is, for example, a rectangular frame.

  In addition, the control unit 193 controls the position of the AF lens 112 to focus on each label region of a plurality of frame images at different captured times. Here, the control unit 193 controls the position of the AF lens 112 in the lens barrel 111 by outputting a drive control signal to the barrel control unit 118.

  The control unit 193 (feature amount comparison unit) was photographed at a relatively later time with a label region (main subject region) of a frame image photographed at a relatively earlier time among a plurality of frame images. A comparison is made with the label area of the frame image to determine whether or not the position of the label area has changed in the frame image. Here, the control unit 193 detects the contrast evaluation value regardless of whether or not the position of the label area in the frame image has changed. This is because, even if the position of the label area in the frame image does not change, the contrast evaluation value changes according to the subject distance when the distance between the subject and the imaging device 100 (subject distance) changes. Because.

  When the position of the label area in the frame image captured at the relatively previous time is different from the position of the label area in the frame image captured at the relatively later time, the control unit 193 relatively moves backward. The focus is again focused on the label area in the frame image (subsequent frame image) captured at the time of.

Next, the operation procedure of the focus adjustment device will be described.
FIG. 2 is a flowchart showing an operation procedure of the focus adjustment apparatus. FIG. 3 is a diagram for explaining the timing for focusing when the position of the label area changes in the image.
It is assumed that the operation unit 180 receives an instruction to change the state to the tracking mode by an operation input. Here, the tracking mode is an operation mode for tracking a subject in a moving image or a through image. The operation unit 180 outputs a signal indicating that the tracking mode has been received to the control unit 193. Thereby, the control unit 193 makes a state transition to the tracking mode (FIG. 2: step S1).

  Further, it is assumed that the operation unit 180 accepts designation of a subject to be tracked by the control unit 193 by an operation input (camera user interface (camera UI)). Here, the subject to be tracked by the control unit 193 may be specified based on the coordinates of the subject in the moving image or the through image, or information indicating the subject distance (for example, whether or not the macro shooting is performed). Information)) (FIG. 2: Step S2).

  The control unit 193 performs hill-climbing autofocus on the area where the designated subject is imaged, and focuses the area once or more. Further, the control unit 193 extracts a label area from the frame image based on the color component (RGB value) of the subject (FIG. 2: step S3). Further, the control unit 193 determines a window (ranging frame) circumscribing the label area as a frame image (FIG. 2: step S4). Here, it is assumed that the window 500 circumscribing the label region 400 (see FIG. 3) is defined as a frame image 300 (initial frame image) captured at a relatively previous time.

  The control unit 193 integrates the contrast evaluation value in the window 500 for each color component (FIG. 2: Step S5). In addition, the control unit 193 stores the contrast evaluation value integrated for each color component in the storage unit 160 (FIG. 2: step S6). In addition, the control unit 193 controls the position of the AF lens 112 by performing contrast autofocus within the window 500, and focuses the label area 400 (FIG. 2: Step S7).

  And the control part 193 acquires the image (following frame image) imaged at the later time relatively (FIG. 2: step S8). Here, it is assumed that the control unit 193 has acquired the frame image 301 (see FIG. 3).

  The control unit 193 determines whether or not the position of the label region 400 has changed in the frame image 301 acquired this time (FIG. 2: step S9). When the position of the label region 400 has not changed (step S9—NO), the control unit 193 acquires the frame image 301 acquired this time based on the same condition as the condition of the image feature amount used for the frame image 300 acquired last time. Are labeled (FIG. 2: Step S10). Further, the control unit 193 integrates the contrast evaluation value in the window 500 for each color component (FIG. 2: step S11), and returns the process to step S8 in FIG.

  On the other hand, when the position of the label area 400 has changed in step S9 (FIG. 2: step S9-YES), the control unit 193 executes contrast autofocusing within the window 500, whereby the AF lens 112 is moved. The position is controlled to focus on the label area 400 (FIG. 2: step 12). In the frame image 301 (see FIG. 3), the control unit 193 controls the position of the AF lens 112 by executing contrast / autofocus within the window 500, and matches the label area 400 whose position has changed. To burn. And the control part 193 returns a process to step S8 of FIG.

As described above, the focus adjustment device 191 acquires a plurality of images captured through the AF lens 112 at different shooting times, and the feature amount of the image acquired by the acquisition unit 192. Then, a control unit 193 (subject region extraction unit) that extracts a main subject region, and the image taken at a later time relative to the main subject region of an image taken at a relatively earlier time among the plurality of images. A control unit 193 (feature amount comparison unit) that compares the main subject region that has been performed, and a control unit 193 that controls the AF lens 112 according to the comparison result by the control unit 193 (feature amount comparison unit).
With this configuration, the focus adjustment device can easily detect a subject as compared with image matching based on the difference between images (tolerant to movement or rotation of the subject), and a label extracted based on an image feature amount Since the area is focused, it is possible to focus on the subject with high accuracy. Further, the focus adjustment device can track the subject captured in the image even when the subject moves or rotates.

In addition, the control unit 193 (subject region extraction unit) includes at least one of the distribution of the region that is the image feature amount, the area of the region, the number of regions, the defocus amount, the motion vector, the color component, the saturation, the brightness, and the texture. The label area is extracted based on the one.
As a result, the focus adjustment device can easily detect the subject and accurately focus on the subject even when the apparent area, shape, or amount of light of the subject changes, compared to image matching based on the difference between images. be able to.

In addition, the control unit 193 differs in the position of the label area (main subject area) of the image taken at a relatively previous time and the position of the main subject area of the image taken at a later time. In this case, the AF lens 112 is controlled to focus on the main subject area of the image (subsequent frame image) taken at a later time.
With this configuration, when the position of the label area changes, the focus adjustment device refocuses the label area. Accordingly, the focus adjustment apparatus focuses the subject at an appropriate timing, and focuses the subject more frequently than necessary. Therefore, the moving image and the through image can be displayed easily.

[Second Embodiment]
A second embodiment of the present invention will be described in detail with reference to the drawings. The second embodiment is different from the first embodiment in that the subject distance changes instead of the subject position in the image not changing. Only the differences from the first embodiment will be described below.

  The control unit 193 (calculation unit) calculates the contrast evaluation value of the extracted label area (main subject area). The control unit 193 (see FIG. 1) moves the AF lens 112 back and forth in the optical axis direction (wobbling). The control unit 193 is in a state of focusing on the foreground of the subject (hereinafter referred to as “front pin”) or focusing on the background of the subject based on the increase or decrease of the contrast evaluation value due to wobbling. It is determined whether it is in a state (hereinafter referred to as “rear pin”).

  The control unit 193 compares the contrast evaluation value of the label area (main subject area) in the frame image captured at a relatively previous time and the frame image (subsequent frame image) captured at a relatively later time. When the contrast evaluation value of the label area (main subject area) is different, the label area in the subsequent frame image is focused again. Here, the control unit 193 restricts the movement direction of the AF lens 112 to either the front pin direction or the rear pin direction based on the determination result of whether it is the front pin or the rear pin, and the window The label area is focused by executing contrast autofocus on the inside.

  In addition, the control unit 193 sets the contrast evaluation value of the label area in the frame image captured at a relatively previous time as an initial value, and in a frame image (subsequent frame image) captured at a relatively later time. By controlling the position of the AF lens 112 until the contrast evaluation value of the label area becomes equal to or higher than the initial value, the label area is focused again.

  Here, S0 is “initial value of contrast evaluation value”, S1 is “contrast evaluation value in subsequent frame image”, ΔS is (S1−S0) or (S1 / S0), and K1 is “predetermined. Threshold ”.

  When ΔS> K1 is satisfied, the control unit 193 determines that focus is not achieved (out of focus). Further, the control unit 193 controls the position of the AF lens 112 until the expression is not satisfied, thereby refocusing the label area.

Next, the operation procedure of the focus adjustment device will be described.
FIG. 4 is a flowchart showing an operation procedure of the focus adjustment apparatus. It is assumed that the operation unit 180 receives an instruction to change the state to the tracking mode by an operation input. The operation unit 180 outputs a signal indicating that the tracking mode has been received to the control unit 193. Thereby, the control unit 193 makes a state transition to the tracking mode (FIG. 4: step Sa1). Further, it is assumed that the operation unit 180 accepts designation of a subject to be tracked by the control unit 193 by an operation input (camera UI) (FIG. 4: step Sa2).

  The control unit 193 performs hill-climbing autofocus on the area where the designated subject is imaged, and focuses the area once or more by hill-climbing autofocus. Further, the control unit 193 extracts a label area from the frame image based on the color component (RGB value) of the subject (FIG. 4: step Sa3). Further, the control unit 193 determines a window (ranging frame) circumscribing the label area as an image (FIG. 4: Step Sa4).

  The control unit 193 integrates, for each color component, the contrast evaluation value in the window defined for the frame image (initial frame image) captured at a relatively previous time (FIG. 4: step Sa5). Also, the control unit 193 stores the contrast evaluation value integrated for each color component in the storage unit 160 (FIG. 4: step Sa6).

  Then, the control unit 193 acquires a frame image (subsequent frame image) captured at a relatively later time (FIG. 4: step Sa8). Further, the control unit 193 integrates the contrast evaluation value in the window defined for the frame image acquired this time for each color component (FIG. 4: step Sa9).

  The control unit 193 determines whether or not the contrast evaluation value is reduced (deteriorated) in the frame image acquired this time (FIG. 4: Step Sa10). When the contrast evaluation value is reduced (step Sa10-YES), the control unit 193 performs labeling on the frame image acquired this time based on the same condition as the condition of the image feature amount used for the previously acquired frame image ( FIG. 4: Step Sa11). Further, the control unit 193 integrates the contrast evaluation value in the window defined for the frame image acquired this time for each color component (FIG. 4: step Sa12).

  Further, the control unit 193 determines whether or not the contrast evaluation value is reduced in the frame image acquired this time (FIG. 4: step Sa13). When the contrast evaluation value is reduced (step Sa13—YES), the control unit 193 determines whether it is the front pin or the rear pin based on the increase / decrease in the contrast evaluation value due to wobbling.

  Further, based on the determination result, the control unit 193 restricts the moving direction of the AF lens 112 to either the front pin direction or the rear pin direction, and executes contrast autofocus for the inside of the window. Focus on the area. Here, the control unit 193 focuses on the label area by executing contrast autofocus while limiting the movement amount of the AF lens 112 based on the defocus amount (blur amount) of the front pin or the rear pin. (FIG. 4: Step Sa15). In this way, the focus adjustment device limits the amount of movement of the AF lens 112, so that the label area can be focused with high accuracy in a short time.

On the other hand, when the contrast evaluation value is not reduced in step Sa10 (step Sa10-YES), the control unit 193 returns the process to step Sa8 in FIG.
If the contrast evaluation value is not reduced in step Sa13 (step Sa13-YES), the control unit 193 returns the process to step Sa8 in FIG.

As described above, the focus adjustment device 191 further includes a calculation unit that calculates the contrast evaluation value of the main subject region extracted by the control unit 193 (subject region extraction unit). The control unit 193 contrasts the main subject region of an image (subsequent frame image) captured at a later time relative to the contrast evaluation value of the main subject region of an image captured at a relatively earlier time. If the evaluation value is different, the main subject area of the image taken at the later time is focused.
With this configuration, when the contrast evaluation value changes, the focus adjustment device focuses on the label area, so that the subject can be focused accurately and in a short time. Further, the focus adjustment device can track the subject captured in the image even when the subject moves or rotates.

In addition, the control unit 193 sets the contrast evaluation value of the main subject area of the image captured at a relatively previous time as an initial value, and the contrast of the main subject area of an image captured at a relatively later time. By controlling the position of the AF lens 112 until the evaluation value becomes equal to or higher than the initial value, the label area (main subject area) is focused again.
With this configuration, the focus adjustment device determines the optimum AF lens position for focusing the label evaluation area on the basis of the contrast evaluation value based on the initial value. it can.

[Third Embodiment]
A third embodiment of the present invention will be described in detail with reference to the drawings. The third embodiment is different from the first and second embodiments in that focusing is performed based on the contrast evaluation value of the outline of the label region. Only the differences from the first and second embodiments will be described below.

First, the configuration of the focus adjustment device will be described.
The control unit 193 performs an operation using an edge detection operator (differentiation operator), and extracts the outline of the label area from the frame image for each color component. In addition, when the overall contrast evaluation value of the outline of the label area changes, the control unit 193 refocuses the label area.

  FIG. 5 is a diagram for explaining an edge detection operator. In FIG. 5, it is assumed that the pixel 600 and the pixel 601 are adjacent to each other. The image feature amount of the pixel 600 is “+1”, and the image feature amount of the pixel 601 is “0”. The edge detection operator 700 has a coefficient “−1” for the image feature amount of one adjacent pixel and a coefficient “+1” for the image feature amount of the other adjacent pixel.

  The control unit 193 (calculation unit) detects (calculates) a difference (inclination) in the image feature amount between adjacent pixels, and extracts a contour line of the label area based on the absolute value of the detected difference. Specifically, the control unit 193 multiplies the image feature amount “+1” of the pixel 600 by the coefficient “−1” of the edge detection operator 700. In addition, the control unit 193 multiplies the image feature amount “0” of the pixel 601 by the coefficient “+1” of the edge detection operator 700. Further, the control unit 193 adds the multiplication results and calculates an absolute value of the addition result. Hereinafter, the output value of the edge detection operator is referred to as “contrast evaluation value”. The contrast evaluation value is a value corresponding to the difference in the image feature amount (edge power) of the contour line of the label area.

  The control unit 193 extracts the outline of the label area from the frame image based on the calculated contrast evaluation value. FIG. 6 shows a contour line of the label area and a window including the whole contour line. FIG. 6 shows that the window 520a includes the entire outline 420a of the label area.

  The control unit 193 refocuses the label area based on the frequency distribution (histogram) of contrast evaluation values determined by the image feature amount for each pixel. In FIG. 7, the frequency distribution of contrast evaluation values is represented for each color component. The horizontal axis represents the contrast evaluation value. The vertical axis represents the frequency of contrast evaluation values. Here, frequency distributions of contrast evaluation values of red (R channel), green (G channel), and blue (B channel), which are color components included in the contour line, are represented.

  In determining whether or not to perform focus adjustment, a color component (color channel) having a frequency equal to or higher than a certain level is selected by the control unit 193. Note that the color component used for the determination may be selected based on the average value (Ave) or variance (σ) of the frequency distribution of the contrast evaluation values.

  The storage unit 160 (see FIG. 1) stores the contrast evaluation value and the position of the AF lens 112 in advance in association with each other. In addition, the storage unit 160 stores a contrast evaluation value for each color component in accordance with control by the control unit 193.

  In FIG. 8, the frequency distribution of contrast evaluation values is represented for each subject distance. FIG. 8A shows a frequency distribution at a subject distance of 2 [m]. FIG. 8B shows a frequency distribution at a subject distance of 2.2 [m]. FIG. 8C shows a frequency distribution at a subject distance of 2.5 [m]. FIG. 8D shows a frequency distribution at a subject distance of 4 [m]. As shown in FIG. 8, the dispersion of the frequency distribution increases as the subject distance increases.

FIG. 9 shows the relationship between the peak frequency of the contrast evaluation value and the subject distance. As shown in FIGS. 8 and 9, the peak frequency of the contrast evaluation value increases as the subject distance is shorter .

  The control unit 193 refers to the storage unit 160 that stores the contrast evaluation value and the position of the AF lens 112 in advance in association with each other, and acquires the prescribed number of steps of the pulse voltage from the storage unit 160. The control unit 193 limits the amount of movement of the AF lens 112 based on the number of steps of the pulse voltage defined for each contrast evaluation value. Accordingly, the control unit 193 can prevent the AF lens 112 from greatly exceeding the optimum position for focusing on the label area.

  In addition, the control unit 193 (feature amount comparison unit) selects a contrast evaluation value having a frequency equal to or greater than a certain value for each color component, and a frame image captured at a relatively previous time for all the selected color components. It is compared with the contrast evaluation value of the label area in (the frame image acquired last time). When the contrast evaluation value of the label area of the frame image captured at a later time (frame image acquired this time) is inferior, the control unit 193 performs a frame image captured at a later time (acquired this time). The frame area) is again focused.

Next, the operation procedure of the focus adjustment device will be described.
10 and 11 are flowcharts showing the operation procedure of the focus adjustment apparatus. FIG. 12 is a diagram for explaining the outline of the label area. It is assumed that the operation unit 180 receives an instruction to change the state to the tracking mode by an operation input. The operation unit 180 outputs a signal indicating that the tracking mode has been received to the control unit 193. As a result, the control unit 193 makes a state transition to the tracking mode (FIG. 10: step Sb1). Further, it is assumed that the operation unit 180 accepts designation of a subject to be tracked by the control unit 193 by an operation input (camera UI) (FIG. 10: step Sb2).

  The control unit 193 performs hill-climbing autofocus on the area where the designated subject is imaged, and focuses the area once or more. Furthermore, the control unit 193 extracts a label area from the frame image based on the color component (RGB value) of the subject (FIG. 10: Step Sb3). Further, the control unit 193 determines a window (ranging frame) circumscribing the label area as an image (FIG. 10: Step Sb4).

  The control unit 193 integrates the contrast evaluation value in the window of the frame image (initial frame image) captured at a relatively previous time for each color component (FIG. 10: Step Sb5). The control unit 193 integrates the contrast evaluation value in the window 510 of the frame image 310 (initial frame image) for each color component (see FIG. 12). Further, the control unit 193 stores the contrast evaluation value integrated for each color component in the storage unit 160 (FIG. 10: Step Sb6).

  The control unit 193 extracts the outline of the label area for each color component. The control unit 193 extracts the outline 410a of the label area for each color component (see FIG. 12). In addition, the control unit 193 stores the color component amount (edge power) of the contour line 410a in the storage unit 160 (FIG. 10: Step Sb7).

  The control unit 193 detects a frequency distribution (histogram) of contrast evaluation values (FIG. 10: Step Sb8). Further, the control unit 193 detects the peak frequency of the contrast evaluation value for each color component. Further, the control unit 193 calculates the variance of the frequency distribution (FIG. 10: Step Sb9).

  Then, the control unit 193 acquires a frame image (subsequent frame image) captured at a relatively later time (FIG. 10: step Sb10). Further, the control unit 193 labels the currently acquired frame image based on the same condition as the image feature amount used for the previously acquired frame image (FIG. 10: step Sb11). Further, the control unit 193 extracts a contour line from the currently acquired frame image based on the same condition as the image feature amount used for the previously acquired frame image (FIG. 10: step Sb12).

  The control unit 193 detects the average value and variance of the frequency distribution for each color component having a frequency equal to or greater than a certain value (FIG. 10: Step Sb13). Further, the control unit 193 determines whether or not the edge power has been reduced for all the detected color components (FIG. 10: Step Sb14). When the edge power is reduced for all the detected color components (FIG. 10: Step Sb14-YES), the control unit 193 outputs a drive control signal as wobbling to drive the AF lens 112 in the direction of the front pin (see FIG. 10). 11: Step Sb15).

  Then, the control unit 193 acquires a frame image (subsequent frame image) captured at a relatively later time (FIG. 11: step Sb16). Further, the control unit 193 labels the currently acquired frame image based on the same condition as the image feature amount used for the previously acquired frame image (FIG. 11: step Sb17). Further, the control unit 193 extracts a contour line from the currently acquired frame image based on the same condition as the image feature amount used for the previously acquired frame image (FIG. 11: step Sb18).

  The control unit 193 detects the average value and variance of the frequency distribution for each color component having a frequency equal to or higher than a certain level (FIG. 11: Step Sb19). Further, the control unit 193 determines whether or not the edge power has increased for all the detected color components (FIG. 11: Step Sb20). When the edge power increases (FIG. 11: Step Sb20—YES), the control unit 193 sets a front pin flag indicating that it is a front pin (FIG. 11: Step Sb21). On the other hand, when the edge power has not increased (FIG. 11: Step Sb20-NO), the control unit 193 sets a rear pin flag indicating that it is a rear pin (FIG. 11: Step Sb22).

  Further, the control unit 193 determines a window (ranging frame) circumscribing the label area as an image (FIG. 11: Step Sb23). The control unit 193 refers to the storage unit 160 that stores the contrast evaluation value and the position of the AF lens 112 in advance in association with each other, and acquires the prescribed number of steps of the pulse voltage from the storage unit 160. The control unit 193 limits the amount of movement of the AF lens 112 based on the specified number of steps of the pulse voltage.

  When the front pin flag is set in step Sb21, the control unit 193 drives the AF lens 112 in the rear pin direction by the number of steps of the specified pulse voltage as preparation for executing the contrast autofocus. On the other hand, when the rear pin flag is set in step Sb22, the control unit 193 drives the AF lens 112 in the front pin direction by the number of steps of the specified pulse voltage as preparation for executing the contrast autofocus (FIG. 11: Step Sb24).

  And the control part 193 acquires the frame image (following frame image) imaged at the later time relatively (FIG. 11: step Sb25). Further, the control unit 193 extracts a contour line from the currently acquired frame image based on the same condition as the image feature amount used for the previously acquired frame image (FIG. 11: step Sb26).

  The control unit 193 detects the average value and variance of the frequency distribution for each color component in which the frequency is equal to or greater than a certain level (FIG. 11: Step Sb27). Further, the control unit 193 executes contrast scan in contrast autofocus, and integrates the contrast evaluation value in the window of the frame image acquired this time for each color component (FIG. 11: Step Sb28). The control unit 193 determines whether or not the contrast evaluation value is greater than or equal to the contrast evaluation value in the focused state (FIG. 11: step Sb29).

  When the contrast evaluation value is less than the contrast evaluation value in the in-focus state (FIG. 11: Step Sb29-NO), whether or not the control unit 193 has detected the peak of the contrast evaluation value when executing the contrast scan. (FIG. 11: Step Sb30).

  When the peak of the contrast evaluation value is not detected (FIG. 11: Step Sb30-NO), the control unit 193 determines whether or not the scan area that is the range in which the AF lens 112 is driven in the contrast scan is finished ( FIG. 11: Step Sb31). When the scan area is completed (FIG. 11: Step Sb31-YES), the control unit 193 responds to the interpolation point determined by the interpolation method based on the contrast evaluation value corresponding to the position of the AF lens 112 in the contrast scan. The focus position is detected (FIG. 11: Step Sb32), and the AF lens 112 is moved to the detected focus position (FIG. 11: Step Sb33). The control unit 193 ends the contrast / autofocus (FIG. 11: Step Sb34), and returns the process to Step Sb10 of FIG.

On the other hand, when the edge power is reduced in all the detected color components in step Sb14 (FIG. 10: step Sb14-NO), the control unit 193 returns the process to step Sb10 in FIG.
In step Sb29, when the contrast evaluation value is greater than or equal to the contrast evaluation value in the in-focus state (FIG. 11: step Sb29-YES), the control unit 193 advances the process to step Sb34 in FIG.
Further, when the peak of the contrast evaluation value is detected in step Sb30 (FIG. 11: step Sb30-YES), the control unit 193 advances the process to step Sb32 in FIG.
In step Sb31, when the scan area has not ended (FIG. 11: step Sb31-NO), the control unit 193 returns the process to step Sb24 in FIG.

As described above, when the overall contrast evaluation value of the outline of the label area (main subject area) changes, the control unit 193 focuses the label area again.
With this configuration, the focus adjustment device clarifies the boundary between the subject and the background (the contour of the subject) based on the overall contrast evaluation value of the contour of the label area. It is possible to stably focus on the label area with high accuracy.

Further, the control unit 193 refers to the storage unit 160 that stores the contrast evaluation value and the position of the AF lens 112 in advance in association with each other, and limits the movement amount of the AF lens 112 based on the contrast evaluation value.
With this configuration, the focus adjustment device limits the movement amount of the AF lens based on the defocus amount of the front pin or the rear pin, so that the AF lens does not exceed the optimum position for focusing on the label area. The label area can be focused with high accuracy in a short time.

In addition, the control unit 193 (feature amount comparison unit) selects the contrast evaluation value for each color component, and images of all the selected color components are captured at a relatively previous time (a frame image acquired last time). ), When the contrast evaluation value of the main subject region of the image captured at a later time (the frame image acquired this time) is inferior, The main subject area of the image captured at a later time is refocused.
With this configuration, the focus adjustment device determines whether or not to perform focus adjustment based on the contrast evaluation value for each color component having a frequency equal to or greater than a certain value. Thus, the label area is accurately focused. be able to.

  In addition, the control unit 193 (calculation unit) detects a difference in image feature amount between adjacent pixels, and extracts a contour line of the main subject region based on an absolute value of the detected difference.

[Fourth Embodiment]
A fourth embodiment of the present invention will be described in detail with reference to the drawings. The fourth embodiment is different from the first to third embodiments in that focusing is performed based on the contrast evaluation value of the outline portion of the label area. Only the differences from the first to third embodiments will be described below.

  Further, in the following, for the sake of simplicity, the description will be made assuming that there is no change in the position of the contour line and no change in the amount of light among the plurality of frame images. The gist is unchanged. Similarly, in the following description, it is assumed that the characteristics of the contour line (for example, the composition of the color components) are the same characteristics over the entire circumference of the contour line, but even if these are not the same characteristics, the gist of the present invention Will not change.

  When the contrast evaluation value of a part of the outline of the label area changes (deteriorates), the control unit 193 refocuses the label area. FIG. 13 shows a window including a part of the outline of the label area. FIG. 13 shows that the window 530a and the window 530b include a part of the outline of the label region 420.

  Here, Sa0 is “initial value of contrast evaluation value in window 530a”, Sa is “contrast evaluation value in window 530a”, ΔSa is (Sa−Sa0) or (Sa / Sa0), and K2 is “predetermined. Threshold value ”. Similarly, Sb0 is “initial value of contrast evaluation value in window 530b”, Sb is “contrast evaluation value in window 530b”, ΔSb is (Sb−Sb0) or (Sb / Sb0), and K2 is “predetermined. Threshold value ”.

  When ΔSa> K2 and ΔSb> K2 are satisfied, the control unit 193 determines that the label area is not focused. The control unit 193 controls the position of the AF lens 112 until at least one of these expressions does not hold, thereby refocusing the label area.

  Further, K3 is assumed to be a “predetermined threshold value”. The control unit 193 determines that the label area is not focused even when (ΔSa + ΔSb)> K3 is satisfied. The control unit 193 controls the position of the AF lens 112 until the expression is not established, thereby refocusing the label area.

  When the contrast evaluation values of a plurality of portions in the outline of the label area change (deteriorate), the control unit 193 refocuses the label area. FIG. 14 shows a plurality of windows including the entire outline of the label area. FIG. 14 shows that the windows 540 a to 540 d include the entire outline of the label region 420.

  When the depth of the subject is within the depth of field (DOF) and the contrast evaluation value changes (deteriorates) in all windows, the control unit 193 refocuses the label area. The determination in this case is the same as the determination described with reference to FIG.

  In addition, when the contrast evaluation value is lower than the first threshold value in all of the plurality of portions in the outline of the label area, the control unit 193 refocuses the label area. Further, the control unit 193 refocuses the label area when the contrast evaluation value falls below the second threshold in a part of the plurality of portions in the outline of the label area.

  FIG. 15 shows a plurality of windows each including a contour line portion of the label area. FIG. 15 shows that the windows 550a to 550g each include the outline portion of the label region 420.

  FIG. 16 shows the change in contrast evaluation value for each window. In FIG. 16, a to g represent windows 550a to 550g. FIG. 16A shows the contrast evaluation value for each window in the initial stage of focusing. Here, an average value of the contrast evaluation values of the windows 550a to 550g is set as an initial value K4.

  FIG. 16B shows the contrast evaluation value for each window in which the in-focus state has changed (deteriorated). In FIG. 16B, it is assumed that the contrast evaluation values of the windows other than the window 550g have deteriorated with a change amount less than the initial value K4. Here, in all of the plurality of portions (windows 550a to 550f) in the outline of the label region 420, the contrast evaluation value does not fall below the second threshold value (the amount of change is less than the initial value K4), so the control unit 193 Determines that contrast autofocus is not executed.

  FIG. 16C shows the contrast evaluation value for each window in which the in-focus state has changed (deteriorated). For example, it is assumed that the contrast evaluation values of the windows 550c to 550f are deteriorated by a change amount equal to or larger than the initial value K4 in FIG. Here, since the contrast evaluation value is lower than the second threshold value (deteriorated by the amount of change equal to or greater than the initial value K4) in a part of the contour line of the label area 420 (windows 550c to 550f), the control unit In step 193, it is determined that contrast autofocus is to be executed, and the label area 420 is focused again.

  FIG. 16D shows the contrast evaluation value for each window in which the in-focus state has changed (deteriorated). For example, when the subject moves back and forth with respect to the imaging apparatus 100 (see FIG. 1), in FIG. 16D, the contrast evaluation values of the windows 550a to 550f are deteriorated by a change amount equal to or greater than the threshold value K5. To do. Here, in all of the plurality of portions (windows 550a to 550f) in the contour line of the label region 420, the contrast evaluation value is lower than the first threshold value (the change amount is equal to or greater than the threshold value K5). It is determined that the contrast / autofocus is to be executed, and the label area 420 is focused again.

As described above, the control unit 193 causes the label region 420 to be focused again when the contrast evaluation value of a part of the outline of the label region 420 changes.
With this configuration, the focus adjustment device obtains each contrast evaluation value from a different part in the label area (subject), so that even when the subject is tilted, deformed, or rotated, the subject can be accurately focused. .

When the contrast evaluation values of a plurality of portions in the contour line of the label area 420 change, the control unit 193 focuses the label area 420 again.
With this configuration, the focus adjustment device obtains each contrast evaluation value from different parts of the label area (subject) in the vertical and horizontal directions, so that the subject can be accurately focused even when the subject is tilted, deformed, or rotated. Can be made.

When the contrast evaluation value falls below the first threshold value in all of the plurality of portions (windows 550a to 550g), the control unit 193 focuses the label area 420 again.
With this configuration, the focus adjustment device can focus the subject with high accuracy even when the subject moves back and forth with respect to the imaging device 100.

The control unit 193 refocuses the label area 420 when the contrast evaluation value falls below the second threshold in a part of the plurality of portions in the outline of the label area 420.
With this configuration, the focus adjustment device can focus on the subject with high accuracy even when the subject is inclined obliquely.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, the control unit 193 may extract a contour line using another edge detection operator (for example, a Laplacian filter).

  The program for realizing the focus adjustment apparatus described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed, thereby executing the execution process. You may go. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Imaging part, 111 ... Lens barrel, 112 ... AF lens, 116 ... Lens drive part, 117 ... AF encoder, 118 ... Lens-barrel control part, 119 ... Imaging element, 120 ... A / D conversion , 130 ... Buffer memory unit, 140 ... Image processing unit, 150 ... Display unit, 160 ... Storage unit, 170 ... Communication unit, 180 ... Operation unit, 190 ... CPU, 191 ... Focus adjustment device, 192 ... Acquisition unit, 193 ... Control unit, 200 ... Storage medium, 300-302, 310 ... Frame image, 400,410 ... Label area, 410a, 420a ... Outline of label area, 520, 530a-530b, 540a-540d ... Window, 550a-550g ... ranging frame, 600, 601 ... pixel, 700 ... edge detection operator

Claims (10)

An extraction unit that extracts a specific area from the first image of the photographed subject and the second image of the subject photographed after the first image;
If the evaluation value based on the contrast of the specific area extracted from the first image is different from the evaluation value based on the contrast of the specific area extracted from the second image, the specific area extracted from the second image A control unit for controlling a lens used for focus adjustment so as to be focused on,
The control unit is configured so that the evaluation value based on the contrast of the specific region extracted from the second image and the correspondence relationship between the contrast evaluation value and the lens position stored in advance in the storage unit are Focus adjustment device that limits the amount of movement .   The extraction unit is based on at least one of distribution, area, number, defocus amount, motion vector, color component, saturation, brightness, and texture of regions included in the first image and the second image. The focus adjustment apparatus according to claim 1, wherein the specific area is extracted.   When the position of the specific area of the first image in the first image is different from the position of the specific area of the second image in the second image, the control unit is configured to specify the specific area of the second image. The focus adjusting apparatus according to claim 1, wherein the lens is controlled so as to be focused on.   When the evaluation value based on the contrast of the outline of the specific area extracted from the first image is different from the evaluation value based on the contrast of the outline of the specific area extracted from the second image 4. The focus adjustment apparatus according to claim 1, wherein the lens is controlled so as to focus on a specific area extracted from the second image. 5.   The control unit evaluates based on an evaluation value based on a contrast of a part of the contour line of the specific area extracted from the first image and an evaluation based on a contrast of a part of the contour line of the specific area extracted from the second image. The focus adjustment apparatus according to claim 4, wherein when the value is different, the lens is controlled to focus on a specific area extracted from the second image.   The controller is configured to evaluate an evaluation value based on a contrast of a plurality of portions of a contour line of a specific region extracted from the first image and an evaluation based on a contrast of a plurality of portions of the contour line of a specific region extracted from the second image. The focus adjustment apparatus according to claim 4 or 5, wherein when the value is different, the lens is controlled to focus on a specific area extracted from the second image.   The control unit according to claim 6, wherein the control unit controls the lens to focus on a specific region extracted from the second image when the evaluation values of the plurality of portions are below a first threshold value. Focus adjustment device.   The said control part controls the said lens so that it may focus with respect to the specific area | region extracted from the said 2nd image, when the said evaluation value of a part of said several part is less than a 2nd threshold value. The focus adjustment device described in 1. An extraction unit that extracts a specific area from the first image of the photographed subject and the second image of the subject photographed after the first image;
If the evaluation value based on the contrast of the specific area extracted from the first image is different from the evaluation value based on the contrast of the specific area extracted from the second image, the specific area extracted from the second image A control unit for controlling a lens used for focus adjustment so as to be focused on,
The control unit is configured so that the evaluation value based on the contrast of the specific region extracted from the second image and the correspondence relationship between the contrast evaluation value and the lens position stored in advance in the storage unit are An imaging device that limits the amount of movement . To the computer of the focusing device,
A procedure for extracting a specific area from a first image of a photographed subject and a second image of a subject photographed after the first image;
If the evaluation value based on the contrast of the specific area extracted from the first image is different from the evaluation value based on the contrast of the specific area extracted from the second image, the specific area extracted from the second image Controlling the lens used for focus adjustment to focus on
A focus adjustment program for executing
The procedure for controlling the lens is based on the evaluation value based on the contrast of the specific area extracted from the second image and the correspondence relationship between the contrast evaluation value and the position of the lens stored in advance in the storage unit. Limit the amount of lens movement
Focus adjustment program.

Images