JP6635764B2 - Imaging device, control method thereof, and control program - Google Patents

Description

  The present invention relates to an image pickup apparatus including a tiltable lens or a tiltable image pickup device, a control method thereof, and a control program.

  2. Description of the Related Art Conventionally, the following first or second method has been used to obtain images focused on a plurality of subjects at different distances from an imaging device during photographing. The first method is a method of obtaining a depth of field that can cover a plurality of subjects by an appropriate aperture value (see Patent Document 1). Hereinafter, photographing using the first technique is referred to as normal photographing.

  On the other hand, in the second method, the imaging optical system is tilted with respect to the imaging element surface by tilting the imaging optical system using the Scheimpflug's law described later, so that the distance from the imaging element differs. A focused image is obtained for the subject (see Patent Document 2). Hereinafter, photographing using the second technique is referred to as tilt photographing.

JP-A-62-52538 JP-A-5-53166

  By the way, in an imaging device such as a surveillance camera, in order to recognize a subject even in a dark place (low illuminance environment), an aperture is opened as much as possible and an image is taken at a small F value to obtain a high signal to noise ratio (SNR). It is necessary to secure. However, when there are a plurality of subjects, as described later, which of the normal shooting and the tilt shooting can focus on all the subjects with a smaller F value changes depending on the arrangement state of the subjects.

  Therefore, in an environment in which the state of the subject changes moment by moment, as described in Patent Documents 1 and 2, an imaging apparatus using either normal shooting or tilt shooting may correspond to the situation of the subject. Have difficulty. As a result, especially in a dark place, shooting is continued in an inappropriate shooting mode.

  Therefore, an object of the present invention is to provide an imaging apparatus capable of automatically selecting a shooting mode having a good SNR in shooting in a dark place regardless of the situation of a subject, a control method thereof, and a control program. is there.

In order to achieve the above object, an imaging device according to the present invention includes an imaging device in which an optical image is formed via an imaging optical system, and is an imaging device that obtains an image in accordance with an output of the imaging device. A first photographing mode for performing photographing by performing a tilt control for controlling an angle between a lens main surface of the imaging optical system and an image forming surface, and a state where an optical axis of the imaging optical system is orthogonal to the image forming surface; Selecting means for selecting any one of a second shooting mode for shooting in; a distance measuring means for obtaining distance information by measuring a distance from the imaging device to a subject present in a shooting range of the image; A first calculating unit for obtaining a tilt angle and a first aperture value for focusing on a plurality of subjects existing in the shooting range in the first shooting mode; Focus on multiple subjects in the shooting range And second calculating means for obtaining a second aperture value for the first photographing mode based on a comparison result between the first aperture value and the second aperture value. It is characterized in that one of the second photographing modes is selected as a selective photographing mode.

A control method according to the present invention is a control method for an imaging apparatus that includes an imaging element on which an optical image is formed via an imaging optical system and obtains an image in accordance with an output of the imaging element. A first imaging mode for performing imaging by performing tilt control for controlling an angle between a lens main surface and an imaging plane, and a second imaging mode for performing imaging in a state where an optical axis of the imaging optical system is orthogonal to the imaging plane. A step of selecting one of the two photographing modes; a step of measuring a distance from the image pickup apparatus to a subject existing in a photographing range of the image to obtain distance information; A first calculation step for obtaining a tilt angle and a first aperture value for focusing on a plurality of subjects existing in the shooting range in the shooting mode; and a tilt angle in the shooting range in the second shooting mode. Focus on multiple subjects A second calculation step of calculating a second aperture value of the first and second photographing modes. In the selecting step, the first photographing mode and the second photographing mode are selected based on a comparison result between the first aperture value and the second aperture value. It is characterized in that one of the second photographing modes is selected as a selective photographing mode.

A control program according to the present invention is a control program used in an imaging device that includes an imaging device on which an optical image is formed via an imaging optical system and obtains an image in accordance with an output of the imaging device. A first photographing mode in which a computer provided with a computer performs a tilt control for controlling an angle between a lens main surface of the imaging optical system and an image forming surface to perform photographing, and an optical axis of the imaging optical system and the image forming. A selection step of selecting one of a second shooting mode in which shooting is performed in a state where the surfaces are orthogonal to each other, and obtaining distance information by measuring a distance from the imaging device to a subject existing in a shooting range of the image. A distance measurement step; a first calculation step for obtaining a tilt angle and a first aperture value for focusing on a plurality of subjects existing in the shooting range in the first shooting mode; and the second calculation step. Shooting mode Performing a second calculation step for obtaining a second aperture value for focusing on a plurality of subjects present in the shooting range at the time of shooting, and selecting the first aperture value and the second aperture value in the selection step. And selecting one of the first photographing mode and the second photographing mode as a selective photographing mode based on a comparison result with the aperture value.

  According to the present invention, it is possible to appropriately select an aperture value while focusing on a subject existing in a shooting range. As a result, an image having a good SNR in shooting in a dark place or the like is obtained regardless of the situation of the subject. be able to.

It is a figure for explaining Scheimpflug's law, (a) is a figure showing arrangement of an image sensor and a lens in normal photography, and a positional relationship of a focal plane, and (b) inclined a lens with respect to an image sensor. It is a figure showing a state. FIG. 3 is a diagram illustrating an example of an image obtained in a first shooting scene. FIG. 3 is a diagram showing a model of an arrangement relationship of subjects in the shooting scene shown in FIG. 2. FIG. 4 is a diagram illustrating a state in which focusing is performed on all subjects by normal shooting with respect to the arrangement of the subjects illustrated in FIG. 3. FIG. 4 is a diagram showing a state in which focusing is performed on all the subjects by tilting photographing with respect to the arrangement of the subjects shown in FIG. 3. FIG. 9 is a diagram illustrating an example of an image obtained in a second shooting scene. FIG. 7 is a diagram showing a model of an arrangement relationship of subjects in the shooting scene shown in FIG. 6. FIG. 8 is a diagram illustrating a positional relationship between a center of a focal plane and a subject during normal shooting in the arrangement of the subject illustrated in FIG. 7. FIG. 8 is a diagram illustrating a state in which focusing is performed on all the subjects by tilt-up photographing with respect to the arrangement of the subjects illustrated in FIG. 7. FIG. 1 is a block diagram illustrating a configuration of an example of a camera according to a first embodiment of the present invention. 11 is a flowchart illustrating an example of a photographing process performed by the camera illustrated in FIG. FIG. 2 is a diagram illustrating an example of an apparent position (apparent coordinate) set by a subject coordinate calculation unit illustrated in FIG. 1. FIG. 2 is a diagram illustrating an example of z-axis coordinates set by a subject coordinate calculation unit illustrated in FIG. 1. FIG. 2 is a diagram illustrating an arrangement relationship between an image sensor illustrated in FIG. 1, a lens, and a focal plane. It is a block diagram showing the composition about an example of the camera by a 2nd embodiment of the present invention. 16 is a flowchart illustrating an example of a shooting process performed by the camera illustrated in FIG. It is a block diagram showing composition of a photographic subject detection part used by a camera by a 3rd embodiment of the present invention. 11 is a flowchart illustrating a shooting process performed by a camera according to a third embodiment of the present invention. It is a block diagram showing the composition about an example of the camera by a 4th embodiment of the present invention. 20 is a flowchart illustrating a shooting process performed by the camera illustrated in FIG. 19. FIG. 20 is a diagram illustrating an example of a mask range set in an image displayed on the display unit illustrated in FIG. 19.

  Hereinafter, an example of an imaging device according to an embodiment of the present invention will be described with reference to the drawings.

  Here, before describing the imaging apparatus according to the embodiment of the present invention, Scheimpflug's law will be outlined.

  FIG. 1 is a diagram for explaining Scheimpflug's law. FIG. 1A is a diagram showing an arrangement of an image sensor and a lens and a positional relationship between a focal plane and a normal imaging. FIG. 1B is a diagram illustrating a state in which the lens is tilted with respect to the imaging element.

  In FIG. 1A, the image sensor 200a and the lens 201a are arranged so that the image sensor surface (imaging surface) and the lens main surface are parallel to each other, and the focusing surface 202a, the image sensor 200a, and the lens 201a are all parallel. On the other hand, FIG. 1B shows the positional relationship between the arrangement of the image sensor 200b and the lens 201b and the focusing surface 202b in the tilt photographing using Scheimpflug's law, and the lens 201b with respect to the image sensor 200b. Is tilted.

  The point at which the extension of the image sensor 200b and the extension of the focusing surface 202b intersect is referred to as CP. At this time, if the lens 201b is tilted so that an extension of the lens main surface passes through the point CP, it is possible to focus on the focusing surface 202b. This is called Scheimpflug's law.

  Here, the above-mentioned Scheimpflug's law will be described using two shooting scenes as an example.

  FIG. 2 is a diagram illustrating an example of an image obtained in the first shooting scene.

  In an image (photographed image) 10 obtained in the first photographing scene, subjects 11, 12, 15, and 16 stand on the first floor area of the building, and subjects 13, 14, 17, and 18 represent the buildings. Standing on the second floor area.

  FIG. 3 is a diagram showing a model of the arrangement relationship of subjects in the shooting scene shown in FIG.

  In the illustrated example, the subjects 11 to 18 are approximated by black spheres having a diameter w1. Each of the subjects 11 to 18 is positioned at the vertex of a cube having a side length of d1 and captures an image of the illustrated cube by an imaging device (hereinafter simply referred to as a camera) 19.

  FIG. 4 is a diagram illustrating a state in which all the subjects are focused by normal shooting with respect to the arrangement of the subjects shown in FIG.

  In normal photographing (second photographing mode), the center 20 of the focal plane is set at the midpoint of the side defined by the subjects 11 and 12, and the subjects 13 and 14, the subjects 15, 16 and the subjects 17 and 18, respectively. Set to a plane passing through the midpoint of the specified side. At this time, the depth of field L1 necessary for keeping all the subjects within the focusing range is L1 = d1 + w1.

  FIG. 5 is a diagram showing a state in which focusing is performed on all the subjects by tilt-up photographing with respect to the arrangement of the subjects shown in FIG.

  In the tilt shooting (first shooting mode), the focal plane center 21 is set to a plane passing through the subjects 11, 14, 15, and 18. At this time, the depth of field L2 necessary for keeping all the subjects within the focusing range is L2 = √2 × d1 + w1.

  Comparing the depths of field L1 and L2, since the depth of field L1 <the depth of subject L2, in the first shooting scene, normal shooting can be performed with a smaller F-number (aperture value). it can.

  Note that, in the illustrated example, a special situation is assumed to explain the difference between the normal shooting and the tilt shooting, but there are various similar situations in an actual shooting scene. This also applies to a second shooting scene described later.

  FIG. 6 is a diagram illustrating an example of an image obtained in the second shooting scene.

  In the image 10 obtained in the second shooting scene, the subjects 11 to 16 stand on the first floor area of the building. These subjects 11 to 16 are arranged in a line in the depth direction of the screen.

  FIG. 7 is a diagram showing a model of the arrangement relationship of subjects in the shooting scene shown in FIG.

  Here, the subjects 11 to 16 are approximated by circles having a diameter w1. The subjects 11 to 16 are arranged at a fixed distance d3.

  FIG. 8 is a diagram illustrating the positional relationship between the center of the focal plane and the subject during normal shooting in the arrangement of the subject shown in FIG.

  As shown in the figure, the center 23 of the focal plane is located between the subjects 13 and 14, and the depth of field d4 required to keep all the subjects within the focusing range is d4 = 5 × d3 + w1.

  FIG. 9 is a diagram showing a state in which all the subjects are focused on by the tilting photographing for the arrangement of the subjects shown in FIG.

  Here, the center 24 of the in-focus plane includes a straight line connecting the centers of the subjects 11 to 16. The depth of field d5 necessary for keeping all the subjects within the focusing range is d5 = w1. Comparing the depths of field d4 and d5, since depth of field d5 <subject depth d4, in the second photographing scene, the tilting photographing can be performed with a smaller F-number.

  As described above, which of normal shooting and tilt shooting can focus on all subjects with a smaller F value changes depending on the arrangement of the subjects.

[First Embodiment]
FIG. 10 is a block diagram illustrating a configuration of an example of the camera according to the first embodiment of the present invention.

  The camera 100 has an imaging optical system 101 having a tilt mechanism, and an optical image (subject image) is formed on the imaging device 102 such as a CCD, a CMOS sensor, or an InGaAs sensor via the imaging optical system 101. . The imaging optical system control unit 111 performs focus control, tilt control, exposure control, and the like of the imaging optical system 101 under the control of the system controller 112.

  The image sensor 102 outputs an image signal (analog signal) corresponding to the optical image. The image processing unit 103 converts the image signal into a digital image signal by A / D conversion. Then, the image processing unit 103 performs predetermined image processing such as gamma correction and color balance adjustment on the digital image signal to generate image data such as JPEG.

  Image data output from the image processing unit 103 is sent to the subject detection unit 104 and the system controller 112. Then, the system controller 112 displays the image data on the display unit 113 as a video.

  The subject detection unit 104 detects the subject by analyzing the image data. Then, the subject detection unit 104 sends the image data together with the subject detection result to the subject coordinate calculation unit 105. The subject coordinate calculation unit 105 holds in advance the vertical and horizontal dimensions of the image sensor 102, the pixel pitch dimension, and the permissible circle of confusion, and also holds focal length information indicating the focal length of the imaging optical system 101.

  The subject coordinate calculation unit 105 first generates subject information indicating the position of the subject on the image using the subject detection result and the image data. Then, the subject coordinate calculation unit 105 sends the subject information to the system controller 112.

  The system controller 112 controls the distance measuring unit 110 based on the subject information to measure each subject. Then, the distance measurement unit 110 sends the distance measurement result obtained by measuring the distance to the object to the object coordinate calculation unit 105. The subject coordinate calculation unit 105 obtains three-dimensional coordinates for each of the subjects with reference to the distance measurement result. The subject coordinate calculation unit 105 sends the three-dimensional coordinates together with the image data to the tilt-focus imaging plane calculation unit 106 and the normal shooting depth-of-field calculation unit 108.

  The tilt-focus imaging plane calculation unit 106 obtains a plane formula indicating a focus plane necessary for performing tilt-shift imaging based on the three-dimensional coordinates of the subject. Then, the tilt-focus imaging plane calculation unit 10 sends the plane expression indicating the focus plane and the three-dimensional coordinates of the subject to the tilt-shift depth-of-field calculation unit 107. The tilt shooting depth-of-field calculation unit 107 calculates an F value (hereinafter, referred to as tilt shooting F value) necessary for focusing on all subjects based on a planar expression indicating a focal plane and three-dimensional coordinates of the subject. ).

  In the tilt photographing which is the first photographing mode, tilt control for controlling an angle (tilt angle) between a lens main surface of the imaging optical system 101 and an imaging plane formed by the imaging optical system 101 is performed. Shooting is performed. Note that by controlling the angle between the lens main surface and the image plane, the angle between the optical axis of the image sensor 102 and the image plane also changes. Further, the image forming surface is formed on the image sensor 102.

  On the other hand, the normal photographing depth-of-field calculating unit 108 calculates an F-number (hereinafter, referred to as a normal photographing F-number) necessary for focusing on all subjects when performing normal photographing based on the three-dimensional coordinates of the subject. Ask. The tilt shooting depth-of-field calculating unit 107 and the normal shooting depth-of-field calculating unit 108 send the tilt shooting F-number and the normal shooting F-number to the shooting mode selecting unit 109, respectively.

Note that, in the normal imaging which is the second imaging mode, imaging is performed in a state where the optical axis of the imaging optical system 101 is orthogonal to the imaging plane formed by the imaging optical system 101.

  The photographing mode selection unit 109 compares the F value for tilt photographing and the F value for normal photographing, and selects the smaller F value as the selected F value (selected aperture value). Then, the shooting mode selection unit 109 selects a shooting mode corresponding to the selected F value, and sends the selected shooting mode to the system controller 112 as the selected shooting mode. The system controller 112 controls the imaging optical system control unit 111 to drive the imaging optical system 101 based on the selected shooting mode.

  FIG. 11 is a flowchart illustrating an example of a photographing process performed by the camera illustrated in FIG. The processing according to the illustrated flowchart is performed under the control of the system controller 112.

  When the photographing process is started, the system controller 112 determines whether or not a subject has been detected by the subject detection unit 104 (Step S2). If no subject is detected in the image (NO in step S2), system controller 112 waits. On the other hand, if a subject is detected in the image (YES in step S2), system controller 112 controls distance measuring section 110 to measure the distance of all detected subjects (step S3).

  Subsequently, under the control of the system controller 112, the subject coordinate calculation unit 105 calculates the actual subject position according to the apparent position of the subject on the image (screen) (step S4).

  FIG. 12 is a diagram illustrating an example of an apparent position (apparent coordinate) set by the subject coordinate calculation unit illustrated in FIG. 1.

  In the captured image shown in FIG. 12, a plurality of subjects 301 to 304 exist. When calculating the subject positions of the subjects 301 to 304 from the apparent positions of the subjects 301 to 304, the subject coordinate calculation unit 105 sets the X axis in the horizontal direction and the Y axis in the vertical direction with the center of the captured image 300 as the origin. . Note that the unit of the X axis and the Y axis is the number of pixels.

  The subject coordinate calculation unit 105 assigns coordinates (X1, Y1), (X2, Y2), (X3, Y3), and (X4, Y4) to the subjects 301 to 303 in XY coordinates, respectively. Hereinafter, these coordinates will be referred to as apparent coordinates on the image.

  FIG. 13 is a diagram illustrating an example of z-axis coordinates set by the subject coordinate calculation unit illustrated in FIG.

  The subject coordinate calculation unit 105 assigns the distance values obtained by the distance measurement unit 110 to the subjects 301 to 304 as Z coordinates Z1, Z2, Z3, and Z4, respectively. Then, the subject coordinate calculation unit 105 calculates the subject position (subject coordinate) that is the actual coordinates of the subject from the apparent XY coordinates of the subjects 301 to 304 using the Z coordinate.

  Here, an example of the subject position calculation will be described using the subject 301 as an example.

First, the subject coordinate calculation unit 105 calculates the coordinates on the image sensor surface by multiplying the apparent XY coordinates on the screen (that is, the image) by the pixel pitch of the image sensor 102. Assuming that the pixel pitch is p (mm), the actual coordinates of the subject on the imaging element surface are (Xp, Yp ). The coordinates are in mm.

  Subsequently, the subject coordinate calculation unit 105 calculates the subject 301 based on the focal length f (mm) of the imaging optical system 101 (lens) and the vertical dimension V (mm) and the horizontal dimension H (mm) of the image sensor 102. The XY coordinates (X1, Y1) in the real space are obtained by the following equations (1) and (2).

  Accordingly, the three-dimensional coordinates of the subject 301 in the real space are (X1, Y1, Z1). Similarly, when the three-dimensional coordinates of the objects 302 to 304 in the real space are obtained, they are (X2, Y2, Z2), (X3, Y3, Z3), and (X4, Y4, Z4), respectively.

  Referring again to FIG. 11, after the calculation of the subject position, under the control of the system controller 112, the tilt shooting focusing plane calculation unit 106 obtains a focusing plane at the time of tilt shooting (step S5). For example, least squares approximation is used to find the focal plane during tilt shooting.

  Here, the three-dimensional coordinates (X1, Y1, Z1) to (X4, Y4, Z4) of the objects 301 to 304 in the real space are regarded as a sequence of points, and the tilt plane focusing plane calculation unit 106 performs these operations. A plane obtained by performing a least-squares approximation of the point sequence at a distance in the z-axis direction is obtained.

  First, the tilt plane focusing plane calculation unit 106 sets the expression of the approximate plane to Expression (3).

  When considering the distance in the Z direction, the distance di between the point (Xi, Yi, Zi) (where i = 1 to 4) and the plane represented by the equation (3) is expressed by the following equation (4). You.

  Then, considering the sum of squares Ω of the distance between all the point sequences and the plane, the sum of squares Ω is expressed by the following equation (5).

  In order to obtain the coefficients a, b, and c for minimizing the expression (5), the tilt-and-focus photographing surface calculation unit 106 calculates the expression (5) for each of the counts a, b, and c by using the expression Partial differentiation is performed as shown in (6) to (8).

^{_{Here, As = ΣX i 2, Bs}} = ΣY i 2, Cs = ΣX i Y i, Ds = ΣX i z i, Es = ΣY i z i, Fs = ΣX i, Gs = ΣY i, and Hs = Σz _{When i} is set, Expressions (6) to (8) are expressed by Expressions (9) to (10), respectively.

  Solving the simultaneous equations expressed by Expressions (9) to (11), the coefficients a, b, and c are expressed by Expressions (12) to (14), respectively.

  Then, by substituting Equations (12) to (14) into Equation (3), the plane equation of the least squares approximation plane is obtained. This least-squares approximation plane is the plane equation of the focal plane in tilt shooting. Thereafter, z = ax + by + c is used as the plane expression of the focal plane in the tilt photographing.

  Next, the tilt plane focusing plane calculation unit 106 calculates a plane equation in which the lens principal surface necessary to obtain the focusing plane z = ax + by + c is to be arranged.

  FIG. 14 is a diagram illustrating an arrangement relationship between the image sensor illustrated in FIG. 1, a lens, and a focal plane.

  The illustrated focal plane 300 is the focal plane obtained by the above-described equations (3) and (12) to (14). Further, a lens, which is the imaging optical system 101, is arranged on the Z-axis separated from the imaging element 102 by the focal length f. The imaging element surface is a plane of z = 0, and the common line with the plane expression of the focal plane can be obtained by the following expression (15).

  Assuming that an intersection E1 with the X axis and an intersection E2 with the Y axis are taken as two points on the common line represented by the equation (15), the coordinates are E1 = (− c / a, 0, 0) and E2 = (0 , -C / b, 0). The coordinates of the center point E3 of the lens are E3 = (0, 0, f), and a plane passing through these three points corresponds to a plane formula of a lens main surface necessary to obtain a desired focal plane. By generating vectors K1 = (c / a, -c / b, 0) and K2 = (c / a, 0, f) on a plane from three points of the coordinates E1 to E3, and taking an outer product of them. As shown in Expression (16), a normal vector to the plane of the lens main surface is calculated.

  Further, through the point E3, the equation of the plane having the normal vector of the equation (16) is given by the following equation (17).

  If the tilt of the lens is controlled so that the plane given by the above equation (18) and the principal surface of the lens coincide with each other, a desired focal plane z = ax + by + c in the tilt photographing can be obtained from the aforementioned Scheimpflug's law. Can be

  Referring again to FIG. 11, under the control of the system controller 112, the tilt photographing depth-of-field calculating unit 107 calculates an F value necessary for focusing on all the subjects in the tilt photographing (step S6). . Here, first, the tilt photographing depth-of-field calculating unit 107 calculates the distance between the subject 301 to 304 and the focal plane z = ax + by + c in the tilt photographing using the following equation (19).

However, i = 1 to 4.

  Then, the tilt shooting depth of field calculation unit 107 calculates the sum Sm of the positive maximum value and the absolute value of the negative minimum value at the distance to the focal plane. This sum Sm is the depth of field necessary to bring all the subjects 301 to 304 into the focusing range.

  Next, the tilt-depth-of-field calculating unit 107 calculates an F-number at which the depth of field becomes the sum Sm. The depth of field T is obtained by adding the front depth of field Tf and the rear depth of field Tr. Let R be the permissible circle of confusion, and F the aperture value of the lens. Further, assuming that the distance from the center of the image sensor surface to the focal plane in tilt shooting is Zv, the depth of field T can be obtained by the following equation (20). Here, Zv is given by equation (21).

  Then, the tilt-depth-of-field calculating unit 107 calculates the F when the depth of field T expressed by the equation (20) is equal to the sum (the required depth of field) Sm by the following equation (21). Calculate the value.

  When equation (22) is solved with respect to the F value, the F value required for keeping the entire subject within the focusing range in tilt shooting can be calculated as shown in equation (23).

  Subsequently, under the control of the system controller 122, the normal-image-depth-of-field calculating unit 108 calculates an F-number necessary for focusing on all subjects when using normal imaging (step S7). As described above, the distance information obtained by the distance measuring unit 110 is assigned to each of the subjects 301 to 304 as the z coordinate. The Z coordinates are Z1, Z2, Z3, and Z4, respectively.

  The normal imaging depth of field calculation unit 108 selects the maximum value Zmax and the minimum value Zmin from among these Z coordinates Z1 to Z4, and calculates the difference (Zmax−Zmin). This difference (Zmax-Zmin) is the depth of field necessary for focusing on all subjects in normal shooting.

  Next, in the same manner as the method used in the tilt shooting, the normal shooting depth-of-field calculating unit 108 calculates an F value necessary for the depth of field to be (Zmax-Zmin). At this time, the depth of field T is obtained by adding the front depth of field Tf and the rear depth of field Tr. Assuming that the permissible circle of confusion is R, the aperture value of the lens is F, and the distance from the center of the image sensor surface to the focal plane in normal shooting is Zv, the depth of field T is given by the following equation (24). can get. Here, Zv is given by equation (25).

  When the F values are arranged, an F value (normal photographing F value) in normal photographing is obtained by Expression (26).

  The F value shown in Expression (26) is an F value necessary for focusing on all subjects in normal shooting.

  Subsequently, under the control of the system controller 112, the shooting mode selection unit 109 compares the tilt shooting F value with the normal shooting F value, and based on the comparison result, the mode capable of shooting with the minimum F value is tilt shooting. Is determined (step S8). That is, the photographing mode selection unit 109 selects a photographing mode that allows photographing with a smaller F value.

  If the mode selection result indicates tilt shooting (YES in step S8), the system controller 112 controls the imaging optical system control unit 111 to drive the imaging optical system 101 in a tilt (TILT) manner (step S9). At this time, the system controller 112 performs tilt control so that the main surface of the lens becomes the in-focus surface determined by Expression (3) and Expressions (12) to (14).

  Subsequently, the system controller 112 controls the imaging optical system control unit 111 to control the focusing of the imaging optical system 101 (step S10). Then, the system controller 112 executes tilt shooting. After that, the system controller 112 returns to the process of step S2.

  On the other hand, if the mode selection result indicates normal shooting (NO in step S8), the system controller 112 proceeds to the process in step S10, and controls the focusing of the imaging optical system 101. Then, the system controller 112 executes normal shooting.

  As described above, in the first embodiment of the present invention, it is determined whether the entire subject can be focused with a smaller F value in either tilt shooting or normal shooting. Since the photographing mode is switched based on the determination result, the subject can be photographed in the optimal photographing mode even when the situation of the subject changes.

[Second embodiment]
Subsequently, an example of the camera according to the second embodiment of the present invention will be described.

  FIG. 15 is a block diagram illustrating a configuration of an example of a camera according to the second embodiment of the present invention. In FIG. 15, the same components as those of the camera shown in FIG. 10 are denoted by the same reference numerals.

  The illustrated camera 100 has an image sensor control unit 114. The image sensor 102 has a tilt mechanism, and the tilt mechanism can tilt the image sensor surface with respect to the image pickup optical system 101. The image sensor control unit 114 controls the tilt of the image sensor 102 under the control of the system controller 112.

  FIG. 16 is a flowchart illustrating an example of a photographing process performed by the camera illustrated in FIG. In FIG. 16, steps that are the same as the steps in the flowchart shown in FIG. 11 are given the same reference numerals, and descriptions thereof will be omitted.

  If the mode selection result indicates tilt shooting (YES in step S8), the system controller 112 causes the image sensor 102 to perform tilt drive (tilt control) by the image sensor controller 114 (step S11).

  Here, calculation of the drive amount at the time of tilt control of the image sensor 102 will be described.

  First, let z = ax + by + c be the plane equation of the focal plane in tilt shooting. In the second embodiment, the tilt control of the imaging optical system 101 is not performed, and as a result, the plane expression of the lens main surface is z = f. The common line between the lens main surface and the focal plane z = ax + by + c is 0 = ax + by-f + c. Then, the plane expression of the imaging surface of the imaging device 102 to be obtained includes the public line and the coordinates I1 (0, 0, 0) of the center of the imaging device.

  Considering an intersection I2 with the X axis and an intersection I3 with the Y axis as two points on the public line 0 = ax + by-f + c, the intersection I2 is ((fc) / a, 0, f) and the intersection I3 is ( 0, (fc) / b, f). The system controller 112 generates a vector T1 using the intersections I1 and I2, and generates a vector T2 using the intersections I1 and I3. Then, the system controller 112 obtains the cross product of the vectors T1 and T2 by equation (27), and calculates the normal vector of the plane of the imaging surface.

  An equation of a plane having the normal vector indicated by the equation (27) and including the point I1 (0, 0, 0) is represented by the following equation (28).

  By rearranging equation (28), equation (29) is obtained.

  The system controller 112 tilts and drives the image sensor 102 so that the image sensor surface (imaging surface) matches the plane represented by Expression (29). This makes it possible to obtain a focal plane z = ax + by + c in tilt shooting. After the processing in step S11, the system controller 112 proceeds to the processing in step S10.

  As described above, in the second embodiment of the present invention, similarly to the above-described first embodiment, by switching the shooting mode based on the determination result, even if the situation of the subject changes, the SNR is good. Shooting can be performed in the shooting mode. Furthermore, since the tilt control is performed on the image pickup device that is smaller than the image pickup optical system, the camera can be downsized as compared with the case where the image pickup optical system 101 is tilt controlled.

[Third Embodiment]
Subsequently, an example of the camera according to the third embodiment of the present invention will be described. The configuration of the camera according to the third embodiment is the same as that of the camera shown in FIG.

  FIG. 17 is a block diagram illustrating a configuration of a subject detection unit used in a camera according to the third embodiment of the present invention.

  The illustrated subject detection unit 104 includes a moving object detection unit 114 and a face detection unit 115. Then, the image data is sent from the image processing unit 103 to the moving object detection unit 114 and the face detection unit 115. The moving body detection unit 114 analyzes the image data, detects a moving body that is a moving subject, and outputs a moving body detection result to the subject coordinate calculation unit 105. The face detection unit 115 analyzes the image data, detects the face area (specific area) of the person who is the subject, and outputs the face detection result to the subject coordinate calculation unit 105.

  The subject coordinate calculation unit 105 determines a subject to be focused and a subject to be out of focus using the moving object detection result and the face detection result. Here, the subject coordinate calculation unit 105 determines that a subject whose face has not been detected, although the subject has been detected as a moving object, is out of focus. By selecting the subject to be focused in this way, the number of focused subjects is reduced, and the possibility that the depth of field can be made shallower than that of focusing on all subjects is increased. As a result, imaging can be performed with a smaller F value.

  FIG. 18 is a flowchart illustrating a photographing process performed by the camera according to the third embodiment of the present invention. In FIG. 18, steps that are the same as the steps in the flowchart shown in FIG. 11 or FIG. 16 are given the same reference numerals, and descriptions thereof will be omitted.

  When the photographing process is started, the subject detection unit 104 detects a moving object under the control of the system controller 112. Then, the system controller 112 determines whether a moving object has been detected by the subject detection unit 104 (step S12). If no moving object is detected (NO in step S12), system controller 112 waits.

  When a moving object is detected (YES in step S12), under the control of system controller 112, subject detection unit 104 performs face detection on the moving object. Then, the system controller 112 determines whether or not a face has been detected by the subject detection unit 104 (step S13). If no face is detected (NO in step S13), system controller 112 returns to the process in step S12.

  On the other hand, if a face is detected (YES in step S13), system controller 112 measures the distance of moving object (subject) in which the face is detected by distance measuring section 110 (step S14). After that, the system controller 112 proceeds to the process of step S4.

  As described above, in the third embodiment of the present invention, the moving object, that is, the face of the person who is acting is detected, and the photographing process is performed while paying attention to the person. In other words, only a subject to be focused on when performing suspicious person monitoring or abnormal behavior monitoring is set as a focusing target, and even a moving object, such as a person other than a person, which is less important in monitoring is excluded from focusing. This makes it possible to reduce the required focus width. As a result, the required F value can be further reduced, and an image with a good SNR can be obtained.

[Fourth embodiment]
Subsequently, an example of the camera according to the fourth embodiment of the present invention will be described.

  FIG. 19 is a block diagram showing a configuration of an example of a camera according to the fourth embodiment of the present invention. In FIG. 19, the same components as those in the camera shown in FIG. 15 are denoted by the same reference numerals.

  In the camera shown in FIG. 19, the imaging object coordinate calculation unit 105 and the display unit 113 are connected to the system controller 112 in two directions. As described above, the system controller 112 displays an image corresponding to the image data on the display unit 113. The user looks at the image displayed on the display unit 113, and selectively designates a three-dimensional range to be masked from a focusing target in the image as a mask range using an operation unit (not shown).

  When the user specifies a mask range, information indicating the mask range is sent from the display unit 113 to the system controller 112. Then, the system controller 112 sends the mask range information to the subject coordinate calculation unit 105. The subject coordinate calculation unit 105 calculates the subject coordinates excluding the mask range specified by the mask range information, as described later.

  FIG. 20 is a flowchart for explaining the photographing processing performed by the camera shown in FIG. In FIG. 20, steps that are the same as the steps in the flowcharts shown in FIGS. 11 and 16 are given the same reference numerals, and descriptions thereof are omitted.

  When the photographing process is started, the user sets a mask range as necessary as described above (step S15).

  FIG. 21 is a diagram illustrating an example of a mask range set in the image displayed on the display unit illustrated in FIG.

  In the illustrated example, it is assumed that the user has set the three-dimensional mask range 20 in order to exclude the object from focusing. As a result, the subject 16 is excluded from the focus target. Then, the system controller 112 sends mask range information (the coordinates of the mask range) indicating the three-dimensional mask range 20 to the subject coordinate calculation unit 105.

  Subsequently, the system controller 112 determines whether or not a subject has been detected by the subject detection unit 104 in step S2. After the process of step S4, under the control of the system controller 112, the subject coordinate calculation unit 105 determines whether or not the detected subject exists in a mask range (also referred to as a mask area) (step S16).

  If the subject exists in the mask range (YES in step S16), system controller 112 returns to the process in step S2. On the other hand, if the subject does not exist in the mask range (NO in step S16), the system controller 112 proceeds to the process in step S5, and the in-focus imaging focal plane calculation unit 106 calculates the in-focus plane for the in-tilt imaging. .

  In the fourth embodiment, the tilt of the image sensor is controlled in step S11. However, the tilt of the imaging optical system 101 may be controlled. In this case, the image sensor control unit 114 shown in FIG. 19 becomes unnecessary.

  As described above, according to the fourth embodiment of the present invention, when a user masks an image, that is, a range captured by another camera in a shooting range, unnecessary subjects can be combined. Can be removed from the focus. As a result, the possibility that the F value required for focusing can be further reduced can be increased.

  As described above, according to the embodiment of the present invention, a shooting mode with a good SNR can be automatically selected in shooting in a dark place or the like, regardless of the situation of the subject.

  As is apparent from the above description, in the example shown in FIG. 10, the system controller 112 and the imaging mode selection unit 109 function as selection means, and the system controller 112, the subject coordinate calculation unit 105, the tilt plane focus plane calculation unit 106 and the tilt shooting depth-of-field calculating unit 107 function as first calculating means. In addition, the system controller 112, the subject coordinate calculation unit 105, and the depth-of-field calculation unit for normal shooting function as a second calculation unit. The distance measuring unit 110 and the system controller 112 function as distance measuring means, and the system controller 112 and the imaging optical system control unit 111 function as control means.

  As described above, the present invention has been described based on the embodiments. However, the present invention is not limited to these embodiments, and various forms without departing from the gist of the present invention are also included in the present invention. .

  For example, the function of the above-described embodiment may be used as a control method, and the control method may be executed by the imaging apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable recording medium, for example.

[Other Embodiments]
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

104 Subject detection unit 105 Subject coordinate calculation unit 106 Focusing plane calculation unit for tilt shooting 107 Depth of field calculation unit for tilt shooting 108 Normal shooting depth of field calculation unit 109 Shooting mode selection unit 110 Distance measuring unit 111 Imaging optics System controller 112 System controller 114 Image sensor controller

Claims (9)

An image pickup apparatus that includes an image pickup element on which an optical image is formed via an image pickup optical system and obtains an image in accordance with an output of the image pickup element,
A first photographing mode for performing photographing by performing a tilt control for controlling an angle between a lens main surface of the imaging optical system and an image forming surface, and a state where an optical axis of the imaging optical system is orthogonal to the image forming surface; Selecting means for selecting one of a second shooting mode for shooting with
A distance measuring means for measuring a distance from the imaging device to a subject existing in a shooting range of the image to obtain distance information;
First calculating means for calculating a tilt angle and a first aperture value for focusing on a plurality of subjects existing in the shooting range in the first shooting mode;
A second calculating unit that calculates a second aperture value for focusing on a plurality of subjects existing in the shooting range in the second shooting mode,
The selector selects one of the first shooting mode and the second shooting mode as a selected shooting mode based on a comparison result between the first aperture value and the second aperture value. Imaging device.   The selection means compares the first aperture value and the second aperture value, and selects a shooting mode corresponding to the smaller aperture value as the selected shooting mode. The imaging device according to claim 1. 3. The apparatus according to claim 1, further comprising a control unit configured to control a tilt of at least one of the image pickup optical system and the image pickup element to set the tilt angle when performing photographing in the first photographing mode. 4. Imaging device. The imaging apparatus according to any one of claims 1 to 3, characterized in that it comprises a detection means for detecting an object in the shooting range. The detecting means, when detecting the subject, detects whether the subject is a moving object, and detects whether a predetermined specific region exists in the subject,
The distance measuring means is the subject body by said detecting means, and when the specific area is detected to be present, according to claim 4, characterized in that the distance measuring the distance to the subject Imaging device. The imaging device according to claim 5 , wherein the specific area is a face area of a person. Masking means for selectively masking a subject present in the photographing range of the image,
Each of said first arithmetic means and said second computing means according to any one of claims 1 to 6, characterized in that the masked object covered by the focusing by the mask means Imaging device. A control method of an image pickup apparatus including an image pickup element on which an optical image is formed via an image pickup optical system, and obtaining an image in accordance with an output of the image pickup element,
A first photographing mode for performing photographing by performing a tilt control for controlling an angle between a lens main surface of the imaging optical system and an image forming surface, and a state where an optical axis of the imaging optical system is orthogonal to the image forming surface; A selection step of selecting one of a second shooting mode in which shooting is performed in
A distance measuring step of measuring a distance from the imaging device to a subject existing in a shooting range of the image to obtain distance information;
A first calculation step of obtaining a tilt angle and a first aperture value for focusing on a plurality of subjects existing in the shooting range in the first shooting mode;
A second calculation step of obtaining a second aperture value for focusing on a plurality of subjects existing in the shooting range in the second shooting mode,
In the selecting step, one of the first shooting mode and the second shooting mode is selected as a selected shooting mode based on a comparison result between the first aperture value and the second aperture value. And control method. A control program used in an imaging device that includes an imaging device on which an optical image is formed via an imaging optical system and obtains an image in accordance with an output of the imaging device,
In the computer provided in the imaging device,
A first imaging mode for performing imaging by performing tilt control for controlling an angle between a lens main surface of the imaging optical system and an imaging plane, and a state in which an optical axis of the imaging optical system is orthogonal to the imaging plane; A selection step of selecting one of a second shooting mode for shooting in
A distance measuring step of measuring a distance from the imaging device to a subject existing in a shooting range of the image to obtain distance information;
A first calculation step of obtaining a tilt angle and a first aperture value for focusing on a plurality of subjects existing in the shooting range in the first shooting mode;
Performing a second calculation step of obtaining a second aperture value for focusing on a plurality of subjects existing in the shooting range in the second shooting mode,
In the selecting step, one of the first shooting mode and the second shooting mode is selected as a selected shooting mode based on a comparison result between the first aperture value and the second aperture value. Control program.