JP4764896B2 - Camera calibration apparatus, camera calibration method, camera calibration program, and recording medium recording the program - Google Patents

Description

  The present invention can be used for camera calibration of a monocular camera or a multi-view camera arranged so as to surround a subject, and a small number of reference points or a part of the reference points is used for the purpose of reducing the cost of camera calibration work. The present invention relates to a technique for photographing a calibration target including the same and calibrating a camera based on the projection relationship (determining camera parameters).

  Camera calibration (camera calibration) is performed to obtain internal parameters such as focal length and image center as external information inside the camera, and external parameters such as posture and position information at the time of camera placement. In the conventional method, a two-dimensional object having a lattice pattern as shown in FIG. 2 is photographed with respect to the camera to be calibrated, and from the geometric information about the reference object and the projection information on the image observed by the camera, Internal and external parameters (both are referred to as camera parameters) are estimated (see Non-Patent Document 1). Furthermore, as an example using a simpler reference object, a method of using a one-dimensional object with a known length as shown in FIG. 3 and shooting with the point A fixed and calibrating the camera from the image has been disclosed. (Refer nonpatent literature 2).

In computer vision, there is a basic principle called Structure From Motion, and there is a method for restoring the three-dimensional shape and movement of a subject by moving the camera relative to a stationary subject (for moving subjects). On the other hand, the principle is the same even if the camera is stationary). Self-calibration (also called auto-calibration) is one of the methods based on this principle. It does not require the above-mentioned known reference object and uses a feature point observed from a static video scene. Estimate the parameters.
Z. Zhang, “A flexible new technology for camera calibration”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, no. 11, pp. 1330-1334, 2000. Z. Zhang, “Camera calibration with one-dimensional objects”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 26, no. 7, pp. 892-899, 2004.

  In conventional camera calibration methods, it is common practice to use a reference object as represented by Non-Patent Documents 1 and 2. In FIG. 2, the two-dimensional coordinates of the reference point on the two-dimensional object are known (Non-Patent Document 1). In FIG. 3, the length L of the one-dimensional object, the distance from point A to point C, and the point B to point C are shown. It is a premise that the ratio of the distance to is known (Non-Patent Document 2).

  In the case of camera calibration using such a reference object, it is necessary to photograph a sufficient amount of reference points. In order to respond to this, a lot of time is taken for the work of photographing the reference object before the camera calibration. A fixed-point camera such as a surveillance camera cannot always capture a sufficient amount of reference points for camera calibration, and does not have enough time to capture them. In some cases, a situation occurs in which camera calibration is performed from a small number of reference points.

  On the other hand, according to the self-calibration, the camera can be calibrated according to the feature points observed from the static video scene without requiring a known reference object. However, it is necessary to measure the image coordinates of a sufficient amount of feature points from the time series image even for the camera calibration, and if a certain amount of error is mixed in the measurement of image coordinates at the feature points, the calibration accuracy is greatly deteriorated. Has been pointed out.

  The present invention solves the above-mentioned problems, and its purpose is to use a special reference object in order to reduce the cost of photographing work for calibration, and from a small number of reference points observed from the object, such as a surveillance camera. It is an object to provide a camera calibration apparatus, a camera calibration method, a camera calibration program, and a recording medium in which the program is recorded, which can easily calibrate a fixed-point camera or a multi-viewpoint camera arranged so as to surround a subject.

In order to describe the projective relationship between the camera viewpoint and the three-dimensional point, a reference coordinate system is set. This coordinate system may be set arbitrarily, but for convenience of explanation, a world coordinate system (Euclidean space) with the origin at a certain point on the ground or floor (or may be a point on an arbitrary plane) Assuming Assume that the geometric projection by a camera can be modeled as a perspective projection according to a pinhole camera. If the three-dimensional point (reference point) expressed in the world coordinate system is P _j = (X _j , Y _j , Z _j ) and the camera viewpoint position is (Tx, Ty, Tz), the point P _{j on} the image The two-dimensional coordinate values (x _j , y _j ) observed in (1) are given by equations (1) and (2).

Here, α is a focal length in units of pixels, and R ₁₁ , R ₁₂ ,..., R ₃₃ correspond to elements of a 3 × 3 rotation matrix shown in Expression (3).

When the X coordinate of the target reference point P _j is 0 (X _j = 0) and the camera is located on the Y axis (Tx = 0), the relationship between the camera viewpoint and the world coordinate system XYZ is shown in FIG. (X-axis is in the direction from the front to the back of the page). In this coordinate system, the point O is the origin, Ty corresponds to the camera depth, Tz corresponds to the camera height, and φ corresponds to the camera tilt angle. Further, as shown in FIG. 1, a camera coordinate system X′Y′Z ′ centered on the camera viewpoint is provided (the X ′ axis is the front direction from the back of the page). The Z-axis is defined as the camera optical axis, and the camera optical axis passes through a point Q (arbitrary) on the Z-axis, and the projected coordinate is the image center. In general, a fixed point camera such as a surveillance camera can be modeled as a camera arrangement as shown in FIG.

  φ is defined as φ = 0 in the vertically downward direction (−Z axis direction) and φ = −π / 2 when directed in the opposite direction to the Y axis, and the tilt angle of the camera is within this range (−π / 2 < The rotation angle is defined as φ <0). Considering a simple camera arrangement in which there is no rotation about the Y ′ axis and the Z ′ axis for the camera of FIG. 1, the elements of the rotation matrix of Expression (1) can be given by Expression (3).

Furthermore, since X _j = 0 is considered for the reference point P _j and Tx = 0 is considered at the camera viewpoint, x _j = 0 may always be set in equation (1). Therefore, assuming the above camera arrangement and reference point, it can be concluded that only the y-coordinate of equation (4) is handled as the projected coordinate on the image of the reference point.

The present invention focuses on the simple calibration of the camera using the reference point projected on the y-axis on the image. That is, the basic principle of the present invention is that Y _j and Z _j are given by the prepared reference point P _j and y _j is observed, and the focal length α and the inclination angle φ are obtained in the geometrical constraint related by the equation (4). The viewpoint position (Ty, Tz) in the YZ coordinate system is uniquely determined.

  The camera calibration device according to claim 1, which solves the above-described problem, calibrates external information including internal information, posture, and position of an image input device from an image acquired using an eye or a plurality of image input devices. An image input device to be calibrated to shoot a subject whose shape is known, set an image coordinate system in the acquired image, and arbitrarily set a line segment formed from a reference point on the subject Subject control means for changing the orientation / position of the subject or the orientation / position of the image input device so that the point is projected onto the origin of the image coordinate system, the reference axis set on the image coordinate system, and the reference on the subject The image is rotated around the origin of the coordinate system set on the acquired image so that the line segments formed from the points match, and the other reference points are also points on the reference axis. Posture / position or front Image correction means for positioning the projected coordinates of all reference points on the reference axis by changing the posture / position of the image input device, and from the three-dimensional points of all reference points to the reference axis of the image coordinate system From the geometrical information of the reference point and the image coordinates of the reference point on the reference axis, the relational expression between the internal information and the external information of the image input device derived from the perspective projection conditions at the time of projection, the image input device Camera calibration processing means for calibrating internal information or external information.

  The camera calibration device according to claim 2 is a device that calibrates external information including the internal information, posture, and position of the image input device from an image acquired using a single eye or a plurality of image input devices, A reference object with a disk that forms a plurality of circles centered on a reference point on a foundation and a vertical bar that forms a line segment that is lowered to the reference point so as to be perpendicular to the circle is used for calibration photography. An image coordinate system is set in the image of the subject acquired using the image input device to be calibrated as the subject, and the posture of the subject is projected so that an arbitrary point on the vertical bar is projected to the origin of the coordinate system A subject control means for changing the position or posture / position of the image input device, a reference axis set on the image coordinate system and the reference by rotating the image around the origin of the coordinate system set on the acquired image The vertical bar of the object A plurality of points, reference points, and intersections between a plurality of concentric circles and the reference axis set as reference points, and reference point measuring means for acquiring image coordinates of the reference points; and all the reference points The geometric information of the reference point and the reference point measurement means based on the relational expression between the internal information and the external information of the image input device derived from the perspective projection conditions when projecting from the three-dimensional point onto the reference axis of the image coordinate system Camera calibration processing means for calibrating the internal information or external information of the image input device from the image coordinates of the reference point on the reference axis acquired by the above.

  The camera calibration device according to claim 3 is a device that calibrates external information including the internal information and orientation and position of the image input device from an image acquired using a single eye or a plurality of image input devices, A disk that forms a circle centered on a reference point on a base, a vertical bar that is installed perpendicular to the circle and that forms a line segment extending from the reference point, and a vertical line that extends along the circumference of the circle A reference object provided with a rotating rod that forms a line segment that rotates around a bar is set as a subject for calibration shooting, and an image coordinate system is set in the image of the subject acquired using an image input device to be calibrated. Subject control means for changing the posture / position of the subject or the posture / position of the image input device so that an arbitrary point of the vertical bar is projected on the origin of the coordinate system, and coordinates set on the acquired image Around the origin of the system The image is rotated so that the reference axis set on the image coordinate system and the vertical bar of the reference object coincide with each other, and when the rotation bar is rotated around the vertical bar and the rotation bar coincides with the reference axis, the vertical axis Reference points measuring means for acquiring a plurality of points set on the bar, reference points, and a plurality of points set on the rotating bar as reference points, and acquiring image coordinates of the reference points, and images from the three-dimensional points of all the reference points The reference information obtained from the geometric information of the reference point and the reference point measurement means by the relational expression between the internal information and the external information of the image input device derived from the perspective projection condition when projecting onto the reference axis of the coordinate system Camera calibration processing means for calibrating internal information or external information of the image input device from image coordinates of a reference point on the axis.

The camera calibration device according to claim 4 is a device that calibrates external information including the internal information and orientation and position of the image input device from an image acquired using a single eye or a plurality of image input devices,
It consists of a base with a flat surface and a semi-disc, and the semi-disc has a plurality of identical semi-circles centered on the reference point, and the straight portions of the semi-circles coincide with the straight portions of the semi-disc The half disk is in contact with the plane of the base and the straight part of the half disk, and it is a mechanism that allows the half disk to rotate around the straight part that touches, and on the half disk passing through the center of multiple semicircles The reference object with the center line as the center axis is taken as the subject for calibration photography, and the image coordinate system is set in the subject image acquired using the image input device to be calibrated. Rotate the image around the origin of the coordinate system set on the acquired image and subject control means for changing the orientation / position of the subject or the orientation / position of the image input device to be projected onto the origin of the coordinate system The reference axis set on the image coordinate system and the reference A plurality of points when the center axis of the object is matched, and a plurality of points, reference points, and semi-disks installed on the center axis are inclined with respect to the plane of the base or rotated with respect to the rotation axis of the semi-disk. Reference point measuring means for obtaining the image coordinates of the reference point using the point where the semicircle intersects the reference axis as a reference point, and when projecting from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system From the relational expression between the internal information and external information of the image input device derived from the perspective projection conditions, from the geometric information of the reference point and the image coordinates of the reference point on the reference axis acquired by the reference point measuring means, Camera calibration processing means for calibrating internal information or external information of the image input device.

  The camera calibration method according to claim 5 is a method for calibrating external information including an internal information and an attitude and a position of an image input device from an image acquired using a single eye or a plurality of image input devices, The subject control means captures a subject whose shape is known using an image input device to be calibrated, sets an image coordinate system in the acquired image, and selects an arbitrary line segment formed from a reference point on the subject. A step of changing the posture / position of the subject or the posture / position of the image input device so that a point is projected on the origin of the image coordinate system; and the reference axis set on the image coordinate system and the subject Rotate the image around the origin of the coordinate system set on the acquired image so that the line segments formed from the reference points match, and so that the other reference points are also points on the reference axis Subject posture / position Alternatively, by changing the posture / position of the image input device, the projected coordinates of all the reference points are positioned on the reference axis, and the camera calibration processing unit is configured to generate an image from the three-dimensional points of all the reference points. The geometric information of the reference point and the image coordinates of the reference point on the reference axis are derived from the relational expression between the internal and external information of the image input device derived from the perspective projection conditions when projecting onto the reference axis of the coordinate system. And calibrating internal information or external information of the image input device.

  The camera calibration method according to claim 6 is a method for calibrating external information including the internal information and orientation and position of the image input device from an image acquired using a single eye or a plurality of image input devices, A reference object in which the subject control means comprises a disk that forms a plurality of circles centered on a reference point on a certain base, and a vertical bar that forms a line segment drawn down to the reference point so as to be perpendicular to the circle Is the subject for calibration shooting, and an image coordinate system is set in the image of the subject acquired using the image input device to be calibrated, and an arbitrary point on the vertical bar is projected to the origin of the coordinate system Changing the posture / position of the subject or the posture / position of the image input device, and the reference point measuring means rotates the image around the origin of the coordinate system set on the acquired image to obtain an image coordinate system Set above A reference in which a quasi-axis and a vertical bar of the reference object are matched, and a plurality of points, reference points, and intersections between a plurality of concentric circles and the reference axis are set as reference points to obtain image coordinates of the reference point Internal information and external information of the image input device derived from the perspective projection conditions when the point measurement step and the camera calibration processing means project from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system Calibrating internal information or external information of the image input device from geometric information of a reference point and image coordinates of the reference point on the reference axis acquired by the reference point measurement step It is characterized by providing.

  The camera calibration method according to claim 7 is a method of calibrating external information including an internal information and an attitude and a position of an image input device from an image acquired using a single eye or a plurality of image input devices, A subject control means is a disk that forms a circle centered on a reference point on a base, a vertical bar that is installed perpendicular to the circle and extends from the reference point, and a circumference of the vertical bar. A reference object with a rotating bar that forms a line segment that rotates around the vertical bar along the vertical axis is taken as the subject for calibration shooting, and the image coordinates in the image of the subject acquired using the image input device to be calibrated Setting a system, changing the posture / position of the subject or the posture / position of the image input device so that an arbitrary point of the vertical bar is projected on the origin of the coordinate system, and a reference point measuring means , The acquired image Rotate the image around the origin of the coordinate system set in step 1, match the reference axis set on the image coordinate system with the vertical bar of the reference object, and rotate the rotary bar around the vertical bar to A reference point measurement step for obtaining the image coordinates of the reference point by using a plurality of points set on the vertical bar, a reference point, and a plurality of points set on the rotating bar as reference points when matching with the reference axis, and camera calibration The reference point is obtained from the relational expression between the internal information and the external information of the image input device derived from the perspective projection condition when the processing means projects from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system. And calibrating internal information or external information of the image input device from the geometric information of the reference point and the image coordinates of the reference point on the reference axis acquired in the reference point measurement step.

  The camera calibration method according to claim 8 is a method for calibrating external information including the internal information and orientation and position of the image input device from an image acquired using a single eye or a plurality of image input devices, The subject control means is composed of a base with a flat surface and a semi-disc, and the semi-disc has a plurality of identical semi-circles centered on the reference point, and the straight portions of the semi-circles are straight lines of the semi-disc The semi-disc is in contact with the base plane and the straight part of the semi-disk, and the semi-disk can rotate around the straight part that touches the axis of rotation. A reference object centered on a line on a semicircular disk passing through is a subject for calibration shooting, and an image coordinate system is set in the image of the subject acquired using the image input device to be calibrated, Arbitrary point projected to the origin of the coordinate system And changing the posture / position of the subject or the posture / position of the image input device so that the reference point measuring means rotates the image around the origin of the coordinate system set on the acquired image, The reference axis set on the coordinate system coincides with the central axis of the reference object, and a plurality of points, reference points, and semi-disks installed on the central axis are inclined with respect to the plane of the base or the rotational axis of the semi-disk The reference point measuring step for obtaining the image coordinates of the reference point using the points where a plurality of semicircles intersect with the reference axis when rotated with respect to the reference axis, and the camera calibration processing means include all the reference points. The geometric information of the reference point and the reference point measurement step based on the relational expression between the internal information and the external information of the image input device derived from the perspective projection conditions when projecting from the three-dimensional point onto the reference axis of the image coordinate system Obtained by From the image coordinates of the reference point on the semi-axis, it is characterized by comprising the steps of: calibrating the internal information or external information of the image input device.

  A camera calibration program according to a ninth aspect is a program for causing a computer to execute each step according to any one of the fifth to eighth aspects.

  A recording medium according to claim 10 is a computer-readable recording medium in which the camera calibration program according to claim 9 is recorded.

  According to the first to tenth aspects of the present invention, since camera calibration is performed using a small number of reference points, a large amount of reference is used for calibration as in conventional camera calibration (Non-Patent Documents 1 and 2). Do not need a point. In the present invention, the use of a special reference object for smoothly measuring the coordinate value of the reference point on the image simplifies the camera calibration work and shortens the processing time. Furthermore, if the plurality of cameras are arranged so as to surround the reference object, the plurality of cameras can be calibrated simultaneously, which contributes to seamless 3D video generation or arbitrary viewpoint video generation using a multi-viewpoint camera.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments.

Example 1
FIG. 4 shows the camera arrangement and reference points when reference points A and B on the Z-axis are given in FIG. The present embodiment estimates the focal length α, the tilt angle φ, and the viewpoint position (Ty, Tz) in the YZ coordinate system in the camera arrangement of FIG. 4 (these are collectively referred to as camera parameters). To do). First, the camera arrangement and reference points in FIG. 4 will be described. Points A and B are points given heights h ₁ and h ₂ from the origin O, respectively, and the points A, B and origin O are arranged so that x = 0 on the image. In such a situation, the X coordinate of the camera can be set to Tx = 0 with respect to the points A, B, and the origin O, and there is no rotation in the Y axis direction with respect to the reference points arranged in the Z axis direction. Even generality is not lost. In FIG. 4, perspective projection images p ₁ , p ₂ , and p ₀ are drawn on the image planes of the reference points A, B, and O, and this projection relationship is assumed to follow the perspective projection of Expression (4). Assume that the image coordinates of the image coordinate points p ₁ , p ₂ , and p ₀ are (0, y ₁ ), (0, y ₂ ), and (0, y ₀ ), respectively.

In principle, the camera parameters cannot be determined only by the points A, B, and O on the Z axis. In addition to these points, points arranged so that x = 0 on the image plane are also used as reference points. FIG. 5 shows an example in which the points C and D are arranged as reference points at positions separated from the origin O by distances d ₃ and d ₄ along the Y axis. Points C and D are also projected onto p ₃ and p ₄ , respectively, according to the perspective projection equation of equation (4). The image coordinates of p ₃ and p ₄ are (0, y ₃ ) and (0, y ₄ ), respectively.

Here, a calculation formula for calibrating the camera is derived using the prior information on the reference points A, B, O, C, and D described above. Since the origin O = (0, 0, 0) in FIG. 4 is projected onto p ₀ on the image plane, Expression (5) is obtained using Expression (4).

Similarly, since the three-dimensional coordinates of the points A and B in FIG. 4 are given by (0, 0, h ₁ ) and (0, 0, h ₂ ), respectively, the image coordinates p ₁ and p ₂ are expressed by Equation (6). , (7) is obtained.

On the other hand, since the three-dimensional coordinates of the points C and D in FIG. 5 are given by (0, d ₃ , 0) and (0, d ₄ , 0), respectively, the image coordinates p ₃ and p ₄ are expressed by the equations (8), The relationship (9) is obtained.

  Since equation (5) can be transformed into equation (10), if the relational expression of equation (10) is used in equations (6) to (9), equations (11) to (14) are transformed, respectively. can do.

  Furthermore, equation (15) can be derived from equations (11) and (12), and equation (16) can be derived from equations (13) and (14).

Therefore, when equation (16) is substituted into equation (15), a quadratic equation relating to tan (φ) is obtained. When this is solved in the range of −π / 2 <φ <0, equation (17) is obtained for tan (φ). (Φ can be obtained by taking tan ⁻¹ (inverse function of tan) of the value on the right side of equation (17)).

  When Expression (17) is substituted into Expression (16), α can be expressed as Expression (18).

  Further, simultaneous equations (10) to (14) yield a simultaneous linear equation of equation (19).

Therefore, if φ and α are given in equation (19), Ty and Tz can be obtained. Thus, in this embodiment, the spatial arrangement of the points A, B, C, D, and O and the observation coordinates of each point: y ₁ , y ₂ , y ₃ , y ₄ , y ₀ , and the formula (5 ) To Expression (9), the focal length α, the inclination angle φ, and the viewpoint position (Ty, Tz) in the YZ coordinate system are obtained. In the present invention, camera parameters are obtained from five points A, B, O, C, and D, but the present invention is also applied to five or more reference points by increasing the points on the Y-axis or Z-axis. It can be easily expanded (the case of nine points will be described in Example 2).

  FIG. 6 is a basic configuration diagram relating to the camera calibration invention described in claim 1, and this embodiment will be described with reference to FIG. The present invention presents a reference object for calibration, a subject control unit 101 as subject control means for controlling the subject position in conjunction with the image correction unit 104, a camera 102 as a calibration target image input device, and calibration. An image correction unit 104 that obtains an image to be used; an image correction unit 104 that performs preprocessing for camera calibration processing in conjunction with the subject control unit 101 with respect to the image from the image input unit 103; A reference point measuring unit 105 serving as a reference point measuring unit that measures image coordinates necessary for the calibration process from the processed image, and a camera calibration processing unit 106 serving as a camera calibration processing unit that executes camera calibration are included.

  In this configuration, the camera to be calibrated does not necessarily have to be connected as a component, and a sample image necessary for calibration may be acquired through the image input unit 103. The image input unit 103 includes a hard disk and a RAID device. Either a recording medium such as a CD-ROM or a remote data resource via a network may be used. Furthermore, it is possible to acquire an image in real time according to the necessity of processing, and the present invention does not necessarily require a storage device.

The subject control unit 101 in FIG. 6 presents the target designed in FIG. 13 in front of the camera 102 as a photographing subject for calibration. The target shown in FIG. 13 is composed of a vertical bar 11, a horizontal bar 12, and a plane board 13 that serves as a foundation for them. The vertical bar 11 is provided with points A and B at the heights h ₁ and h ₂ from the point O as the origin. The subject control unit 101 can freely rotate the horizontal bar 12 around the vertical bar 11 in response to a command. On the horizontal bar 12, points C and D are installed at distances d ₃ and d ₄ from the point O, respectively. A sample image is obtained from the image input unit 103 through the camera 102 (FIG. 15). In FIG. 15, the center of the image is the center of the image coordinate system, and the xy image coordinate system with the point as the origin is displayed in an overlapping manner. Camera calibration is performed using this sample image, but preprocessing is performed by the image correction unit 104 in FIG. 6 before measuring the image coordinates of each reference point. The processing is continued according to the processing flow of FIG. 8 until the vertical bar 11 (one-dimensional object composed of points A, B, and O) shown in FIG. 13 matches the y-axis on the screen. What does not match is that the vertical bar 11 of the target is not installed along the Y axis (in other words, the X coordinate value of the points A, B, and O cannot be set to 0) and the optical axis of the camera 102 This is due to the fact that the rotation of is not zero.

  The subject control unit 101 performs the processing flow shown in FIG. 8 so that an arbitrary point on the vertical bar (corresponding to the point Q in FIG. 1) intersects the origin (image center) on the image plane as shown in FIG. To change the position of the target. First, the subject is moved by ΔX along the X axis set as the base in accordance with a command (steps S1 and S2). At this time, it is determined whether or not an arbitrary point on the vertical bar 11 matches the image center in the sample image obtained from the image input unit 103. For example, a straight line is detected from the sample image, the largest line segment is extracted, and it is determined whether or not the image center overlaps the straight line (steps S3 and S4). If a matching point is not found even if it is moved within the range that can be moved in the X-axis direction, this time, the subject is moved by ΔY along the Y direction (steps S3 and S5), and based on the position of the subject on the X-axis. Return (step S1), and again move the subject by ΔX along the X axis. In this way, the subject is moved on the XY plane, and it is determined whether or not the point on the vertical bar overlaps the image center. If they match, the image shown in FIG. 16 is obtained, and the process proceeds to the processing of the image correction unit 104 in FIG. In FIG. 10, the rotation around the optical axis of the camera is detected and correction is performed to cancel the rotation. 10, the pixels of the vertical bar 11 are first voted (step S11), the linear parameter is detected (step S12), and the angle (camera optical axis) formed by the vertical bar 11 and the y-axis shown in FIG. (Rotation around) θ is detected (step S13). In this detection process, the image coordinates forming the vertical bar 11 are Hough transformed to detect a straight line, and an angle θ formed with the y-axis is obtained. Finally, the image is rotated around the center of the image so as to cancel the detected rotation angle (step S14), and the image of FIG. 17 is obtained.

When the processing in the image correction unit 104 is completed, the reference point measurement unit 105 in FIG. 6 measures the image coordinates of each reference point. In FIG. 17, since the target vertical bar 11 coincides with the y axis, y ₁ , y ₂ , and y ₀ are obtained as y coordinates on the image corresponding to the points A, B, and O, respectively. On the other hand, in order to arrange the x-coordinate values on the images of the reference points C and D to be always 0, it is necessary to arrange the reference points C and D so as to face the direction of the camera as shown in FIG. is there. The circles on the broken line in FIG. 17 indicate points C and D that do not face the camera. Therefore, returning to the subject control unit 101, according to the processing flow of FIG. 12, the horizontal bar 12 on which the points C and D are arranged is rotated by Δω around the vertical bar 11 (steps S21 and S22), and the horizontal bar 12 at that time A straight line is detected from the projected image (step S23), and when the line segment most closely matches the y-axis (steps S24 and S25), the image coordinates of the points C and D: y ₃ and y ₄ are used as the reference point measuring unit 105. Measure.

If the image coordinates of the reference point can be measured as described above, the camera calibration processing unit 106 shown in FIG. 6 calculates camera parameters along the processing flow shown in FIG. Since the derivation of the calculation formula and the basic principle have been described above, the camera parameters are obtained here using the calculation formula. First, in step S31, geometric information regarding each reference point: h ₁ , h ₂ , d ₃ , d ₄ and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y _3. , Y ₄ is substituted into equation (17) to obtain tan (φ) (the camera tilt angle φ can be obtained by taking the inverse function of tan of the value). Next, in step S32, geometric information about each reference point: h ₁ , h ₂ , d ₃ , d ₄ and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y ₃ and y ₄ are substituted into the equation (18) to obtain the focal length α (in this case, the order of obtaining tan (φ) and α may be switched). Finally, in step S33, the camera coordinate position (Ty, Tz) is obtained by substituting the y-coordinate value of each reference point, the inclination angle φ and the focal length α obtained so far, into equation (20).

  That is, when using Expression (20), it is necessary to obtain a Moore-Penrose type generalized inverse matrix. On the other hand, the calculation without obtaining the generalized inverse matrix is described below. The right side of Expression (20) can be transformed into Expression (21) by replacing Expression (24) to Expression (27) and Expression (29) to Expression (31). Furthermore, the equation (22) can be rewritten (ΔW is defined by the equation (28)), and finally, the right side of the equation (20) can be rearranged into the equation (23).

  Therefore, equation (23) can also be used to calculate the camera viewpoint position (Ty, Tz).

  The overall processing flow of the first embodiment is as shown in FIG. First, the subject control unit 101 presents a target that is the subject shown in FIG. 13 (step S41), and an image of the subject is acquired by the camera 102 and the image input unit 103 (step S42).

  Next, the image correction unit 104 determines whether or not the reference points A and B on the target vertical bar 11 overlap the image center (step S43), and the subject control unit 101 determines the subject position according to the flowchart of FIG. The image correction unit 104 rotates the image according to the flowchart of FIG. 10 so as to cancel the angle (rotation around the camera optical axis) θ between the vertical bar 11 and the y-axis (step S44) (step S44). Step S45).

  Next, the subject control unit 101 determines whether or not the reference points C and D on the target horizontal bar 12 coincide with the optical axis of the camera 102 (step S46), and follows the flowchart of FIG. 12 is rotated so as to coincide with the optical axis of the camera 102 (step S47).

  Next, the reference point measurement unit 105 measures the reference points A to D (step S48), and the camera calibration processing unit 106 calculates camera parameters according to the flowchart of FIG. 11 (step S49).

  As described above, the target designed in FIG. 13 is set as a photographic subject for calibration, and processing is performed according to the basic configuration in FIG. 6, so that the camera viewpoint position (Ty, Tz) shown in FIG. The tilt angle φ and the focal length α of the camera can be obtained.

(Example 2)
In the present embodiment, a target designed as shown in FIG. 14 is used as a photographing subject for calibration. 14, the same parts as those in FIG. 13 are denoted by the same reference numerals. In FIG. 14, the point O is the origin, the points A ₁ , A ₂ , B ₁ , B ₂ are the points on the vertical bar 11, and the points C ₁ , C ₂ , D ₁ , D ₂ are the points on the horizontal bar 12. (In this example, a total of 9 points are used as reference points). The basic configuration of the second embodiment is realized by the same configuration as that of FIG. 6, and the camera 102 can be calibrated by the same processing as that of the first embodiment except that the processing is slightly different depending on the reference point. Hereinafter, the difference will be mainly described.

Points A ₁ , A ₂ , B ₁ , B ₂ are located at heights h ₁ , h ₅ , h ₂ , h ₆ from the origin O, respectively, and their three-dimensional coordinates are (0, 0, h ₁ ), When (0, 0, h ₅ ), (0, 0, h ₂ ), (0, 0, h ₆ ), the coordinates on the image plane are (0, y ₁ ), (0, y ₅ ), respectively. , (0, y ₂ ), (0, y ₆ ) are observed. On the other hand, the points C ₁ , C ₂ , D ₁ , and D ₂ are set at positions d ₃ , d ₇ , d ₄ , and d ₈ from the origin O along the Y-axis direction. When (0, d ₃ , 0), (0, d ₇ , 0), (0, d ₄ , 0), (0, d ₈ , 0), the coordinates on the image plane are (0, y _{3), (0, y 7} ), (0, y 4), and is observed (0, y _8). Assuming that all these reference points follow the perspective projection formula of Formula (4), camera parameters are obtained using each reference point as in the first embodiment.

First, the subject control unit 101 in FIG. 6 presents the target in FIG. 14, and if necessary, the sample image is aligned with the y-axis according to the processing flow of the subject control unit 101 in FIG. 8 so that the vertical bar 11 in FIG. It is processed. It is assumed that the reference points C ₁ , C ₂ , D ₁ , D ₂ are also arranged on the y-axis as a result of rotating the horizontal bar 12 by the processing flow of FIG. Next, the reference point observation unit 105 in FIG. 6 measures the y-coordinate y _j (j = 0, 1, 2,..., 8) on the image plane from the image image corrected. Subsequently, the camera calibration processing unit 106 extracts the y-coordinates of the points A ₁ , B ₁ , C ₁ , and D _{1 from} the observed nine reference points, substitutes them into the equation (17), and tan (φ) (The tilt angle φ of the camera can be obtained by taking the inverse function of tan of the value). Next, geometric information about each reference point: h ₁ , h ₂ , d ₃ , d ₄ and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y ₃ , y Substituting ₄ into Equation (18) to obtain the focal length α (tan (φ) and α may be calculated in reverse order). Up to this point, the same calculation formula as in the case of using the five points in the first embodiment is used. On the other hand, the equation (32) is used to calculate the camera viewpoint position (Ty, Tz) (this equation can be easily derived by the same procedure as that derived from equation (20), and its details are omitted). ).

  At this time, when the substitution of the formulas (24) to (27) is performed, the formula (32) is expressed as the formula (33).

  Further, when the substitution of the equations (34) to (39) is performed, the equation (33) can be rewritten as shown in the equation (40).

  Therefore, the camera viewpoint position (Ty, Tz) can also be obtained using the formula (40).

  As described above, when the target shown in FIG. 14 is used in the present invention, the calculation formula using five points of the first embodiment is used to calculate tan (φ) and α, and the camera viewpoint position (Ty, Tz) is used. As for the calculation of, the camera parameters can be calculated by solving simultaneous equations (equation (32)) corresponding to the number of reference points.

(Example 3)
This embodiment relates to camera calibration according to the second aspect of the present invention, and performs camera calibration using a calibration photographing target shown in FIGS. FIG. 20 is an overview when viewed from the vertical bar 11 of FIG. Although the base supporting this target is a disk shape, it may be a flat board in which concentric circles 14 centering on the point O are drawn on the base. A concentric circle 14 having radii d ₃ and d ₄ centered on the origin O is drawn on the base in FIG. 20, and arbitrary points on the respective circumferences correspond to points C and D, respectively. If this concentric circle 14 is used, when a straight line composed of points A, B, and O coincides with the y-axis on the image plane, the points C and D are simply measured by measuring the points that intersect the y-axis on the circumference. The y coordinate value on the image plane can be measured. Thereby, the step (S22, S47) of rotating the horizontal bar 12 in the subject control unit 101 can be omitted.

  In the present embodiment, when the camera is calibrated based on the calculation principle as described in the first embodiment, the camera calibration process is performed more easily and smoothly by using the reference objects of FIGS. 19 and 20. be able to. The third embodiment has the same basic configuration as that of the first embodiment, and the portion that uses the reference object shown in FIGS. 19 and 20 in the subject control unit 101 in FIG. 6 is different. Hereinafter, only the location where the reference object is imaged and the image coordinates of each reference point are measured will be described.

  The subject control unit 101 in FIG. 6 presents the calibration shooting target shown in FIG. 19 in front of the camera 102 as a subject. A sample image is obtained from the image input unit 103 through the camera 102. In this sample image, the vertical bar 11 (one-dimensional object composed of points A, B, and O) shown in FIG. 19 is followed on the screen in accordance with the processing flow of FIG. The target position is changed until the image shown in FIG. 21 is obtained. Next, when it is determined in FIG. 21 that the vertical bar 11 and the y-axis do not coincide with each other, the rotation around the camera optical axis is detected as in the first embodiment, and the image rotation is performed so as to cancel the rotation. (Flowchart in FIG. 10). When the processing of the image correction unit 104 in FIG. 6 is completed, the rotation-corrected image in FIG. 22 is obtained.

In this rotation-corrected image, the points C and D are points on the circumference of FIG. 20, so the point intersecting the y-axis in FIG. 22 is perpendicular to the vertical bar 11 and faces the direction of the camera 102. It can be seen that the points are C and D. According to this knowledge, y coordinate values: y ₃ and y ₄ of points C and D are measured from points on the circumference intersecting the y axis in FIG. If the image coordinates of each reference point can be measured, camera parameters can be calculated by the camera calibration processing unit 106 in FIG. 6 as in the first embodiment.

  As described above, the image coordinates of the reference point are measured using the calibration shooting target shown in FIGS. 19 and 20, and the camera parameters can be obtained by the same calculation procedure described in the first embodiment.

Example 4
In this embodiment, a method of calibrating a plurality of cameras (multi-viewpoint cameras) using the calibration shooting target shown in FIGS. 23 and 24 will be described. FIG. 23 is an overview of the target, and the shape is designed with the same specifications as FIG. However, it is assumed that the azimuth needle 15 in FIG. 23 operates so as to freely rotate around the center O according to a command from the subject control unit 101.

An example in which a total of seven cameras 0 to 6 are arranged so as to surround the target is shown in FIG. This figure is an outline view seen from directly above, and the reference camera is camera 0. In this camera arrangement, it is assumed that the position or posture of each camera is adjusted in advance so that an arbitrary point on the vertical bar 11 intersects the image center of the sample image. In this camera arrangement, the images of FIG. 21 are obtained through the image input unit 103 in the cameras 0 to 6. For the sample image obtained from each camera, the image correction unit 104 detects the angle between the vertical bar 11 and the y-axis on the image plane, and cancels the angle as in the previous embodiments. Are rotated (flow chart of FIG. 10) to obtain the images of FIG. 22 respectively (there is no need to prepare images that are all in the same state, as long as the vertical bar 11 coincides with the y-axis of each image. ). Further, the subject control unit 101 rotates the azimuth needle 15 by Δω after obtaining the rotation image, and when the azimuth needle 15 and the y axis on the image plane coincide with the sample images from the camera 0 to the camera 6. The rotation angle ω _i of the reference point measurement unit 105 measures the rotation angle ω _i one after another. This azimuth angle ω _i (i = 0, 1,..., 6) is temporarily stored.

If the image of FIG. 22 is obtained with respect to the sample image of each camera, the reference point measuring unit 105 determines y coordinates corresponding to the points A, B, O, C, and D for each camera: y _ij (i = 0, 1,..., 6, j = 0, 1, 2,. First, a sample image of the camera 0 is taken out and image coordinates of points A, B, and O are obtained. If the y-coordinates y _ij corresponding to the points A, B, O, C, and D can be measured for each camera, the camera calibration processing unit 106 uses the geometric information about each reference point: h ₁ , as in the first embodiment. h ₂ , d ₃ , d ₄ and y coordinate values on the image of the corresponding reference points: y _ij (i = 0, 1,..., 6, j = 0, 1,..., 4) And tan (φ _i ) (camera tilt angle φ _i can be obtained by taking the inverse function of tan of the value) and focal length α _i . Further, the camera viewpoint position (Ty _i ) according to the equation (20) or the equation (23) using the y-coordinate value of each reference point and the inclination angle φ _i and the focal length α _i obtained so far for each camera. , Tz _i ), i = 0, 1,.

Here, the camera viewpoint position (Ty _i , Tz _i ), i = 0, 1,..., 6 is obtained independently as Tx _i = 0 in each camera. Therefore, the viewpoint positions of other cameras are restored with the camera viewpoint (0, Ty ₀ , Tz ₀ ) obtained by the camera ₀ as a reference. Since the azimuth angle ω _i of each camera is obtained based on the azimuth angle (ω = 0) of the camera 0 when the image coordinate measurement is performed by the reference point measurement unit 105, the camera viewpoint position (for each camera) ( _{Ty i, Tz i), i} = 1,2, ..., removed Ty _i of 6, a camera viewpoint azimuth omega _i and position Ty _i referenced to camera 0 into equation (41) (Tx ' _i , Ty' _i , Tz _i ), i = 1, 2,.

By this restoration, the viewpoint positions (Tx ′ _i , Ty ′ _i , Tz _i ), i = 1, 2,..., 6 of the cameras 0 to 6 are obtained in the same coordinate system. This is useful for generating seamless 3D video or arbitrary viewpoint video.

  According to the embodiment described above, it is possible to calibrate a plurality of cameras arranged so as to surround the target for calibration photographing shown in FIGS. 23 and 24 at a time.

(Example 5)
While the first to fourth embodiments are camera calibrations based on the arrangement of reference points in FIG. 5, this embodiment is an embodiment relating to camera calibration according to claim 3, and see FIG. It is an example of camera calibration on the premise of point arrangement. In FIG. 25, points C and D are arranged at a position translated from the Z axis by a distance d. These points C and D are also projected onto p ₃ and p ₄ , respectively, according to the perspective projection formula of formula (4). Since the three-dimensional coordinates of points C and D in FIG. 25 are given as (0, d, h ₁ ) and (0, d, h ₂ ), respectively, the image coordinates of p ₃ and p ₄ are (0, y ₃₎ , respectively. ), (0, y ₄ ), the relationships of equations (42) and (43) are obtained.

  If the relational expression of the expression (10) is used in the expression (42) and the expression (43), they can be transformed into the expressions (44) and (45), respectively.

  Here, since Expression (46) can be derived from Expression (44) and Expression (45) (B is replaced by Expression (47)), Expression (46) is substituted into Expression (15). Then, a quadratic equation relating to tan (φ) shown in Expression (48) is obtained.

  Equation (48) can be solved by using the solution formula, but a solution in the range of −π / 2 <φ <0 is obtained as a solution of tan (φ) among the two solutions. By substituting the obtained tan (φ) into the equation (46), α can be obtained. Further, as in the previous examples, when the equations (10) to (12), and the equations (44) and (45) are combined, the simultaneous linear equations with Ty and Tz in the equation (49) as unknowns ( (Substitution of the formula (24) to the formula (27) is performed). If φ and α are given in this equation (49), Ty and Tz can be obtained. It should be noted that by replacing the equations (28) to (31), the equation (49) can be transformed into the equation (50), and finally can be rearranged into the equation (51).

  Therefore, Ty and Tz can be obtained using the equation (51).

As described above, the present embodiment has the spatial arrangement of the points A, B, C, D, and O shown in FIG. 25 and the observation coordinates of each point: y ₁ , y ₂ , y ₃ , y ₄ , y ₀ , In addition, in both geometrical constraint conditions expressed by the equations (5) to (7), and the equations (42) and (43), the focal length α, the inclination angle φ, and the viewpoint position in the YZ coordinate system ( Ty, Tz).

  This embodiment has a basic configuration shown in FIG. 6, and camera calibration is performed according to the processing described in the first embodiment. However, the difference from the previous embodiments is that the subject control unit 101 in FIG. 6 performs camera calibration using the calibration shooting target shown in FIG. When the camera is calibrated based on the calculation method as described above, the camera calibration process can be performed more easily and smoothly by using this reference object. In the following, the reference object is photographed, the image coordinates of each reference point are measured, and the point at which the camera is calibrated using the calculation method described above will be described.

Although the base 16 that supports the target of FIG. 26 has a disk shape, it may be a flat board in which a circle with a radius d centered on the point O is drawn on the base 16. The rotating bar 17 can be arbitrarily rotated around the vertical bar 11 along the circumference of the base 16, and the points of heights h ₁ and h ₂ on the circumference of the radius d are C 1. , D. As in the previous examples, the subject control unit 101 in FIG. 6 has the calibration photography target in FIG. 26 installed, and the sample image obtained through the camera 102 and the image input unit 103 is processed by the image correction unit 104 to produce light. It is assumed that the axis rotation is canceled and the vertical bar 11 coincides with the y axis on the image plane. Next, if the rotating rod 17 is rotated to a position intersecting the y axis on the circumference, the y coordinate value on the image plane of the points C and D can be easily measured by the reference point measuring unit 105. it can. Thus, if the disk-shaped base 16 or the plane board in which a circle is drawn on the base 16 and the rotating rod 17 are used, the work of directing points C and D to the camera optical axis can be reduced.

If the image coordinates of the reference point can be measured as described above, the camera calibration processing unit 106 in FIG. 6 calculates camera parameters along the processing flow in FIG. Since the derivation of the calculation formula and the basic principle have been described above, the camera parameters are obtained here using the calculation formula. First, geometric information about each reference point: h ₁ , h ₂ , d and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y ₃ , y ₄ ) Is substituted into the coefficient of the quadratic equation related to tan (φ), and the quadratic equation is solved within a reasonable range of −π / 2 <φ <0 to obtain tan (φ) (the tilt angle φ of the camera is It is obtained by taking the inverse function of tan of the value). Next, geometric information regarding each reference point: h ₁ , h ₂ , d and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y ₃ , y ₄ are obtained. The focal length α is obtained by substituting tan (φ) into the equation (46). Finally, the y-coordinate value of each reference point, the tilt angle φ and the focal length α obtained so far are substituted into equation (51) to obtain the camera viewpoint position (Ty, Tz).

  As described above, the target designed in FIG. 26 is used as a photographic subject for calibration, and processing is performed according to the basic configuration in FIG. 6, so that the camera viewpoint position (Ty, Tz) shown in FIG. The tilt angle φ and the focal length α of the camera can be obtained.

(Example 6)
The present embodiment is an embodiment relating to camera calibration according to the fourth aspect of the present invention, and is an example of camera calibration based on the reference point arrangement of FIG. In FIG. 27, the points C and D are arranged at positions where the points A and B are rotated at the rotation angle β around the X axis (the axis is directed from the front to the back of the page). These points C and D are also projected onto p ₃ and p ₄ , respectively, according to the perspective projection formula of formula (4). The three-dimensional coordinates of points C and D in FIG. 27 are given by (0, h ₁ sin (φ), h ₁ cos (φ)), (0, h ₂ sin (φ), h ₂ cos (φ)), respectively. Therefore, if the image coordinates of p ₃ and p ₄ are (0, y ₃ ) and (0, y ₄ ), the relations of equations (52) and (53) are obtained.

  If the relational expression of Expression (10) is used in Expression (52) and Expression (53), it can be transformed into Expression (54) and Expression (55), respectively.

  Here, since Formula (56) can be derived from Formula (54) and Formula (55) (B and C are replaced by Formula (57)), Formula (15) is replaced by Formula (56). Is substituted, a quadratic equation relating to tan (φ) shown in Expression (58) is obtained.

  Equation (48) can be solved by using the solution formula, but a solution in the range of −π / 2 <φ <0 is obtained as a solution of tan (φ) among the two solutions. Further, as in the previous examples, when the equations (10) to (12), and the equations (54) and (55) are combined, the simultaneous linear equations with the unknown values of Ty and Tz in the equation (59) ( (Substitution of the formula (24) to the formula (27) is performed). If φ and α are given in this equation (59), Ty and Tz can be obtained. It should be noted that by replacing the equations (28) to (31), the equation (59) can be transformed into the equation (60), and finally can be rearranged into the equation (61).

  Therefore, Ty and Tz can be obtained by calculating according to the equation (61).

As described above, in this embodiment, the spatial arrangement of the points A, B, C, D, and O shown in FIG. 27 and the observation coordinates of each point: y ₁ , y ₂ , y ₃ , y ₄ , y ₀ , In addition, the focal length α, the inclination angle φ, and the viewpoint position in the YZ coordinate system in the geometrical constraint conditions expressed by the equations (5) to (7) and the equations (52) and (53) ( Ty, Tz).

  This embodiment has a basic configuration shown in FIG. 6, and camera calibration is performed according to the processing described in the first embodiment. However, the difference from the previous embodiments is that the subject control unit 101 in FIG. 6 performs camera calibration using the calibration photographing target shown in FIG. When the camera is calibrated based on the calculation method as described above, the camera calibration process can be performed more easily and smoothly by using this reference object. In the following, the reference object is photographed, the image coordinates of each reference point are measured, and the point at which the camera is calibrated using the calculation method described above will be described.

In FIG. 28, a semi-disc 18 (target plane disc) on which concentric circles having a radius h ₁ and a radius h ₂ centered on the point O are drawn is erected on a base 19, and this semi-disc 18 (which is necessarily a semi-disc). (It is not necessary, and it may be a plane board on which a concentric circle centered on the point O is drawn) is designed to rotate around the rotation axis. When the semicircular disk 18 stands vertically with respect to the base 19, the intersections of the concentric circle and the central axis become points A and B, and the points on the circumference when rotated at the rotation angle β correspond to C and D. . As in the previous embodiments, the target for calibration photography in FIG. 28 is installed in the subject control unit 101 in FIG. 6, and the sample image obtained through the camera 102 and the image input unit 103 is processed by the image correction unit 104. It is assumed that the rotation of the axis is canceled and the center axis coincides with the y axis on the image plane. Next, the semicircular disk 18 is rotated at a rotation angle β (β is a fixed angle or a rotation angle is appropriately recorded), and the intersection of the concentric circle and the y axis at that time is pointed by the reference point measuring unit 105. Obtained as y-coordinate values on the C and D image planes.

If the image coordinates of the reference point can be measured as described above, the camera calibration processing unit 106 in FIG. 6 calculates camera parameters along the processing flow in FIG. Since the derivation of the calculation formula and the basic principle have been described above, the camera parameters are obtained here using the calculation formula. First, geometric information about each reference point: h ₁ , h ₂ , β and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y ₃ , y ₄ ) Is substituted into the coefficient of the quadratic equation related to tan (φ), and the quadratic equation is solved within a reasonable range of −π / 2 <φ <0 to obtain tan (φ) (the tilt angle φ of the camera is It is obtained by taking the inverse function of tan of the value). Next, geometric information regarding each reference point: h ₁ , h ₂ , β and y coordinate values on the image of the corresponding reference point: y ₀ , y ₁ , y ₂ , y ₃ , y ₄ are obtained. Substituting tan (φ) into equation (56), the focal length α is obtained. Finally, the y coordinate value of each reference point, the tilt angle φ and the focal length α obtained so far are substituted into the equation (61) to obtain the camera viewpoint position (Ty, Tz).

  As described above, the target designed in FIG. 28 is used as a photographic subject for calibration, and processing is performed according to the basic configuration in FIG. 6, so that the camera viewpoint position (Ty, Tz) shown in FIG. The tilt angle φ and the focal length α of the camera can be obtained.

  Further, a part or all of the functions of each means in the camera calibration apparatus of the present embodiment can be configured by a computer program, and the program can be executed using the computer to realize the present invention. Needless to say, the procedure in the camera calibration method can be constituted by a computer program and the program can be executed by the computer, and the program for realizing the function by the computer can be read by a computer-readable recording medium such as an FD. (Floppy (registered trademark) Disk), MO (Magneto-Optical disk), ROM (Read Only Memory), memory card, CD (Compact Disk) -ROM, DVD (Digital Versatile) Disk) -ROM, CD-R, CD-RW, HDD, removable disk, etc., and can be stored and distributed. It is also possible to provide the above program through a network such as the Internet or electronic mail.

It is explanatory drawing which shows the camera viewpoint used by this invention, and a world coordinate system. It is explanatory drawing which shows the camera calibration operation | work using the two-dimensional object in the conventional camera calibration method. It is explanatory drawing which shows the camera calibration operation | work using the one-dimensional object in the conventional camera calibration method. Reference point A used in the present invention, B, is an explanatory diagram showing the relationship between O and the projection point _{p 1, p 2, p 0} . Reference point C used in the present invention, is an explanatory diagram showing the relationship between the projection point p _3, p ₄ and D. 1 is a basic configuration diagram showing an embodiment of a camera calibration apparatus of the present invention. It is the whole flowchart in the embodiment of this invention. It is a flowchart explaining the process of the subject movement in the subject control unit in the embodiment of the present invention. It is explanatory drawing which shows the example of arrangement | positioning of the multiview camera in Example 4 of this invention. It is a flowchart explaining the process in the image correction part in the embodiment of this invention. It is a flowchart explaining the process in the camera calibration process part in the embodiment of this invention. It is a flowchart explaining the process of horizontal bar rotation in the to-be-photographed object control part in the example of embodiment of this invention. It is explanatory drawing of the target for calibration photography used in Example 1 of this invention. It is explanatory drawing of the target for calibration imaging | photography used in Example 2 of this invention. It is explanatory drawing which shows a sample image when image | photographing the calibration imaging | photography target of FIG. It is explanatory drawing which shows the sample image for a calibration when the target of FIG. 13 is moved. It is explanatory drawing which shows the sample image for calibration when the image of the target of FIG. 13 is rotated. It is explanatory drawing which shows the relationship of the reference points C and D and the camera optical axis in the example of embodiment of this invention. It is explanatory drawing of the target for calibration imaging | photography used in Example 3 of this invention. It is a general-view figure from right above the target for proofreading used in Example 3 of the present invention. It is explanatory drawing of the sample image for a calibration when the target is moved in Example 3 of this invention. It is explanatory drawing of the sample image for a calibration when the image of FIG. 21 is rotated around the image center. It is explanatory drawing of the target for calibration imaging | photography used in Example 4 of this invention. It is a general-view figure from right above the target for proofreading used in Example 4 of the present invention. Reference point C on the assumption in Example 5 of the present invention, is an explanatory diagram showing the relationship between the projection point p _3, p ₄ and D. It is explanatory drawing of the target for calibration photography used in Example 5 of this invention. Reference point C on the assumption in Example 6 of the present invention, is an explanatory diagram showing the relationship between the projection point p _3, p ₄ and D. It is explanatory drawing of the target for calibration photography used in Example 6 of this invention.

Explanation of symbols

  0-6, 102 ... camera, 11 ... vertical bar, 12 ... horizontal bar, 13 ... flat board, 14 ... concentric circle, 15 ... azimuth needle, 16, 19 ... foundation, 17 ... rotating bar, 18 ... semi-disc.

Claims (10)

A device for calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
A subject whose shape is known is photographed using an image input device to be calibrated, an image coordinate system is set in the acquired image, and an arbitrary point on a line segment formed from a reference point on the subject is an image coordinate system Subject control means for changing the posture / position of the subject or the posture / position of the image input device so as to be projected onto the origin of
The image is rotated around the origin of the coordinate system set on the acquired image so that the line segment formed from the reference axis set on the image coordinate system and the reference point on the subject coincides, and another reference Image correction means for positioning the projected coordinates of all reference points on the reference axis by changing the posture / position of the subject or the posture / position of the image input device so that the point is also a point on the reference axis When,
The geometric information of the reference point and its relational expression are derived from the relational expression between the internal information and external information of the image input device derived from the perspective projection conditions when projecting from the three-dimensional points of all reference points onto the reference axis of the image coordinate system. Camera calibration processing means for calibrating internal information or external information of the image input device from image coordinates of a reference point on a reference axis;
A camera calibration device comprising: A device for calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
A reference object with a disk that forms a plurality of circles centered on a reference point on a foundation and a vertical bar that forms a line segment that is lowered to the reference point so as to be perpendicular to the circle is used for calibration photography. An image coordinate system is set in the image of the subject acquired using the image input device to be calibrated as the subject, and the posture of the subject is projected so that an arbitrary point on the vertical bar is projected to the origin of the coordinate system Subject control means for changing the position or posture / position of the image input device;
The image is rotated around the origin of the coordinate system set on the acquired image, the reference axis set on the image coordinate system and the vertical bar of the reference object are matched, and a plurality of points set on the vertical bar, the reference A reference point measuring means for obtaining the image coordinates of the reference point, using the point, and the intersection of the plurality of concentric circles and the reference axis as a reference point;
The geometric information of the reference point is derived from the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when projecting from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system. Camera calibration processing means for calibrating internal information or external information of the image input device from the image coordinates of the reference point on the reference axis acquired by the reference point measuring means;
A camera calibration device comprising: A device for calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
A disk that forms a circle centered on a reference point on a base, a vertical bar that is installed perpendicular to the circle and that forms a line segment extending from the reference point, and a vertical line that extends along the circumference of the circle A reference object provided with a rotating rod that forms a line segment that rotates around a bar is set as a subject for calibration shooting, and an image coordinate system is set in the image of the subject acquired using an image input device to be calibrated. Subject control means for changing the posture / position of the subject or the posture / position of the image input device so that an arbitrary point of the vertical bar is projected onto the origin of the coordinate system;
Rotate the image around the origin of the coordinate system set on the acquired image to match the reference axis set on the image coordinate system with the vertical bar of the reference object, and further rotate the rotary bar around the vertical bar When the rotating bar coincides with the reference axis, a plurality of points set on the vertical bar, a reference point, and a plurality of points set on the rotating bar are used as reference points, and reference point measuring means for acquiring image coordinates of the reference points When,
The geometric information of the reference point is derived from the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when projecting from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system. Camera calibration processing means for calibrating internal information or external information of the image input device from the image coordinates of the reference point on the reference axis acquired by the reference point measuring means;
A camera calibration device comprising: A device for calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
It consists of a base with a flat surface and a semi-disc, and the semi-disc has a plurality of identical semi-circles centered on the reference point, and the straight portions of the semi-circles coincide with the straight portions of the semi-disc The half disk is in contact with the plane of the base and the straight part of the half disk, and it is a mechanism that allows the half disk to rotate around the straight part that touches, and on the half disk passing through the center of multiple semicircles The reference object with the center line as the center axis is taken as the subject for calibration photography, and the image coordinate system is set in the subject image acquired using the image input device to be calibrated. Subject control means for changing the posture / position of the subject or the posture / position of the image input device so as to be projected onto the origin of a coordinate system;
The image is rotated around the origin of the coordinate system set on the acquired image, the reference axis set on the image coordinate system and the central axis of the reference object are matched, and a plurality of points set on the central axis, the reference The point and the image coordinates of the reference point when the semi-disk is inclined with respect to the base plane or when the semi-circle intersects the base axis when the semi-disk is rotated with respect to the rotation axis of the semi-disk. A reference point measuring means for obtaining
The geometric information of the reference point is derived from the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when projecting from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system. Camera calibration processing means for calibrating internal information or external information of the image input device from the image coordinates of the reference point on the reference axis acquired by the reference point measuring means;
A camera calibration device comprising: A method of calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
The subject control means captures a subject whose shape is known using an image input device to be calibrated, sets an image coordinate system in the acquired image, and selects an arbitrary line segment formed from a reference point on the subject. Changing the posture / position of the subject or the posture / position of the image input device so that the point is projected to the origin of the image coordinate system;
The image correcting means rotates the image around the origin of the coordinate system set on the acquired image so that the line segment formed from the reference axis set on the image coordinate system and the reference point on the subject coincides, In addition, by changing the posture / position of the subject or the posture / position of the image input device so that other reference points are also points on the reference axis, the projected coordinates of all reference points are placed on the reference axis. A step of positioning;
According to the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when the camera calibration processing means projects from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system, Calibrating internal information or external information of the image input device from the geometric information of the reference point and the image coordinates of the reference point on the reference axis;
A camera calibration method comprising: A method of calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
A reference object in which the subject control means comprises a disk that forms a plurality of circles centered on a reference point on a certain base, and a vertical bar that forms a line segment drawn down to the reference point so as to be perpendicular to the circle Is the subject for calibration shooting, and an image coordinate system is set in the image of the subject acquired using the image input device to be calibrated, and an arbitrary point on the vertical bar is projected to the origin of the coordinate system Changing the posture / position of the subject or the posture / position of the image input device;
The reference point measuring means rotates the image around the origin of the coordinate system set on the acquired image, aligns the reference axis set on the image coordinate system with the vertical bar of the reference object, and installs on the vertical bar A plurality of points, reference points, and intersections between a plurality of concentric circles and the reference axis as reference points, and a reference point measurement step for obtaining image coordinates of the reference points;
According to the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when the camera calibration processing means projects from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system, Calibrating internal information or external information of the image input device from geometric information of a reference point and image coordinates of the reference point on the reference axis acquired by the reference point measurement step;
A camera calibration method comprising: A method of calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
A subject control means is a disk that forms a circle centered on a reference point on a base, a vertical bar that is installed perpendicular to the circle and extends from the reference point, and a circumference of the vertical bar. A reference object with a rotating bar that forms a line segment that rotates around the vertical bar along the vertical axis is taken as the subject for calibration shooting, and the image coordinates in the image of the subject acquired using the image input device to be calibrated Setting a system and changing the posture / position of the subject or the posture / position of the image input device so that an arbitrary point of the vertical bar is projected onto the origin of the coordinate system;
The reference point measuring means rotates the image around the origin of the coordinate system set on the acquired image so that the reference axis set on the image coordinate system coincides with the vertical bar of the reference object, and the rotation bar Is rotated around the vertical bar, and the rotating bar coincides with the reference axis, the multiple points set on the vertical bar, the reference point, and the multiple points set on the rotating bar are used as reference points, and the image coordinates of the reference point are A reference point measurement step to be acquired;
According to the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when the camera calibration processing means projects from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system, Calibrating internal information or external information of the image input device from geometric information of a reference point and image coordinates of the reference point on the reference axis acquired by the reference point measurement step;
A camera calibration method comprising: A method of calibrating external information including internal information and posture and position of an image input device from an image acquired using a single eye or a plurality of image input devices,
The subject control means is composed of a base with a flat surface and a semi-disc, and the semi-disc has a plurality of identical semi-circles centered on the reference point, and the straight portions of the semi-circles are straight lines of the semi-disc The semi-disc is in contact with the base plane and the straight part of the semi-disk, and the semi-disk can rotate around the straight part that touches the axis of rotation. A reference object centered on a line on a semicircular disk passing through is a subject for calibration shooting, and an image coordinate system is set in the image of the subject acquired using the image input device to be calibrated, Changing the posture / position of the subject or the posture / position of the image input device so that an arbitrary point is projected onto the origin of the coordinate system;
The reference point measuring means rotates the image around the origin of the coordinate system set on the acquired image, and the reference axis set on the image coordinate system and the center axis of the reference object are coincident and installed on the center axis The reference points are the multiple points, reference points, and points where the semi-circles intersect the reference axis when the semi-disks are tilted with respect to the base plane or rotated with respect to the rotation axis of the semi-disks. A reference point measuring step for obtaining image coordinates of the reference point;
According to the relational expression between the internal information and the external information of the image input device, derived from the perspective projection conditions when the camera calibration processing means projects from the three-dimensional points of all the reference points onto the reference axis of the image coordinate system, Calibrating internal information or external information of the image input device from geometric information of a reference point and image coordinates of the reference point on the reference axis acquired by the reference point measurement step;
A camera calibration method comprising:   9. A camera calibration program characterized in that a program for causing a computer to execute each step according to any one of claims 5 to 8 is provided.   A computer-readable recording medium in which the camera calibration program according to claim 9 is recorded.

Images