JP4153322B2 - Method and apparatus for associating measurement points in photogrammetry - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention is for associating measurement points in photographic analysis performed using two or more stereo images, and in particular, in three-dimensional stereoscopic measurement by computer analysis performed using digital photographic images, The present invention relates to a system for performing association.
[0002]
[Prior art]
Conventionally, a technique for analyzing stereo images, in particular, computer analysis of a subject captured in a digital image, has been used in the field of photogrammetry, and is particularly used for creating topographic maps. .
[0003]
In the computer analysis of the stereo image, it is necessary to derive the positional relationship between at least two cameras and to associate the measurement points imprinted on the photographic images taken by the two cameras. Specifically, a measurement point imprinted in one photographic image is designated in another photographic image, and the measurement point is measured using two-dimensional coordinates on a plurality of photographic images and three-dimensional coordinates of a camera. Deriving 3D coordinates.
[0004]
Since it is difficult to correlate one measurement point with a plurality of photographs by manual operation by visual observation, Japanese Patent No. 3316682 (Patent Document 1) discloses pattern matching using a computer. A technique performed by correlation processing is disclosed.
[0005]
However, in the correlation processing for performing this association, performing all regions of the photographic image as a target lengthens the processing time, and, for example, the association is incorrect for photographic images that repeatedly contain similar patterns. There is also the problem of doing it in a different position. In order to solve this problem, various techniques for narrowing down the area where correlation processing is performed to solve the above problem have been proposed.
[0006]
In Japanese Patent Laid-Open No. 2001-280955 (Patent Document 2), a search area is set based on at least three segment points, and correlation processing is performed on images of the search areas to perform correlation processing. A technique for reducing the above is disclosed.
[0007]
Also, in the paper by Mosquito et al. (Non-patent Document 1) and the publication by Xu Tsuyoshi (Non-patent Document 2), the correlation processing is performed by narrowing the region to be correlated on the epipolar line using the epipolar constraint condition. A technique for reducing the number of search areas to be performed is disclosed (Non-Patent Document 1, page 528, left column, lines 9-14, Non-Patent Document 2, lines 31-32).
[0008]
However, as shown in FIG. 13, an epipolar line in which a half line (a light beam with respect to a measurement point) starting from the optical axis center 100a and passing through the measurement point 75 is projected onto another photographic image is usually the other photographic image. It exists from end to end. Therefore, even in this technique, the area for performing correlation processing is still extensive. Therefore, if this can be further narrowed down, the correlation processing can be performed at higher speed and higher accuracy. On the other hand, if the area is too narrowed, the photogrammetry itself can no longer be performed if the measurement point on the other image deviates from the search area.
[0009]
[Patent Document 1]
Japanese Patent No. 3316682 [Patent Document 2]
JP 2001-280955 A [Non-Patent Document 1]
Hiroshi Mosquito, 3 others, “Realization of Camera Geometric Correction Function in Video Rate Stereo Machine”, Journal of the Robotics Society of Japan, May 1998, Vol. 16, No. 4, p527-532
[Non-Patent Document 2]
Xu Tsuyoshi, “Three-dimensional CG from photographs”, Modern Science, January 2001, p. 31-33
[0010]
[Problems to be solved by the invention]
Accordingly, the technical problem to be solved by the present invention is to perform correlation at a higher speed and with a higher accuracy by reducing the area in which correlation processing is performed in association of measurement points in photogrammetry.
[0011]
[Means for Solving the Problems]
In order to solve the above technical problem, the present invention provides a method for associating measurement points in the following photogrammetry.
[0012]
The method for associating measurement points in photogrammetry includes the following: (1) a reference image and an inspection image respectively photographed by a reference camera whose positional relationship is known and an inspection camera arranged at a position different from the reference camera; A step of creating, (2) a step of designating a plurality of reference points appearing in both the reference image and the inspection image, and (3) from the coordinate value of the reference point on the reference image and the inspection image, Obtaining three-dimensional coordinates, determining a distance between the three-dimensional coordinates of at least one reference point and the optical axis center of the reference camera, and obtaining a reference distance from the optical axis center; (4) on the reference image; (5) determining an epipolar line for the measurement point projected on the inspection image, and (5) calculating the epipolar line and the determined reference distance Each of the steps of: dividing a polar line into line segments or half line segments to determine a segmented section; (6) performing image correlation processing on both the measurement points on the reference image and the neighboring areas of the segmented section. Comprising steps.
[0013]
In the above method, the positional relationship between the reference camera and the inspection camera, that is, the coordinate value indicating where the other camera (inspection camera) is located and the direction in which the other camera is facing as viewed from the coordinate system of the reference camera. The angle indicating whether or not is known. Therefore, if any point that is imprinted on both the reference image and the inspection image can be specified in association with each other, the three-dimensional coordinates of the point can be specified.
[0014]
Of the three-dimensional coordinates obtained from the reference point, the distance between at least one three-dimensional coordinate and the optical axis center of the reference camera is determined, and the distance from the optical axis center (reference distance) is obtained. On the other hand, a measurement point is designated on the reference image, and an epipolar line projected on the inspection image for the measurement point is determined. The epipolar line can be divided into a line segment or a half line segment based on the epipolar line and the determined reference distance, and the epipolar line segment can be determined. One or more reference points may be projected, and as will be described later, it is preferable to use a reference point closer to the measurement point and two reference points farther from the measurement point.
[0015]
Therefore, for example, if it is considered that the 3D coordinates of the measurement point are on the near side compared to the distance between the 3D coordinates obtained from the reference point and the center of the optical axis of the camera, the line segment before that distance Only the epipolar line obtained by projecting the image onto the inspection image may be subjected to correlation processing. Usually, the segment is recognized as a line segment or a half line segment by providing a divided section on the epipolar line extending linearly to infinity. be able to.
[0016]
Therefore, according to the present method, it is not necessary to perform processing on all regions on the epipolar line extending from end to end on the inspection image, and the region on which the correlation processing is performed can be narrowed down, thereby shortening the time required for processing. In addition, the accuracy of processing can be improved.
[0017]
Preferably, the divided section includes the optical axis of a near reference point that is closer to the measurement point and a far reference point that is farther than the measurement point. Each reference distance from the center is measured, and the epipolar line is divided into sections corresponding to the respective reference distances.
[0018]
In the above method, when the epipolar line is divided, the epipolar line is divided into these reference distances using the reference point that is proximal to the measurement point and the reference point that is far from the measurement point and the reference distance from the optical axis center of the reference camera. Is divided into line segments whose end points are the same distance. According to the above method, it is clear that there is a corresponding point of the measurement point on the line segment, and by performing correlation processing only on this area, the area is reduced, and the processing speed and accuracy are improved. Can be achieved.
[0019]
Preferably, the positional relationship between the reference camera and the inspection camera is derived based on coordinate values on both images of a plurality of reference points designated on the reference image and the inspection image.
[0020]
Further, the present invention provides a storage unit for storing a reference image and an inspection image taken from different positions whose mutual positional relationships are known, and a reference for receiving an input of a reference point reflected in both the reference image and the inspection image. From the point designating means, and the coordinate value of the reference point on the reference image and the inspection image, respectively, three-dimensional coordinates of the reference point are obtained, and at least one of the reference point three-dimensional coordinates and the optical axis center of the reference camera, A reference distance calculation means for determining a reference distance from the center of the optical axis, and an epipolar line for designating a measurement point on the reference image and determining an epipolar line for the measurement point projected on the inspection image A calculation section, a division section calculation section for dividing the epipolar line into a line segment or a half line segment based on the epipolar line and the determined reference distance, and determining a division section; and on the reference image In both neighboring region of the divided sections and the measurement point, the correlation processing means for performing image correlation processing, characterized in that it comprises a city, to provide a mapping device of the measuring points in the photogrammetry.
[0021]
Furthermore, the present invention provides a computer for mapping measurement points in photogrammetry with a plurality of reference points appearing in both a reference image and an inspection image taken from different positions whose mutual positional relationships are known. Reference point designating means for receiving an input, respectively, obtains the three-dimensional coordinates of the reference point from the reference image and the coordinate value of the reference point on the inspection image, and at least one of the three-dimensional coordinates of the reference point and the light of the reference camera Reference distance calculation means for determining a distance from the axis center, obtaining a reference distance from the optical axis center, specifying a measurement point on the reference image, and determining an epipolar line for the measurement point projected on the inspection image An epipolar line calculating means, and a divided section calculating unit for determining a divided section by dividing the epipolar line into a line segment or a half line segment based on the epipolar line and the determined reference distance; , A program for associating measurement points in photogrammetry to function as correlation processing means for performing image correlation processing in both the measurement point on the reference image and the vicinity of the divided section, and a storage medium storing the program I will provide a.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a photogrammetry system according to an embodiment of the present invention will be described with reference to the drawings.
[0023]
This apparatus is composed of a computer for inputting a photographic image and performing an analysis operation. The photograph image is preferably taken by a digital camera, but it may be a photograph obtained by digitizing a photograph taken by a film camera with a scanner or the like. The camera may be for photo measurement with internal orientation or a general-purpose digital camera with no internal orientation. A general-purpose computer can be used, and is driven by installing a program that causes the computer to perform processing as described later using a storage medium that stores the program.
[0024]
FIG. 1 shows a block diagram of a photogrammetry apparatus according to an embodiment of the present invention. The photogrammetry apparatus 1 is analyzed by an interface unit 2 for connection with a camera 7 and an input device 8 such as a keyboard and a mouse, a control calculation unit 3 for controlling the entire apparatus, and a display unit 4 for displaying photographic images. A storage unit 5 that functions as a storage area and a calculation area for photographs and data, and a system program storage unit 6 that functions as a system program by installing the software.
[0025]
As will be described later, the control calculation unit 3 is provided with blocks 3a to 3e that are driven by a system program and execute photogrammetry. Each of these blocks functions as a functional block that performs the following processing alone or in cooperation with other blocks in the apparatus.
[0026]
Next, the flow of processing when performing photogrammetry using the photogrammetry apparatus of FIG. 1 will be described. FIG. 2 is a flowchart showing the flow of processing when performing photogrammetry using the photogrammetry apparatus of FIG. When photogrammetry is performed using this apparatus, the processing of each step shown below is mainly performed by each block of the control calculation unit of the apparatus. First, a stereo image is taken (# 10), the photographic image is transmitted to the apparatus, stored in the image storage unit 5, and the image is displayed on the display unit 4 during operation. The stereo image is two or more photographic images obtained by photographing the same subject from different positions using the camera 7. Hereinafter, in this embodiment, a case where two photographic images are used will be described. A photographic image is distinguished as one reference image and the other as an inspection image. Which of the two photographic images is used as a reference image is arbitrary, and any one can be selected.
[0027]
FIG. 3 is a diagram showing a coordinate system for performing photogrammetry. The two photographic images 110 and 210 photographed by the two cameras A and B have the respective optical axis centers 100a and 200a as the origin, the optical axis direction of each camera as the Z axis, and the longitudinal direction of the photographic image is The X axis and the short direction are the Y axis. Therefore, the X axis is not always parallel to the ground surface E according to the tilt of the camera. In the present embodiment, the three-dimensional coordinates are specified based on the camera that captured the reference image. That is, when the photographic image 110 taken by the camera A is used as the reference image, the X1, Y1, and Z1 axes are adopted as the reference coordinate system. Hereinafter, a description will be given of a case where the camera A is a reference camera and a photographic image taken by the camera A is a reference image.
[0028]
FIG. 4 shows an example of a photographic image used for photogrammetry. 4A shows a photographic image (reference image) taken by the camera A, and FIG. 4B shows a photographic image (inspection image) taken by the camera B. FIG. 4 shows an image in which the designation of reference points 11a to 13a, 11b to 13b and measurement points 16a, which have already been described later, is completed for the convenience of understanding. Needless to say, the designated reference points 11a to 13a, 11b to 13b and the measurement point 16a are not displayed in the image transmitted from the camera 7 and not yet processed.
[0029]
Next, reference points are designated for these reference image 110 and inspection image 210 (# 11). For the reference points 11a to 13a and 11b to 13b, a position such as a corner that is easy to discriminate is selected on the appearance of the subject, and the same point is designated in both images while referring to the two photographic images 110 and 210. This input is performed by the user operating the input device 8 such as a mouse, and the coordinate point designating unit 3a performs the main processing. At this time, it is preferable to include a point closer to the measurement point and a point farther from the measurement point with the reference image 110 as a reference. In the present embodiment, the nearest proximal reference point 13 and the farthest far reference point 11 are designated among the plurality of reference points. FIG. 5 is a distribution diagram in which only the reference points attached to the reference image are extracted.
[0030]
Next, five or more reference points are selected in both images, and the positional relationship between the camera A100 and the camera B200 is calculated using the coordinate values of each reference point on the photographic image (# 12). In this embodiment, as a positional relationship, a coordinate value indicating where the optical axis center of the camera B200 (inspection camera) is located in the reference coordinate system having the optical axis center of the camera A100 (reference camera) as the origin, and the camera B200. An angle indicating which direction the optical axis of the light is directed is derived.
[0031]
When the positional relationship between the camera A100 and the camera B200 is calculated, the three-dimensional coordinate value of the reference point in each reference coordinate system is calculated using this value (# 13). These processes are mainly controlled by the position calculation unit 3b.
[0032]
Next, the measurement point 16a is input on the reference image by operating an input device such as a mouse (# 14). As described above, the measurement point 16a is set such that the distance from the camera A100 is not the nearest or farthest from the plurality of reference points 11a to 13a and 11b to 13b input in the previous step (# 11). It is preferable. These processes are mainly controlled by the coordinate point designating unit 3a.
Next, the depth of the reference point as seen from the camera A100 is calculated from the three-dimensional coordinates obtained in the previous steps (# 12, # 13). The following processing is mainly governed by the epipolar line calculation unit 3c. As shown in FIG. 5, first, the farthest reference point 11a and the nearest reference point 13a are specified across the measurement point 16a (# 15). As shown in FIG. 6, when viewed from the camera A100 in the direction of the optical axis L (Z1 axis direction), the subject has a plurality of reference points 11, 12, and 13 having different depth positions. Among the reference points, the nearest proximal reference point 13 and the farthest far reference point 11 are designated on the reference image 110. For these two designated points, the distance (reference distance) from the optical axis center 100a of the camera A100 to these two points is calculated (# 16), and a line 20 having the same distance as the reference distance for the two points, 30 is assumed.
[0034]
Next, as shown in FIG. 7, a line (light ray with respect to the measurement point 16) L2 passing through the measurement point 16a on the reference image 110 and the optical axis center 100a of the camera A100 is assumed. Since the corresponding point on the inspection image 210 of the measurement point 16 always exists on the epipolar line 40 projected on the inspection image by the epipolar constraint condition described in Non-Patent Document 1, the epipolar line By performing the correlation process along 40, the association can be performed at high speed and with high accuracy. However, the light beam L2 passing through the center of the optical axis and the measurement point 16 is a straight line extending to infinity, and the epipolar line 40 projected on the inspection image 210 of the inspection camera B200 as shown in FIG. Extending to both ends in the X-axis direction. Therefore, it is still extensive to specify the corresponding point that is one point on the line. Therefore, in this embodiment, the search area for performing the correlation process is narrowed down as follows.
[0035]
That is, as shown in FIG. 9, the intersections 11x and 13x between the equidistant lines 20 and 30 assumed in the above step and the light ray L2 are obtained (# 16). The equidistant lines 20 and 30 have the same distance as the proximal reference point 13 that is the most proximal among the reference points 11a to 13a that are imprinted in the reference image 110 and the far reference point 11 that is the farthest (depth side). The actual measurement points are always present in the region L3 delimited by these equidistant lines 20 and 30.
[0036]
It is clear that the divided division L3 is shorter than the epipolar line 40 projected on the inspection image 210 when projected onto the inspection image 210 of the inspection camera B200 as indicated by arrows 21 and 31. It is. That is, as shown in FIG. 10, on the inspection image 210, the epipolar line 40 extends to both ends, and a part 42 thereof becomes a region corresponding to the divided section L <b> 3. Therefore, it is possible to determine the corresponding point of the measurement point 16a on the inspection image by performing the correlation process as described later by limiting to the divided area 42 (# 17). This process is mainly controlled by the divided section determination unit 3d.
[0037]
Next, correlation processing is performed using the divided section as a search area (# 18). This process is mainly controlled by the correlation processing unit 3e. Specifically, pattern matching is performed as follows. First, as shown in FIG. 11A, a region 18 near the measurement point 16 of the reference image 110 is extracted. Then, as shown in FIG. 11 (b), pattern matching is performed on the neighborhood region 18 with the section divided in the previous step in the inspection image 210. In this way, the associated region 28 is searched, the corresponding point 16b is determined (# 19), and the coordinate value on the inspection image is obtained.
[0038]
In the present embodiment, in step 15, by specifying the most proximal reference point 13 a and the farthest reference point 11 a on the reference image, and uniformly setting the section 42 divided in step 17 as a search region. The section 42 can also be used when measuring three-dimensional coordinates of a plurality of measurement points.
[0039]
Finally, as shown in FIG. 12, the measurement point 16 includes a light beam L2 connecting the optical axis center 100a of the camera A100 (reference camera) and the measurement point 16, and an optical axis center 200a of the camera B200 (inspection camera). It is represented as an intersection with a straight line M2 connecting the corresponding point 16b on the image 210, and its three-dimensional coordinates are calculated by the principle of triangulation.
[0040]
As described above, according to the photogrammetry apparatus according to the present embodiment, the search area narrowed down using the epipolar constraint condition can be obtained by providing information on the reference points proximal and far from the measurement point. It can narrow down greatly.
[0041]
In addition, this invention is not limited to the said embodiment, It can implement in another various aspect.
[0042]
The designation of the reference point for determining the end point of the divided section is not limited to the most proximal point and the farthest point on the reference image, and may be individually selected for each measurement point. That is, by selecting the most proximal point and the farthest point on the reference image, all measurement point distances exist between both reference points. While the correlation process can be performed in the area, the search area in the correlation process is widened. On the other hand, when individually selecting a reference point for each measurement point, if a reference point that is close to the measurement point is selected, the search area in the correlation process can be made smaller, and the processing speed and accuracy can be increased. Can be achieved.
[Brief description of the drawings]
FIG. 1 is a block diagram of a photogrammetry apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart showing the flow of processing when photogrammetry is performed using the photogrammetry apparatus of FIG. 1;
FIG. 3 is a diagram showing a coordinate system when photogrammetry is performed using the apparatus of FIG. 1;
4A and 4B are diagrams illustrating examples of photographic images used for photogrammetry, where FIG. 4A is a photographic image (reference image) of camera A100, and FIG. 4B is an example of a photographic image (inspection image) of camera B200. .
FIG. 5 is a diagram in which only reference points and measurement points designated on a reference image are extracted.
FIG. 6 is a diagram showing a positional relationship between a proximal point and a far point.
FIG. 7 is a diagram showing light rays at a measurement point.
FIG. 8 is a diagram showing an epipolar line for a measurement point on an inspection image.
FIG. 9 is an explanatory diagram when a segmented light beam is projected onto an inspection image.
FIG. 10 is a diagram showing a segmented region on an epipolar line projected on an inspection image.
FIG. 11 is an explanatory diagram of a correlation process.
FIG. 12 is a principle diagram of processing for calculating three-dimensional coordinates of measurement points.
FIG. 13 is an explanatory diagram of epipolar constraint conditions.
[Explanation of symbols]
L Optical axis L2 of the camera A Light beam L3 Split section M Optical axis of the camera B 1 Photogrammetry device 2 Interface unit 3 Control calculation unit 4 Display unit 5 Image storage unit 6 System program 7 Digital camera 8 Input device 11 Distant reference point 11a Distant Projection point 11b of reference point on reference image Projection point 11x on inspection image of remote reference point Equal distance point 12 on epipolar line of remote reference point Reference point 12a Projection point 12b of reference point on reference image Inspection image of reference point Projection point 13 to the proximal reference point 13a Projection point 13b to the reference image of the proximal reference point Projection point 13x to the examination image of the proximal reference point Equidistant point 14 on the epipolar line of the proximal reference point Measurement object 16 Measurement Point 16a Projection point 16b of measurement point on reference image Corresponding point (projection point of measurement point on inspection image)
20, 30 equidistant line 40 epipolar line 42 segmented area (search area)
100 Camera A (reference camera)
110 Reference image 200 Camera B (inspection camera)
210 Inspection image

Claims (8)

Creating a reference image and an inspection image respectively photographed by a reference camera having a known positional relationship with each other and an inspection camera arranged at a position different from the reference camera;
Designating a plurality of reference points appearing in both the reference image and the inspection image;
From the reference image and the coordinate value of the reference point on the inspection image, a three-dimensional coordinate of the reference point is obtained, and a distance between at least one of the reference point three-dimensional coordinates and the optical axis center of the reference camera is determined. Obtaining a reference distance from the center of the optical axis;
Designating a measurement point on the reference image and determining an epipolar line for the measurement point projected on the inspection image;
Dividing the epipolar line into a line segment or a half line segment from the epipolar line and the determined reference distance, and determining a division section;
Performing image correlation processing on both the measurement points on the reference image and the vicinity of the divided section; and
A method of associating measurement points in photogrammetry characterized by comprising:   The divided section has a distance from the optical axis center of the reference camera from the optical axis center of a near reference point that is closer to the measurement point and a far reference point that is farther than the measurement point. 2. The method of associating measurement points in photogrammetry according to claim 1, wherein the reference distances are measured, and the epipolar lines are divided to correspond to the respective reference distances.   The positional relationship between the reference camera and the inspection camera is derived based on coordinate values on both images of each of a plurality of reference points designated on the reference image and the inspection image. 3. A method for associating measurement points in the photogrammetry described in 2. A storage unit for storing a reference image and an inspection image taken from different positions whose mutual positional relationships are known;
A reference point specifying means for receiving an input of a reference point reflected in both the reference image and the inspection image;
From the reference image and the coordinate value of the reference point on the inspection image, a three-dimensional coordinate of the reference point is obtained, and a distance between at least one of the reference point three-dimensional coordinates and the optical axis center of the reference camera is determined. A reference distance calculation means for obtaining a reference distance from the center of the optical axis;
Epipolar line calculation means for specifying a measurement point on the reference image and determining an epipolar line for the measurement point projected on the inspection image;
Dividing the epipolar line into a line segment or a half line segment from the epipolar line and the determined reference distance, and a divided section calculation means for determining a divided section;
An apparatus for associating measurement points in photogrammetry, comprising correlation processing means for performing image correlation processing in both the measurement points on the reference image and the regions in the vicinity of the divided sections. The divided section calculation means is configured such that the distance from the photographing position of the reference image is from the optical axis center of a near reference point that is proximal to the measurement point and a far reference point that is far from the measurement point. 5. The apparatus for associating measurement points in photogrammetry according to claim 4, wherein a distance obtained by dividing each epipolar line and dividing the epipolar line so as to correspond to each reference distance is defined as a divided section. Before SL positional relationship, photogrammetry according to claim 4 or 5, characterized in that guided on the basis of the coordinate values on each of the two images of a plurality of reference points specified on the reference image and the inspection image Device for associating measurement points . Reference point designating means for accepting the input of a plurality of reference points appearing in both the reference image and the inspection image taken from different positions with known positional relations in order to associate measurement points in photogrammetry ,
From the reference image and the coordinate value of the reference point on the inspection image, a three-dimensional coordinate of the reference point is obtained, and a distance between at least one of the reference point three-dimensional coordinates and the optical axis center of the reference camera is determined. , A reference distance calculation means for obtaining a reference distance from the center of the optical axis,
Divided section calculation means for determining the divided section by dividing the epipolar line into a line segment or a half line segment from the epipolar line and the determined reference distance;
A program for associating measurement points in photogrammetry for functioning as correlation processing means for performing image correlation processing in both the measurement points on the reference image and the regions near the divided sections.   A storage medium storing the program according to claim 7.

Images