JP3735248B2 - Spatial data processing method, spatial data processing apparatus, and recording medium recording program describing spatial data processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention provides spatial data that can correctly associate acquired spatial data and real space when acquiring, for example, the shape or structure of a building in real space from time-series image data acquired by an image input device or the like. The present invention relates to a processing method, a spatial data processing apparatus, and a recording medium on which a program describing the spatial data processing method is recorded.
[0002]
[Prior art]
Conventionally, when a camera with a fixed angle of view is used, in the case of aerial images , if sufficient altitude is taken into consideration, the target object can be photographed as one image at a time. In the angle, it is difficult to capture the entire overview, for example, a desired object to be photographed may be framed in and out of the frame.
[0003]
In addition, an omnidirectional or all-around camera has been developed, and the entire landscape can be acquired at once, and is often used for an application of acquiring a virtual space as a 2.5-dimensional space.
[0004]
In addition, conventionally, spatial data is projected onto an image, internal parameters and external parameters are adjusted by an optimization algorithm, and the position coordinates of the camera viewpoint, camera posture information, etc. are calibrated and corrected from the spatial data, Reconciling spatial data with real space is carried out.
[0005]
[Problems to be solved by the invention]
In the present invention, the difference between the spatial shape in the actual landscape with various spatial data, detected by the indication that the phase difference from the omnidirectional image, so that the omnidirectional image and spatial data are aligned, the space The main purpose is to correct and calibrate the three-dimensional coordinate values constituting the data to improve the accuracy.
[0006]
[Means for Solving the Problems]
The present invention uses an omnidirectional camera characterized by projecting the entire landscape, and provides positional information (external parameters) of the camera itself, internal parameters specific to the omnidirectional camera, and optical projectivity of the omnidirectional camera. In consideration of the above, the phase difference between the phase when various spatial data are projected on the omnidirectional image and the corresponding target point in the image is detected, and the phase difference is reduced according to the time-series image. Thus, the spatial data is calibrated and corrected, and more accurate spatial data is constructed.
[0007]
By using the method of the present invention, an omnidirectional camera is mounted and moving while shooting in the city space and landscape, the stored spatial data is combined with the real space to make it more realistic. It is possible to acquire, update, and construct near spatial data.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Example 1
An embodiment relating to the first claim will be described. In the following, for the sake of explanation, it is assumed that the structure or shape in the space is a rectangular parallelepiped, and the omnidirectional camera is a camera equipped with a fisheye lens, and an omnidirectional image is acquired while the camera is mounted on a vehicle and moved. To do. Note that the present invention is valid even if the shape of the object in the space is an arbitrary shape.
[0009]
FIG. 1 is an apparatus configuration diagram of the present invention. The apparatus includes a fisheye lens unit 1, an optical system 2 including an optical adjustment unit, an image generation unit 3 obtained by imaging from the optical system 2, a position information acquisition unit 6 that senses position information, and stored spatial data in all directions. A trajectory prediction unit 11 that projects on an image or predicts a trajectory of a projection point at each position, and further detects a phase difference between a target point of a target object in an omnidirectional image and a spatial data. Unit 7, a data correction processing unit 8 that calibrates and corrects the spatial data by reducing these phase differences, and a correction data update processing unit 9 that holds the corrected spatial data as updated spatial data. . In the figure, 4 represents a synchronization signal generator, 5 represents a target point search unit, 10 represents a spatial data holding unit, and 12 represents a projection processing unit.
The above-mentioned 10, 11, 5, 12 function as the phase angle φ _{p (i)} searching means in the present invention, and the above 4, 3, 5, 6 as the phase angle φ _{q (i)} searching means. The above 7, 8, and 9 serve as position coordinate acquisition means.
[0010]
In preparation for explaining the present invention, the coordinate system shown in FIG. 2 is set.
FIG. 2 is a diagram showing a coordinate system set for explanation.
[0011]
In FIG. 2, in the three-dimensional coordinates, the spatial data held for the target object (in real space) is acquired in a state that is slightly displaced due to a measurement error at the time of acquisition. In the present invention, both of these are slid together to correct this misalignment.
[0012]
In FIG. 2, it is assumed that the omnidirectional camera is present at the position A (X _a , Y _a , Z _a ) and is moving in the direction of the traveling axis shown in the figure. The direction of the traveling axis is an angle φ _F with respect to the X axis. The illustrated coordinate system (i, j) is defined for the field of view of the omnidirectional camera. Further, assuming that the omnidirectional camera has a fisheye lens having a hemispherical surface, (i) P that represents spatial data _{(X p, Y p, Z} p) of the line of sight and fisheye lens when observed hemispherical The position where the intersection with the surface is projected onto the coordinate system (i, j) plane is defined as (I _ap , J _ap ), and (ii) a point Q (X _q , Y _q , Z _q ) representing the target object is observed. (I _aq , J _aq ) is the position where the intersection of the line of sight and the hemispherical surface of the fisheye lens is projected on the coordinate system (i, j) plane, and (iii) the direction of the line of sight when the point P is observed is shown in the figure It is assumed that the angle φ _AP with respect to the traveling axis and (iv) the direction of the line of sight when the point Q is observed is the angle φ _AQ with respect to the illustrated traveling axis.
[0013]
In the projection onto the coordinate system (i, j) plane, the projection formula (i ² + j ² ) ^1/2 = f · θ ────── (1)
(Where f is the focal length; θ is the angle between the line of sight and the optical axis)
Is used.
[0014]
That is, the position information of the omnidirectional camera at the position A (the three-dimensional coordinate value of the camera viewpoint of the omnidirectional image or the point where the point in the space is condensed) is A (X _a , Y _a , Z _a ). At this time, the data having the point P is the spatial data held, and the point in the data is P (X _p , Y _p , Z _p ). Further, it is assumed that the spatial coordinates in the actual cityscape have a point Q, and that point is Q ( _Xq , _Yq , _Zq ). Also, the two-dimensional coordinate values (projection points) on which the P and Q points are projected on the omnidirectional image are (I _ap , J _ap ) and (I _aq , J _aq ), respectively, and the progression axis and the projection point The angles formed by a straight line drawn from the center of the image to the center of the image are φ _AP and φ _AQ , respectively. Further, the angle between the traveling axis and the world coordinate system (X axis in FIG. 2) is φ _F.
[0015]
In the present invention, the XY coordinate value mismatch between the P point and the Q point is automatically detected from the phase difference on the omnidirectional image, and the XY coordinate value of the P point is corrected so that the P point and the Q point match. The processing flow for achieving the Q point will be described (the height information of the spatial data is correct with no positional deviation, that is, Z _p = Z _q ).
[0016]
First, when position information of an omnidirectional camera is given or acquired, a certain range (for example, three-dimensional coordinates A (X _a , Y _a at position A) is selected from the spatial data that is owned or accumulated. , Z _a ) is loaded into the entire spatial data existing in the radius R). Among these, in order to narrow down the target, the semi-circular region as viewed from the advancing axis in the omnidirectional image is targeted. In FIG. 2, the left semicircle region is the target.
[0017]
The spatial data P point in the narrowed range is projected onto an omnidirectional image by camera parameters such as internal parameters and external parameters of the omnidirectional camera, and two-dimensional coordinate values (I _ap , J _ap ) as well as the traveling axis And the angle φ _{AP formed} by a straight line drawn from the two-dimensional coordinate values (I _ap , J _ap ) to the image center. (I _ap , J _ap ) is calculated by the following equation.
I _ap = [f {π / 2−tan ⁻¹ ((Z _p −Z _a ) / ((X _p −X _a ) ²
+ (Y _p −Y _a ) ² ) ^1/2 )} cos φ _AP ] / M + I (2)
J _ap = [f {π / 2−tan ⁻¹ ((Z _p −Z _a ) / ((X _p −X _a ) ²
+ (Y _p −Y _a ) ² ) ^1/2 )} sin φ _AP ] / M + J (3)
φ _AP = tan ⁻¹ ((Y _p −Y _a ) / (X _p −X _a )) − φ _F (4)
However, M [m]: physical length in the space corresponding to one pixel
(I, J): Coordinate value of the image center
f [m]: Focal length Further, when position information at an arbitrary time ti is given or acquired, the projection onto the omnidirectional image at that time is unique according to the following equation (5). It can be predicted as a curve. In the case of polar coordinate display (L _x , φ _x ) in an omnidirectional image, the length L _x to the image center and the phase φ _{x formed} by the straight line from the image center to the projection point and the traveling axis are predicted. .
L _x (φ _x ) = f tan ⁻¹ (tan {π / 2−tan ⁻¹ ((Z _p −Z _a ) / ((X _p −X _a ) ²
+ (Y _p −Y _a ) ² ) ^1/2 )} sin φ _AP ) (5)
On the other hand, the actual target object that is the target of the spatial data is projected as a Q point. This omnidirectional coordinate value is obtained by using the fact that the projection point becomes (I _ap , J _ap ) when φ _x = φ _AP with respect to the area in the vicinity of the point P, and when the position coordinates are given. The trajectory of the target point passing through the point (I _ap , J _ap ) is calculated according to the equations (6), (7), (8) (this trajectory group is referred to as an Rh curve).
[0018]
L = ((I−I _ap ) ² + (J−J _ap ) ² ) ^1/2 (6)
φ _A = tan ⁻¹ { ((I−I _ap ) ² / (J−J _ap ) ² ) ^1/2 } (7)
L _x (φ _x ) = f × tan ⁻¹ ((tan (L / f) sin φ _A ) / sin φ _x )
0 <φ _x <π (8)
When the position coordinates are given, the search range is narrowed down based on the omnidirectional trajectory prediction from the Rh curve (L _x (φ _x ) in the equation (8)) using the variable of (L _x , φ _x ), Image processing such as edge detection is performed on a straight line extending from the center of the image, and two-dimensional coordinate values (I _aq , J _aq ) are detected accurately at high speed. At the same time, an angle (phase) φ _{AQ formed} by the traveling direction and the straight line from the image center to the two-dimensional coordinate value (I _aq , J _aq ) is acquired.
[0019]
In this phase acquisition process, if the position of the spatial data and the target object are different (in FIG. 4, the point P and the point Q are different), the phase difference φ _AP −φ _AQ in the time series.
Can be detected. This phase difference is not uniform and changes depending on the position information of the omnidirectional camera.
[0020]
The phase difference φ _{pq (i)} according to each position information is detected according to the following equation.
Φ _{q (i)} = π / 2−tan ⁻¹ {(I−I _{q (i)} ) / (J−J _{q (i)} )} (9)
Φ _{p (i)} = tan ⁻¹ ((Y _p −Y _{p (i)} ) / (X _p −X _{p (i)} )) −φ _Fi (10)
φ _Fi = tan ^-1 ((Yp _(i) -Yp _{(i- 1)} )) / ( _{Xp (i)} -Xp _{( i- 1)} )) (11)
Φ _{pq (i)} = | Φ _{p (i)} − Φ _{q (i)} | (12)
The omnidirectional coordinate value of Q point, the phase, and the phase difference φ _{pq (i)} are sequentially acquired as the position information of the camera changes.
[0021]
FIG. 4 is a diagram showing the phase difference between the spatial data and the target object at each position of the camera. As the omnidirectional camera moves from position A to position T along position along the axis of travel, point P and point Q are (I _{p (1)} , J _{p (1)} ), (I _{q (1)} , J _{q (1)} )
( _{Ip (i)} , _{Jp (i)} ), ( _{Iq (i)} , _{Jq (i)} )
(I _{p (j)} , J _{p (j)} ), (I _{q (j)} , J _{q (j)} )
It is observed as follows.
[0022]
In the correction of the spatial data, the whole in a data correction processing unit 8 shown in FIG. 1, by changing the XY coordinates of the point P, the A, S, the total phase difference phi _pq in time series in T a _(i) Find the difference (dX, dY) that minimizes the target. That is, a difference (dX, dY) is obtained such that (X _p + dX, Y _p + dY, Z _p ) ≈ (X _q , Y _q , Z _q ) for point _P.
[0023]
FIG. 3 is a processing flow of the embodiment.
(S1): Processing is started.
(S2): Time t is first set corresponding to the movement of the camera, and dt is added thereafter.
(S3): Camera position information is acquired.
(S4): A spatial data database is accessed.
(S5): Extract spatial data existing within a predetermined (set) radius R from the position of the camera.
(S6): An omnidirectional projection from the camera position is obtained.
(S7): It is checked whether or not the projection is within the semicircle range to be processed.
If yes, go to step (S8), but if no, go to the process to get the next omnidirectional projection.
(S8): The coordinates (I _ap , J _ap ) of point P are measured.
(S9): φ _p (t) corresponding to point P is measured.
(S10): causing the phase phi _x so as to then search for the point Q is slightly changed.
(S11): The trajectory of the point P (Rh curve) is predicted to search for the point Q.
(S12): The target Q point is searched in consideration of the trajectory.
(S13): It is checked whether or not the target point has been detected. In the case of NO returns to step (S10), by changing the phase phi _x only d.phi, towards the step (S12).
(S14): The phase φ _Q (t) of the Q point corresponding to the target object in the real space is measured.
[0024]
In this state, φ _Q (t) −φ _P (t) obtained under a certain camera position (for example, position A in FIG. 4).
Is obtained. Hereinafter, the time in step (S2) is set to (t + dt) so as to correspond to the movement of the camera, and in step (S3), the camera position is newly (moved—for example, moved from position A to position S in FIG. 4). The steps (S6), (S7), (S8), (S9), (S10), (S11), (S12), (S13), and (S14) are performed in the same manner as described above, and φ _Q obtained under the new camera position is obtained. (T + dt) −φ _P (t + dt)
Get. Of course, in the same manner, φ _Q (t + 2dt) −φ _P (t + 2dt) at the camera position T in FIG.
Get.
[0025]
In this way, when the position P corresponding to the spatial data is at P (X _p , Y _p , Z _p ), when the camera advances from position A to position T as shown in FIG. Phase difference φ _Q (t) −φ _P (t)
φ _Q (t + dt) − φ _P (t + dt)
φ _Q (t + 2dt) − φ _P (t + 2dt)
...
By obtaining, for example, the root mean square of these, “the mean square phase difference under the condition that the position P exists P (X _p , Y _p , Z _p )” is obtained. Next, the position P is moved to P (X _p + dx, Y _p + dy, Z _p ) (note that Z _p = Z _{q as} described above), and a similar mean square phase difference is obtained. go. That is,
(S15): moving the position _{P P (X p + dx,} Y p + dy, Z p) in. Then, the above-mentioned “square average phase difference” is obtained.
(S16): The obtained mean square phase difference is compared with the previously obtained “mean square phase difference with minimum value”.
(S17): When the position P is moved as described above, the “root mean square phase difference” having the smallest value is obtained.
(S18): A correction amount (dx, dy) for the position P when the “smallest mean square phase difference” is obtained is acquired. Then, the coordinates of the position P are changed on the spatial database.
(S19): An acquisition result that includes the correction obtained as described above is output.
[0026]
In the above description, the method and apparatus for correcting the deviation between the landscape in the real space and the spatial data acquired from the real space have been described. The method is recorded in the form of a program. It can be recorded on a medium. Therefore, the present invention also covers the recording medium.
[0027]
【The invention's effect】
As described above, according to the present invention, even when the spatial data is different from the actual position coordinates of the target object, automatic correction can be performed and the spatial data can be updated. Moreover, it can apply with respect to several coordinate values other than P point and Q point, and can correct the coordinate value of spatial data.
[0028]
Thus, by performing data correction, data can be updated even when the actual target object (or its shape) changes, and if there is an error in the spatial data, the data can be corrected. it can.
[Brief description of the drawings]
FIG. 1 is an apparatus configuration diagram of an embodiment.
FIG. 2 is a diagram illustrating a coordinate system in which an example is set for explanation.
FIG. 3 is a processing flow of the embodiment.
FIG. 4 is a diagram illustrating a phase difference between spatial data and a target object at each position of the camera.
[Explanation of symbols]
1: Fisheye lens (omnidirectional camera)
2: Optical adjustment unit (optical system)
3: Image generation unit 4: Synchronization signal generation unit 5: Target point search unit 6: Position information acquisition unit 7: Phase difference detection unit 8: Data correction processing unit 9: Correction data update processing unit 10: Spatial data holding unit 11: Trajectory prediction unit 12: projection processing unit

Claims (3)

The position coordinates of the target point in the real space from the time-series image acquired by photographing the real space while moving , using an image input device that is mounted on a moving vehicle and captures all directions at once to obtain an omnidirectional image In the spatial data processing device that acquires
Using the unique projection formula of the image input device, the locus of the projected coordinates on the photographed image of the position coordinate P of the target point held in the spatial data is obtained, and the search range is narrowed down by this locus, and photographing at time i is performed. and phase angle φ _{q (i)} searching means for obtaining the polar phase angle φ _{q (i)} of the target point on the image,
Phase angle determined by the phase angle of the polar coordinate phi _{p (i)} of the target point on the photographed image at time i with respect to the position coordinate P, using the position coordinates of the image input apparatus of the projection type and time i φ _{p (i)} search means;
By moving the position coordinate P, for φ _{p (i)} and φ _{q (i)} obtained at a plurality of times, the position coordinate P at which the difference between φ _{p (i)} and φ _{q (i)} is minimized as a whole. A spatial data processing device characterized by comprising: a position coordinate acquisition means for obtaining. From a time-series image acquired by shooting a real space while moving , using an image input device that is mounted on a moving vehicle and captures all directions at once to obtain an omnidirectional image,
Using the unique projection formula of the image input device, the locus of the projected coordinates on the photographed image of the position coordinate P of the target point held in the spatial data is obtained, and the search range is narrowed down by this locus, and photographing at time i is performed. and phase angle φ _{q (i)} searching means for obtaining the polar phase angle φ _{q (i)} of the target point on the image,
Phase angle determined by the phase angle of the polar coordinate phi _{p (i)} of the target point on the photographed image at time i with respect to the position coordinate P, using the position coordinates of the image input apparatus of the projection type and time i φ _{p (i)} search means;
By moving the position coordinate P, for φ _{p (i)} and φ _{q (i)} obtained at a plurality of times, the position coordinate P at which the difference between φ _{p (i)} and φ _{q (i)} is minimized as a whole. And a data processing device for acquiring the position coordinates of the target point in the real space.
The phase angle φ _{q (i)} searching means obtains the locus of the projected coordinates on the captured image of the position coordinates P of the target point held in the spatial data using the unique projection formula of the image input device, The search range is narrowed down by this trajectory, and the polar angle phase φ _{q (i)} of the target point on the captured image at time i is obtained ,
The phase angle phi _{p (i)} searching means, the phase angle of the polar coordinate of the target point phi _{p (i)} on the captured image at the time i to the position coordinate P, an image input of the projection type and time i Using the position coordinates of the device,
Entire position coordinate acquisition means, try to move the position coordinate P, the determined at a plurality of time φ _{p (i)} and φ _{q (i),} the difference between φ _{p (i)} and φ _{q (i)} is A spatial data processing method characterized in that the minimum position coordinate P is obtained. From a time-series image acquired by shooting a real space while moving , using an image input device that is mounted on a moving vehicle and captures all directions at once to obtain an omnidirectional image,
Using the unique projection formula of the image input device, the locus of the projected coordinates on the photographed image of the position coordinate P of the target point held in the spatial data is obtained, and the search range is narrowed down by this locus, and photographing at time i is performed. and phase angle φ _{q (i)} searching means for obtaining the polar phase angle φ _{q (i)} of the target point on the image,
Phase angle determined by the phase angle of the polar coordinate phi _{p (i)} of the target point on the photographed image at time i with respect to the position coordinate P, using the position coordinates of the image input apparatus of the projection type and time i φ _{p (i)} search means;
By moving the position coordinate P, for φ _{p (i)} and φ _{q (i)} obtained at a plurality of times, the position coordinate P at which the difference between φ _{p (i)} and φ _{q (i)} is minimized as a whole. A spatial data processing method for acquiring spatial data using a data processing device for acquiring the positional coordinates of a target point in real space, and recording a program describing the processing method In the medium,
In this processing method, in the phase angle φ _{q (i)} search means, the projection coordinate of the position coordinate P of the target point held in the spatial data on the photographed image is obtained by using a unique projection formula of the image input device. The trajectory is obtained, the search range is narrowed down by this trajectory, and the polar angle phase φ _{q (i)} of the target point on the captured image at time i is obtained ,
In the phase angle phi _{p (i)} searching means, the position coordinates P polar phase angle of the point of interest on the photographed image at time i phi relative to _{p (i),} the image input projection type and time i Using the position coordinates of the device ,
In said position coordinate acquisition means, try to move the position coordinate P, the determined at a plurality of time φ _{p (i)} and φ _{q (i),} the difference between φ _{p (i)} and φ _{q (i)} the entire A recording medium on which is recorded a program describing a spatial data processing method characterized in that a minimum position coordinate P is obtained.

Images