JPH0614355B2 - Information extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Outline] In order to obtain the moving direction of a camera from a captured image of the camera, spherical mapping is performed on a plurality of points of interest photographed in time series to obtain a plurality of points. The intersections are obtained, and the obtained plurality of intersections are spherically mapped to obtain the intersections, and the moving direction information of the camera is extracted based on the intersection information.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information extraction method for extracting moving direction information of a camera required when performing three-dimensional measurement using motion stereoscopic vision.

In recent years, object recognition technology has been actively developed in order to realize a function similar to that of the human eye by a machine. In the recognition of such an object, a method of recognizing the object by capturing the object as an image and extracting information such as the feature of the object from the image is used. Therefore, it is generally necessary to capture an original image by capturing an image of an object with an image capturing unit such as a television camera and perform electrical processing based on this image to extract information.

On the other hand, such an original image often contains noise or has bleeding or blurring, and it is necessary to remove these noise components and extract the structure lines or the like constituting the original image.

As a method of extracting such an image, an information extraction method by mapping has been conventionally known. A method using a spherical map as an information extraction method using this map (JP-A-60-218183).
Issue gazette).

In the case of using the spherical mapping, the original image is projected on a spherical surface (spherical projection), and as shown in FIG. 5 (a), the point is centered for each point P of the original image projected on the spherical surface. Is drawn on the image plane as shown in FIG. 5 (b), and information is extracted from the mutual relation of the drawn great circles. By following such a procedure, a normal vector of a plane including each point of the original image and the center of the spherical surface can be obtained, a line segment can be extracted at the same time, and a line segment that is discontinuous or distorted can be reproduced. Furthermore, by mapping the change information of the projection on the sphere to one point on the sphere,
The trajectory of a point-like object that moves in a straight line is extracted.

By the way, in this spherical mapping, one camera is moved to a plurality of locations by using motion stereoscopic vision, and the principle of triangulation is used in the target space from the relationship between the captured image and the camera movement pitch at each location. The three-dimensional information of the line segment is measured.

Therefore, when obtaining three-dimensional information by using such a motion stereoscopic vision, it is necessary to accurately obtain the moving direction information of the camera.

[Conventional technology]

Conventionally, as a method of extracting the moving direction information of the camera,
A method of extracting the mounting position of a camera in a three-dimensional positioning device such as a robot equipped with a camera based on the positioning information for the three-dimensional positioning device has been used.

[Problems to be solved by the invention]

However, the positioning accuracy of the current three-dimensional positioning device is not sufficiently high, the position where the camera is actually positioned and the positioning accuracy do not always match, and when the camera is attached to the three-dimensional positioning device, the positioning accuracy is not accurate. It cannot be installed, and there is always an installation error.

Therefore, if the moving direction of the camera is simply determined based on the positioning information for the three-dimensional positioning device, an accurate moving direction cannot be obtained, and accurate three-dimensional information such as a line segment in the target space can be obtained. There was a problem that I could not do it.

In view of the above-mentioned conventional drawbacks, an object of the present invention is to obtain an accurate moving direction of a camera, and thus to obtain moving direction information of a camera for obtaining accurate three-dimensional information such as a line segment in a target space. It is to provide an information extraction method that can be accurately extracted.

[Means for solving problems]

An object of the present invention is to provide a camera for photographing a space to be measured, a means for moving the camera in a three-dimensional direction, a means for generating a spherical mapping function for a photographed image of the camera, and the spherical mapping function. And a mapping plane A to which a great circle is added, and a plurality of points of interest P ₀ ^{t to} P ₂ ^t are set in the measurement target space, and each of the points of interest is linearly moved. At each different position, an image is taken, and each attention point P ₀ for each different position
¹ , P ₀ ²・ ・ ・, P ₁ ¹ , P ₁ ²・ ・ ・, P ₂ ¹ , P ₂ ²・ ・ ・ generate a spherical mapping function and write it on the mapping surface. The obtained first spherical circles L _P0 ¹ ..., L _P1 ¹ ..., L _P2 ¹ ... Intersection Σ _0, Σ _1, Σ ₂ ...
Is extracted, a spherical mapping function is further generated with respect to the obtained first intersections corresponding to respective attention points, the spherical mapping function is written on the mapping surface, and the spherical mapping function on the mapping surface obtained by the writing is generated. The intersection v of the plurality of second great circles L ₀ , L ₁ , L ₂ ... Is extracted and the moving direction information of the camera is extracted based on the extracted intersection information. It is achieved by the information extraction method.

FIG. 1 is a diagram for explaining the basic principle of the information extraction method according to the present invention.

In the figure, P ₀ ^t , P ₁ ^t , and P ₂ ^t (t is time series information, t = 0, 1, 2, ...) Are arbitrary points of interest in the measurement target space, and the spherical surface A is of interest. A spherical memory for recording the information of the camera that captures the points P ₀ ^t , P ₁ ^t , and P ₂ ^t is shown.

Now, when the plurality of points of interest P ₀ ^t , P ₁ ^t , and P ₂ ^t that are stationary while the camera is moving linearly are seen, it seems that the plurality of points are moving linearly in parallel. Therefore, for example, as shown in FIG. 1, a plurality of points are spherically mapped (great circle information generation) at each time point, and the results are accumulated in the spherical memory A. A sequence of points Σ ₀ , Σ ₁ , Σ ₂ , ... Is obtained. When a spherical map is applied to this Σ point sequence and great circle information is written on the spherical surface, their intersection v is obtained.

Here, it can be seen that the vector V connecting the intersection v and the center of the sphere is parallel to a plurality of straight lines connecting the point sequences projected on the sphere due to the nature of the sphere.

On the other hand, since the plurality of straight lines projected on the spherical memory A and the moving direction of the camera coincide with each other, the moving direction of the camera coincides with this vector V.

Therefore, the three-dimensional movement direction of the camera can be known from the address of the intersection v on the spherical memory A.

〔Example〕

An embodiment of an information extraction method according to the present invention will be described in detail below with reference to the drawings.

FIG. 2 is an overall configuration diagram of a spherical mapping device to which the present invention is applied.

In the figure, 30 is a microprocessor (hereinafter referred to as MPU), 31 is a spherical camera, 32 is an original image memory, 34
Is a moving direction measurement data creation unit, 35 is a moving direction determination unit, 36 and 37 are mapping processors, 38 and 39 are mapping memories, and 40 is a parallel interface.

In the above configuration, the MPU 30 controls the spherical camera 31, the original image memory 32, the moving direction measurement data creation unit 34, the moving direction determination unit 35, and the like. The spherical camera 31 images an object and obtains image data In the polar coordinate (r, θ) format, and the original image memory 32 stores the image data in the polar coordinate format output from the spherical camera 31 and subjected to the contour extraction processing in association with each other in a grid pattern. Yes, with a storage area divided into N in the longitude direction and M in the latitude direction, the moving direction measurement data creation unit 34 selects any of a plurality of points on the measurement target space plane as the attention points P ₀ ^t ,
It is set as P ₁ ^t , P ₂ ^t ... For example, in a contour-extracted original image, vertices at which a plurality of line segments intersect are set as points of interest, or a plurality of points are set in a measurement target space. The difference between the original image when the light spot is illuminated and the original image when the light spot is not illuminated is obtained, and the coordinates of the plurality of obtained light spots are set as the points of interest.
The moving direction of the spherical camera 31 is determined based on which coordinate the intersection point developed in 9 is stored. The mapping processor 36 processes the image data for each pixel stored in the original image memory 32. The one that generates great circle information and sequentially stores the mapping result in the mapping memory inside, and the mapping memory 38 that the peak point (intersection point) which is the mapping result in the mapping processor 36 is transferred and stored. The processor 37 generates great circle information for the pixels having the peak points stored in the mapping memory 38 and sequentially stores the mapping result in the internal mapping memory, and the mapping memory 39 is the mapping result in the mapping processor 37. A certain peak point is transferred and stored, the parallel interface 40 is an MPU
Reference numeral 30 and the like are for accessing the mapping memories 38 and 39.

A method of extracting the moving direction information of the spherical camera 31 in the configuration described above will be described.

This moving direction information extraction method has three steps:
It consists of moving direction measurement data creation, moving trajectory extraction, and moving direction information extraction, which will be described below in this order.

(a) Creation of moving direction measurement data First, the spherical camera 31 captures an image of the measurement target space under the control of the MPU 30 and outputs image data in polar coordinate (r, θ) format. The image data (coordinate position and density value) not subjected to the extraction process is stored in a corresponding storage area of the first internal memory in the moving direction measurement data creation unit 34 in pixel units.

Next, the MPU 30 operates a spot light irradiation unit (not shown) while keeping the spherical camera 31 stationary,
A plurality of spot lights are emitted onto the same measurement target space.

Then, in the same manner as in the above item, the spherical camera 31 captures an image of the target space, and the obtained image data that is not subjected to the contour extraction processing corresponds to the second internal memory in the moving direction measurement data creation unit 34 in pixel units. Store in the storage area.

In the moving direction measurement data creation unit 34, the first and second
When the image data not irradiated with the spot light and the image data irradiated with the spot light are stored in the internal memory of, the difference between the two is calculated.

As a result, only the point irradiated with the spot light is taken out as a difference, and therefore the moving direction measurement data creation unit 34 extracts the coordinate value (r, θ) of this point and stores this point in the internal memory. Write the coordinate values.

Next, the MPU 30 gives an instruction to a three-dimensional positioning device (not shown) on which the spherical camera 31 is mounted so as to move linearly (translationally) by a predetermined pitch, and the spherical camera 3
The position of 1 in the three-dimensional space is changed.

Then, the ~ by sequentially repeatedly executing the operation section, time-series information P ₀ ¹ point of attention ^{_{P 0, P 0 2, P}} 0
³ , ..., Time series information P ₁ ¹ , P ₁ ² , for the point of interest P ₁
P ₁ ^3, ..., time-series information P ₂ ¹ point of attention ^{_{P 2, P 2 2, P}} 2
³ , ... Are written as shown in FIG.

In this way, measurement data for measuring the three-dimensional movement direction of the spherical camera 31 is obtained.

(b) Movement Trajectory Extraction First, the MPU 30 includes the movement direction measurement data creation unit 3
The measurement data is written from the internal memory in 4 into the original image memory 32.

Then, the mapping processor 36 is operated to generate a spherical mapping function (great circle) for the pixel of each point of interest written as shown in FIG. 3 and write it into the internal mapping memory.

FIG. 3 is a diagram showing a state in which a great circle corresponding to each attention point is written in this internal mapping memory. For the attention point P ₀ ^1, for the great circle L _P0 ¹ and the attention point P ₁ ^1. It is written as a great circle L _P1 ¹ , ...

Then, the mapping processor 36 causes the intersection points Σ ₀ , Σ ₁ , Σ of the great circles L _P0 ¹ ... Written in the internal memory in this way.
₂ are extracted and the intersections Σ ₀ , Σ ₁ , Σ ₂ , ... Are written in the mapping memory 38 as shown in FIG.

As a result, the intersection, which is the pole of the locus of the point of interest that moves in a straight line, is obtained.

(c) Extraction of moving direction information Intersection points Σ ₀ X, Σ ₁ , written as shown in FIG.
A spherical mapping function (great circle) is generated by the mapping processor 37 for the pixels of Σ ₂ , ... And written in the internal mapping memory.

Figure 4 is an intersection sigma ₀ to the internal memory, sigma _1, sigma _2, a diagram showing a state where the great circle is written with respect ..., intersection sigma _0, the large circle L ₀ with respect to the intersection sigma _1, Is written as a great circle L ₁ and an intersection Σ ₂ is written as a great circle L ₂ .

Then, the mapping processor 37 extracts the intersection v of the great circles L ₀ , L ₁ , ... Written in the internal memory in this way, and writes this intersection v in the mapping memory 39.

The MPU 30 operates the movement direction determination unit 35 at the time when the intersection V is written in the mapping memory 39.

The moving direction determination unit 35 is a parallel interface 40.
The intersection point v in the mapping memory 39 is read out via to determine the moving direction of the spherical camera 31 corresponding to the read position.

That is, the mapping memory 39 stores the image data in polar coordinates (r, θ) format, and the coordinate position itself of the image data represents the position of the point where the center position of the sphere intersects with the spherical surface. can do.

Then, the moving direction information of the camera obtained as described above and the moving direction information of the camera are extracted, and at the same time, the spherical data is applied to the image data when the image is captured in the target space and the information is stored in the mapping memory. The third-order information of the line segment is obtained from the intersection information indicating the line segment that has been set.

〔The invention's effect〕

As described above, according to the present invention, a plurality of points of interest photographed in time series are subjected to spherical mapping to obtain a plurality of intersections, and further a plurality of intersections are subjected to spherical mapping to obtain intersections. Since the movement direction information of the camera is extracted based on this intersection information, it is accurate without being affected by the positioning error of the three-dimensional positioning device, the mounting error of the camera on the three-dimensional positioning device, and the like. It is possible to measure the moving direction of the camera and obtain accurate three-dimensional information such as a line segment in the target space.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of the principle of the information extraction method, FIG. 2 is a block diagram showing the configuration of a spherical mapping device to which the present invention is applied, and FIG.
FIG. 4 is a diagram showing a state in which a great circle is written for a target point, FIG. 4 is a diagram showing a state in which a great circle is written for an intersection, and FIG. 5 is an explanatory diagram of the principle of spherical mapping. In the figure, 30 is an MPU, 31 is a spherical camera, 32 is an original image memory, 34 is a moving direction measurement data creating section, and 35 is
Is a moving direction determination unit, 36 and 37 are mapping processors, 3
Reference numerals 8 and 39 are mapping memories.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yusuke Yasukawa 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Fujitsu Limited Examiner Shigekazu Ishii

Claims (1)

[Claims] 1. A camera for photographing a space to be measured, a means for moving the camera in a three-dimensional direction, a means for generating a spherical mapping function with respect to a photographed image of the camera, and a spherical mapping function. And a mapping surface (A) to which a great circle is added, and a plurality of points of interest (P ₀ ^{t to} P ₂ ^t ...) Are set in the measurement target space, and each point of interest is set to the camera. An image is taken at each different position when moved linearly, and each attention point (P ₀ ¹ , P ₀ ² ..., P ₁ ¹ , P ₁ ² ..., P ₂ ¹ , P ₂ ² ...)
On the other hand, a spherical mapping function is generated and written in the mapping surface, and a plurality of first great circles (L _P0 ¹ ..., L _P1 ¹⁾ obtained by the writing are created by the spherical mapping function on the mapping surface. ..., L _P2 ¹ ...
・) The first intersection (Σ _0, Σ _1, Σ) corresponding to each attention point
₂ ) is extracted, a spherical mapping function is further generated with respect to the obtained first intersections corresponding to respective attention points, the spherical mapping function is written in the mapping surface, and the mapping surface obtained by the writing is generated. The intersection (v) of the plurality of second great circles (L ₀ , L ₁ , L ₂ ...) Is extracted by the above spherical mapping function, and the moving direction information of the camera is obtained based on the extracted intersection information. An information extraction method characterized by being extracted.