JPH0827838B2 - Multi-view stereoscopic device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a stereoscopic device used for stereoscopically recognizing a three-dimensional object, and in particular, the present invention relates to a three-dimensional system for a large number of visual systems. The present invention relates to a multi-view stereoscopic device having a function equivalent to stereoscopically viewing an object.

<Prior Art> Conventionally, in order to stereoscopically view a three-dimensional object, for example, a trinocular stereoscopic device as shown in FIG. 7 is generally used.

The device shown in the figure has three TV cameras 9 mounted on a fixed machine base 90.
A, 9B and 9C are arranged to configure the image pickup means 9, the object images obtained by each television camera are taken into the image processing device 91, and the image points in each object image are associated with each other, and The three-dimensional coordinates of the point (hereinafter referred to as “object point”) are calculated.

Each of the television cameras 9A, 9B, 9C has
It has its own visual field ranges a, b, and c that overlap each other, and the overlapping part (indicated by diagonal lines in the figure) of each of these visual field ranges is the range in which stereoscopic viewing is possible with this trinocular stereoscopic device.

<Problems to be Solved by the Invention> According to the three-eye stereoscopic device, the possible range of the stereoscopic view is remarkably narrowed, and therefore, there is a strong demand in the industry to expand the range.

Therefore, in order to expand the stereoscopic viewable range by applying the principle of the conventional three-eye stereoscopic device, there is a method such as increasing the number of TV cameras to increase the number of eyes, but with this method, the equipment becomes large, There is also a problem that the equipment cost becomes high.

The present invention is to solve the above problems, and provides a multi-view stereoscopic device having a function equivalent to stereoscopically viewing an object with a large number of visual systems with a small number of visual systems. The objective is to greatly expand the range of stereoscopic vision possible with various facilities.

<Means for Solving Problems> In order to achieve the above object, according to the present invention, two television cameras that capture an object at different spatial positions to obtain a plurality of object images, and the two televisions described above. Equipped with a camera,
Moving means for changing the imaging point, measuring means equipped on the moving means for measuring a spatial moving vector of the moving means, and each of the television cameras based on the moving vector measured by the measuring means. A camera model calculating means for calculating a camera model at a moving position, image data of an object image obtained by picking up an object by each TV camera, and a moving position at the time of image pickup of the object calculated by the camera model calculating means. Storage means for storing the camera model of each television camera in association with each other, arbitrary three pieces of image data of all image data stored in the storage means, and camera models associated with the respective image data And are taken out from the storage means while changing the combination of the image data, and the trinocular stereoscopic processing is performed for each combination of the image data. A multi-view stereoscopic device including a stereoscopic means for executing and calculating three-dimensional data of an object point and a data integrating means for integrating three-dimensional data of an object point for each combination of image data calculated by the stereoscopic means. Are configured.

<Operation> While moving two TV cameras, an image of an object is taken by each TV camera. The measurement means measures the movement vector at each movement position, and the camera model calculation means
A camera model at each moving position of each TV camera is calculated based on the measured moving vector. The image data of the object image obtained by each TV camera is stored in association with the camera model of each TV camera at the moving position when the object is imaged. After the image capturing is completed, arbitrary three pieces of image data among the obtained image data and the camera models associated with the respective image data are taken out while changing the combination of the image data, and 3 for each combination of the image data. After the stereoscopic vision process is executed to calculate the three-dimensional data of the object point, the calculated three-dimensional data of the object point are integrated.
Therefore, if the number of times of imaging is increased, an effect equivalent to stereoscopically viewing an object with a large number of visual systems can be obtained with a small number of visual systems.

<Embodiment> FIG. 2 shows a schematic configuration of a mobile robot RB incorporating the multi-eye stereoscopic vision device of the present invention.

The mobile robot RB of the illustrated example is equipped with an image pickup means 4 composed of two TV cameras 1 and 2 having both eyes at the upper position of the robot body 3, and forward and reverse driven by a motor at the lower position of the robot body 3. There are four wheels 30 deployed.

Each of the TV cameras 1 and 2 has a diagonal downward field of view,
While the robot body 3 moves in the direction of the arrow S in the figure, it picks up an image of an object at a plurality of arbitrary image pickup points and generates an image of the object on an image pickup surface made of a CCD, for example.

FIG. 1 shows a specific circuit configuration example of a multi-view stereoscopic device according to an embodiment of the present invention. CPU (Central Proce
ssing Unit) 50 is RAM (Randam Access Memory) 51 or R
The microcomputer 5 is configured with an OM (Read Only Memory) 52, and each input / output of the image pickup unit 4, the image memories 61 and 62, the line drawing processing unit 7, the rotary encoders E _{1 to} E ₆ , and the motors M _{1 to} M ₆ is performed. The operation is controlled in series, and various calculations and processes relating to multi-view stereoscopic vision are executed. In the ROM 52, in addition to a program for executing the multi-view stereoscopic processing, data defining a reference coordinate system is stored in advance, and in the RAM 51, various data related to the stereoscopic processing is set. The reference coordinate system is for giving a camera model that defines the positions and orientations of the TV cameras 1 and 2, and this camera model uses the movement vector each time the position of the TV cameras 1 and 2 moves. It will be updated.

The line drawing processing unit 7 performs contour tracking on the object images generated by the television cameras 1 and 2, and applies straight lines to the contours in order to generate a line drawing image.
The image memories 61 and 62 are for storing object images obtained by the television cameras 1 and 2, and reading and writing of these images are controlled by the CPU 50 via the control port 53.

Among the motors M _{1 to} M ₆ , the motors M _{1 to} M ₄ are mobile robots.
Drives the four wheels 30 of the RB to generate thrust. The remaining motors M ₅ and M ₆ drive the front and rear axles for steering operation. The CPU 50 controls the forward / reverse driving of each of the motors M _{1 to} M ₆ via the motor control unit 8 to move the mobile robot RB by a required distance in a desired direction.

The rotary encoders E _{1 to} E ₆ are connected to the motors M _{1 to} M ₆ , respectively, and output a number of pulses corresponding to the rotation angles of the motors M _{1 to} M ₆ . The output of each rotary encoder E _{1 to} E ₆ is
This is taken into the CPU 50 via the I / O port 54, and the CPU 50 updates the camera model by calculating the movement vector (movement amount and movement direction) to each imaging point of the mobile robot RB based on these input data.

FIG. 3 shows a control procedure of multi-view stereoscopic vision processing by the above apparatus example.

At the start point in the figure, the mobile robot RB stands by at a predetermined position, and the TV cameras 1 and 2 for both eyes have their visual fields directed in a predetermined direction. First, in step 1, CPU50 is ROM52
The data that defines the reference coordinate system is read and RA
Set it on the M51. Next, in step 2, when the CPU 50 gives an imaging command to the TV cameras 1 and 2 via the control port 53, each TV camera 1 and 2 images the object and generates an object image on the imaging surface. Each object image is stored in the image memories 61 and 62.

In the next step 3, the CPU 50 takes in the outputs of the rotary encoders E _{1 to} E ₆ via the I / O port 54, and outputs C
The PU 50 calculates the movement vector (movement amount and movement direction) of the mobile robot RB based on these input data, and updates the camera model based on the calculation result and the data related to the reference coordinate system. In this case, since the mobile robot RB is stationary, the camera model is not updated and the process proceeds to the next step 4. In step 4, the CPU 50 gives a line drawing command to the line drawing processing unit 7, performs contour tracing, straight line fitting, and the like on the object images already set in the image memories 61 and 62 to form a line image of the object image. The line drawing image data (hereinafter referred to as “line drawing data”) and the camera model are paired and stored in the RAM 51.

In the next step 5, it is determined whether or not the number of times of image capturing by the television cameras 1 and 2 has reached a prescribed number n (where n ≧ 2), and if “NO” is determined, the process proceeds to step 6. In step 6, the CPU 50 controls each motor M ₁ via the motor controller 8.
The drive of M ₆ is controlled to move the mobile robot RB in a desired direction. In this case the rotation of the motors M ₁ ~M ₆ is transmitted to the rotary encoder E ₁ to E ₆ corresponding, respective rotary encoders E ₁ to E ₆ output a number of pulses corresponding to the rotation angle (rotational speed) To do.

Next, returning to step 2, the CPU 50 gives an image pickup command to the television cameras 1 and 2 again, an object is imaged at an arbitrary image pickup point, and the object image is stored in the image memories 61 and 62. In the case of this embodiment, the above-mentioned imaging operation is performed while the mobile robot RB is moving, but not limited to this, for example, the mobile robot RB
The RB may be temporarily stopped and performed in a stationary state. Then, in the next step 3, the CPU 50 calculates the movement vector to the imaging point of the mobile robot RB based on the outputs of the rotary encoders E _{1 to} E ₆ , and uses the calculation result and the data related to the reference coordinate system to calculate the camera. Perform a model update. Next, in step 4, the CPU 50 causes the line drawing processing unit 7 to execute the line drawing of the object image, and then the line drawing data and the updated camera model are paired to the RAM 51.
To memorize.

In this case, since the two TV cameras 1 and 2 are used, a total of four line drawings are obtained at this point.

Thus, the number of times of imaging by the TV cameras 1 and 2 is the specified number n
The same processing is repeatedly executed until. As a result, the determination of “Is the number of imaging times n?” In step 5 is “YES”.
If so, the process proceeds to step 7, and the CPU 50 gives a stop command to the motor control unit 8 to stop the driving of the motors M _{1 to} M ₆ and stops the movement of the mobile robot RB.

Next, in step 8, the CPU 50 selects any three line drawings from the obtained 2n line drawings, and then, in the next step 9, executes the processing of three-eye stereoscopic vision.

FIG. 4 shows a method of associating image points on three line drawing images with each other in the three-eye stereoscopic view.

In the figure, A, B, and C represent the selected three line drawing images, and the respective images are positioned in correspondence with the image pickup surface position of the television camera at the time of image pickup. In each image A, B, C,
In addition to including the image points P ₁ , P ₂ , and P ₃ for the object P, the image points R ₂ and R ₃ for other object points R also exist.

Also, on the image B, the focus of the TV camera for imaging
F ₁ and the image point P ₁ linear F ₁ P ₁ of the image (this is called "epipolar line") connecting the l ₂ is set, similarly to the the image C,
The epipolar lines l ₃ , m ₃ , and n _{3 of the} straight line F ₁ P ₁ , the straight line F ₂ P _2, and the straight line F ₂ R ₂ are set.

FIG. 5 shows the images A, B and C described above. According to the figure, the image P ₂ in the image B is located on the epipolar line l ₂ , and the image point P ₃ in the image C is located on the intersection of the epipolar lines l ₃ and m ₃ .

Thus, the CPU 50 determines that the image points P ₁ , P ₂ , P ₃ are points corresponding to each other as an image of the object point P, and then determines the three-dimensional coordinates of the object point P as straight lines F ₁ P ₁ , F ₂ P. _It is obtained as the intersection of ₂ and F ₃ P ₃ and is stored in the RAM 51. 5 (2) and (3), the image points R ₂ and R ₃ of the other object point R are also shown. In this case, the image point R ₃ corresponds to the epipolar lines l ₃ and m ₃ . Not located on the intersection.

The above-described trinocular stereoscopic processing (correspondence of image points and calculation of coordinates of object points) is performed for all image points. When this is completed, next, three line drawings for another combination are selected. Then, similar multi-view stereoscopic vision processing is performed.

Thus, when all combinations of 3 line drawings from 2n line drawings are selected and trinocular stereoscopic vision is performed, the determination of “Are all combinations completed?” In step 10 is “YES”,
In step 11, the three-dimensional coordinates of all object points within the stereoscopic viewable range are integrated, and a series of processing is ended.

Fig. 6 shows a television camera when a total of two images have been taken.
The visual field ranges a, b, c, d by 1, 2 are shown. In the figure, the shaded area f where the visual field ranges a, b and d overlap and the shaded area e where the visual field ranges b, c and d overlap are the possible range for three-eye stereoscopic vision. It is easily understood that the possible range of stereoscopic vision is greatly expanded as compared with FIG.

<Effects of the Invention> As described above, the present invention stores image data of an object image obtained by capturing images while moving two television cameras and the camera model calculated at each moving position in association with each other. , After taking out arbitrary three pieces of image data of the obtained image data and the camera models corresponding to them while changing the combination of the images and performing the trinocular stereoscopic processing,
Since the three-dimensional data of object points for each combination of the calculated image data is integrated, the same effect as stereoscopically viewing an object with a large number of visual systems can be obtained with a small number of visual systems, and simple equipment can be used. The remarkable effect that the object of the invention is achieved is achieved, such that the possible range of stereoscopic vision can be greatly expanded.

[Brief description of drawings]

FIG. 1 is a circuit block diagram of a multi-view stereoscopic device according to an embodiment of the present invention, FIG. 2 is a schematic explanatory view of a mobile robot incorporating the multi-view stereoscopic device of the present invention, and FIG. FIG. 4 is a flow chart showing a processing procedure of the stereoscopic vision apparatus, FIG. 4 is a principle explanatory view showing a method of associating image points, FIG. 5 is an explanatory view showing each selected image, and FIG. 6 is an apparatus of the present invention. 7 is an explanatory view showing a stereoscopic viewable range in FIG.
FIG. 8 is an external view showing an eye stereoscopic device, and FIG. 8 is an explanatory diagram showing a stereoscopic viewable range in a conventional device. 1,2 ...... TV camera, 3 ...... Mobile robot 4 ...... Imaging means, E _{1 to} E ₆ ...... Rotary encoder 5 ...... Microcomputer, 50 ...... CPU 51 ...... RAM

Continuation of the front page (56) Reference JP 58-90268 (JP, A) JP 61-114381 (JP, A) JP 61-122509 (JP, A) JP 60-199293 (JP , A) JP 60-199292 (JP, A) JP 60-199291 (JP, A) JP 60-201792 (JP, A) JP 60-27085 (JP, A) JP 60-55479 (JP, A)

Claims (1)

[Claims] 1. Two TV cameras for picking up an object at different spatial positions to obtain a plurality of object images, and a moving means for mounting the two TV cameras and changing an imaging point. Measuring means mounted on the moving means, for measuring a spatial moving vector of the moving means, and a camera model for calculating a camera model at each moving position of each TV camera based on the moving vector measured by the measuring means. Calculating means, image data of an object image obtained by imaging an object with each TV camera, and the camera model of each TV camera at the moving position during object imaging calculated by the camera model calculating means are associated with each other. Storage means for storing, corresponding to any three pieces of image data among all the image data stored in the storage means and respective image data Stereoscopic means for extracting the selected camera model from the storage means while changing the combination of the image data, and executing three-eye stereoscopic processing for each combination of the image data to calculate the three-dimensional data of the object point, A multi-view stereoscopic device comprising: a data unifying unit that unifies three-dimensional data of object points for each combination of image data calculated by the stereoscopic viewing unit.