JP2554615B2 - Image registration device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to an image registration apparatus capable of interactively and easily performing registration processing of input images necessary for image processing of satellite images, medical images, and the like.

[Technical Background of the Invention and its Problems] When image processing is performed on a satellite image or a medical image to obtain necessary image information, alignment between a plurality of images obtained from different viewpoints is performed as preprocessing. It is necessary to do it. For example, when a side scanner image obtained by capturing and inputting the ground surface from a satellite is made to correspond to a map and the ground surface information is searched for, a side scanner image in which the same ground surface is viewed obliquely with respect to the map viewed from directly above is displayed. Alignment is required. Similarly, in order to obtain diagnostic information of the diagnosis target region from a plurality of X-ray imaged medical images at different angles, it is necessary to align the medical images.

So, in the past, for multiple input images with different viewpoints,
The parallel movement and rotation are used to perform the alignment. However, improvement in image processing efficiency could not be expected, and improvement in image processing accuracy could not be expected.

[Object of the Invention]

The present invention has been made in consideration of such circumstances.
The purpose is to easily and accurately perform alignment between a plurality of images necessary for various image processing without inviting complication of the apparatus, and improving the efficiency of image processing and its processing. An object of the present invention is to provide an image alignment device capable of improving accuracy.

[Outline of Invention]

An image registration apparatus according to the present invention stores an input image storage unit that stores two pieces of image information obtained by observing an object including a common feature point, and an input image storage unit that stores the input image storage unit. Display means for displaying the two pieces of image information on a display screen, and instruction input means for inputting the position coordinates of the characteristic points in the displayed two pieces of image information by interactive instructions on the display screen. , Means for obtaining a conversion coefficient necessary for two-dimensional projective transformation between the two pieces of image information from each position coordinate of the feature point inputted by this means, and one using the transformation coefficient obtained by this means It is provided with a means for coordinate-converting the image information and an output image storage means for storing the one image information coordinate-converted by this means.

That is, the position coordinates of the feature points in the two input images are
Directly inputting and inputting each on the displayed image, the position coordinates of the characteristic points are used to calculate the conversion coefficients required for the two-dimensional projective conversion processing used for the image alignment.

〔The invention's effect〕

Thus, according to the present invention, there is provided an instruction input means for inputting the position coordinates of the feature points of each image information imaged on the display screen by an interactive instruction on the surface screen, and the feature points input by this means are provided. Since the conversion coefficient necessary for the two-dimensional projective conversion between the two pieces of image information is obtained from each position coordinate of, the position coordinates of the feature points are detected and input by, for example, an electronic feature extraction method. Unlike the above, not only can the device be simplified, but the position coordinates of common feature points in multiple images can be interactively input for each image under the operator's judgment. Can be accurately specified, and the conversion coefficient necessary for the two-dimensional projective conversion used for alignment between a plurality of images can be easily and effectively calculated from the position coordinates. Therefore, the processing efficiency of the image alignment can be dramatically improved, and the processing accuracy thereof can be greatly improved.

Moreover, as described above, it is only necessary to interactively input and input the position coordinates of the feature points necessary for the alignment process, so that the position coordinate input process is easy and does not place much burden on the operator. Therefore, while ensuring the ease of handling of the device, the processing efficiency and the processing accuracy can be dramatically improved, and it is possible to greatly contribute to various image processings, which is a great practical effect. Sent.

Example of Invention

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows the basic concept of image registration according to the present invention.

Surfaces with slight irregularities such as the surface of the earth viewed from hygiene, X
In an image taken from a position separated by a distance like a line-taken medical photograph, the component in the vertical (height) direction in the observed image can be almost ignored. That is, it can be assumed that the vertical direction component in the observed image of the target object viewed from the positions separated by the viewpoint distance is negligible as compared with the horizontal direction (plane) component.

Now, as shown in FIG. 1, a target object P is
The image Ap viewed from the viewpoint position A at a distance such that its vertical component is negligible can be considered as the perspective projection image, and its coordinate system is, for example, (X, Y) {X (k), It can be expressed as Y (k); k = 1,2,3 to k}. An image Bp of the same target object P viewed from a viewpoint position B different from the viewpoint position A is
The coordinate system can be expressed as (x, y) as {x (k), y (k); k = 1,2,3 to k}.

Since these two images Ap and Bp are images of the same target object P, the image information indicated by the same parameter k can be captured as corresponding to the same point (position), and thus the above The mutual relationship between the two images Ap and Bp can be represented by a two-dimensional projective transformation of X = (b0 + b1x + b2y) / (1 + a1x + a2y) Y = (c0 + c1x + c2y) / (1 + a1x + a2y). Therefore, if the two-dimensional projective transformation is applied to the image Bp, the image Bp can be superimposed (positioned) on the image Ap.

The eight unknowns (conversion coefficients; a1, a2, b0, b2, b3, c0, c1, c2) required for this two-dimensional projective transformation are equations showing the relationship between the two images Ap, Bp. 4 because there are two
By giving the position coordinates of two corresponding points, the simultaneous 1
If the solution is obtained as the solution of the following equation, and the corresponding point position coordinates of four or more points are given, the solution can be uniquely obtained by introducing the method of least squares, for example.

That is, a set of four or more position coordinates corresponding to each other between two images Ap and Bp is {x (k), y (k), X (k), Y (k); k = 1,2 , 3 to k}, it is possible to solve the simultaneous equations described above based on each of these data and obtain the conversion coefficients thereof.

After that, each of these transformation coefficients is introduced into the two-dimensional projective transformation formula, and if the two-dimensional projective transformation process is performed on the image Bp using this formula, the image Bp aligned with the image Ap is obtained. ′ Can be obtained.

In image processing, it is general to perform pixel coordinate calculation while performing raster scanning. Therefore, it is more advantageous to specify the pixels of the output image and calculate the corresponding pixels of the input image in terms of the sampling processing and the like. This means that the above-described coordinate system conversion process may be executed in reverse, and therefore the relationship between the images Ap and Bp may be obtained in reverse and the conversion process may be executed.

That is, in terms of projective geometry, the inverse function described above can be obtained from the duality in exactly the same manner. You should do it.

This device is based on such a technical idea, and is configured as shown in FIG. 2, for example.

The first and second input image memories 1a and 1b respectively input and store two images Ap and Bp including the same target object and having different viewpoints.
The input images Ap and Bp stored in a and 1b are displayed on the display device 9.
Is displayed. The input image Ap is an image in the (X, Y) coordinate system as shown in FIG. 3 (a) when the ground surface such as a map is viewed from directly above, and the input image is the same ground surface in an oblique direction from a satellite or the like. Viewed, for example, Figure 3 (b)
It consists of an image in the (x, y) coordinate system as shown in.

The position coordinate input device 2 cooperates with the display device 9 under the control of the control device 3 to interactively input the position coordinate information of the feature points that are common to the two images Ap and Bp. And is realized by, for example, a cursor or a light pen device. Via the position coordinate input device 2, at least four or more characteristic points (k that are common to the two images Ap and Bp are common.
= 1,2,3 to k) position coordinate information {x (k), y (k), X (k), Y (k); k = 1,2,3 to k} .

The information of the position coordinates of the feature points thus input is given to the coefficient calculation circuit 4 and used for the coefficient calculation of the coordinate calculation formula used for the position alignment of the images by the above-described two-dimensional projective transformation. In this coefficient calculation circuit 4, for example, when position coordinate data of 4 points (k = 1,2,3,4) is input, X (1) = {b0 + b1x (1) + b2y (1)} / {1 + a1x ( 1) + a2y (1)} Y (1) = {C0 + C1x (1) + C2y (1)} ÷ {1 + a1x (1) + a2y (1)} X (2) = {b0 + b1x (2) + b2y (2)} ÷ { 1 + a1x (2) + a2y (2)} Y (2) = {c0 + c1x (2) + c2y (2)} ÷ {1 + a1x (2) + a2y (2)} X (3) = {b0 + b1x (3) + b2y (3)} ÷ {1 + a1x (3) + a2y (3)} Y (3) = {c0 + c1x (3) + c2y (3)} ÷ {1 + a1x (3) + a2y (3)} X (4) = {b0 + b1x (4) + b2y (4) )} ÷ {1 + a1x (4) + a2y (4)} Y (4) = {c0 + c1x (4) + b2y (4)} ÷ {1 + a1x (4) + a2y (4)}, which is the unknown conversion coefficient. (A1, a
2, b0, b2, b3, c0, c1, c2). These conversion coefficients (a1, a2, b0, b2, b3, c0, c1, c2) are set in the coefficient register 5 and given to the X coordinate calculation circuit 6 and the Y coordinate calculation circuit 7, respectively. Each calculation circuit 6,7
X = (b0 + b1x + b2y) / (1 + a1x + a2y) Y = (c0 + c1x + c2y) / (1 + a1x + a2y) X-coordinate conversion and Y-coordinate conversion based on the two-dimensional projective transformation.

The X-coordinate calculation circuit 6 and the Y-coordinate calculation circuit 7 assign to each coordinate point (x (k), y (k)) of the coordinate system (x, y) of the input image Bp according to the above-described two-dimensional coordinate conversion formula. On the other hand, the transformation coordinate point (X, Y) of the transformation coordinate system (X
(K) and y (k)) are respectively calculated, and the address of the calculated coordinate position on the output image memory 8 is designated. The output image memory 8 receives this address designation and writes the image information of the corresponding coordinate position (x (k), y (k)) of the input image Bp stored in the input image memory 1b at the address.

Such coordinate conversion processing is sequentially performed on all coordinate points of the input image memory 1b, and the coordinate conversion is performed on the output image memory 8, that is, the input image Bp is aligned with the coordinate system of the input image Ap. The output image Bp 'will be stored.

It should be noted that, in this coordinate conversion, interpolation of the coordinates indicated by the decimal point component of the position coordinates obtained by the coordinate conversion processing occurs. This interpolation may be dealt with by appropriately selecting the coordinates of the shortest distance or introducing an interpolation process. Further, image information in which a certain region is converted into the same integer coordinate may appear due to the inclination of the image caused by the shift of the viewpoint between the images Ap and Bp. It is sufficient to deal with this by introducing, for example, an averaging process or a weighting addition process for image information.

Thus, according to the present apparatus configured as described above, the conversion coefficient necessary for the alignment processing between the two images is easily obtained according to the position coordinate data interactively instructed, and the two images are acquired according to this conversion coefficient. By executing the coordinate conversion calculation for position alignment of the input image and writing the input image Bp stored in the input image memory 1b to the output image memory 8 according to the result of the coordinate conversion calculation, the input image is stored in the output image memory 8. It is possible to obtain the image Bp ′ aligned with Ap.

Moreover, since the conversion coefficient described above is performed based on the position coordinate information interactively instructed, it can be easily and accurately obtained. Especially, unlike the case of mechanically searching for common feature points from the two input images Ap and Bp, it is possible to appropriately specify the feature points that serve as a clue for alignment, using the operator's judgment. Therefore, the processing load on the apparatus can be significantly reduced and the processing accuracy can be improved without significantly increasing the operator's load.

By the way, in the above-described embodiment, the two displayed images are displayed.
Although the position coordinates of four or more feature points were directly designated and input to Ap and Bp, the relationship between the above two images Ap and Bp is shown as the relationship of the coordinate system. The conversion coefficient described above may be obtained by giving the amount of axial translation, the amount of three-dimensional reduction and expansion, and the like.

For example, consider the simple case where the input image Bp is rotated by θ ° around the x-axis, and h is the viewpoint distance. Therefore, the relation X = X ′ / w ′ = x / (1-y sin θ / h) Y = Y ′ / w ′ = y cos θ / (1-y sin θ / h) can be obtained. From this relationship, the conversion coefficient can be obtained as a1 = 0 a2 = -sin θ / h b0 = b2 = 0 b1 = 1 c0 = c2 = 0 c1 = -cos θ.

Therefore, using an interactive information input device such as a joystick or a mouse, the angle information is sequentially input from 0 to θ °, and the converted images obtained by this are sequentially monitored, thereby instructing the alignment of the images. Can be input accurately.

Information regarding the amount of parallel movement and enlargement / reduction can be basically input in the same manner, and the conversion coefficient for aligning the images can be similarly obtained according to the input information.

As described above, according to the present invention, it is possible to easily and accurately perform the alignment between the two images, and it is possible to achieve a great effect on the ground surface exploration based on the satellite image, the medical image processing, and the like. .

The present invention is not limited to the above embodiment. For example, an interactive input method of position coordinates of common feature points among a plurality of images, a conversion coefficient calculation algorithm, and the like may be set according to the specifications of the apparatus. Further, the calculation algorithm for coordinate conversion can be changed as appropriate. In short, the present invention can be variously modified and implemented without departing from the scope of the invention.

[Brief description of drawings]

FIG. 1 is a diagram showing the basic concept of the present invention, FIG. 2 is a schematic configuration diagram of an apparatus of an embodiment of the present invention, and FIGS. 3 (a) and 3 (b) are diagrams showing two input image examples. . 1a, 1b …… input image memory, 2 …… position coordinate input device,
3 ... Control device, 4 ... Coefficient calculation circuit, 5 ... Coefficient register, 6 ... X coordinate calculation circuit, 7 ... Y coordinate calculation circuit,
8 ... Output image memory, 9 ... Display device.

Claims (3)

(57) [Claims] 1. Input image storage means for respectively storing two pieces of image information obtained by observing an object including common feature points, and the two pieces of image information stored in the input image storage means. Is displayed on the display screen, instruction input means for inputting the position coordinates of the feature points in the displayed two pieces of image information by an interactive instruction on the display screen, and the input means. A means for obtaining a conversion coefficient necessary for the two-dimensional projective transformation between the two pieces of image information from each position coordinate of the feature point, and one of the image information is subjected to coordinate transformation using the transformation coefficient obtained by this means. An image alignment apparatus comprising: a means and an output image storage means for storing the one image information coordinate-converted by the means. 2. The instruction input means includes means for inputting parameters consisting of a triaxial rotation amount, a three-dimensional translation amount, a three-dimensional scaling amount, and a viewpoint distance between the two images. The image registration device according to claim 1. 3. The two pieces of image information input to the input image memory are obtained by observing a three-dimensional three-dimensional object from points separated by a distance where the height of the object can be ignored. The image registration apparatus according to claim 1, wherein the image registration apparatus is an image registration apparatus.