JP3539632B2 - Image change extraction method and image processing program thereof - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image change extraction method for extracting a change between images and a program thereof.
[0002]
[Prior art]
In fields such as updating a city map and analyzing the situation of a disaster area, a method for quickly detecting a change in geographical situation is required.
Conventionally, as this type of method, a method of manually comparing two aerial photographs and high-resolution satellite photographs at different photographing times has been known. In this method, an expert who can observe images individually with the left and right eyes detects differences between images one by one.
[0003]
Normally, a non-linear displacement occurs between two images due to a difference in a photographing position, an angle, and a lens aberration. The above-mentioned comparison work by the expert is a very excellent method in that this deviation can be unconsciously removed and only a change in the geographical situation can be accurately detected.
[0004]
Meanwhile, when such a comparison operation is mechanized, how to correct a non-linear correspondence deviation between images becomes an important problem. Correction of the correspondence shift requires a non-linear mapping that partially expands and contracts the image on the rubber film.
[0005]
The present inventor has proposed a nonlinear mapping suitable for such processing in Japanese Patent Application Laid-Open No. 7-239935. This non-linear mapping is generally performed according to the following procedures (1) to (4).
{Circle around (1)} A step of searching for corresponding points between images for a plurality of points in the image space and provisionally determining a plurality of movement vectors. [2] A group of movement vectors obtained based on the assumption that the images are continuously deformed. To increase the spatial continuity of the motion vector group. {Circle over (3)} The above operation is repeated to increase the detection accuracy of the motion vector group and to determine the motion vector group. Step of Correcting Correspondence Mismatch Between Images Using Correction of Correspondence Mismatch Between Images Using This Nonlinear Mapping, Image Comparison is Performed, It is possible to accurately detect a change in geographical situation.
[0006]
[Problems to be solved by the invention]
By the way, the occurrence of a fault has been reported in large-scale earthquakes such as the Great Hanshin-Awaji Earthquake. Such fault detection is important in elucidating the mechanism of earthquakes and obtaining preliminary information on earthquake prediction.
Most of this type of tomogram appears as a discontinuous image shift of about 2 to 5 pixels bordering the tomographic portion on an image of an aerial photograph or a satellite photograph.
[0007]
In the above-described nonlinear mapping, the discontinuous image shift of the tomographic portion spreads to the periphery in the process of smoothing the movement vector. Therefore, the corrected image data is likely to be blurred image data of the tomographic part, and it becomes difficult to clearly detect the boundary of the tomographic part in the image comparison.
[0008]
Therefore, an object of the present invention is to provide an image change extraction method for clearly detecting a tomographic change generated between images and a program therefor.
[0009]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the present invention is configured as follows.
[0010]
<< Claim 1 >>
The invention according to claim 1 is an image change extraction method for obtaining an image change for two input images A and B, wherein a sample step of extracting a plurality of minute regions from the image A and a correspondence between the minute regions A search step of searching for a portion from the image B to obtain a motion vector of a plurality of minute regions, and smoothing a group of the motion vectors obtained in the search step into a unit of a flat block predetermined in an image space. The coordination step of increasing the continuity of the movement vector group, the search step and the coordination step are repeated to increase the search accuracy of the corresponding part, the iteration step of determining the movement vector group, and the movement vector group determined in the iteration step, A step of detecting a change in the short axis direction of the flat block and obtaining a tomographic change between the images A and B.
[0011]
<< Claim 2 >>
According to a second aspect of the present invention, in the image change extracting method according to the first aspect, a search step, a coordination step, a repetition step, and a detection step are executed for a plurality of types of flat blocks having different short-axis directions, respectively. It is characterized in that a tomographic change between A and B is detected in a plurality of directions.
[0012]
<< Claim 3 >>
According to a third aspect of the present invention, there is provided an image change extraction method for obtaining an image change for two input images A and B, wherein a sample step of extracting a plurality of minute regions from the image A and a correspondence between the minute regions A search step of searching for a portion from the image B to obtain the motion vectors of the plurality of minute regions, and a group of the motion vectors obtained in the search step is divided by clustering, and the motion vector group is smoothed for each of the sections. , A coordination step to increase the continuity of the movement vector group, an iterative step of repeating the search step and the coordination step to increase the search accuracy of the corresponding portion, and to determine the movement vector group, and a space for the movement vector group determined in the iteration step. Detecting a change and obtaining a tomographic change between the images A and B.
[0013]
<< Claim 4 >>
According to a fourth aspect of the present invention, in the image change extraction method according to any one of the first to third aspects, a sample step, a search step, and a subdivided image obtained by dividing the images A and B into boundaries are provided. The coordination step, the repetition step, and the detection step are each performed.
[0014]
<< Claim 5 >>
According to a fifth aspect of the present invention, there is provided a program for causing a computer to execute a sample step, a search step, a coordination step, an iterative step, and a detection step according to any one of the first to fourth aspects.
[0015]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described.
[0016]
<< 1st Embodiment >>
The first embodiment is an embodiment corresponding to claims 1, 2, and 5.
FIG. 1 is a flowchart illustrating an image change extraction method (image processing program) executed on a computer. FIG. 2 is a diagram showing a state of the processing process.
Hereinafter, the image change extraction method will be described in the order of the step numbers shown in FIG.
[0017]
Step S1: The computer acquires two images A and B at different photographing times.
[0018]
Step S2: The computer extracts minute regions from the image A at predetermined intervals.
[0019]
Step S3: The computer creates a movement vector on the memory in association with each minute area. This movement vector is vector data having a micro area on the image A as a start point and a corresponding point on the image B as an end point. The computer initializes the elements of these motion vectors to (0,0).
[0020]
Step S4: The computer sets a search range on the image B near the end point of each movement vector. The computer searches for a corresponding portion that substantially matches the minute area from within the search range. For such a search, for example, a template matching method such as a residual sequential test method is used. The computer updates the movement vector for each minute area based on the search results.
[0021]
Step S5: The computer determines whether or not the group of movement vectors has converged from the update status of the movement vectors in step S4.
Here, when the movement vector group does not converge, the computer shifts the operation to step S6.
On the other hand, when the movement vector group has converged, the computer shifts the operation to step S7.
[0022]
Step S6: The computer sets a horizontally long flat block H (see FIG. 2) for each movement vector. The computer obtains the median of the movement vector for each flat block H, and increases the spatial continuity of the movement vector in the horizontal direction. After this operation, the computer returns the operation to Step S4. As a result, the operations of steps S4 to S6 are repeated until the movement vector group converges.
[0023]
Step S7: On the other hand, when the movement vector group has converged, the computer obtains a spatial differential (difference) in the vertical direction for the movement vector group. From this spatial derivative, the computer obtains a point where the movement vector discontinuously changes in the vertical direction (tomographic line F2 in FIG. 2).
[0024]
Step S8: The computer initializes the elements of the movement vector group to (0, 0).
[0025]
Step S9: On the image B, the computer sets a search range near the end point of each movement vector. The computer searches for a corresponding portion that substantially matches the minute area from within the search range. For such a search, for example, a residual sequential test method or the like is used. The computer updates the movement vector for each minute area based on the search results.
[0026]
Step S10: The computer determines whether or not the movement vector group has converged based on the update state of the movement vector in step S9.
Here, when the movement vector group does not converge, the computer shifts the operation to step S11.
On the other hand, when the movement vector group has converged, the computer shifts the operation to step S12.
[0027]
Step S11: The computer sets a vertically long flat block V (see FIG. 2) for each movement vector. The computer obtains the median of the movement vector for each flat block V, and increases the spatial continuity of the movement vector in the vertical direction. After this operation, the computer returns the operation to step S9. As a result, the operations of steps S9 to S11 are repeated until the movement vector group converges.
[0028]
Step S12: On the other hand, when the movement vector group has converged, the computer obtains a spatial differential (difference) in the movement vector group in the horizontal direction. From this spatial derivative, the computer obtains a position where the movement vector changes discontinuously in the horizontal direction (tomographic line F1 in FIG. 2).
[0029]
Step S13: The computer synthesizes the spatial derivatives obtained in steps S7 and S12, and specifies the position of the tomographic change.
[0030]
By the above-described series of operations, it is possible to obtain the location of the tomographic change occurring between the images A and B (tomographic lines F1 and F2 in FIG. 2).
Next, another embodiment will be described.
[0031]
<< 2nd Embodiment >>
The second embodiment is an embodiment corresponding to claims 3 and 5.
FIG. 3 is a flowchart illustrating an image change extraction method (image processing program) executed on a computer. FIG. 4 and FIG. 5A are conceptual diagrams showing the state of the processing process.
Hereinafter, the image change extraction method will be described in the order of the step numbers in FIG.
[0032]
Step S21: The computer acquires two images A and B at different photographing times.
[0033]
Step S22: The computer extracts minute regions from the image A at predetermined intervals.
[0034]
Step S23: The computer creates a movement vector on the memory in association with each minute area. This movement vector is vector data having a micro area on the image A as a start point and a corresponding point on the image B as an end point. The computer initializes the elements of these motion vectors to (0,0).
[0035]
Step S24: The computer sets a search range on the image B near the end point of each movement vector. The computer searches for a corresponding portion that substantially matches the minute area from within the search range. For such a search, for example, a template matching method such as a residual sequential test method is used. The computer updates the movement vector for each minute area based on the search results.
[0036]
Step S25: The computer determines whether or not the movement vector group has converged based on the update state of the movement vector in step S24.
Here, when the movement vector group does not converge, the computer shifts the operation to Step S26.
On the other hand, when the movement vector group has converged, the computer shifts the operation to Step S28.
[0037]
Step S26: As shown in FIG. 4, the computer executes clustering based on the XY component values of the movement vector. By this clustering, the movement vector group is divided into a plurality of sections (sections R1 and R2 shown in FIG. 5A). For the clustering here, for example, a K-means algorithm is used.
[0038]
Step S27: The computer obtains the median of the movement vector for each section. Note that the median processing here may be performed over the entire area of one section, or may be executed for each local block in the section. By such processing, the spatial continuity of the movement vector is independently enhanced for each section.
After such an operation, the computer returns the operation to step S24. As a result, the operations of steps S24 to S27 are repeated until the movement vector group converges.
[0039]
Step S28: On the other hand, when the movement vector group has converged, the computer obtains the spatial differentiation (difference) in the vertical and horizontal directions for the movement vector group. From this spatial derivative, the computer obtains a portion where the motion vector changes discontinuously (tomographic lines F3 and F4 shown in FIG. 5A).
[0040]
<< About the processing result of the second embodiment >>
6A to 6C show specific processing steps in the second embodiment.
FIG. 6A is an image A before a tomogram is generated. On the other hand, FIG. 6B is an image B after a tomographic image has been generated.
FIG. 6C is a diagram in which the above-described processing (the flowchart shown in FIG. 3) is performed on these images A and B, and a tomographic change is detected. From FIG. 6C, it can be seen that a rectangular tomographic change between the images A and B occurred at the lower left of the screen.
[0041]
<< Supplementary information of the embodiment >>
In the first and second embodiments, the movement vector group is smoothed by the median process. However, the present invention is not limited to this. For example, the movement vector group may be smoothed by averaging processing or majority processing.
[0042]
In the first and second embodiments described above, the case where the entire image is processed at one time has been described. However, the present invention is not limited to this. For example, as described in claim 4, the processing may be executed in units of subdivided images obtained by dividing the images A and B into boundaries.
[0043]
In the first embodiment, a tomographic change is detected using a vertically long or horizontally long flat block. However, the present invention is not limited to this. For example, a flat block inclined at an angle may be used.
[0044]
In the second embodiment, as shown in FIG. 5A, the case where the image is divided into two sections R1 and R2 has been described. However, the present invention is not limited to this. For example, as shown in FIG. 5B, it may be divided into three or more sections R1 to R3 by clustering. In this case, it is possible to detect a tomographic change more finely.
[0045]
In the second embodiment, clustering is performed based on the XY component values of the movement vector. However, the present invention is not limited to this. For example, clustering may be performed based on the polar coordinate values of the movement vector. In this case, clustering can be performed based on the direction or magnitude of the movement vector.
[0046]
【The invention's effect】
<< Claim 1 >>
According to the image change extraction method of the first aspect, the motion vector is smoothed in flat block units.
Usually, the contradiction (discontinuous change) of the movement vector caused by a mistake in the search step occurs only sporadically, and does not show a directionality like a change in tomography. Therefore, the inconsistency of the one-shot motion vector can be sufficiently eliminated by smoothing the flat block unit regardless of the uneven shape of the flat block.
On the other hand, in the tomographic portion, a discontinuous change occurs in the movement vector group at the tomographic line. Here, it is assumed that the long axis direction of the flat block substantially matches the direction of the tomographic line. At this time, the direction of the change of the movement vector group substantially coincides with the short axis direction of the flat block. Therefore, even if smoothing is performed in units of flat blocks mainly in the major axis direction, discontinuous changes on the tomographic line are hardly eliminated.
As described above, in the present invention, a single discontinuous change due to a search error is reduced, and a discontinuous change on a tomographic line extending in the longitudinal direction is well preserved.
Therefore, after determining the motion vector, by detecting discontinuous changes in the motion vector group in the short axis direction of the flat block, it is possible to accurately detect a tomographic change occurring between images.
[0047]
<< Claim 2 >>
In the image change extraction method according to the second aspect, a search step, a coordination step, a repetition step, and a detection step are executed for a plurality of types of flat blocks having different short-axis directions. As a result, it is possible to accurately detect a tomographic change running in a plurality of directions on the image space.
[0048]
<< Claim 3 >>
According to the third aspect of the present invention, the movement vector group is divided by clustering, and the movement vector group is independently smoothed for each of the divisions.
Usually, the inconsistency (discontinuous change) of the movement vector caused by a mistake in the search step occurs sporadically. Therefore, it can be sufficiently reduced by the smoothing process for each section.
On the other hand, when there is a tomographic change between images, the movement vector group changes at the tomographic line. Therefore, in the clustering described above, there is a high possibility that the movement vector group is divided by the tomographic line. Therefore, in the smoothing processing for each section, there is little possibility that the movement vector is smoothed over the tomographic line, and discontinuous changes on the tomographic line are hardly eliminated.
As described above, according to the present invention, a single discontinuous change due to a search error is reduced, and the discontinuous change on the tomographic line is well preserved.
Therefore, after determining the movement vector, by detecting the discontinuous change of the movement vector group, it is possible to accurately detect a tomographic change occurring between images.
[0049]
<< Claim 4 >>
In the image change extraction method according to the fourth aspect, a sample step, a search step, a coordination step, a repetition step, and a detection step are executed in units of subdivided images obtained by dividing the images A and B into boundaries.
In particular, when clustering is performed as in claim 3, it is possible to reduce the processing load of clustering by subdividing the processing units.
Furthermore, in the case of claim 3, the size of the area after clustering can be limited in advance by the size of the subdivided image. In this case, there is no possibility that the smoothing process is performed over a wide range more than necessary, and it is possible to appropriately correct the correspondence deviation between images by nonlinear mapping.
[0050]
<< Claim 5 >>
A program described in claim 5 causes a computer to execute a sample step, a search step, a coordination step, a repetition step, and a detection step described in any one of claims 1 to 4. Therefore, it is possible to execute the image change extraction method according to any one of claims 1 to 4 on a computer.
[Brief description of the drawings]
FIG. 1 is a flowchart illustrating an image change extraction method according to a first embodiment.
FIG. 2 is a diagram illustrating a processing process according to the first embodiment.
FIG. 3 is a flowchart illustrating an image change extraction method according to a second embodiment.
FIG. 4 is a diagram illustrating clustering according to a second embodiment.
FIG. 5 is a diagram illustrating a process of the second embodiment.
FIG. 6 is a photograph of a halftone image on a display for showing a processing result of the second embodiment.
[Explanation of symbols]
H Horizontal flat block V Vertical flat block R1 to R3 Regions F1 to F5 divided by clustering

Claims (5)

An image change extraction method for obtaining an image change for two input images A and B,
A sample step of extracting a plurality of minute regions from the image A;
A search step of respectively searching the image B for a corresponding location of the minute area to obtain a movement vector of the plurality of minute areas;
A coordination step of smoothing the group of the motion vectors obtained in the search step into units of flat blocks predetermined on an image space to increase the continuity of the motion vector group;
An iterative step of repeating the search step and the coordination step to increase the search accuracy of the corresponding portion, and determining the movement vector group;
Detecting a change in the short axis direction of the flat block with respect to the movement vector group determined in the repetition step, and obtaining a tomographic change between the images A and B. . The image change extraction method according to claim 1,
Executing the search step, the coordination step, the repetition step, and the detection step for each of the plurality of types of the flat blocks having different short-axis directions to detect a tomographic change between the images A and B in a plurality of directions. An image change extraction method characterized by the following. An image change extraction method for obtaining an image change for two input images A and B,
A sample step of extracting a plurality of minute regions from the image A;
A search step of respectively searching the image B for a corresponding location of the minute area to obtain a movement vector of the plurality of minute areas;
Coordinating the movement vector group obtained in the search step by clustering, smoothing the movement vector group for each of the sections, and increasing the continuity of the movement vector group;
An iterative step of repeating the search step and the coordination step to increase the search accuracy of the corresponding portion, and determining the movement vector group;
Detecting a spatial change with respect to the movement vector group determined in the iterative step, and obtaining a tomographic change between the images A and B. The image change extraction method according to any one of claims 1 to 3,
An image change extraction method, wherein each of the sample step, the search step, the coordination step, the repetition step, and the detection step is executed in units of subdivided images obtained by dividing the images A and B into boundaries. On the computer,
An image processing program for executing the sample step, the search step, the coordination step, the iterative step, and the detection step according to any one of claims 1 to 4.

Images