JP4560434B2 - Change region extraction method and program of the method - Google Patents

Description

  The present invention provides a change area extraction method and a change area extraction program for determining whether or not there is a change according to a pattern in an original image when detecting a change in the image in a monitoring camera, an image encoding step, or the like. In particular, after preliminarily defining the amount of movement of the pattern to be detected in the image, each point of the image between the two images is determined as three types: change, no change, and no change determination. A highly reliable region extraction is obtained.

  In the conventional method for determining whether or not there is a change between two images, the similarity between the images is calculated, and it is determined whether or not a change has been made by threshold processing using a preset threshold (for example, Patent Document 1, Patent). Reference 2).

In many cases, the sum of squares of pixel values of two images is subjected to threshold processing, and if it is large, it is determined that there is a change. Also, when using correlation values for similarity calculation, set a threshold for each image, such as determining that there is no change by setting another parameter in that there is no feature in the image on both images. A binarization process for determining only the region that has been performed is performed.
JP-A-5-94536 JP-A-11-120349

  In the first place, considering the comparison of two images, there is a case in which it is impossible to determine whether there is a change or not depending on the characteristics of the image. That is, there is a limit in judging whether or not there is a change by performing threshold processing. If parameters are set independently of image features, accurate change extraction is impossible. If the user makes settings and adjustments for each image, some improvement may be expected, but in principle there are still risks such as oversight and over-extraction.

  An object of the present invention is to provide a change region extraction method and a program for the method that solve the above-described problems.

  In the present invention, the presence or absence of movement of an object is determined by actively using local features of a captured image. At this time, the processing is performed using a threshold value quantitatively determined using the local feature amount of the image. In particular, the region in which movement cannot be determined in principle is identified, and the third category of change, no change, and determination is not provided, thereby improving the reliability of the result.

  As a result, when two images are compared to determine whether or not there is a change, it is possible to make three types of determinations that include the determination that there is no change or not, instead of two types of determination that the change has not occurred. Realization of high change area extraction.

  In addition, it has not been determined that identification is impossible until now, and thus it is impossible to estimate a moving area with high accuracy. However, if the result of the present invention is used, by using another image processing, there is a change with the unidentifiable region, the positional relationship between the regions without change is analyzed, and the region determined to be an unidentifiable region, It is also possible to determine whether there is a change.

  Further, the user or the like does not set a parameter serving as a threshold according to the image, but can only give a movement amount on the image whose change is desired to be detected in advance. This realizes easy-to-use change area extraction in which no other parameters are required.

  As described above, the present invention is characterized by the following method and program.

(1) A changed region extraction method for comparing two images of the same size and extracting a changed region and a non-changed region,
The apparent prescribed movement amount to be detected as the movement on the image in advance is preset, and the movement similarity when the object photographed in the first image moves the prescribed movement amount is the maximum value at each point. A moving similarity map creation step obtained as a pair value of (best value) and minimum value (worst value);
An image similarity map calculation step for obtaining the similarity of each corresponding point of the two images by the same processing method as the calculation of the movement similarity;
At each point of the image, the image similarity obtained in the image similarity map calculation step is in any range of the threshold with the maximum value and the minimum value obtained in the moving similarity map creation step as threshold values. The movement presence / absence determination step for classifying into three categories, which are similar, unsimilar, or unspecified,
A change area extraction method characterized by comprising:

  (2) In the above (1), the movement similarity map creating step sets a small area including each point of the image, and obtains the movement similarity while shifting the predetermined movement amount around the center point of the small area. The change area extraction method characterized by this.

(3) In (1) above, the similarity is
Calculating the gradient of the image after cutting off the high frequency components of the image, and then calculating the correlation value between the images;
Obtaining a similarity map by calculating two eigenvalues and eigenvectors obtained by calculating a second derivative (Hessian: 2 × 2 matrix) after cutting high-frequency components of the image at each point;
And a change area extraction method characterized by comprising:

  (4) The change region extraction method according to (3), wherein the cut parameter of the high frequency component in the similarity calculation is changed according to the specified movement amount.

(5) A change area extraction program for causing a computer to execute the change area extraction method according to any one of (1) to (4) above .

  As described above, according to the present invention, by determining a movement amount to be determined in advance, when comparing two images, the movement of the pattern is larger or smaller than the prescribed movement amount, or depending on the characteristics of the image. Classifies each point of the image into three types that cannot be determined. For this reason, since there is no parameter setting that is difficult to interpret given by a user or the like, change extraction can be performed automatically and with high accuracy.

  In an area where change cannot be determined, it is possible to divide the area with high accuracy by adopting the result of having determined whether there is a change in the vicinity within a consistent range.

  The procedure of the change area extraction method corresponding to claim 1 and the like will be described with reference to FIG. In the figure, reference numerals 1 and 2 denote images to be subjected to change area extraction. Reference numeral 3 is a prescribed movement amount setting unit, 4 is a movement similarity map of image 1, 5 is an image similarity map calculated from images 1 and 2, and 6 is a ternary image storage unit, which is set and stored. Information is reserved in a computer storage device (memory, hard disk, etc.).

  S1 is a movement similarity map creation step, S2 is an image similarity map calculation step, and S3 is a movement presence / absence determination step. These are mounted on a computer with a software configuration described in a program, and can be executed sequentially. deep.

  Image 1 and image 2 are images having the same resolution (number of pixels) given in advance. The specified movement amount set by the setting unit 3 is a length in units of pixels, and is prepared in advance as a movement amount of an object to be extracted. This may be given by a user or a setter, or may be fixed as a program. For example, the movement amount to be detected is defined in advance, for example, when there is a movement of 2 dots or more, it is determined that “movement is present”.

  The movement similarity map creation step S1 obtains, as map data, how the similarity of the image changes when the pattern of each point of the image 1 moves to a point in the vicinity of the range of the specified movement amount. This result is accumulated in the movement similarity map 4 as values of two pairs of a maximum value and a minimum value at each point.

  The image similarity calculation map S2 calculates the similarity at each point at the same position by using two images of the image 1 and the image 2. Here, the similarity calculation method is a processing method that coincides with the movement similarity map creation step S1. The result is accumulated in the image similarity map 5.

  In the movement presence / absence determination step S3, the movement similarity map 4 and the image similarity map 5 are compared to determine whether each point has changed, has not changed, or whether the change cannot be determined. It is determined that there is no change when the similarity is high, and there is a change when the similarity is low. As the determination criterion at this time, the values of two pairs of the maximum value and the minimum value accumulated in the movement similarity map 4 are used.

  The ternary image storage unit 6 stores the image in which each point of the image is classified into three categories as a result of determination in the movement presence / absence determination step S3. One is that it is determined that the movement amount is equal to or less than the predetermined amount, that is, there is no change. The next point is that the movement amount is determined to be equal to or greater than the prescribed movement amount. Others are areas where it is impossible to determine whether or not there is movement.

  Of the above steps, a specific method for realizing the movement similarity map creation step S1 will be described in detail. Since the movement similarity map creation step S1 depends on the similarity calculation method, it is always the same as the similarity calculation method in the image similarity map calculation step S2.

  In the movement similarity map creation step S1, the specified movement amount of the setting unit 3 in which the movement amount range to be determined that the photographed object has moved is used. Hereinafter, this value is represented by d. The prescribed movement amount d here is not the movement amount in the actual scene but the length in units of pixels in the image. This value need not be an integer, but is a positive value. Zero is basically not specified because of the influence of image noise.

  A first example of a method for creating the movement similarity map 4 will be described. This method corresponds to claim 2 and the like.

  First, an image is expressed as g (x, y). Among these, one point (x, y) on the image is selected. A small area is set around this point. Although the size of the small area is not specified, it is a condition that the similarity is 3 × 3 or more that can be calculated. A correlation value is calculated by moving the small region by an amount corresponding to the prescribed movement amount d around the center point (x, y) of the small region. Naturally, the degree of similarity is maximum (for example, 1 for the correlation value and zero for the sum of squared differences) at the position (center point) that matches the original image. When moved by the specified amount of movement d, the center position is shifted and the images do not match. Therefore, the similarity usually decreases (a value of 1 or less for the correlation value and a value of 0 or more for the sum of squared differences).

  In this way, the degree of similarity is calculated at all points, limited to the prescribed movement amount d (that is, the equidistant position with the center point as the center). Then, the best value M (x, y) and the worst value m (x, y) of the similarity are calculated. As described above, the movement similarity map is obtained by calculating the two values of the best value and the worst value for each point. Therefore, the movement similarity map 4 stores two values for each point. This is expressed as, for example, T (x, y) = (m (x, y), M (x, y)).

  FIG. 2 shows three types of results when the similarity is calculated around the center point, and the brighter the higher the similarity. This is an example in which the degree of similarity is different depending on the characteristics of the small area. A is an example in which the degree of similarity greatly decreases even with a slight shift. This is generally seen as a feature point. B is an example in which the decrease in similarity due to deviation is small. This is seen when the feature of the center point is not remarkable. C is an example in which the degree of similarity decrease greatly differs depending on the direction of deviation. This occurs, for example, when the center point has a vertically long feature. For this reason, the decrease in similarity is large in the movement in the horizontal direction, and the decrease in similarity is small in the vertical direction.

  In these examples, the best value M (x, y) and the worst value m (x, y) are both small for A, large for B, and the best value M (x, y) for C. The difference between the worst values m (x, y) increases (when the similarity is higher as the similarity is higher).

  Note that a discussion of noise superimposed on an actual image may be added to the above discussion. Specifically, when calculating the similarity by shifting a small area, a method of adding the assumed noise to the image in advance and performing the same calculation can be considered. Also, assuming the amount of noise given to the degree of similarity, the worst value m (x, y) is reduced in advance by a predetermined numerical value, and the best value M (x, y) is determined in advance (the same as described above). It is also possible to specify a larger size.

  Next, a second example of a method for creating a movement similarity map will be described. This method corresponds to claim 3 and the like. In claim 3 and the like, the similarity calculation method is fixed. In this method, smoothing and gradient calculation are performed as preprocessing on images, and then correlation values between images are calculated. Here, the degree of smoothing can be determined by one parameter. For example, a convolution process using a Gaussian function with σ as a parameter can be used. Hereinafter, the similarity calculation method will be described.

Two images are denoted by g ₁ (x, y) and g ₂ (x, y). In this case, the result of the convolution processing and the gradient calculation by the Gaussian function ▽ Gσ using σ as a parameter is ▽ Gσ * g ₁ (x, y). Correlation values within a small region of ＧGσ * g ₁ (x, y) and ▽ Gσ * g ₂ (x, y) are calculated according to the following formula, and this correlation value becomes the similarity between images.

  Next, in order to calculate the movement similarity map, an integral value in the same region of the second derivative (Hessian: 2 × 2 matrix) after cutting the high frequency component of the Hessian image is calculated by the following formula.

Since this Hessian (H) is originally a symmetric matrix, two eigenvalues (λ ₁ , λ ₂ ) can be calculated (λ ₁ = λ ₂ , but there are two orthogonal eigenvectors, so 2 Two eigenvectors). Actually, it can be calculated by the following formula.

  Here, trH is a trace of H (diagonal sum of matrices), and detH is a determinant.

  Next, for i = 1, 2, the following calculation is performed.

  Of these two values, the larger one is the best value M (x, y) of the similarity, and the smaller one is the worst value m (x, y). Here, d is the prescribed movement amount as described above.

Furthermore, as in the above example, noise superimposed on an actual image may be assumed. However, in this case, since the similarity is not calculated by actually moving the small area, an expression that assumes the amount of noise is used. Specifically, N ₁ and N ₂ are fixed values indicating the amount of noise, and the larger one of the values expressed by the following equations is the best value M (x, y), and the smaller one is the worst value m (x, y).

In the above, it is assumed that the processing is performed in a small area, but the size of the area is actually free. Furthermore, for example, a method of performing calculation by enlarging the region and increasing the weight of the central portion can be considered. If | で あ れ ば Gσ * g ₁ (x, y) | ≠ 0, the small area can be considered as one point and processed.

Here, the magnitude of σ can be freely set independently of d. However, it is meaningless if only d is set large with σ fixed, and the worst value m (x, y) becomes smaller than −1 (because the correlation value is always larger than −1). In that case, a method of increasing σ by using the fact that λ _i becomes smaller depending on σ as the smoothing of the image proceeds. For example, it is conceivable to set σ to the same value as the prescribed movement amount d or a value of a constant multiple such as twice d, or the square of d (corresponding to claim 4). The results and effects in these will vary depending on how they are determined, but the method is not specified.

  When σ is large, the same effect can be obtained by reducing the image in advance by a constant value and reducing σ and d by the same ratio. For example, the image can be reduced to 1 / σ, and d and σ can be set to 1 / σ.

Next, in the image similarity map calculation step S2, the same method as the similarity calculation method used in the movement similarity map creation step S1 is used, and the images 1, 2 (g ₁ (x, y), g ₂ (x , Y)). This value is denoted as an image similarity map R (x, y).

  In the movement presence / absence determination step S3, the presence / absence of movement or inability to determine is determined according to the movement similarity map T and the image similarity map R. Specifically, for example, it is assumed that the similarity value R (x, y), such as a correlation value, shows a larger value as it is similar. If C (x, y) is a ternary image, it can be defined by the following equation. Note that the numerical value of C (x, y) = 1, 0, −1 has no meaning and is merely a value for classification.

  In this example, if C (x, y) = 1, the motion is d or less, if −1, the motion is detected, and if 0, the interpretation is impossible. This result is stored in the ternary image storage unit 6. The magnitudes of the best value M (x, y) and the worst value m (x, y) depend on the feature of the point.

  For example, when the image pattern is a small point “•”, the similarity is greatly reduced regardless of the direction. On the other hand, in the edge portion of the pattern, the similarity decrease is large in the vertical direction of the edge, and the similarity decrease is small in the edge direction. The non-characteristic point has little decrease in similarity regardless of the direction of movement. In the movement similarity map, such features are calculated in advance. The probability that the determination is impossible at a point where the best value M (x, y) -worst value m (x, y) is small is low, and the probability is high at a point where the best value M (x, y) is large.

  In the present invention, the case where two images 1 and 2 are processed has been described. These images may be time-series images (two frames after the moving image) that are close in time but different. Further, when it is desired to make the specified movement amount d very small, the same processing may be performed by increasing the specified movement amount d and at the same time extending the time interval by the ratio. The image 1 may be an image separated in time, such as a past background image and the image 2 a current image. It is also possible to prepare a new third image and perform the same processing on the image 2 and the third image to increase the accuracy.

  Since the resulting ternary image does not determine the presence or absence of a change by simple threshold processing, it is also possible to collect a change area and a non-determinable area that follows this as a change area. Is possible.

  In the present invention, some or all of the processing functions of the method shown in FIG. 1 can be configured as a program and executed by a computer.

The procedure figure of the change area | region extraction method which shows embodiment of this invention. The example of the similarity map at the time of calculating a similarity degree centering on the center point in embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image 2 Image 3 Specified movement amount setting part 4 Movement similarity map 5 Image similarity map 6 Three-value image memory | storage part S1 Movement similarity map creation step S2 Image similarity map calculation step S3 Movement presence determination step

Claims (5)

A change region extraction method for comparing two images of the same size and extracting a changed region and a non-changed region,
The apparent prescribed movement amount to be detected as the movement on the image in advance is preset, and the movement similarity when the object photographed in the first image moves the prescribed movement amount is the maximum value at each point. A moving similarity map creation step obtained as a pair value of (best value) and minimum value (worst value);
An image similarity map calculation step for obtaining the similarity of each corresponding point of the two images by the same processing method as the calculation of the movement similarity;
At each point of the image, the image similarity obtained in the image similarity map calculation step is in any range of the threshold with the maximum value and the minimum value obtained in the moving similarity map creation step as threshold values. The movement presence / absence determination step for classifying into three categories, which are similar, unsimilar, or unspecified,
A change area extraction method characterized by comprising:   2. The movement similarity map creating step according to claim 1, wherein the movement similarity map creating step sets a small area including each point of the image, and obtains the movement similarity degree while shifting the specified movement amount around the center point of the small area. Change area extraction method. In claim 1, the similarity is:
Calculating the gradient of the image after cutting off the high frequency components of the image, and then calculating the correlation value between the images;
Obtaining a similarity map by calculating two eigenvalues and eigenvectors obtained by calculating a second derivative (Hessian: 2 × 2 matrix) after cutting high-frequency components of the image at each point;
And a change area extraction method characterized by comprising:   4. The change area extraction method according to claim 3, wherein the cut parameter of the high frequency component in the similarity calculation is changed according to the specified movement amount. The change area extraction program for making a computer perform the change area extraction method of any one of the said Claims 1-4.