JP2914441B2 - Motion detection method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion detecting method used for encoding a moving image, and more particularly to a motion detecting method for detecting a macroscopic motion of a specific subject in an image or the entire image with a small amount of calculation. About the method.

[0002]

2. Description of the Related Art Conventionally, a block matching method has been widely known as a method for detecting a motion of a moving image. This method is a method in which an image for which motion is to be detected is divided into small blocks, block matching processing is performed on the reference image for each small block, and motion is detected from a coordinate position where an error is minimized. Such a block matching method is widely used in the IS117172-2 standard internationalized by ISO / IECJTC1 / SC29 / WG11.

[0003]

However, the block matching process generally requires a large amount of calculation. In particular, when there is a large motion in an image due to a camera operation (viewpoint movement) or the like, the search range for performing block matching needs to be widened, so that there is a problem that a larger amount of calculation is required.

As a method of detecting the movement of the entire image due to a camera operation or the like, there is a method described in Japanese Patent Application Laid-Open No. Hei 4-3595. However, even in this method, first, a motion vector is detected for each small block by performing a block matching process, and a motion parameter characterizing the motion of the entire image is calculated by an arithmetic operation from a large number of detected motion vectors. It is. That is,
Also in this method, there is a problem that a large amount of calculation is required because the block matching process is used to detect the motion of the entire image.

As another method for detecting the motion of the entire image, there is a method described in Japanese Patent Laid-Open No. 6-96210. In this method, first, many contour components in an image are detected. Next, a large number of detected contours are put together, and the motion of the entire image is detected by neural network processing.

However, the detection of the shape and position of the contour is based on a change in the positional relationship between the objects, deformation of the object itself,
Alternatively, there is a problem that an error is likely to occur due to the influence of noise or the like of the imaging system, and the detection accuracy is likely to deteriorate.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a motion detecting method capable of accurately detecting a macroscopic motion of a specific subject in an image or the entire image with a small amount of calculation.

[0008]

According to the present invention, in a motion detecting method for detecting a motion between frames of a specific subject in a moving image, a current image frame and a reference image frame having a time difference therefrom are provided. and of extracting the region of the specific subject from the respective weight of the extracted subject region
The center position, the inclination of the main axis, and the number of pixels are detected as macroscopic feature amounts , respectively, and based on the amount of change in the macroscopic feature amount between the current image frame and the reference image frame. A motion detection method is provided, wherein a motion parameter characterizing the motion of the subject area is calculated.

Further, according to the present invention, in a motion detecting method for detecting a motion between frames of an entire image of a moving image, a method for identifying a current image frame and a reference image frame having a time difference from the current image frame is provided. Each of the above areas is extracted , and the position of the center of gravity, the inclination of the main axis, and the image
Respectively detect the prime as macroscopic feature amounts, to calculate the motion parameters characterizing the entire image of the motion based on the change amount of the macroscopic feature amounts at between the current image frame and the reference image frame Thus, a motion detection method characterized by the following is obtained.

Still further, according to the present invention, in a motion detecting method for detecting a motion between frames of a moving image, a method which can be identified from each of a current image frame and a reference image frame having a time difference from the current image frame. The above area is extracted , and the position of the center of gravity of the extracted area , the inclination of the main axis, and the number of pixels
Detecting respectively the macroscopic feature amounts, the based on the change amount of the macroscopic feature amounts between the current image frame and the reference image frame, calculating a motion parameter characterizing the entire image of the motion, the The current image frame is divided into small blocks, a motion amount indicating a rough motion of each small block is calculated based on the motion parameter, and the motion of each small block is set to the motion amount as an initial value. A motion detection method characterized by performing detection by processing is obtained.

Further, according to the present invention, in an image processing apparatus for detecting a motion of a specific subject between frames in a moving image, the present invention is characterized in that a current image frame and a reference image frame having a time difference from the current image frame are used. Area extracting means for extracting an area of a specific subject, and a center of gravity position, a principal axis inclination, and a pixel of the subject area extracted by the area extracting means
Macroscopic feature amount detecting means for detecting a number as a macroscopic feature amount , and the subject based on a change amount of the macroscopic feature amount between the current image frame and the reference image frame. And a motion parameter calculating means for calculating a motion parameter characterizing the motion of the area.

Further, according to the present invention, in an image processing apparatus for detecting motion between frames of an entire image of a moving image, one image that can be identified from a current image frame and a reference image frame having a time difference from the current image frame. One or more of the area extracting means region is extracted, respectively, and macroscopic characteristic quantity detecting means the position of the center of gravity of the region where the region extraction unit and extracted, the inclination of the main shaft, and which detect the number of pixels as the macroscopic feature amounts,
Motion parameter calculation means for calculating a motion parameter that characterizes the motion of the entire image based on the amount of change in the macroscopic feature amount between the current image frame and the reference image frame. Thus, an image processing apparatus can be obtained.

[0013] In addition, according to the present invention, in an image processing apparatus for detecting motion between frames of a moving image, one image that can be identified from each of a current image frame and a reference image frame having a time difference from the current image frame. Region extracting means for extracting the above region, and the center of gravity of the region extracted by the region extracting means
Position, wherein between the inclination of the main shaft, and a macroscopic feature amount detecting means for detecting, respectively Re their <br/> the number of pixels as the macroscopic feature amounts, the current image frame and the reference image frame macroscopic Parameter calculation means for calculating a motion parameter characterizing the motion of the entire image based on the change amount of the statistical feature amount,
A block in which the current image frame is divided into small blocks, a motion amount indicating a rough motion of each small block is calculated based on the motion parameter, and the motion of each small block is set to the motion amount as an initial value. Image processing comprising: block matching processing means for detecting by block matching processing detected by matching processing.
A device is obtained.

[0014]

Embodiments of the present invention will now be described with reference to the drawings.

The image processing apparatus according to the present embodiment is capable of processing a subject area or a predetermined area extracted from an image in advance.
A macroscopic feature amount is detected for each region, and motion of an image is detected based on a change in the feature amount between frames. Hereinafter, the operation of the image processing apparatus according to the embodiment of the present invention will be described with reference to FIG.

Here, as shown in FIG. 1, an image of a certain frame is used as a reference image, and the motion of the input image one frame after that is detected. In the reference image and the input image, a subject area is detected in advance. Then, a macroscopic feature amount is obtained for each of the detected subject regions.

As the macroscopic feature in the subject area, the barycentric coordinate position Gi of the area Ri (i: area number) =
(Xgi, Ygi), the inclination θi of the principal axis of the region Ri,
The size of the region Ri (ie, the number of pixels Si) can be used. For example, in the example of FIG. 1, for each of the regions 1 and 2, G1r = (Xg1r, Yg
1r), θ1r, S1r, and G2r = (Xg2r, Yg2r), θ
2r and S2r are G1c = (Xg1c, Yg1) on the input image.
c), θ1c, S1c, and G2c = (Xg2c, Yg2c), θ2
c and S2c are respectively obtained as macroscopic feature amounts.

These feature amounts can be obtained by a simple calculation relating to the coordinate positions of all the pixels belonging to the region of interest. Also, since these feature values are macroscopic values, they are less susceptible to erroneous detection and the like than local and fine feature values such as the contour of the subject. Therefore, even when the subject itself has some deformation or the imaging system has some bad influence such as noise, it is possible to easily and accurately detect these feature amounts.

Next, a motion parameter that characterizes the motion of the entire region is obtained for each region based on the amount of change in the feature amount between frames. That is, as a motion vector indicating the parallel movement amount of the entire area, the change amount ΔGi = (Ei,
Fi), the change amount Q of the principal axis inclination θ indicating the angle of rotation of the entire area, and the square root Z of the change rate of the size S of the area indicating the enlargement / reduction rate of the entire area. In the case of FIG.
(E1, F1) = (Xg1c-Xg1r, Yg1c-Yg1
r), (E2, F2) = (Xg2c-Xg2r, Yg2c-Y
g2r), Q1 = θ1c−θ1r, Q2 = θ2c−θ2r, and Z
1 = √ (S1c / S1r) and Z2 = √ (S2c / S2r) are obtained.

The motion vector (Vcx, Vcy) indicating the movement from the reference image to the input image at an arbitrary pixel position in each area is obtained by setting the arbitrary pixel position to (Xc, Yc) and setting the motion parameter to It can be expressed as in the following Expressions 1 and 2.

[0021]

(Equation 1)

[0022]

(Equation 2) In the above formulas 1 and 2, (Hi, Vi) includes a term for correcting the amount of parallel movement of each region caused by scaling and rotation of each region. When these formulas are arranged, they are expressed as formulas 3 and 4, respectively.

[0023]

(Equation 3)

[0024]

(Equation 4) Further, Equations 3 and 4 can also be expressed as Equations 5 and 6.

[0025]

(Equation 5)

[0026]

(Equation 6) Even when the entire image moves uniformly due to a camera operation or the like, the motion parameters (Z, Q, E, F) or (A, B, C, D, E,
F) can be obtained by calculation. In this case, one or more regions are extracted from the image, macroscopic feature amounts are detected from each of the regions, and motion parameters are calculated. Then, the motion parameters (Zi,
Qi, Ei, Fi) or (Ai, Bi, Ci, Di,
Ei, Fi) are arithmetically averaged or weighted arithmetically averaged for each element, and motion parameters are calculated for the entire image. Here, in the case of performing weighting, it is sufficient to increase a parameter having high reliability of parameter detection, such as an area having a large number of pixels. When detecting the motion parameters of the entire image by this method, the detection accuracy is improved by using a plurality of regions rather than by using one region.

In the case where the whole image is largely moving due to a camera operation or the like, and there is a portion which locally moves, the following operation may be performed.

That is, first, as described above, the motion parameters (Z, Q, E, F) or (A, B,
C, D, E, F) are detected. Next, the image is divided into small blocks, and a motion vector is detected for each small block by block matching processing. At this time, the initial value of the search range of the block matching is calculated from the coordinate position (Xc, Yc) of the small block by using Equations 3 and 4 or Equations 5 and 6.
Is determined as the value (Vcx, Vcy) of the motion vector obtained in step (1). By determining the search range in this way, rough motion of the entire image can be compensated in advance within the range of the initial value, so that accurate motion detection can be realized even if the range of block matching is reduced.

[0029]

Next, a first embodiment of the present invention will be described with reference to FIG.

First, in step 201, a subject area is extracted from a moving image. A conventional method can be used as a method for extracting a region. For example, a Laplacian process is performed on the luminance component of the image, and a zero crossing point of the processed image is detected as a region boundary line. Then, the detected boundary line may be traced, and a portion closed by the continuous boundary line may be extracted as an independent region. Also, as another method of region extraction, a method using limited color expression is widely known. In this method, an input color image is analyzed, and a continuous portion having a small hue change is replaced with a representative color and extracted as one region. If the input moving image has been divided into regions in advance, step 201 can be omitted.

Next, in step 202, the macroscopic feature of the extracted area is detected. As the macroscopic feature of each area, the center of gravity of the area Gi = (Xgi, Ygi
), The inclination θi of the main axis of the area, the size of the area (number of pixels)
Use Si or the like. The center of gravity position Gi can be easily obtained by arithmetically averaging the coordinate positions of all the pixels belonging to the attention area. As the main axis of the area, a linear axis that passes through the center of gravity of the area and that has the smallest moment about that axis is selected. This is also obtained by arithmetic calculation on the coordinate positions of all the pixels belonging to the area. Furthermore, the inclination θi of the main axis is an axis predetermined with respect to the screen,
For example, it may be defined as a tilt with respect to the vertical axis. In addition, other than these, the length of the principal axis of the region can be used as a feature amount indicating the size of the region as the macroscopic feature amount, and the magnitude of the moment around the principal axis of the region can be used as the feature amount for identifying the region.

Next, in step 203, step 20
The change amount between the frames of the macroscopic feature amount detected in 2 is detected. The amount of change (E, F) of the center of gravity Gi = (Xgi, Ygi) of each region is a motion vector indicating the amount of parallel movement of the entire region. The variation Q of the inclination θi of the main shaft is
The rotation angle of the entire area is shown. The square root Z of the rate of change of the number of pixels Si in the region indicates the scale of the size of the region.

Finally, in step 204, the detected variation (Z, Q, E, F) is output as a motion parameter of the area. By using this parameter, the value of the motion vector can be obtained by calculation at an arbitrary pixel position in the area. The calculation of the motion vector is obtained according to the above-described equations 3 and 4. Alternatively, the motion parameters (A, B, C, D,
E, F).

Next, a second embodiment of the present invention will be described with reference to FIG.

First, in step 301, a plurality of regions Ri (identifiable regions) are extracted in the same manner as in step 201 of FIG. And steps 302 and 30
3, in the same manner as in steps 202 and 203 of FIG.
The motion parameters (Zi,
Qi, Ei, Fi) are detected.

Next, at step 304, the motion parameters (Zi, Qi, Ei, Fi) detected for each region Ri.
An arithmetic average is calculated for each of the elements to calculate a motion parameter (Z, Q, E, F) of the entire image. At this time, weighting may be performed based on the reliability of the motion parameter for each region to take an arithmetic average.

Then, in step 305, the obtained motion parameters of the entire image are output.

Next, a third embodiment of the present invention will be described with reference to FIGS.

Steps 401 to 4 of this embodiment
Until 05, steps 301 to 305 in FIG.
This is the same process as above. Thereafter, in this embodiment, in step 406, the input image is divided into small blocks. Then, a rough motion vector is calculated for each block based on the motion parameters obtained in step 405. This calculation is performed according to Equations 3 and 4 or Equations 5 and 6.

Then, in step 407, a block matching process is performed for each small block using the motion vector obtained by the above calculation as an initial value, and a more accurate motion vector is searched. At this time, by performing the initial value setting as described above, since the rough motion as the entire image has been searched, the search range in the block matching process can be reduced, and the amount of calculation is reduced as compared with the conventional case. can do.

[0041]

According to the present invention, a macroscopic feature is detected from an input image and a reference image, respectively, and a motion parameter is calculated based on a change in the macroscopic feature between frames. Thus, the motion of the image can be detected with a small amount of calculation.

Also, by using the motion parameters obtained based on the amount of change in the macroscopic feature amount for block matching, the search range can be narrowed without deteriorating the accuracy of motion detection and reducing the amount of calculation. Can be reduced.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the operation of an embodiment of the present invention.

FIG. 2 is a flowchart of the first embodiment of the present invention.

FIG. 3 is a flowchart of a second embodiment of the present invention.

FIG. 4 is a flowchart of a third embodiment of the present invention.

FIG. 5 is a diagram for explaining the flowchart of FIG. 4;

Claims (6)

(57) [Claims] 1. A motion detecting method for detecting a motion of a specific subject between frames in a moving image, wherein a region of the specific subject is determined from each of a current image frame and a reference image frame having a time difference from the current image frame. The position of the center of gravity of the extracted subject area , the inclination of the main axis, and the number of pixels
As the respectively detected as a macroscopic feature amounts, calculates a motion parameter characterizing the macroscopic feature amounts of variation of the movement area of the object based on the between the current image frame and the reference image frame A motion detection method characterized in that: 2. A motion detection method for detecting motion between frames of an entire image of a moving image, wherein one or more identifiable regions are respectively extracted from a current image frame and a reference image frame having a time difference from the current image frame. Of the extracted area
The center of gravity position, the inclination of the main shaft, and the number of pixels and the macroscopic feature amounts
Respectively detected Te, and characterized in that to calculate the motion parameters characterizing the entire image of the motion based on the change amount of the macroscopic feature amounts at between the current image frame and the reference image frame Motion detection method. 3. A motion detection method for detecting motion between frames of a moving image, wherein one or more identifiable regions are extracted from each of a current image frame and a reference image frame having a time difference from the current image frame. Centroid position of extracted area
Group location, the inclination of the main shaft, and it <br/> respectively detect the number of pixels as the macroscopic feature amounts, the amount of change in the macroscopic feature amounts at between the current image frame and the reference image frame Calculating a motion parameter characterizing the motion of the entire image, dividing the current image frame into small blocks, and calculating a motion amount indicating a rough motion of each small block based on the motion parameter; A motion detection method wherein a motion of each block is detected by a block matching process using the motion amount as an initial value. 4. An image processing apparatus for detecting a motion of a specific subject between frames in a moving image, wherein an area of the specific subject is determined from each of a current image frame and a reference image frame having a time difference from the current image frame. Region extracting means for extracting, and a subject area extracted by the region extracting means.
The center of gravity position, the inclination of the main shaft, and the number of pixels and the macroscopic feature amounts
Calculating a macroscopic feature amount detecting means for detecting each of the motion parameters characterizing the movement of areas of the subject of change based on the macroscopic feature amounts between the current image frame and the reference image frame Te An image processing apparatus comprising: 5. An image processing apparatus for detecting movement between frames of an entire image of a moving image, wherein one or more identifiable regions are respectively extracted from a current image frame and a reference image frame having a time difference from the current image frame. Region extracting means, and the position of the center of gravity of the region extracted by the region extracting means ,
A macroscopic feature amount detecting means for detecting the inclination and the number of pixels as macroscopic feature amounts , respectively , and the image based on a change amount of the macroscopic feature amount between the current image frame and the reference image frame. An image processing apparatus comprising : a motion parameter calculating unit configured to calculate a motion parameter characterizing the entire motion. 6. An image processing apparatus for detecting a motion between frames of a moving image, wherein one or more identifiable regions are extracted from each of a current image frame and a reference image frame having a time difference therefrom. Extraction means, the position of the center of gravity of the area extracted by the area extraction means , the inclination of the main axis, and
Macroscopic feature amount detecting means for detecting the number of pixels as a macroscopic feature amount, and a motion of the entire image based on a change amount of the macroscopic feature amount between the current image frame and the reference image frame. A motion parameter calculation unit that calculates a motion parameter characterizing the current image frame; and divides the current image frame into small blocks, calculates a motion amount indicating a rough motion of each small block based on the motion parameter, and An image processing apparatus comprising: a block matching processing unit configured to detect a motion of each image by a block matching process that detects the motion amount as an initial value by a block matching process.