JPH0620305B2 - Motion vector detector - Google Patents

Description

Description: TECHNICAL FIELD OF THE INVENTION The present invention relates to a motion vector detection device for detecting a translation amount (motion vector) of an entire image within a screen.

[Technical background of the invention and its problems]

Generally, the movement of an image on a screen such as a television screen is roughly classified into movement of an object in the image and movement of the camera in parallel (panning). The former is the local movement of the image, while the latter is the movement of the entire image while maintaining the mutual relationship. The latter amount of translation (direction and size) of the entire image is called a motion vector. That is, the image on the screen in FIG.
When parallel movement is made in the direction of the arrow as shown in (b) to (c),
The motion vector is given by the deviations (ξ, η) in the horizontal and vertical directions as shown in FIG. Detecting this motion vector, using this, for example, by shifting and superimposing the image of the previous frame on the image of the current frame, the actual motion between the image obtained by superimposing and the image of the previous frame can be calculated. It can be made smaller, resulting in higher quality images with less blur. Further, when the image signal is encoded and transmitted, the fact that the motion of the image superimposed by using the motion vector becomes small means that the predictive coding is performed by utilizing the time correlation of the images between two consecutive frames. This means that the prediction error at that time becomes small, which can contribute to the improvement of coding efficiency.

However, the actual movement of the image is the parallel movement of the whole image,
Since the motions of the object in the image are mixed, there is a problem that the motion vector is erroneously detected due to the motion of the object. That is, the conventional motion vector detection method specifically performs a correlation calculation of image information between two temporally continuous screens, and among the correlation values obtained thereby, a correlation value having a relatively high correlation is obtained. And a method of calculating a motion vector from the shift between two points giving the correlation value. In this method, since the deviation that gives the highest correlation value than the predetermined motion vector detection range is detected as the motion vector, there is no means for judging the accuracy of the obtained motion vector. Generally, when the image is a still image or close to it, the motion vector is detected as zero. However, even if there is no translation of the camera, if the proportion of the movement of the object in the image on the screen increases,
A shift other than zero may have a high correlation, and if such a shift is also used as a motion vector, false detection will result.

[Object of the Invention]

An object of the present invention is to provide a motion vector detection device that can reduce erroneous detection of a motion vector due to the motion of an object in an image.

[Outline of Invention]

In order to achieve the above-mentioned object, the present invention is continuous in time.
In a motion vector detection device that detects a motion vector indicating the amount of parallel movement of the entire image from the image information of the frame, the correlation calculation of the image information between the screens of two temporally consecutive frames is performed, and the first correlation value is calculated. Between the first means for outputting and the second correlation value having the highest correlation among the plurality of first correlation values obtained by the first means and the two points giving the second correlation value. Effectiveness of the correlation between the second means for obtaining the deviation and the second correlation value obtained by the second means.
Third determination of invalidity by referring to the plurality of first correlation values
And when the second correlation value is determined to be invalid by the third means, the motion vector detection output is set to zero, and when the second correlation value is determined to be valid, it is obtained by the second means. And a fourth means for detecting and outputting the shift.

〔The invention's effect〕

According to the present invention, the correlation value having the highest correlation obtained by the second means is determined to be invalid for the local movement of the image caused by the movement of the object on the screen, and By setting the vector detection output to zero, it is possible to effectively reduce the erroneous detection of the motion vector due to the motion of the object or the like.

Example of Invention

FIG. 1 is a block diagram showing a schematic configuration of a motion vector detecting device according to an embodiment of the present invention. The image information input from the input terminal 1 is branched into two, one of which is guided to the frame memory 2 as the image information of the previous frame, and the other is guided to the correlator 3 as the image information of the current frame. The input image information is, for example, a serial scanning signal such as television image information, and is generally digitized. The frame memory 2 receives the read address data 11 from the address controller 4,
The image information of the same pixel as the pixel of the image information from the input terminal 1 of the image information of the previous frame is sequentially supplied to the correlator 3. As a result, the correlator 3 performs known correlation calculation such as square error calculation, difference absolute value calculation, or cross-correlation calculation between image information of two frames that are temporally continuous. .

The correlation value output from the correlator 3 is input to the cumulative adder 5 and is added to the corresponding cumulative adder according to the address data 12 given from the address controller 4. That is, the read address data 11 supplied from the address controller 4 to the frame memory 2 is set to the pixel of the image information of the current frame input to the input terminal 1,
Since the address data 12 supplied from the address controller 4 to the cumulative adder 5 corresponds to the read address data 11 corresponding to the deviation of the pixel of the previous frame, the cumulative adder 5 has a correlator. The correlation value corresponding to a certain deviation newly input from 3 will be added to the cumulative addition value corresponding to the same deviation up to that time.

When the image information for one frame is input to the input terminal 1 and the correlation calculation and the cumulative addition of the correlation values are completed, the address controller 4 sends the address data 12 to the cumulative adder 5 as
The address data corresponding to each of the above deviations is supplied, whereby the cumulative adder 5 outputs a plurality of cumulative addition correlation values (first correlation values) 13. The correlation value 13 output from the cumulative adder 5 is input to the correlation validity / invalidity determination circuit 6 and the correlation search circuit 7.

The correlation validity / invalidity determination circuit 6 determines validity / invalidity of the correlation value having the highest correlation among the correlation values 13 from the cumulative adder 5, and outputs the determination result 14 to the determination circuit 9. Further, the correlation search circuit 7 searches the correlation value 13 for the second correlation value 15 having the highest correlation and outputs the second correlation value 15 to the determination circuit 9 and the address data 1 to the address holding circuit 8.
The instruction signal 16 holding the address data corresponding to the correlation value 15 of 2 is output. The address holding circuit 8 converts the address data designated by the designation signal 16 in the address data 12 into a shift, and outputs the shift 17 to the determination circuit 9.

The determination circuit 9 receives three inputs, that is, the determination result 14 from the correlation validity / invalidity determination circuit 6, the correlation value 15 from the correlation search circuit 7, and the shift 17 from the address holding circuit 8, and determines whether or not there is a motion vector. judge. FIG. 6 shows a specific example of the determination circuit 9, which is a zero generator 61 and a selection circuit 6.
2 and. That is, the selection circuit 62 receives the determination result 14 from the correlation validity / invalidity determination circuit 6, and when the determination circuit 6 determines that the correlation is valid, the shift supplied from the address holding circuit 8 in FIG. 17 is output to the output terminal 10 as the motion vector detection output 18,
When the determination circuit determines that the correlation is invalid, the output of the zero generator 61 is selected and the motion vector detection output to the output terminal 10 is set to zero.

Next, a specific example of the correlation validity / invalidity determination circuit 6 will be described. FIG. 2 shows a first concrete example of the correlation validity / invalidity judgment circuit 6, and the correlation value 1 from the cumulative adder 5 in FIG.
3 is input, and the average value is calculated by the average value calculation circuit 21. This average value is input to the comparison circuit 22 and compared with a preset setting value C ₁ . When the average value falls below the set value C ₁ , the comparison circuit 22 determines that the correlation of the correlation value 13 is invalid, and otherwise determines that the correlation is valid, and the determination result 14
Is output. This example focuses on the fact that when the correlation is low, the magnitude of the correlation value decreases on average. This example is particularly effective when the pattern of the image has little change and the correlation value is small overall.

FIG. 3 shows a second concrete example of the correlation validity / invalidity determination circuit 6. The process until the average value calculation circuit 31 calculates the average value of the correlation value 13 is the same as in FIG. 2, but in this example, the calculated average value is multiplied by, for example, a constant by the average value correction circuit 32, and then compared. It is supplied to one input of the circuit 33. The other input of the comparison circuit 33 is given the correlation value 15 having the highest correlation from the correlation search circuit 7 in FIG. Here, the correlation calculation in the correlator 3 in FIG. 1 is such that the smaller the correlation value is, the higher the correlation is, such as the calculation of the absolute difference value or the square error between the image information of two consecutive frames. In the case of being configured by calculation, when the calculated average value is lower than the correlation value 15 having the highest correlation, the correlation of the correlation value 13 from the cumulative adder 5 is determined to be invalid, and otherwise it is determined to be valid. The determination is made and the determination result 14 is output. On the other hand, when the correlation calculation is configured by a calculation such that the correlation value increases as the correlation value increases, such as the cross correlation calculation, when the calculated average value exceeds the correlation value 15 having the highest correlation, , The correlation of the correlation value 13 from the cumulative adder 5 is determined to be invalid, and otherwise it is determined to be valid. That is, in the example of FIG. 3, it is determined that the one having a deeper correlation value peak is effective, and therefore, it is particularly effective when there is a moving object in the image although the pattern of the image changes.

FIG. 4 is a third specific example of the correlation validity / invalidity determination circuit 7, in which the minimum value and the maximum value of the correlation value 13 are obtained by the minimum value search circuit 41 and the maximum value search circuit 42, and the ratio calculation circuit 43. At, the ratio of the two, that is, the minimum value / maximum value is calculated. This ratio is a predetermined threshold value C in the comparison circuit 44.
₂ (0 ≦ C ₂ ≦ 1). Here, when the correlation is low, the minimum value and the maximum value are close to each other. Therefore, when the ratio exceeds the threshold value C ₂ , the correlation is determined to be invalid, and in other cases, it is determined to be valid and the determination result 14 is output. The difference may be calculated instead of calculating the ratio between the maximum value and the minimum value in the example of FIG.

FIG. 5 shows a fourth concrete example of the correlation validity / invalidity judgment circuit 7, which is a composite form of the examples of FIGS. 2 to 4 explained above. That is, the average value calculation circuit 51 is shown at 21, 31 in FIGS. 2 and 3, the comparison circuits 52, 53, 58 are shown at 22, 33, 44 in FIGS. 2 to 4, respectively, and the correction circuit 53 is shown. Is 32 in FIG. 3, and the minimum value search circuit 55 and the maximum value search circuit 56 are 41 in FIG.
42 respectively. In this example, the OR circuit 59 calculates the logical sum of the judgment results of the three comparison circuits 52, 54 and 58, and outputs the result as the judgment output 14.

According to the example of FIG. 5, if any one of the three conditions described in FIGS. 2 to 4 is satisfied, the correlation of the correlation value 13 is determined to be invalid, and any one of the three conditions is determined. If it does not hold, it is determined to be valid.

Next, an embodiment of the present invention will be described with reference to FIG. In this embodiment, a plurality of motion vector detection areas for each frame and a plurality of representative point extraction areas including at least one of the motion vector detection areas are set, and an image of each point of each motion vector detection area in the current frame is set. Information,
This is an example in which the present invention is applied to a motion vector detection device that performs a correlation calculation with image information of a representative point extracted from a representative point extraction area including the motion vector detection area in the frame.

In FIG. 7, the image information input from the input terminal 1 is branched into two, one is led to the latch circuit 71 as the image information of the previous frame, and the other is led to the correlator 3 as the image information of the current frame. When the image information of a predetermined representative point is input to the input terminal 1, the latch circuit 71 latches the image information by a latch pulse 81 generated at the input timing of the image information of the representative point. This latch circuit 7
The image information of the representative point latched at 1 is the transfer permission signal 8
2 is transferred to the representative point storage memory 72 at an appropriate timing and stored in a predetermined address for the representative point.

The representative point storage memory 72 is provided in place of the frame memory 2 in the embodiment of FIG.
The address switching circuit 74 controls write address data 84 generated by the address controller 4 in the write mode and read address data 85 in the read mode, which are controlled by the read mode switching signal 83.
Is provided as an address input 86 via. The image information of the representative point read from the representative point storage memory 72 is latched by the latch circuit 73 and guided to the correlator 3. Correlator 3
Performs a correlation calculation between the image information of the current frame input to the input terminal 1 and the image information of the representative point of the previous frame from the latch circuit 73. Here, in the latch circuit 73, the image information of the representative point read from the representative point storage memory 72 represents the image information of one pixel in the representative point extraction area, and the image information is included in each representative point extraction area. In relation to being used multiple times for the correlation calculation with the image information of each pixel in the motion vector detection area, during the period used multiple times,
It is provided to hold the image information read from the representative point storage memory 72. Further, the latch circuit 71 is provided in order to accept the image information of the representative point in the current frame when the representative point storage memory 72 is in the read mode.

Every time the image information of each pixel of the current frame is input to the input terminal 1, the correlator 3 correlates the image information with the image information of the representative point of the representative point extraction region in the previous frame supplied from the latch circuit 73. Calculate. In this case, the image information supplied from the latch circuit 73 to the correlator 3 is the image information of the representative point in the previous frame of the representative point extraction region including the motion vector detection region to which the pixel of the image information input to the input terminal 1 belongs. Is. Since the configuration and operation of the correlator 3 and subsequent elements are the same as those of the embodiment shown in FIG.

According to the embodiment of FIG. 7, the number of calculations required for motion vector detection is about the number of pixels constituting one frame at most, which is an advantage. With this amount of calculation, one frame or one
Processing in the field, that is, real-time processing is sufficiently possible, and the hardware, especially the one required for the storage means, is significantly smaller than the number of pixels in one frame, and the image information of the representative point for one frame can be obtained. Only a memory for storing and a memory for cumulative addition of correlation values are significant advantages in realizing the device.

The present invention is not limited to the above embodiment,
Various modifications can be made without departing from the spirit of the invention.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a motion vector detecting apparatus according to an embodiment of the present invention, and FIGS. 2 to 5 are various correlation validity / invalidity determining circuits in the motion vector detecting apparatus according to the present invention. FIG. 6 is a block diagram showing a concrete example, FIG. 6 is a block diagram showing a concrete example of a determination circuit for obtaining a motion vector detection output, and FIG. 7 shows a configuration of a motion vector detection device according to another embodiment of the present invention. Block diagrams, FIGS. 8 and 9 are diagrams for explaining the concept of a motion vector. 1 ... Image information input terminal, 2 ... Frame memory, 3 ...
... Correlator, 4 ... Address controller, 5 ... Cumulative adder, 6 ... Correlation valid / invalid judgment circuit, 7 ... Correlation search circuit, 8 ... Address holding circuit, 9 ... Judgment circuit,
10 ... Motion vector output terminals 21, 31, 51 ...
Average value calculation circuit, 22, 33, 44, 52, 54, 58
…… Comparison circuit, 32,53 …… Correction circuit, 41,55…
... minimum value search circuit, 42, 56 ... maximum value search circuit, 4
3, 57 ... Ratio calculation circuit, 59 ... OR circuit, 61 ...
Zero generator, 62 ... Selection circuit, 71, 73 ... Latch circuit, 72 ... Representative point storage memory, 74 ... Address switching circuit.

Front page continuation (72) Inventor Yuichi Ninomiya 1-10-11 Kinuta, Setagaya-ku, Tokyo Inside the Japan Broadcasting Corporation Broadcasting Technology Laboratory (72) Yoshimichi Otsuka 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Technology Institute, Broadcasting Corporation (72) Inventor Yoshinori Izumi 1-10-11 Kinuta, Setagaya-ku, Tokyo Inside Broadcasting Technology Laboratory, Japan Broadcasting Association (72) Seiichi Koshi 1-10 Kinuta, Setagaya-ku, Tokyo No. 11 Inside Broadcasting Technology Laboratory of Japan Broadcasting Corporation (56) Reference JP-A-60-28392 (JP, A) JP-A-58-107785 (JP, A) JP-A-58-15990 (JP, A) JP-A Sho 61-113377 (JP, A)

Claims (5)

[Claims] 1. A motion vector detecting device for detecting a motion vector indicating a translation amount of an entire image from image information of two frames which are temporally continuous, and a motion vector detecting device for displaying image information between screens of two frames which are temporally continuous. A first means for performing a correlation operation and outputting a first correlation value; and a second correlation value having the highest correlation among the plurality of first correlation values obtained by the first means and its A second means for obtaining a shift between two points giving a second correlation value, and valid / invalid correlation of the second correlation value obtained by the second means. A third means for determining by referring to the value, and when the second correlation value is determined to be invalid by the third means, the motion vector detection output is set to zero, and the second correlation value is determined to be valid. Sometimes the deviation obtained by the second means is detected as a motion vector. A motion vector detecting device, comprising: a fourth means for force. 2. A third means calculates an average value of a plurality of first correlation values obtained by the first means, compares the average value with a predetermined set value, and calculates an average value. The correlation of the second correlation value is determined to be invalid when the value is less than the set value, and the correlation of the second correlation value is determined to be valid otherwise. The motion vector detection device described. 3. A third means calculates a mean value of a plurality of first correlation values obtained by the first means, and multiplies the mean value by a constant to obtain a set value and a second correlation value. When the set value is lower than the second correlation value (the smaller the first correlation value is, the higher the correlation is), or when the set value is higher than the second correlation value (the first correlation value). The second correlation value is determined to be invalid, and the second correlation value is determined to be valid otherwise (when the correlation value is higher as the correlation value is higher). Motion vector detection device. 4. The third means compares the ratio of the minimum value to the maximum value of the plurality of first correlation values obtained by the first means with a predetermined threshold value, and when the ratio exceeds the threshold value. The motion vector detecting device according to claim 1, wherein the second correlation value is determined to be invalid, and the second correlation value is determined to be valid otherwise. 5. A third means is: (a) an average value of a plurality of first correlation values is lower than a predetermined set value, or
(b) whether the set value obtained by multiplying the average value by a constant is lower than the second correlation value (when the smaller the first correlation value is, the higher the correlation is), or the set value is the second correlation value. Exceeds (when the higher the first correlation value is, the higher the correlation is), or (c)
When the ratio of the minimum value to the maximum value of the plurality of first correlation values exceeds a predetermined threshold value or at least one of the three conditions is satisfied, it is determined that the correlation of the second correlation value is invalid, and these 3 The motion vector detecting device according to claim 1, wherein the correlation of the second correlation value is determined to be valid when none of the two conditions is satisfied.