JP3486227B2 - Motion vector interpolation filter - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector interpolation filter for compensating for erroneous detection of motion vectors obtained from image signals such as NTSC signals and high definition signals.

[0002]

2. Description of the Related Art Conventionally, as a filter for this type of motion vector, a filter for performing spatial smoothing between upper, lower, left and right blocks (that is, a spatial low-pass filter for a vector) has been used. The reason why the filter for such a vector is used is that the actual motion vector detection method is not perfect and an erroneous vector is detected in some images, so that the adverse effect thereof is mitigated.

[0003]

However, there is no guarantee that spatial smoothing of a vector will always improve the accuracy of the vector. For example, if spatial smoothing is performed on the edge part of the moving object, even if the motion vector is accurately detected, the average vector of the vector of the moving object and the background vector will be the edge part even if the motion vector is detected accurately. , And the wrong motion compensation will be performed for that part.

Therefore, an object of the present invention is to improve the accuracy of the motion vector by avoiding such unnecessary change of the motion vector value and changing the value of only the motion vector which is likely to be erroneous. It is to provide a motion vector interpolation filter capable of

[0005]

To achieve the above object, the present invention sequentially detects a motion vector and an error of the motion vector for an image signal in units of blocks each of which includes a plurality of pixels in the horizontal and vertical directions. Means for forming a motion vector and an error in blocks on both sides of the block by delaying the detected motion vector and the error in a predetermined direction, and forming an average vector of the formed motion vectors on both sides. Means for outputting a selection signal, and selecting and outputting the formed average vector when the selection signal is input, and selecting the detected motion vector when the selection signal is not input And a switch for outputting the selected signal, the means for outputting the selection signal is Uniformity determining means for outputting a signal when one, vector comparing means for outputting a signal when the detected motion vector and the formed motion vectors on both sides have different values, and the formed both sides Error comparing means for outputting a signal when the detected error is larger than the detected error, and the selection is performed only when signals are output from all of the uniformity determining means, the vector comparing means and the error comparing means. And a selection signal determining means for outputting a signal.

Further, in the present invention, preferably,
The predetermined direction may be a horizontal direction, a vertical direction, or an oblique direction.

[0007]

[0008]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 shows an embodiment of the present invention. The input signal of the entire circuit in the figure is a motion vector V (horizontal direction component V _x , vertical direction component V _y ) and a numerical value E (hereinafter, error E) representing the reliability of the vector V. It is output from the motion vector detection circuit as shown in FIG.

FIG. 4 shows an example of a circuit based on a block matching method which is a typical motion vector detecting method. This circuit obtains one vector V (V _x , V _y ) for each appropriate block of pixels (for example, one block of horizontal 16 pixels × vertical 8 lines) from the input image signal P and a matching error E of the vector. , Output sequentially. P is the frame memory 30
Thus, the signal is delayed by a little less than one frame, and then further delayed by the variable delay circuit 31 by the delay amount Vi (Vi _x , Vi _y ). At this time, the delay amounts of 30 and 31 are set to be exactly one frame time when Vi _x = Vi _y = 0. The output of 31 is P'and the subtractor 3
2 and the absolute value circuit 33 determine e = | PP ′ |, and the in-block accumulating circuit 34 integrates e for each block. The output Ei of 34, which is the integration result, is a matching error with respect to Vi of the block. Vi is also called a candidate vector, and the candidate vector generation circuit 36 sequentially generates preset candidates in a brute force manner. The minimum value detection circuit 35 detects the minimum value for each block from Ei. i = imin (both i and imin are integers)
If Ei is the minimum at that time, 35 outputs Ei _min as the error E of the block. At the same time, the output vector selection circuit 37 selects Vimin from Vi according to the signal imin and outputs it as the motion vector V of the block. Note that although 31 to 34 and 36 are shown as a single circuit in the figure, in reality, many same circuits are often operated in parallel due to processing time constraints.

In FIG. 1, the output signal of the entire circuit is a motion vector V _o (V _ox , V _oy ), and the input signals V and E are sequentially input for each appropriate block of pixels according to the scanning order of the image. . These are block-unit latches (D
^* ) Sequentially latched by the circuits 1 to 6, the vector V _R (V _Rx , V _Ry ) of the right block and its error E _R , the vector V _c (V _cx , V _cy ) of its center block and its error E _C , vector of the left block _{V L (V Lx, V Ly} ) becomes and the error E _L.

The right (R), center (C) and left (L) blocks are arranged in a horizontal row on the screen as shown in FIG.
As shown in FIG. 2, the operation of the circuit shown in FIG.
Regarding the vector value and error from one block, the vector value of the central block is interpolated from the left and right vector values by the horizontal interpolation filter only when a certain condition is satisfied to obtain the output vector V _o . Number 7 in Figure 1
The circuits indicated by ˜25 form a horizontal interpolation filter.

In FIG. 1, an average value V _LRx is calculated from V _Rx and V _Lx by an adder 7 and a 1/2 coefficient unit 8. V _LRx is a horizontal component of the interpolation vector V _LR (V _LRx , V _LRy ) used for interpolation of V _C when the conditions described later are satisfied. From V _Ry and V _Ly , adder 9, 1 /
The vertical coefficient V _LRy is _similarly obtained by the 2-coefficient _multiplier 10. These constituent elements 7 to 10 collectively constitute the vector averaging circuit 11.

In the switch 12, according to the control signal SEL, if SEL is at a logic "1" level, V _LRx , V _LRy
If SEL is at a logic "0" level, then V _Cx and V _Cy are selected and used as the output vector V _o (V _ox , V _oy ). That is, if SEL is "1", the central vector V _{C is changed} to V _LR.
Interpolate with. If SEL is "0", no interpolation is performed.

The components indicated by 13 to 15 collectively form a vector uniformity determining circuit 16, where V _L and V _{R are set.}
To determine the uniformity of. In the difference comparator 13, the difference (V _Rx −
If the absolute value of V _Lx ) is smaller than the threshold value TH _vx , that is,

[0016]

If | V _Rx −V _Lx | <TH _vx (1), the output C _{1 is set} to “1”. C otherwise
₁ is “0”. Similarly, the difference comparator 14

[0017]

If | V _Ry −V _Ly | <TH _vy (2), the output C _{2 is set} to “1”. The AND gate 15 takes the logical product of C ₁ and C ₂ to obtain the signal C ₁₂ . C ₁₂ becomes “1” only when the difference between the vector values of the left and right blocks is smaller than the threshold value in both the horizontal and vertical components. The thresholds TH _vx and TH _vy are determined by quad and try in an actual circuit. Further, in general, the number of bits of V _x and V _y is not large (usually about 4 to 6 bits), and the processing is processing for each block, which can be extremely low speed.
4 can actually be composed of a low-speed, large-capacity ROM or the like.

The components 17 to 19 collectively constitute a vector comparison circuit 20, in which the left and right average vectors (interpolation vectors) V _Lr and the central vector V _C are compared. The subtractors 17 and 18 respectively have V _Cx and V _LRx ,
Differences d _vx and d _vy between V _Cy and V _LRy are obtained.

[0019]

D _vx = V _LRx −V _Cx (3) d _vy = V _LRy −V _Cy (4) In the two-dimensional comparison circuit 19, the two-dimensional difference between V _C and V _Ly.

[0020]

## EQU4 ## d _v = (d _Vx ² + d _Vy ² ) ^1/2 (5) is obtained, and if d _v is equal to or greater than the threshold value TH _dV , the output C
_{Set 3} to “1”. Otherwise, C _{3 is set} to “0”. That is, if V _LR and V _C are significantly different, C ₃ is “1”. TH _dV is determined by cut and try. The circuit 19 can be realized by a low-speed large-capacity ROM, for example.

The components indicated by 21 to 23 constitute the error comparison circuit 24 as a whole. Where E _C and E _R , E
Compare _L. E by the adder 21 and the 1/2 coefficient unit 22
_Calculate the average value E _LR of _R and E _L. The comparator 23 compares E _C and E _LR , and if E _C > E _LR , sets the signal C ₄ to “1”. Otherwise, it is set to "0".

In this embodiment, the AND gate 25 is a selection signal determining circuit that determines the control signal SEL. AN
The D gate 25 calculates the logical product of the signals C ₁₂ , C ₃ and C ₄ ,
It is used as a control signal SEL for the switch 12. That is, the circuit of FIG. 1 as a whole has left and right vectors ( _VL ,
V _R) is uniform, and the left and right of the vector (V _L, V _R) and the center of the greatly different vector (V _C), and the center of vector (V _C) of the error (E _C) the left and right of the vector (V
Only when it is larger than the error (E _L , E _R ) of _L , V _R ), the central vector (V _Cx , V _Cy ) is rejected, and the average value (V _LRx , V _LRy ) of the left and right vectors is _discarded . Interpolate (ie replace) with.

The circuit of FIG. 1 is effective for an image as shown in FIG. 3, for example. This generally represents an image in which an object with diagonal gray edges passes from top right to bottom left in front of a background with black vertical bars on a white background.
The image is in the state at time t = 0, and at time t = -1 / 30s
In ec, the diagonal edge is located at the position indicated by the alternate long and short dash line in FIG. The positions of the blocks L, C and R considered here are indicated by dotted lines in the figure.

As described with reference to FIG. 4, the motion vector detection is performed by the inter-frame difference, that is, the image at t = 0 and t
It is performed based on the image difference of = -1 / 30 sec.
In FIG. 3, regarding the left and right blocks L and R, the image has an extremely simple motion, and a correct motion vector can be detected. Moreover, the values are very close. However, in the central block C, the portion a (vertical lever), which did not exist in the image 1/30 seconds before, appears at t = 0. Therefore, in the block C, the inter-frame difference is also present regarding the correct movement. There is a high possibility that a vector having a value that is not zero and is different from the left and right vector values will be detected. Moreover,
The detected vector error has a large value.

All of the above situations meet the condition that V _C is interpolated in the circuit of FIG. 1 as described above, and the incorrect vector V _C at the center is the average value V _LR of the correct left and right vectors.
Is interpolated with. As a result, even in the central block C, a correct motion vector is output for the diagonal edge image.

The present invention can be realized by a circuit other than the above-mentioned circuits and conditions. In particular, the circuits 16, 20, 24 and 25 are shown in FIG.
It may be configured by other logic similar to the configuration logic of the circuit of FIG.
For example, E to the error comparison circuit 24 E _{C L,} although compared to the average value E _LR of E _R, the E _C as shown in FIG. 5 E
_The comparison may be made with the larger value of _L and E _R.

Further, as shown in FIG. 6, the signals of C _{1 to} C ₄ , which are the determination results of each circuit, are not 1 bit but multi-bit.
Alternatively, all of C _{1 to} C _{4 in the} figure are 2 bits. In the figure, for example, the difference comparators 41 and 42 are respectively

[0028]

Equation 5] _{K x - | V Rx -V Lx} | (K x is appropriate _{constant) K y - | V Ry -V} Ly | calculates the (K _y Suitable constant), quantum the result to 2bit C ₁ and C ₂ signals. The multiplier 43 multiplies C ₁ and C ₂ to obtain a 4-bit C ₁₂ signal. In the two-dimensional comparison circuit 45, d in equation (5)
_V is quantized into 2 bits to obtain a C ₃ signal. Comparison circuit 4
In 7, the E _C -E _LR is quantized into 2 bits to obtain a C ₄ signal. The selection signal determination circuit 49 is actually a low-speed, large-capacity R
It can be realized by OM or the like, and the content (decision logic) is as follows, for example. With TH _SEL as the threshold,

[0029]

[Equation 6]

With the above circuit, a more accurate operation similar to that of the circuit of FIG. 1 can be realized.

Further, although the horizontal interpolation is performed in the embodiment, the present invention can be directly applied to the vertical and diagonal interpolation.

[0032]

According to the present invention, the accuracy of the motion vector can be improved by interpolating the value with the surrounding vector only when the motion vector is highly likely to be erroneous. It is possible to improve the image quality of the image processing apparatus.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a diagram for explaining the operation of the circuit of FIG.

FIG. 3 is another diagram for explaining the operation of the circuit of FIG.

FIG. 4 is an example of a block diagram of a circuit that detects a motion vector and its error.

FIG. 5 is a diagram showing another example of an error comparison circuit.

FIG. 6 is a diagram showing another example of a selection signal determination circuit.

[Explanation of symbols]

1-6 Latch circuit 11 Vector averaging circuit 12 switches 16 Vector uniformity determination circuit 20 Vector comparison circuit 24 Error comparison circuit

Claims (2)

(57) [Claims] 1. A means for sequentially detecting a motion vector and an error of the motion vector in a block unit composed of a plurality of pixels in the horizontal and vertical directions with respect to an image signal, and the detected motion vector and the error in a predetermined direction. Means for forming a motion vector and an error in blocks on both sides of the block by delaying, means for forming an average vector of the formed motion vectors on both sides, means for outputting a selection signal, and the selection signal When the input signal is input, the formed average vector is selected and output, and when the selection signal is not input, a switch that selects and outputs the detected motion vector is output, and the selection signal is output. Means for outputting a signal when the formed motion vectors on both sides are uniform; Vector comparing means for outputting a signal when the detected motion vector and the formed motion vectors on both sides have different values, and a signal when the detected error is larger than the formed error on both sides. And a selection signal determination means for outputting the selection signal only when signals are output from all of the uniformity determination means, the vector comparison means and the error comparison means. Motion vector interpolation filter. 2. The motion vector interpolation filter according to claim 1, wherein the predetermined direction is one of a horizontal direction, a vertical direction, and an oblique direction.