JPH0477516B2 - - Google Patents

Description

[Detailed Description of the Invention] [Field of Industrial Application] This invention is applied to image processing in a high-definition television receiver using the MUSE system, and calculates the amount of motion correction between each field from the transmitted motion information. The present invention relates to a motion compensation subsample interpolation method that performs calculations to interpolate missing points between fields.

[Conventional technology] This type of motion compensation sub-sample interpolation method that has been proposed in the past is based on the ``New transmission method for high-definition television'' announced by the NHK Broadcasting Technology Research Institute at its foundation commemorative lecture in June 1980. (MUSE)" is mentioned in the preliminary materials. FIG. 11 is a block circuit diagram of an example of a configuration for implementing this motion compensation subsample interpolation method.

In the figure, 1 is a video input terminal that inputs a video signal transmitted at a sample rate of 16.2MHz;
is a switch that switches between the video signal input to the input terminal 1 and the output signal of the motion correction field memory 8, which will be described later, at a sub-sample timing of 32.4 MHz. Reference numeral 13 denotes a non-motion compensated field memory that stores one frame worth of signal at a sample rate of 32.4MHz that has passed through switch 2. Reference numeral 8 denotes a motion correction field memory, which stores one frame worth of signals at a sample rate of 32.4 MHz output from the non-correction field memory 13, and performs motion correction using a motion vector. An inter-field interpolation filter 9 interpolates missing sample points based on the output signal of the non-motion compensation field memory 13 and the signal passed through the switch 2. An intra-field interpolation filter 10 interpolates missing sample points based only on the signal that has passed through the switch 2. 1
1 is a switch, when not performing motion compensation,
It is connected to the upper contact 11x to pass the output signal of the inter-field interpolation filter 9, and when performing motion compensation or when motion detection is performed, it is connected to the lower contact 11y to pass the output signal of the intra-field interpolation filter 10. Let it pass. Reference numeral 12 denotes a video output terminal that outputs a signal that has passed through the switch 11 at a sample rate of 64.8 MHz after interpolating missing sample points.

Next, the operation of the above configuration will be explained.

The transmission method for high-definition television is sub-Nyquist sampling, which goes around four fields, and the required bandwidth is 8.1MHz.

Now, the video signal input to the video input terminal 1 is expressed as a _i , b _i , c _i , d _i with 4 fields as one unit.
Expressed as...a ₀ , b ₀ , c ₀ , d ₀ , a ₁ , b ₁ , c ₁ , d ₁ ,
a ₂ , b ₂ , c ₂ , d ₂ ... shall be input in this order,
Assuming that the signal of field A ₂ is now input to the video input terminal 1, the non-motion compensation field memory 13 contains the signal of the second field b ₁ from the previous round, and
On the other hand, the motion compensation field memory ₈ stores the signals of the first field a ₁ and the third field c ₁ from the previous round. When a motion vector is present due to panning of the camera, the contents of the motion correction field memory 8 move two-dimensionally according to the amount of the vector. At this time, if the transmitted motion vector is the amount of motion correction between fields separated by one frame, the motion correction content in the motion correction field memory 8 is the amount of motion correction between fields a 2 that is input to the video input terminal ₁ . This is done based on the signal.

Switch 2 switches at the timing of 32.4 MHz sub-samples, and the phase is inverted for each field and also depending on the motion vector. Therefore, in the above case, the signal of the first field _a2 inputted from the video input terminal 1 and the signal of the third field ₁ , which has been subjected to motion correction one cycle before, pass through the switch 2.

The signal passed through the switch 2 and the output signal of the non-motion compensation field memory 13 are input to an inter-field interpolation filter 9 to perform inter-field interpolation.

When not performing motion compensation, that is, switch 1
1 is connected to the upper contact 11x, 64.8M interpolated between fields is output from the video output terminal 12.
Outputs a video signal with a sample rate of Hz.

In addition, the field signal that has passed through the switch 2 is input to an intra-field interpolation filter 10 to perform intra-field interpolation, and when the switch 11 is connected to the lower contact 11y, the intra-field interpolation is performed from the video output terminal 12. Outputs a video signal with a sample rate of 64.8MHz.

Signals 12a to 12e of each part in FIG. 11 when the _a2 field is input to the video input terminal 1 without motion correction are as shown in FIGS. 12a to 12e, and FIG. A in the middle is b ₁ , c ₁ , d ₁ , a ₂ by interpolation function f _A which is interpolation between fields.
Indicates the interpolated value interpolated from .

Also, motion compensation is performed and the video input terminal 1
The signals 13a to 13e of each part in Fig. 11 when the _a2 field is input are as shown in Fig. 13 a to e.
The horizontal line above the symbol in Fig. 13 indicates that motion compensation has been performed, and B in Fig. ₁₃ is the interpolation function f _B that is intra-field interpolation _. Indicates the interpolated value to be interpolated.

[Problems to be Solved by the Invention] When using the conventional motion compensation subsample interpolation method as described above, the relationship between adjacent fields is not taken into consideration at all when performing motion compensation, so Interpolation of the sample points must be performed within the field by an intrafield interpolation filter 10. Therefore, there was a problem in that the resolution decreased when motion compensation was performed.

The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a motion compensation subsample interpolation method that can suppress a decrease in resolution even when motion compensation is performed.

[Means for Solving the Problems] The motion compensation subsample interpolation method according to the present invention calculates a motion vector between fields separated by one frame by approximation from a transmitted motion vector between fields separated by one frame. The present invention is characterized in that inter-field interpolation can be performed by predicting and motion-correcting all signals from the first field to the fourth field.

[Operation] According to the present invention, by taking advantage of the fact that there is no sudden change in the transmitted motion vectors, the motion vectors between fields separated by one field are converted into two types of motion of adjacent fields on the transmitting side. By predicting by approximating 1/4 of the sum of vectors,
Interpolation can always be performed using an inter-field interpolation filter, thereby suppressing a decrease in resolution during motion correction.

[Embodiment of the Invention] Hereinafter, an embodiment of the present invention will be described based on the drawings.

Figure 1 is a block circuit diagram of this embodiment.
The same reference numerals as in FIG. 1 indicate the same parts. In the figure, switch 2 is video input terminal 1
The video signal input to the 32.4 MHz
Switch at sub-sample timing. 3 is 1
A field delay device delays the transmitted motion vector by one field. 4 is an adder which adds the transmitted motion vector and the motion vector one field before outputted from the one field delay device 3; 5 is a divider, which divides the output of adder 4 into 4
Divide by. A subtracter 6 subtracts the output of the divider 5 from the transmitted motion vector. Reference numeral 7 denotes a first motion correction field memory, which stores one frame worth of signal at a sample rate of 32.4MHz that has passed through switch 2, and performs motion correction using the predicted motion vector between fields output from subtracter 6. . 8 is the second motion compensation field memory,
A signal for one frame of the 32.4 MHz sample rate output from the first motion correction field memory 7 is stored, and motion correction is performed using the predicted motion vector between fields output from the subtracter 6. The inter-field interpolation filter 9 interpolates missing sample points based on the signal passed through the switch 2 and the output signal of the first motion correction field memory 7. The operations of the intra-field interpolation filter 10 and the switch 11 are the same as in the conventional example shown in FIG.

Next, the operation of this embodiment will be explained.

In order to specifically explain the operation shown in Figure 1, Figure 2 shows motion vectors when panning is occurring, and in the figure, the x-axis represents the horizontal axis of the screen, and the y-axis represents the vertical axis of the screen. .

If each field of the video signal is represented by a ₀ ~ d ₂ ...
a ₀ , b ₀ , c ₀ , d ₀ , a ₁ , b ₁ , c ₁ , d ₁ , a ₂ , b ₂ , c ₂ ,
It flows in the order of d ₂ ... In the figure, the motion vectors between fields separated by one frame are...A→ ₁ ,
It is expressed as B→ ₁ , C→ ₁ , D→ ₁ , A→ ₂ , B→ ₂ , ..., and the motion vector between fields separated by one field obtained on the drawing is ... b→ ₁₀ , ca→ ₁₀ , d→ ₁₀ , a→ ₁₀ . . . , the following relational expression holds between the inter-field motion vectors separated by one frame and the inter-field motion vectors separated by one field.

a→ ₁₀ +b→ ₁₀ =A→ ₁ b→ ₁₀ +c→ ₁₀ =BA→ ₁ c→ ₁₀ +d→ ₁₀ =C→ _1Here , for example, b ₁₀ can be rewritten as in the following equation.

b→ ₁₀ =A/→ ₁ +B/→ ₁ /4-1/2(a/→ ₁₀ −
b/→ ₁₀ /2− b/→ ₁₀ −c/→ ₁₀ /2) If the second term on the right side of the above equation is sufficiently small, b ₁₀ can be predicted as follows.

b→ ₁₀ =A/→ ₁ +B/→ ₁ /4 Here, for example, only the relationship between the four fields a ₁ to d ₁ will be considered. However, the motion vector between fields separated by one frame is a digital signal with a horizontal component of 5 bits and a vertical component of 3 bits.
One field is transmitted. In other words, if we describe the magnitude of the motion vector between fields separated by one frame in terms of pixel length, the horizontal component will vary from -15 to +
16, the vertical component is in the range from -3 to +4, and both the horizontal and vertical components have discrete components that are integers. To illustrate this,
As shown in FIG. 3, the inter-field motion vectors separated by one frame are within the range surrounded by broken lines. However, the origin in FIG. 3 is an arbitrary pixel in the _a1 field.

If the parallel movement of the image due to panning is smooth, the inter-field motion vector between the a ₁ field and the b ₁ field separated by one field is within the range surrounded by the dashed-dotted line in FIG. Here, a ₁ ~
The relative relationship between the three fields of _c1 is changed in three ways, as shown in FIGS. 4, 6, and 8, and the prediction accuracy of the position and motion vector of field _d1 is investigated for each.

First, FIG. 4 shows a case where the magnitude of the inter-field motion vector A→ ₁ transmitted one frame apart is maximum. At this time, the inter-field motion vectors a→ ₁₀ and b→ ₁₀ separated by one field are also maximized, and the relative positions of fields _a1 to _c1 are determined. At this time, the d ₁ field exists within the range indicated by the three-dot chain line in the figure, but if there is no sudden change in movement, the d ₁ field exists within the shaded range. Furthermore, if the parallel movement of the image, such as panning, is smooth, the d ₁ field is likely to exist at the position indicated by the black circle in the figure. As an example,
FIG. 5 shows the predicted vector (A→ ₁ +B→ ₁ )/4 and the vector b→ ₁₀ when the d ₁ field is at the position shown. The prediction error at this time is one pixel length, and if the d ₁ field is located at the position indicated by the black circle, the error will be less than that, so such prediction is appropriate.

Next, in Figure 6, the components of the inter-field motion vector A→ ₁ separated by one frame are (x, y) = (10, 2)
The case is shown below. Similarly to Fig. 4, the inter-field motion vector a → ₁₀ exists within the range surrounded by the dashed line, and b If _{the 1} field is at the point shown, the inter-field motion vector one field apart exists within the range surrounded by the dashed line. Vector b→ ₁₀ exists within the range surrounded by the two-dot chain line. In fact, such movement by panning is possible because the c ₁ field is within the range surrounded by the dashed-two dotted line as shown. At this time, the _d1 field exists within the range surrounded by the three-dot chain line in the figure, but as in the explanation in Figure 4, it is within the diagonally shaded range, and within that field, it is possible that it exists at the position indicated by the black circle. Highly sexual.
As an example, FIG. 7 shows the predicted vector (A→ ₁ +B→ ₁ )/4 and the vector b→ ₁₀ when the d ₁ field is at the position shown. The prediction error at this time is one pixel length on the diagonal, and if there is a d ₁ field at the position indicated by the black circle, the error will be less than that, so the 6th
The prediction is also valid in the case of the figure.

Next, in Figure 8, the components of the inter-field motion vector A→ ₁ separated by one frame are (x, y) = (3, 1)
The case is shown below. As in Fig. 4, the inter-field motion vector a → ₁₀ exists within the range surrounded by the dashed-dotted line, and b If _{the 1} field is at the point shown in the figure, the inter-field motion vector a → 10 exists within the range surrounded by the dashed line. b→ ₁₀ exists within the range surrounded by the two-dot chain line. In fact, such movement by panning is possible because the c ₁ field is within the range enclosed by the dash-dotted line as shown. At this time, the d ₁ field exists within the range indicated by the three-dot chain line in the figure, but as in the explanation in Figure 4, it is within the range shaded, and within that field, there is an extremely possibility that it exists at the position indicated by the black circle. expensive. As an example, the predicted vector (A→ ₁ + B→ ₁ )/4 and the vector b→ when the d ₁ field is at the position shown in the figure.
₁₀ is shown in FIG. The prediction error at this time is one pixel length on the diagonal, and if the d ₁ field is located at the position indicated by the black circle, the error will be less than that, so the prediction is valid in the case of FIG. 8 as well.

The prediction is also valid in the case of the figure.

As described above, the inter-field motion vector b→ ₁₀ separated by one field can be expressed as (A→ ₁ +B) using the inter-field motion vectors A→ ₁ and B→ ₁ separated by one frame.
→
Prediction can be made by approximating the integer part of ₁ )/4,
The error is about one pixel on the diagonal or less. Similarly, a→ ₁₀ ≒[(D→ ₀ +A→ ₁ )/4] b→ ₁₀ ≒[(A→ ₁ +B→ ₁ )/4] c→ ₁₀ ≒[(B→ ₁ +C→ ₁ )/4 ] Can be predicted as follows. However, [ ] represents the integer part.

However, in the motion compensation subsample interpolation method that has been proposed in the past, if we consider the timing of the transmitted video signal and motion vector, the motion vector between fields separated by one field is the motion vector between fields separated by one frame. Using vectors,
We have to predict as follows.

a→ ₁₀ =D→ ₀ - [C/→ ₁ +D/→ ₀ /4] b→ ₁₀ =A→ ₁ - [D/→ ₀ +A/→ ₁ /4] c→ ₁₀ =B→ ₁ - [A /→ ₁ +B/→ ₁ /4] d→ ₁₀ =C→ ₁ − [B/→ ₁ +C/→ ₁ /4] Moreover, the error in the prediction of the motion vector between fields separated by one field is the same as that in the later prediction. Therefore, even if there is a temporary large vector change and the error becomes large, predictions with small errors can be made again if the subsequent vector change becomes gentle.

The first motion correction is based on prediction as described above.
This will be explained using the timing chart shown in FIG.

First, when the signal of the _d1 field is input to the video input terminal 1, the signals of the a1 field and the _c1 field are input to the first motion compensation field memory 7, and the signals of the a1 field and _c1 field are input to the second motion compensation field memory 8. The signals of the d ₀ field and the b ₁ field are stored. At this time, when the motion vector B→ ₁ is input, the output of the 1-field delayer 3 is the motion vector A→ ₁ of the previous field, so the adder 4 outputs A→ ₁ +B→ ₁ , Furthermore, it is divided by 4 by the divider 5, and subtracted from B→ ₁ by the subtracter 6, and the integer part of the output B→ ₁ -1/4 (A→ ₁ + B→ ₁ ) becomes 1.
Inter-field motion vector c ₁₀
This is the predicted value. The contents of the first motion correction field memory 7 are the predicted vector B→ ₁ −[A/→ ₁ +B of the motion vector c ₁₀ between fields separated by one field.
/→ ₁ /4] Motion correction is performed based on the d ₁ field which is moved two-dimensionally and input to the video input terminal 1 as a reference.

On the other hand, the signals of the d ₀ field and the b ₁ field stored in the second field memory 8 are 1
At the time before the field, it is stored in the first motion correction field memory 7, and based on the signal of field _c1 inputted to the video input terminal 1 at this time, the motion vector _b10 between fields separated by one field is calculated. Predicted vector A→ ₁ − [D/→ ₀ +A/→ _1/4 ]
Since motion correction has been performed using the d ₁ field as a reference, motion correction can be performed using the predicted vector B→ ₁ − [A/→ ₁ +B/→ ₁ /4] as in the first motion correction field memory 7. .

In this way, signals for four fields, _a1 field, _b1 field, _c1 field, and _d1 field, for which motion compensation has been performed based on field d1, enter the inter-field interpolation filter 9 _, and perform inter-field interpolation. possible. Normally, the switch 11 is connected to the upper contact 11x, and the inter-field interpolated signal passes through it, but inter-field interpolation is not performed for video signals, so when motion is detected, the switch 11 is connected pixel by pixel. Intra-field interpolation is performed only from the signal connected to the lower contact 11y and passed through the switch 2 by the in-field interpolation filter 10.

The sample rate of the signal whose missing sample points are interpolated by the inter-field interpolation filter 9 or the intra-field interpolation filter 10 is 64.8 MHz, and is output from the video output terminal 12.

In an example in which motion vectors between fields separated by one field are determined by such prediction, not only is it stable at the start and end of panning, but even if there is a temporary error in the transmitted motion vector, it is Leaves no negative impact.

It is also stable even in special cases where the scene changes during panning or the image changes during panning. Furthermore, changes in motion vectors caused by panning become more gradual.

FIG. 10 shows the states of the signals 10a to 10f in FIG. 1 when the _A2 field signal is input to the video input terminal 1. In Fig. 10, the horizontal line above the symbol indicates that the movement has been corrected, A is the interpolated value interpolated from ₁ , ₁ , ₁ , _a2 by the interpolation function f _A , and B is the interpolated value ₁ by the interpolation function f _B. , a indicates the interpolated value interpolated from ₂ .

[Effects of the Invention] As described above, according to the present invention, a motion vector for one field is approximately predicted from a transmitted motion vector, and motion correction is performed for each field based on this approximate prediction. Because of this configuration, even when panning occurs, storage can be stabilized using the inter-field interpolation filter, just as in a still image state, and interpolation can be performed with less error, so the drop in resolution during panning can be sufficiently suppressed. The effect that can be achieved can be obtained.

[Brief explanation of the drawing]

Fig. 1 is a block circuit diagram of an embodiment of the present invention, Fig. 2 is a vector diagram showing an example of motion vectors during panning to explain the operation of this embodiment, and Fig. 3 is a diagram of fields separated by one frame. Figures 4, 6, and 8 show the existence ranges of inter-field motion vectors and inter-field motion vectors separated by one field, respectively. Figures 5, 7, and 9 are vector diagrams showing the relationships between the inter-field motion vectors predicted from the inter-field motion vectors separated by one frame, and the actual inter-field motions separated by one field. FIG. 10 is a vector diagram showing an example of vectors, and a timing chart showing the signal flow of this embodiment.
Figure 1 is a block circuit diagram of an example in which the conventional motion correction sub-sample interpolation method is applied, Figure 12 is a timing chart showing the signal flow when motion compensation is not performed in this conventional example, and Figure 13. is a timing chart showing the flow of signals when motion compensation is also performed. 2...Switch, 3...1 field delay device,
4... Adder, 5... Divider, 6... Subtractor, 7
...First motion correction field memory, 8...Second motion correction field memory, 9...Inter-field interpolation filter, 10...Intra-field interpolation filter, 11...Switch, 12...Video input terminal. In each figure, the same reference numerals indicate the same or corresponding parts.

Claims (1)

[Claims] 1. The sample values that are intermittently sub-sampled and transmitted at predetermined sample positions that go around in 4 fields are interpolated for missing sample values received on the receiving side to reproduce the video signal,
In the motion correction subsample interpolation method, which corrects the interpolation position of sample values to be interpolated by delaying the video signal based on motion information between fields separated by one frame of the video signal detected on the transmitting side, the field of interest and 1 from this field
The sum of first motion information between a past field that is separated by a frame, and second motion information between a field that is one field before the field of interest and a past field that is one frame apart from this field. , and subtract this value from the first motion information to approximately predict the motion information of the field of interest, and based only on this approximately predicted motion information, calculate the past three fields from the field of interest. A motion-compensated subsample interpolation method characterized in that interpolation of missing sample values is performed by correcting interpolation positions of sample values.