JPH07114498B2 - Motion-corrected subsample interpolation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention is applied to image processing in a MUSE-type high-definition television receiver, and calculates the amount of motion correction between fields from transmitted motion information. The present invention relates to a motion interpolation positive subsample interpolation method in which a missing point is calculated and interpolated between fields.

[Prior Art] NHK Research Institute of Technology and NHK Science and Technology Research Laboratories announced at the foundation commemorative lecture in June 1984 to the motion compensation subsample interpolation method of this kind that has been proposed so far. There is something mentioned in a preliminary document called "New Transmission System (MUSE) of High Definition Television". FIG. 11 shows the configuration of the motion correction subsample interpolation method.

In FIG. 11, (1) is a video input terminal for inputting a video signal transmitted at a sample rate of 16.2 MHz, (2) is a switch, and the video signal input to the input terminal (1) and the movement described later. The output signal of the correction field memory (4) is switched at the timing of 32.4 MHz sub-sampling.
(3) is a non-motion compensation field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2). Reference numeral (4) is a motion correction field memory, which stores a signal for one frame at a sample rate of 32.4 MHz output from the non-motion correction field memory (3) and performs motion correction by using a motion vector.

(10) is an inter-field interpolation filter, which interpolates the missing sample points based on the output signal of the non-motion correction field memory (3) and the signal passed through the switch (2). (1
1) is an intra-field interpolation filter that interpolates the missing sample points based only on the signal that has passed through the switch (2). (12) is a switch, which is connected to the upper contact when motion compensation is not performed, allows the output signal of the inter-field interpolation filter (10) to pass, and is used as the lower contact when motion compensation or motion detection is performed. It is connected to allow the output signal of the inter-field interpolation filter (11) to pass.
Reference numeral (13) is a video output terminal for outputting a signal which passes through the switch (12) at a sample rate of 64.8 MHz by interpolating the missing sample points.

Next, the operation of the above configuration will be described.

The transmission method for high-definition television is sub-Nyquist sampling, which makes a cycle of four fields, and its required bandwidth is 8.1M.
Hz.

When the signal of the first field is input to the video input terminal (1), the non-motion compensation field memory (3) stores the signals of the second field and the fourth field one cycle before,
On the other hand, in the motion compensation field memory (4), the first
The signals of the field and the third field are stored.
When a motion vector exists due to the panning of the camera, the contents of the motion correction field memory (4) move two-dimensionally according to the vector amount. At this time, if the transmitted motion vector is a motion correction amount between fields separated by one frame, the motion correction content of the motion correction field memory (4) is input to the video input terminal (1) in the first field. The signal is used as a reference.

The switch (2) is switched at the timing of 32.4 MHz sub-sampling, and the phase is inverted for each field and also by the motion vector. Therefore, in the above case, the signal of the first field input from the video input terminal (1) and the signal of the third field one cycle before the motion correction are passed through the switch (2).

The signal that has passed through the switch (2) and the output signal of the non-motion correction field memory (3) are input to the inter-field interpolation filter (10) to perform inter-field interpolation.

When motion compensation is not performed, that is, when the switch (12) is connected to the upper contact (12x), the video output terminal (1
The video signal with a sample rate of 64.8 MHz interpolated between fields is output from 3).

Also, the signal of the field that passed through switch (2) is
It is input to the inter-field interpolation filter (11) to perform inter-field interpolation, and the switch (12) moves to the lower contact (12
When it is connected to y), the video signal with the sample rate of 64.8MHz interpolated within the field is output from the video output terminal (13).

When a ₂ field is input to the video input terminal (1) is not performed motion compensation, the state of 12a~12e signals Figure 11 are shown in FIG. 12, however, the representative of the signal at the field ...... a ₁ , b ₁ , c ₁ , d ₁ ,
It flows in the order of a ₂ , b ₂ , c ₂ , d ₂ . A in FIG. 12 indicates an interpolated value interpolated from b ₁ , c ₁ , d ₁ , a ₂ by an interpolating function f _A which is inter-field interpolation.

Also, when motion compensation is performed and the a ₂ field is input to the video input terminal (1), the signal states of 13a to 13e in FIG. 11 are as shown in FIG. 13, and the symbols in FIG. The horizontal line above indicates that motion compensation has been performed, and B indicates the interpolation value interpolated from ₁ and a ₂ by the interpolation function f _B that is field interpolation.

[Problems to be Solved by the Invention] In the conventional motion compensation sub-sample interpolation method as described above, since the relationship between adjacent fields is not taken into consideration at the time of performing motion compensation, the missing sample points are not taken into consideration. Must be done in the field by an in-field interpolation filter. Therefore, there is a problem that the resolution at the time of performing the motion correction is lowered.

The present invention has been made in order to solve the above problems, and also when the motion vector between the fields is calculated from the transmitted motion vector and the motion correction is performed in the same manner as when the motion correction is not performed. It is an object of the present invention to provide a motion-compensated sub-sample interpolation method capable of suppressing a decrease in resolution by using an inter-field interpolation filter.

[Means for Solving the Problems] In the motion compensation sub-sample interpolation method according to the present invention, sample values transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position. Is a sub-sampling interpolation method for reproducing a video signal by interpolating the missing points from the sample values received in the four-field period on the receiving side, based on 1 frame interval of the video signal detected on the transmitting side. In the motion-correction sub-sample interpolation method in which the position of the sample value one frame before is corrected on the receiving side based on the inter-field motion vector, and the missing point is interpolated to reproduce the video signal, the adjacent fields 2 types of 1 transmitted at the same time as
Generate from the inter-field motion vectors separated by frames by approximating the inter-field motion vectors separated by one field on the receiving side to 1/4 of the sum of the above two types of inter-field motion vectors separated by one frame, and then transmit By using the inter-field motion vector separated by one frame detected on the side and the inter-field motion vector generated by the reception side by one field, relative to the sample value of the past three fields based on the field of interest. The position is corrected and the missing point is interpolated from the sample value received in the four-field period in the moving image as well as the interpolation in the still image.

[Operation] According to the present invention, as described above, by utilizing the fact that the transmitted motion information does not change abruptly, an inter-field motion vector separated by one field at the receiving side is
Generated by approximating to 1/4 of the inter-field motion vector 1 frame apart detected on the transmitting side, and the inter-field motion vector 1 frame apart detected on the transmitting side and the above 1 generated on the receiving side. Since the inter-field motion vectors separated by frames are used, even when a moving image is always used with an inter-field interpolation filter, the inter-field interpolation is performed similarly to a still image to improve the resolution in the case of a moving image. be able to.

[Embodiment] An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a motion compensation sub-sample interpolation method according to an embodiment of the present invention. In FIG. 1, (1) shows a video image input with a video signal transmitted at a sample rate of 16.2 MHz. Input terminal, (2) is a switch,
32.4 the video signal input to the input terminal (1) and the output signal of the first motion compensation field memory (4) described later.
Switch at the timing of the subsample of MHz. Reference numeral (3) is a non-motion correction field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2) and performs motion correction using a motion vector.

(5) is a 1-field delay device, which delays the transmitted motion vector by 1 field. (6) is an adder,
The transmitted motion vector is added to the motion vector one field before, which is the output of the one field delay unit (5). (7) is a divider, which divides the output of the adder (6) by 4. A subtracter (8) subtracts the output of the divider (7) from the transmitted motion vector.

(9) is a second motion compensation field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2), and predictive motion between fields which is an output of the subtracter (8). Motion correction is performed using a vector. (10) is an inter-field interpolation filter, which interpolates a missing sample point based on the signal passing through the switch (2) and the output signal of the second motion correction field memory (9). (11) is an intra-field interpolation filter that interpolates the missing sample points based on only the signal passing through the switch (2). (12) is a switch, which is normally connected to the upper contact (12x) and is connected to the inter-field interpolation filter (1
0) output signal is passed, and when a motion is detected, the output signal of the inter-field interpolation filter (11) is passed by being connected to the lower contact (12y) in pixel units. (13)
Is a video output terminal for outputting a signal which passes through the switch (12) at a sample rate of 64.8 MHz by interpolating the missing sample points.

Next, the operation of the above configuration will be described.

FIG. 2 shows motion vectors during panning in order to specifically explain the operation of FIG.
Represents the horizontal axis of the screen and y represents the vertical axis of the screen.

Each field of the video signal is represented by a _{1 to} d ₂ , and when the signal is represented by a field ... a ₀ , b ₀ , c ₀ , d ₀ , a ₁ , b ₁ , c ₁ , d ₁ , a ₂ , b ₂ ,
It flows in the order of c ₂ , d ₂ . The motion vectors between fields separated by one frame are ... ₁ , ₁ , ₁ , ₁ ,
It is represented by ₂ , ₂ , .... Also, the inter-field motion vectors separated by one field that are obtained in the plot ... ▲
It is represented by ▼, ▲ ▼, ▲ ▼, ▲ ▼ .... The following relational expression holds between the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field.

Here, for example, ▲ ▼ can be rewritten as the following equation.

If the second term on the right side of the above equation is sufficiently small, ▲ ▼ can be predicted as in the following equation.

Here, for example, consider only the four-field relationship of a _{1 to} d ₁ . However, the inter-field motion vectors separated by one frame are transmitted in one field by a digital signal of horizontal component 5 bits and vertical component 3 bits.
In other words, when describing the magnitude of the inter-field motion vector separated by one frame in terms of pixel length, the horizontal component is within the range of −15 to +16, the vertical component is within −3 to +4, and the horizontal component and the vertical component are The direction component also has a discrete component that is an integer. When this is illustrated, as shown in FIG.
The inter-field motion vectors separated by frames are within a range surrounded by a broken line. However, the origin in FIG. 3 is an arbitrary pixel of a ₁ field.

If the parallel movement of the image due to panning is smooth, a ₁
The inter-field motion vector separated by one field between the field and b ₁ field is within the range surrounded by the chain line in FIG. Here, the relative relationships of the three fields a _{1 to} c ₁ are shown in FIGS. 4, 6, and 8 by changing the three relative relationships, and the prediction accuracy of the position and motion vector of the d ₁ field is shown for each of them. Find out.

First, FIG. 4 shows a case in which the magnitude of the inter-field motion vector (1) separated by one frame is maximum. At this time, the inter-field motion vectors ▲ ▼ and ▲ ▼ separated by one field also become maximum, and a ₁ ~
c ₁ The relative position of the field is defined. At this time,
The d ₁ field exists within the range of the three-dot chain line in the figure,
If there is no sudden movement change, the d ₁ field exists within the range of the diagonal line. Furthermore, if the parallel movement of the image such as panning is smooth, the d ₁ field is likely to exist at the position indicated by the black circle in the figure. As an example, the prediction vector when the d ₁ field is at the position shown And vector ▲ ▼ are shown in FIG. The prediction error at this time is one pixel length, and if there is a d ₁ field at the position represented by the black circle, the error will be less than that, so such prediction is appropriate.

Next, FIG. 6 shows the case where the components of the inter-field motion vector ▲ ▼ separated by one frame are (x, y) = (10, 2). Similar to FIG. 4, if the inter-field motion vectors ▲ ▼ separated by one field exist within the range surrounded by the chain line, and b ₁ field is at the point shown in the figure,
Motion vector between fields separated by one field ▲
▼ exists within the range surrounded by the chain double-dashed line. In fact, such movement by panning is possible because the c ₁ field is within the range enclosed by the chain double-dashed line as shown. At this time, the d ₁ field exists within the range of the three-dot chain line in the figure, but within the range of the diagonal lines as in the description in FIG. 4, it is highly possible that it exists at the position indicated by the black circle. . As an example, the prediction vector when the d ₁ field is at the position shown And vector ▲ ▼ are shown in FIG. The prediction error at this time is 1 pixel diagonal, and d ₁
If there is a field, the error will be less than that, so the prediction is valid in the case of FIG.

Next, FIG. 8 shows a case where the components of the inter-field motion vector ▲ ▼ separated by one frame are (x, y) = (3, 1). As in FIG. 4, if the inter-field motion vector ▲ ▼ separated by one field exists within the range surrounded by the chain line and b ₁ field is at the point shown in the figure, 1
Motion vector between fields separated by fields ▲ ▼
Exists within a range surrounded by a chain double-dashed line. In fact, such movement by panning is possible because the c ₁ field is within the range enclosed by the chain double-dashed line as shown. At this time, the d ₁ field exists within the range of the three-dot chain line in the figure, but it exists within the range of the diagonal lines as in the explanation in FIG. 4, and it is highly possible that it exists at the position indicated by the black circle. high. As an example, the prediction vector when the d ₁ field is at the position shown And vector ▲ ▼ are shown in FIG. The prediction error at this time is 1 pixel diagonal, and d ₁
If there is a field, the error will be less than that, so the prediction is valid in the case of FIG.

As described above, the inter-field motion vector ▲ ▼ separated by one field is calculated using the inter-field motion vectors ▲ ▼ and ▲ ▼ separated by one frame. Prediction can be performed by approximation with an integer part of, and the error is about one pixel diagonal or less. Similarly, Can be predicted like. However, [] represents an integer part.

Moreover, in the prediction of the inter-field motion vector separated by one field, the error does not affect the subsequent prediction, so even if there is a temporary large vector change and the error becomes large, the subsequent vector changes will not become gradual. For example, the prediction with small error can be made again.

The motion correction performed by the above prediction will be described with reference to FIG.

First, when a signal of d ₁ field is input to the video input terminal (1) of FIG. 1, the non-motion compensation field memory (3) stores
The signals of d ₁ field and c ₁ field are stored, and the signals of d ₀ field and b ₁ field are stored in the first motion compensation field memory (4). At this time, if the motion vector ▲ ▼ is input, the d ₀ field and the b ₁ field stored by the motion vector ▲ ▼ are two-dimensionally moved in the first motion correction field memory (4), and the image is moved. Motion compensation is performed based on the d ₁ field input to input terminal (1). As a result, the signal passing through switch (2) is d ₁ field and motion vector B ₁
It is a signal with b ₁ field corrected by.

On the other hand, the input motion vector ▲ ▼ is input to the 1-field delay device (5), and the output of the 1-field delay device (5) is the motion vector 1 field before.
Is added to the motion vector ▲ ▼ by the adder (6) and further divided by 4 by the divider (7), and the integer part of the output is the predicted value of the inter-field motion vector ▲ ▼ separated by one field. Become.

The output of the divider (7) is subtracted from the motion vector () by the subtractor (8) to obtain the predicted value of the inter-field motion vector (1) separated by one field. The content of the second motion compensation field memory (9) is the prediction vector of the inter-field motion vector ▲ ▼ separated by one field. Thus, the movement is two-dimensionally performed, and motion correction is performed based on the d ₁ field.

In this way, motion compensation was performed based on the d ₁ field.
Signals for 4 fields of a ₁ field, b ₁ field, c ₁ field, d ₁ field itself enter inter-field interpolation filter (10) to enable inter-field interpolation.
Normally, the switch (12) is connected to the upper contact (12x), and the inter-field interpolated signal passes through. However, since the inter-field interpolation is not performed for the moving image signal, when the motion is detected, the switch (12) is ) Is connected to the lower contact (12y) in pixel units, and the inter-field interpolation filter (1
By 1), in-field interpolation is performed only from the signal that has passed through the switch (2).

The sample rate of the signal in which the missing sample points are interpolated by the inter-field interpolation filter (10) or the intra-field interpolation filter (11) is 64.8 MHz and is output from the video output terminal (13).

In the motion-compensated sub-sample interpolation method that obtains inter-field motion vectors separated by one field by such prediction, not only is it stable at the beginning and end of panning, but also the transmitted motion vector has a temporary error. Even if it does, it will not have any adverse effects.

It is also stable in special cases where the scene changes during panning and the image becomes a panning image. Furthermore, since the change of the motion vector due to panning is gradual, it can be said that this method is very good.

FIG. 10 shows the states of the signals from 10a to 10g in FIG. 1 when the a ₂ field signal is input to the video input terminal (1). In FIG. 10, the horizontal line above the symbol indicates that the motion has been corrected, A is the interpolation value interpolated from ₁ , ₁ , ₁ , ₁ , a ₂ by the interpolation function f _A , and B is ₁ by the interpolation function f _B. , a ₂ is an interpolated value.

[Effects of the Invention] As described above, according to the motion compensation subsample interpolation method according to the present invention, subsamples are transmitted intermittently so as to make a cycle of four fields while maintaining a predetermined sample position. A sub-sample interpolation method for reproducing a video signal by interpolating a missing point from the sample value received in a 4-field period on the receiving side based on the sample value,
The motion of reproducing the video signal by interpolating the missing point after correcting the position of the sample value one frame before on the reception side based on the inter-field motion vector of the video signal detected by the transmission side one frame apart. In the corrected sub-sample interpolation method, an inter-field motion vector separated by one field on the receiving side is calculated from two inter-field motion vectors separated by one frame at the same time as the adjacent fields. Generated by approximating to 1/4 of the sum of inter-field motion vectors separated by frames, and the inter-field motion vector separated by 1 frame detected by the transmitting side and the field separated by 1 field generated by the receiving side. Using the inter-motion vector, the relative values of the sampled values of the past three fields with respect to the field of interest Correcting the location, as with interpolation in a still image, since the from the sample values received in 4-field period even videos to interpolate the missing points,
Similar to a still image, even in a moving image, the missing points can be interpolated from the sample values received in the four-field period, and as a result, even in the case of a panned moving image, an image with excellent resolution can be obtained. There is an effect that can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a schematic configuration of a motion correction sub-sample interpolation method according to an embodiment of the present invention, and FIG. 2 is an example of a motion vector at the time of panning for explaining the operation of FIG. FIG. 3 is a vector diagram showing the existence range of inter-field motion vector separated by one frame and inter-field motion vector separated by one field, FIG. 4, FIG.
FIGS. 8 and 9 are vector diagrams showing the relationship between inter-field motion vectors separated by one frame and inter-field motion vectors separated by one field, FIG. 5, FIG. 7, FIG.
FIG. 10 is a vector diagram showing an example of an inter-field motion vector separated by one field predicted from an inter-field motion vector separated by one frame and an actual inter-field motion vector separated by one field. FIG. 10 is a motion correction according to the present invention. A timing chart showing the signal flow of the sub-sample interpolation method, FIG. 11 is a block diagram showing an example of a schematic configuration of the conventional motion-compensated sub-sample interpolation method, and FIG. 12 is a conventional motion-compensated sub-sample interpolation method. Timing chart showing the signal flow when motion compensation is not performed
FIG. 13 is a timing chart showing a signal flow in the case of performing the motion compensation of the conventional motion compensation sub-sample interpolation method. (1) …… Video input terminal, (2) …… Switch, (3)
...... Non-motion correction field memory, (4) ...... First motion correction field memory, (5) ...... 1 field delay device, (6) …… Adder, (7) …… Divider, (8 ) ……
Subtractor, (9) ... Second motion compensation field memory,
(10) ... Inter-field interpolation filter, (11) ... Inter-field interpolation filter, (12) ... Switch, (13) ...
… Video output terminal. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (1)

[Claims] 1. Based on a sample value transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position, the sample value is received from the sample value received in a four-field period on the receiving side. It is a sub-sampling interpolation method of reproducing a video signal by interpolating a missing point, and based on an inter-field motion vector separated by one frame of the video signal detected on the transmission side, the sample value one frame before on the reception side. In the motion-compensated sub-sample interpolation method that corrects the position of, and reproduces the video signal by interpolating the missing point, two types of inter-field motion vectors separated by one frame and transmitted at the same time as adjacent fields From the inter-field motion vector separated by one field on the receiving side from the inter-field motion vector of the above two types separated by one frame. Generated by approximating to 1/4 of the sum of the tor, and using the inter-field motion vector detected by the transmitter at one frame apart and the inter-field motion vector generated by the receiver at one field apart. Then, the relative position of the sample values of the past 3 fields is corrected based on the field of interest, and the missing point is interpolated from the sample values received in the 4 field period in the moving image as well as the interpolation in the still image. Characteristic motion compensation sub-sample interpolation method.