JP3206413B2 - Variable frame rate video coding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding method for coding a moving picture signal with motion compensation, and more particularly to a method for performing variable frame rate coding which is effective when transmitting at a low bit rate. The present invention relates to a variable frame rate video coding method.

[0002]

2. Description of the Related Art As a method of determining the number of frame skips in a conventional variable frame rate moving picture coding apparatus, a current frame is encoded to determine an actually generated code amount, and based on the generated code amount, a frame skip number is calculated. Control was done.

That is, first, the target number of coded bits of the current frame is obtained from the amount of information generated in the previous frame.
The number of quantization steps is determined from the target number of coding bits, and the current frame is encoded with the number of quantization steps. After actually encoding the current frame in this way, depending on the amount of code stored in the buffer, frame skipping is performed so that transmission at a given bit rate can be performed until the next frame is encoded. Was to determine the number.

[0004]

As described above, according to the conventional method, after a current frame is actually coded, a frame until a next frame is coded according to the amount of code stored in a buffer. The number of skips was determined. That is, the number of frame skips is changed so as to absorb the difference between the target number of bits obtained from the coding result of the previous frame and the number of bits necessary for actually coding the current frame, and the frame after the next frame is changed. Since the rate was controlled, the following problems occurred.

In the conventional method, the magnitude of the amount of information generated in each frame is used for quantization control as much as possible, and the number of frame skips is reduced if the target number of encoded bits obtained from the previous frame cannot be achieved. Because of the control of changing the frame rate, the control of the frame rate is a by-product element after the quantization, and it is impossible to control the frame rate using the characteristics of the image. For this reason, when the actual code amount of the current frame is large, the skip number is automatically increased and the quantization is coarse, so that the image quality is degraded. If the amount of movement is large, the image becomes unnatural as if frame skipping was performed.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a variable frame rate moving picture coding method which maintains a constant picture quality and produces a moving picture having a visually natural motion.

[0007]

According to the present invention, a macroblock of a previous frame is subjected to motion compensation using a motion vector, and the difference between the motion-compensated macroblock and the current macroblock to be coded is obtained. By calculating a prediction error and comparing the calculated prediction error with a threshold, it is determined whether the current macroblock is a non-significant macroblock or a large significant macroblock smaller than the threshold. The present invention relates to a variable frame rate moving image encoding method in which encoding is performed in the case of a non-significant macroblock and only a motion vector is transmitted in the case of a non-significant macroblock. In particular, according to the present invention, the ratio of significant macroblocks occupying the current frame is calculated, and the number of frame skips from the current frame to the next frame is determined before encoding using the calculated ratio.

By determining the number of frame skips before encoding the current frame, it is possible to determine the number of frame skips that makes full use of the characteristics of the image. As a result, the amount of information of the image is predicted in advance,
The frame rate can be controlled, and variable frame rate image coding with constant quality has become possible.
In addition, if the number of frame skips is controlled according to the ratio of significant macroblocks occupying the current frame, it is possible to maintain a constant quality.

The number of frame skips is determined using a function that increases as the ratio of significant macroblocks increases, or a function that increases as the ratio of significant macroblocks increases and saturates when this ratio exceeds a certain value. Is preferred. If the proportion of significant macroblocks occupying the current frame is large, the amount of information necessary to encode the current frame is considered to be large, so the number of frame skips is kept to a minimum until a certain quality can be maintained. I'm making it big. However, a significant increase in the number of frame skips decreases the efficiency of information amount reduction by motion compensation. Therefore, the number of frame skips is saturated rather than being continuously increased with respect to an increase in the ratio of significant macroblocks.

Further, according to the present invention, the magnitude of the motion vector in the current frame is determined, and the number of frame skips from the current frame to the next frame is determined before encoding using the determined magnitude of the motion vector. I have. As described above, since the motion vector information is used for controlling the frame rate, a visually natural motion image can be obtained.

The number of frame skips is preferably determined using a function that decreases as the magnitude of the motion vector increases. When the motion vector scalar length of a significant macroblock is large, an image having visually unnatural motion is caused by increasing the number of frame skips. Therefore, the number of frame skips is slightly reduced.

It is preferable that the number of frame skips is determined by further using the amount of information generated immediately before encoding the current frame. In this case, it is more preferable to use a function that increases as the amount of generated information up to immediately before increases. As described above, when the amount of generated information until immediately before is large, the buffer overflow can be prevented by slightly increasing the skip number and suppressing the generated information amount.

It is also preferable that the quantization step is controlled using the determined number of frame skips. In this case, it is more preferable that the quantization step is performed using a function that becomes finer as the number of frame skips increases. When the number of frame skips is large, there is room in the transmission path, so a slightly finer quantization step is used. Conversely, when the number of frame skips is small, a slightly coarser quantization step is used.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram schematically showing the configuration of an embodiment of the moving picture coding apparatus according to the present invention.

In FIG. 1, reference numeral 1 denotes the configuration of the entire variable frame rate moving picture coding apparatus. The moving picture coding apparatus 1 includes a video signal input unit 2, a frame skip unit 3 connected to the video signal input unit 2 and performing a frame skip on the input video signal, and a frame skip unit 3. , A frame memory 4 for storing at least the video signals of the previous frame and the current frame inputted by skipping the frame, and a current macroblock to be encoded of the current frame outputted from the frame memory 4 (the macro block of interest). ), And an encoding engine unit 6 to which the video signal 5 is input.

The frame memory 4 is connected to the above-described encoding engine unit 6, motion compensation unit 8, and subtraction unit 13. As will be described later, a motion vector generation unit 11 that sequentially generates motion vectors from a small scalar length to a large scalar length and outputs the motion vector information 9 and the scalar length information 10 includes a motion compensation unit 8 and an encoding engine. Part 6
, A VLC (variable length coding unit) 23, a macroblock significant / insignificant determination unit 15, and a frame skip number determination unit 20. The motion compensator 8 stores the video signal 7 of each macroblock of the previous frame in the frame memory 4.
Given by

The subtractor 13 has a video signal 5 of the current macroblock of interest from the frame memory 4 connected to its + input, and a previous frame whose motion has been compensated by the motion compensator 8 connected to the-input. The video signal 12 of the macro block is input. The output of the subtraction unit 13 is connected to a significant / insignificant determination unit 15. The significant / insignificant determination unit 15 includes a difference value 14 between macroblocks from the subtraction unit 13.
The scalar length information 10 is input from the motion vector generator 11. The output of the significant / insignificant determination unit 15 is connected to the frame skip number determination unit 20.

The frame skip number determining section 20 is further connected to the output buffer memory section 17 and the frame skip section 3. The significant / insignificant determining section 15 outputs significant macroblock ratio information 16 to the motion vector generating section 1.
The generated information amount 18 from 1 to the scalar length information 10 from the output buffer memory unit 17 to the immediately before is input, and the frame skip number information 19 is generated and output to the frame skip unit 3 and the encoding engine unit 6.

The variable length coding unit 23 is connected to the significant / insignificant determining unit 15, the coding engine unit 6, and the motion vector generating unit 11, and the macroblock significant information 21, the quantization information 22 and motion vector information 9
And performs encoding. The variable length encoding unit 23 is further connected to an input of the output buffer memory unit 17 and outputs an encoded output 24. The output of the output buffer memory unit 17 is connected to the code output unit 25.

The encoding engine unit 6 has a known encoding device configuration except for a partial configuration described later. That is, the mode switches 6a and 6b that perform switching according to the inter-frame prediction or the intra-frame prediction, the subtraction unit 6c, the addition unit 6d, and the significant / insignificant determination unit 15 determine the significance according to the macroblock significance information 21. , A switch 6e which is turned off if insignificant, a DCT (discrete cosine transform) circuit 6f, a Q (quantization circuit) 6g, a Q ^-1 (inverse quantization circuit) 6h, and an IDCT ( It mainly includes a discrete cosine inverse transform circuit 6i, a frame memory 6j, and a motion compensator 6k.

FIG. 2 is a flowchart for explaining the main operation in this embodiment. Hereinafter, the operation of this embodiment will be described with reference to FIG.

When a video signal is input to the video signal input unit 2 (step S1), it is determined whether or not this frame is the first frame (step S2). When the frame is the first frame, the video signal is input to the frame memory unit 4 and stored in the current frame memory without performing the frame skip operation (step S3) in the frame skip unit 3 (step S4). Next, it is determined again whether this frame is the first frame (step S).
5).

If the frame is the first frame, the frame skip number 19 output from the frame skip number determination section 20 is set to an initial value N (frames) (step S6).
Next, the video signal is input to the encoding engine unit 6 and is subjected to intra-frame encoding. After being converted into a bit stream by the variable length encoding unit 23, the output buffer memory unit 1
7 is output from the code output unit 25 (step S7). Thereafter, the contents stored in the current frame memory of the frame memory unit 4 are moved to the previous frame memory (step S8), and the process returns to step S2.

Thereafter, since the subsequent frames are the second and subsequent frames,
In the frame skip unit 3, the number of frame skips is 1
A frame skip operation of the input video signal according to No. 9 is performed (step S3). The video signal whose frame has been skipped in the frame skip unit 3 is stored in a frame memory 4.
(Step S4). Thereafter, the images stored in the current frame memory and the previous frame memory of the frame memory 4 are divided into a predetermined number of macro blocks (step S9).

Next, it is determined whether or not the processing has been completed for all the macroblocks of the current frame (step S).
10) If not completed, the motion vector generator 11
Then, a motion vector is generated (step S11).
The motion vector is generated by increasing the size of the motion vector from the zero vector (0, 0) by a certain number of pixels so that the magnitude gradually increases.

FIG. 3 is a conceptual diagram of the operation of the motion vector generation unit. Referring to FIG.
1 will be described. Each grid in FIG. 3 indicates a minimum unit of a motion vector. For example, if the increment step of the motion vector is a quarter pixel unit, each grid is a quarter.
Represents a pixel. The circled numbers indicate the order in which the motion vectors are generated. Starting with the zero vector (0,0), (0,1), (1,
0), (0, -1), (-1, 0), (1,
1),... And motion vectors are generated in ascending order of magnitude.

This motion vector is represented by motion vector information 9
Is applied to the motion compensation unit 8, the encoding engine unit 6, and the variable length encoding unit 23. The scalar length is applied as motion vector scalar length information 10 to the macroblock significance determining unit 15 and the frame skip number determining unit 20.

Next, it is determined whether or not all types of motion vectors have been generated in the motion vector generation section 11 (step S12). If no motion vector has been generated, the motion compensation section 8 performs motion compensation processing on the macroblock. (Step S13). In other words, a macroblock that has been subjected to motion compensation with one piece of motion vector information 9 given from the motion vector generation unit 11 to the macroblock of interest of the previous frame stored in the frame memory 4 is extracted,
The motion compensation macro block 12 is output.

Next, the target macroblock 5 and the motion compensation macroblock 12 of the current frame are input to the subtraction unit 13, the difference absolute value 14 (distortion) of each pixel is calculated, and the result is sent to the macroblock significance determination unit 15. (Step S1)
4). The macroblock significance determining unit 15 determines whether the macroblock is significant or non-significant from the difference absolute value 14 (step S15).

FIG. 4 is a conceptual diagram of the operation of the macroblock significance determination section 15. The function of the significance determination section 15 will be described below with reference to FIG. First, a macroblock is divided into four halves vertically and horizontally, and the sum of absolute differences of the divided blocks is calculated. The sum of the four is T1, T
When T2 and T3 are set to T4, the macroblock having the maximum value max (T1, T2, T3, T4) of T1 to T4 exceeding the threshold Th is determined as a significant macroblock, and the others are determined as insignificant macroblocks. I do.

The threshold value Th for this determination is determined by a function including a motion vector scalar length as a parameter, as shown in an example in FIG. This function is not limited to the characteristic curve shown in FIG. 5, but is large when the motion vector scalar length is small, decreases when the motion vector length is large, and saturates when the motion vector exceeds a certain value. Any characteristic curve (straight line) may be used.
As a result, the smaller the motion vector is, the more likely it is to determine a non-significant macroblock.

If it is determined that the motion vector is insignificant, the motion vector and its "insignificant" information are stored (step S16).
The process proceeds to the next macro block (step S17), and the processes of steps S10 to S18 are repeated. If it is determined to be significant, the process returns to step S11 to generate a motion vector of the next magnitude, and performs the same processing in steps S12 to S15.

In step S12, if the motion vector generator 11 determines that all types of motion vectors have been generated, the macroblock has not been determined to be insignificant to the end, and is finally a significant macroblock. It will be. In this case, the motion vector having the smallest inter-frame difference value and the difference value are stored (step S18), and the process proceeds to the next macroblock (step S1).
7), repeat the processing of steps S10 to S18.

As described above, the significance of the current frame is determined for each vector obtained from the motion vector generation unit 11 for each macroblock. When this process is completed for all macroblocks (step S
10), the skip number 19 up to the next frame is calculated by the frame skip number determination unit 20 (step S1).
9). The frame skip number determining unit 20 determines the ratio information 16 of the significant macroblock in the input frame,
Scalar length information 1 of motion vector of significant macroblock
0 and / or a frame skip number 19 by a function including the amount of generated information 18 immediately before encoding as a parameter.
To determine.

FIG. 6 shows an example of a preferable characteristic of the number of frame skips with respect to the ratio of significant macroblocks. When the ratio of significant macroblocks is zero, the number of frame skips is zero, and as the ratio of significant macroblocks increases, the number of frame skips also increases. When the ratio of significant macroblocks exceeds a certain value, the characteristic is saturated. It should be noted that the numerical value of the frame skip number in the figure is merely an example and is not limited to this.
As described above, when the proportion of significant macroblocks occupying the current frame is large, the amount of information necessary to encode the current frame is considered to be large. Is as large as possible. However, a significant increase in the number of frame skips reduces the efficiency of information amount reduction by motion compensation, so that the number of frame skips 19 does not keep increasing but saturates with an increase in the significant macroblock ratio 16. Such a tendency.

FIG. 7 shows an example of a preferable characteristic of the frame skip number with respect to the scalar length of the motion vector. The characteristic is that the number of frame skips decreases as the motion vector scalar length increases. It should be noted that the numerical value of the number of frame skips in the figure is merely an example, and is not limited to this. As described above, when the motion vector scalar length 10 of the significant macroblock is large, if the frame skip number 19 is increased, the image becomes visually unnatural in motion.
9 is slightly reduced.

FIG. 8 shows an example of a preferable characteristic of the number of frame skips with respect to the amount of information generated immediately before (the ratio of the information storage amount to the capacity of the output buffer memory unit 17). As the amount of generated information up to immediately before increases, the number of frame skips also increases. It should be noted that the numerical value of the number of frame skips in the figure is merely an example, and is not limited to this. As described above, when the generated information amount 18 immediately before is large, the skip number is slightly increased to suppress the generated information amount, thereby preventing the buffer overflow.

The number of frame skips is one of the ratio of the significant macroblock, the scalar length of the motion vector of the significant macroblock, and the amount of information generated immediately before encoding.
One function may be a function having one parameter, or a function having all of them as parameters.

FIG. 9 three-dimensionally shows a preferable characteristic of the frame skip number in that case. That is, when the significant macroblock ratio is zero, the number of frame skips is zero. As the significant macroblock ratio increases, the number of frame skips also increases. As the length increases, the number of frame skips decreases, and as the amount of generated information up to immediately before increases, the number of frame skips also increases.

As an example, the significant macro block ratio in the frame of interest is 30%, and the motion vector scalar length is 6
Assuming that the information generation amount up to 0 and immediately before occupies 70% of the output buffer, the skip count is 7 in FIG. 6, the skip count is +1 in FIG. 7, and the skip count is −1 in FIG. So the final number of frame skips in this case is
7 + 1 + (-1) = 7.

After the number of frame skips up to the next frame is determined, the determined number of frame skips 19, the target macroblock 5, the motion vector information 9 of the current frame, and the significant information 21 of the macroblock are respectively transmitted to the encoding engine unit 6. (Step S20), and also inputs the motion vector information 9 and the significant information 21 of the macroblock to the variable length coding unit 23.

In the coding engine unit 6, the determined frame skip number 19 is input to the quantization unit 6g and used as a parameter for selecting a quantizer. When the number of frame skips 19 is large, the transmission path has room, so that a slightly finer quantizer is selected. Conversely, when the frame skip number 19 is small, a slightly coarser quantizer is selected. FIG. 10 shows an example of a preferable characteristic of the quantization step with respect to the number of frame skips. As the number of frame skips increases, the number of quantization steps becomes smaller. It should be noted that the numerical value of the number of frame skips in the figure is merely an example, and is not limited to this.

The significant information 21 of the macroblock is applied to the switch 6e. When the macroblock is a significant macroblock, the switch 6e is turned on to quantize the prediction error relating to the macroblock. Is turned off, and the prediction error for the macroblock is not quantized. Other operations of the encoding engine unit 6 are known, and thus description thereof is omitted.

The quantized data 22 (in the case of a significant macro block) of the frame obtained by the encoding engine unit 6,
Motion vector information 9 and macroblock significant information 21
Is encoded by the variable length encoding unit 23 (step S2).
1) The data is output from the code output unit 25 to the transmission path through the output buffer memory unit 17. Then, the same operation from step S2 is repeated for the next frame.

As described above, according to the present embodiment, the scalar length of a motion vector of a non-significant macro block transmitting only a motion vector is reduced within a range in which the macro block is determined to be non-significant. , The amount of generated motion vector information can be suppressed.

By determining the number of frame skips before coding the frame of interest, it is possible to determine the number of frame skips that makes full use of the features of the image. As a result, the amount of information possessed by the image can be predicted in advance, the frame rate can be controlled, and the variable frame rate image coding with a constant quality can be performed. Further, since the motion vector information can be used for controlling the frame rate, a visually natural motion image can be obtained.

The embodiments described above all show the present invention by way of example and not by way of limitation, and the present invention can be carried out in various other modified and modified forms. Therefore, the scope of the present invention is defined only by the appended claims and their equivalents.

[0048]

As described in detail above, in the present invention,
The ratio of significant macroblocks occupying the current frame is calculated, and the number of frame skips from the current frame to the next frame is determined before encoding using the calculated ratio. By determining the number of frame skips before encoding the frame of interest in this way, it is possible to determine the number of frame skips that fully utilize the features of the image. As a result, the amount of information of the image is predicted in advance,
The frame rate can be controlled, and variable frame rate image coding with constant quality can be performed. In addition, if the number of frame skips is controlled according to the ratio of significant macroblocks occupying the current frame, it is possible to maintain a constant quality.

Further, according to the present invention, the magnitude of the motion vector in the current frame is determined, and the number of frame skips is determined before encoding using the determined magnitude of the motion vector. As described above, since the motion vector information is used for controlling the frame rate, a visually natural motion image can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram schematically illustrating a configuration of an embodiment of a moving image encoding device according to the present invention.

FIG. 2 is a flowchart illustrating a main operation in the embodiment of FIG. 1;

FIG. 3 is a conceptual diagram of the operation of a motion vector generation unit.

FIG. 4 is a conceptual diagram illustrating the operation of a macroblock significance determination unit.

FIG. 5 is a characteristic diagram of a macroblock significance determination threshold.

FIG. 6 is a characteristic diagram illustrating an example of a preferable characteristic of a frame skip number with respect to a significant macroblock ratio.

FIG. 7 is a characteristic diagram illustrating an example of a preferable characteristic of a frame skip number with respect to a scalar length of a motion vector.

FIG. 8 is a characteristic diagram illustrating an example of a preferable characteristic of a frame skip number with respect to a generated information amount up to immediately before;

FIG. 9 is a characteristic diagram three-dimensionally showing preferable characteristics of the frame skip number.

FIG. 10 is a characteristic diagram illustrating an example of preferable characteristics of a quantization step with respect to the number of frame skips.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Variable frame rate moving image encoder 2 Video signal input unit 3 Frame skip unit 4, 6j Frame memory 6 Encoding engine unit 6a, 6b Mode switch 6c, 13 Subtraction unit 6d Addition unit 6e switch 6f DCT (discrete cosine transform) Circuit 6g Q (quantization circuit) 6h Q ^-1 (inverse quantization circuit) 6i IDCT (inverse discrete cosine transform) circuit 6k, 8 Motion compensation unit 11 Motion vector generation unit 15 Macroblock significant / insignificant determination unit 17 Output buffer Memory unit 20 frame skip number determination unit 23 VLC (variable length coding unit) 25 code output unit

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 H03M ^7/ 30-7/50

Claims (10)

(57) [Claims] 1. A motion compensation is performed on a macroblock of a previous frame using a motion vector, and a prediction error that is a difference between the macroblock on which the motion compensation is performed and a coding target macroblock of a current frame is calculated. By comparing the calculated prediction error with a threshold, it is determined whether the current macroblock is a non-significant macroblock or a significant macroblock larger than the threshold, and if the macroblock is a significant macroblock, coding is performed. Performing a variable frame rate video encoding method in which only the motion vector is transmitted if the macroblock is a non-significant macroblock, wherein the ratio of the significant macroblock occupying the current frame is calculated and calculated. Variable to determine the number of frame skips from the current frame to the next frame using the ratio before encoding Frame rate moving picture coding method. 2. The method according to claim 1, wherein the determination of the number of frame skips is performed using a function that increases as the ratio of significant macroblocks increases. 3. The method according to claim 1, wherein the determination of the number of frame skips is performed using a function that increases as the ratio of significant macroblocks increases and saturates when the ratio exceeds a certain value. Method. 4. The method according to claim 1, wherein the size of a motion vector in the current frame is obtained, and the frame skip number is determined before encoding by further using the obtained size of the motion vector. Or the method of claim 1. 5. A macroblock of a previous frame is subjected to motion compensation using a motion vector, and a prediction error, which is a difference between the macroblock subjected to the motion compensation and a current macroblock to be coded, is calculated. By comparing the calculated prediction error with a threshold, it is determined whether the current macroblock is a non-significant macroblock or a significant macroblock larger than the threshold, and if the macroblock is a significant macroblock, coding is performed. Performing a variable frame rate video encoding method in which only the motion vector is transmitted if the macroblock is a non-significant macroblock, wherein the magnitude of the motion vector in the current frame is obtained, and A variable frame, wherein the number of frame skips from the current frame to the next frame is determined before encoding using the size. Mureto moving picture coding method. 6. The method according to claim 4, wherein the determination of the number of frame skips is performed using a function that decreases as the size of the motion vector increases. 7. The method according to claim 1, wherein the determination of the number of frame skips is performed by further using a generated information amount immediately before encoding of a current frame. 8. The method according to claim 7, wherein the determination of the number of frame skips is performed using a function that increases as the amount of generated information until immediately before increases. 9. The method according to claim 1, wherein a quantization step is controlled using the determined frame skip number. 10. The method according to claim 9, wherein the control of the quantization step is performed using a function that makes the quantization step finer as the number of frame skips increases.