JP3872076B2 - Motion compensation device - Google Patents

Description

  The present invention relates to a motion compensation device applied to a digital video transmission device, a digital CATV, a digital broadcasting system, and the like.

FIG. 18 shows the technical report VOL. 95, no. 217, P2-8 (1995), is a block diagram for explaining a conventional digital image coding system shown in "Development of MPEG2 real-time coding system chip set".
In FIG. 18, reference numeral 400 denotes a motion compensation processing unit that inputs an input image 202 and a reference image 152 and outputs a predicted image 204 and an optimal vector 449, and 222 calculates a residual between the input image 202 and the predicted image 204. A difference unit that outputs a signal 223, 401 a differential signal encoding processing unit that encodes an input residual signal 223 and outputs differential encoded data 450, and 402 encodes an input optimal vector 449 to generate a motion vector. It is a motion vector encoding processing unit that outputs encoded data 451.

Next, the operation according to the above configuration will be described.
The motion compensation processing unit 400 inputs an input image 202 that is an image of the current frame and a reference image 152 that is an image of the previous frame, and searches the reference image 152 for an image most similar to the input image 202. In the search method, the sum of absolute differences between the input image 202 and the reference image 152 is calculated, and the image that gives the smallest distortion among the reference images is set as the optimum image, that is, the predicted image 204. At this time, as shown in FIG. 19, a motion vector indicates how much the optimum image (predicted image) moves from the position of the input image in the current frame. This motion vector is transmitted to the motion vector encoding processing unit 402 as an optimum vector 449, and is encoded and output by the motion vector encoding processing unit 402.

  On the other hand, for the prediction error signal after being predicted by the motion compensation processing unit 400, the difference between the optimum predicted image 204 selected by the motion compensation processing unit 400 and the input image 202 is obtained by the subtractor 222, and the residual is obtained. The signal 223 is encoded by the differential signal encoding processing unit 401 and output. The amount of information generated in the encoding process of this process is the information amount of the difference encoded data 450 generated by the difference signal encoding processing unit 401 and the information amount of the motion vector encoded data 451 generated by the motion vector encoding processing unit 402. Is the combined value.

In the conventional apparatus, as described above, the motion compensation processing unit 400 uses the vector indicating the minimum difference sum as the optimal vector, but in the image encoding process, the vector selected as the optimal vector by the motion compensation is encoded. At the same time as (motion vector coding), the residual (prediction error) between the current block and the previous block at the optimum vector position is further coded (residual coding). The information amount obtained by combining the information amount by the motion vector encoding and the information amount by the residual encoding is the encoding amount. For this reason, for example, if the information vector becomes very large if the optimal vector is encoded, the optimal vector selected by the above method does not always minimize the encoding amount.
In general, motion compensation prediction uses only luminance information to determine a vector. However, an incorrect motion vector is extracted from images having the same luminance signal and different color difference signals.

  The present invention has been made to solve the above problems, and in order to determine the optimum vector, not only the sum of absolute differences of prediction differences but also the amount of code for encoding a motion vector is determined. In view of the above, it is an object of the present invention to obtain a motion compensation apparatus, a moving picture coding apparatus, and a method capable of improving the total coding efficiency.

The motion compensator according to the present invention stores the data of the previous frame in the motion compensator that predicts the motion from the data of the previous frame and reduces the amount of information when performing the encoding for moving image encoding. Performs pattern matching from the previous frame memory unit, the block data of the current frame, and the partial image area of the search range data of the previous frame stored in the frame memory unit, and calculates the motion vector and distortion value that give the minimum distortion Input a specific vector, which is a motion vector as comparison reference information for obtaining the current optimal motion vector, by inputting the minimum distortion calculation unit, and the distortion between the block data of the current frame and the block data of the previous frame corresponding to the specific vector A specific vector distortion calculation unit that outputs the specific vector and the calculated distortion, and the minimum distortion calculation unit. An optimal vector output unit that outputs an optimal motion vector based on the strain that is applied and the distortion that is output from the specific vector distortion calculation unit, and the specific vector distortion calculation unit is output from the optimal vector output means The optimal vector output unit is input as a specific vector, and the optimal vector output unit sets the difference value between the motion vector that gives the minimum distortion from the minimum distortion calculation unit and the specific vector output from the specific vector distortion calculation unit. Accordingly, if the difference value is small, a small offset value is given, and if the difference value is large, the offset value is adaptively changed so as to give a large offset value. An adder for adding the offset value from the offset value calculation unit to the distortion value and weighting the distortion from the minimum distortion calculation unit; A comparator that compares the distortion from the calculated minimum distortion calculation unit with the distortion from the specific vector distortion calculation unit, and a motion vector or specific vector that gives the minimum distortion based on the comparison result of the comparator. And a selector that selects and outputs an optimum motion vector .

As described above, according to the present invention, in order to determine the optimal vector, the total coding is performed by considering not only the sum of absolute differences of prediction differences but also the code amount of the vector code amount as the distortion amount. Efficiency can be improved. It is possible to effectively reduce the code amount of the motion vector without greatly increasing the code amount of the prediction error signal. This is particularly true when performing a low bit rate encoding in which the ratio of the code amount of the motion parameter to the code amount of the image data is high, so that the overall code amount and encoding distortion can be well balanced. Is effective.

Reference Example 1
FIG. 1 is a block diagram showing a motion compensation apparatus according to Reference Example 1.
As shown in FIG. 1, the motion compensation device according to the first reference example inputs an input image 202 that is an image of the current frame and a reference image 201 that is an image of the previous frame, and is an image most similar to the input image 202. A motion compensation processing unit 200 that outputs a motion vector 203 and a predicted image 204 between the input image 202 and the reference image 201 when a block is searched from the reference image 201 and a predicted image according to the motion vector is extracted; The difference absolute value sum calculation unit 205 as a distortion amount calculation unit that calculates and outputs the difference absolute value sum 206 as the distortion amount between the input image 202 and the predicted image 204, and the motion vector 203 is input and encoded. The vector value encoding unit 207 that outputs the quantity 208, the motion vector 203, the difference absolute value sum 206, and the vector code quantity 208 are input, and the difference absolute value sum 2 6 and an optimal vector determination unit 209 that outputs a motion vector having an evaluation function having a minimum value as an optimal vector 210 for all the motion vectors 203 that evaluate the evaluation function calculated from the vector code amount 208. .

Next, the operation of Reference Example 1 according to the above configuration will be described.
The motion compensation processing unit 200 extracts a predicted image 204 according to the motion vector 203 from the reference image 201 with respect to the input image 202. The difference absolute value sum calculation unit 205 calculates the difference absolute value sum 206 from the input image 202 and the predicted image 204. On the other hand, the motion vector 203 used to extract the predicted image 204 is input to the vector value encoding unit 207 and the optimal vector determination unit 209. The motion vector 203 input to the vector value encoding unit 207 is encoded and a vector code amount 208 is output. Here, the vector value encoding unit 207 often uses variable length encoding or the like. The optimum vector determination unit 209 receives the difference absolute value sum 206, the vector code amount 208, and the motion vector 203, and evaluates an evaluation function calculated from the difference absolute value sum 206 and the vector code amount 208. All vectors are calculated, and a motion vector whose evaluation function value indicates the minimum value is output as the optimal vector 210. Here, as an evaluation function to be used, for example, a weighted addition is performed on the sum of absolute differences 206 and the vector code amount 208.

As described above, it is possible to improve the total coding efficiency by using the vector code amount 208 in combination instead of using only the absolute difference sum 206 in determining the optimum vector 210. In particular, in low-rate encoding, the code amount of a vector may occupy half or more of the entire code amount, and motion vector selection considering the vector code amount is important.
That is, by selecting the optimal vector using the amount of code for encoding a motion vector and the amount of information for encoding a prediction error signal as an evaluation value, the final code is compared with the conventional motion compensated prediction. The result will be more efficient.

Reference Example 2
FIG. 2 is a block diagram showing a motion compensation apparatus according to Reference Example 2.
The same parts as those of the reference example 1 shown in FIG. As a new code, reference numeral 211 denotes a difference square sum calculation unit used instead of the difference absolute value sum calculation unit 205 used as the distortion amount calculation unit in FIG. Since the sum of squared differences performs a square calculation compared to the sum of absolute differences, a more complicated calculation is required, but the power (power) included in the signal can be evaluated. In general, it is possible to evaluate a prediction error signal with higher accuracy in power, and it is possible to obtain an optimal vector more correctly by performing prediction with higher accuracy than in Reference Example 1.

Reference Example 3
FIG. 3 is a diagram for explaining the motion compensation apparatus according to Reference Example 3, and is an internal configuration diagram of the vector value encoding unit 207 in Reference Examples 1 and 2.
As shown in FIG. 3, the vector value encoding unit 207 obtains a delay unit 213 that delays the input motion vector 203 and a difference between the input motion vector 203 and the motion vector that has passed through the delay unit 213. A difference unit 214 that outputs a difference vector and a difference vector encoding unit 215 that encodes the difference vector and outputs a motion vector code amount 208 are provided.

  The difference between the motion vector 203 input to the vector value encoding unit 207 and the motion vector previously input and accumulated in the delay unit 213 is obtained by the difference unit 214, and this difference vector is obtained by the difference vector encoding unit 215. Encoded. The motion vector often takes a similar value when viewed locally in the screen. Therefore, it is possible to reduce the value to be encoded by taking a difference from a nearby motion vector that has already been used, and to improve the encoding efficiency by reducing the vector code amount.

Reference Example 4
FIG. 4 is a block diagram showing a motion compensation apparatus according to Reference Example 4.
In FIG. 4, the same parts as those of the reference example 1 shown in FIG. As a new code, 216a is a first average value separation unit that obtains an average value separated input image 217 obtained by separating the average value from the input image 202, and 216b is an average value separated predicted image 218 obtained by separating the average value from the predicted image 204. The obtained second average value separation unit 219 receives the average value separation input image 217 and the average value separation prediction image 218 and obtains a distortion amount between the average value separation input image 217 and the average value separation prediction image 219. As in Reference Examples 1 and 2, the distortion amount calculation unit 219 obtains an evaluation value 220 based on the calculation of the sum of absolute differences or the sum of squares of differences, and inputs the evaluation value 220 to the optimal vector determination unit 209. The optimum vector determination unit 209 receives the motion vector 203 and the evaluation value 220 obtained by calculating the sum of absolute differences or the sum of squared differences, and evaluates the evaluation value 220. Seeking for for all motion vectors 203, evaluation value is output as the optimum vector 210 a motion vector indicating the minimum value.

  As shown in FIG. 4, the input image 203 and the predicted image 204 are subjected to average value separation by average value separation units 216a and 216b, respectively. The difference absolute value sum or difference described with reference to FIGS. 1 and 2 between the average value separated predicted image 217 from which the average value has been separated by the distortion amount calculation unit 219 and the average value separated input image 218 from which the average value has been separated. An evaluation value 220 obtained by calculating the sum of squares is obtained and input to the optimum vector determination unit 209 to obtain the optimum vector 210.

Thus, separating and evaluating the average value indicates that motion compensation independent of the image level is possible. For example, in the case of a fade-in image or an image with a sharp change in luminance, the conventional motion search could not detect an accurate motion, but by separating the average value, even in these cases, a more accurate motion vector detection Is possible.
That is, by performing motion prediction evaluation between images obtained by separating the average values of both the input image and the predicted image, motion compensation independent of the level of the image can be made possible, and even more rapidly changing images can be made more accurate. High motion vector detection is possible.

  Although not described here, as described with reference to FIGS. 1 and 2, it is naturally possible to use the motion vector code amount 208 as another evaluation value to help determine the optimum vector. Similarly, as described with reference to FIG. 3, it is also effective to perform differential vector coding for vector coding.

Reference Example 5
FIG. 5 is a block diagram showing a motion compensation apparatus according to Reference Example 5.
In FIG. 5, the same parts as those of the reference example 1 shown in FIG. 1 are denoted by the same reference numerals and the description thereof is omitted. As a new reference numeral 222, a prediction error is obtained by subtracting the input image 202 and the predicted image 204. 223 is a subtractor for calculating 223, 224 is a frequency analysis unit such as DCT or FFT that converts the prediction error 223 into the frequency coefficient 225, and 226 is an evaluation value that generates an evaluation value 227 by performing weighting based on the converted frequency coefficient 225 The optimal vector determination unit 209, which is a generation unit, receives the motion vector 203 and the evaluation value 227, and outputs, as an optimal vector 210, a motion vector having the minimum evaluation value 227 among a plurality of possible motion vectors. To do.

  As shown in FIG. 5, the input image 202 and the predicted image 204 are converted into a prediction error 223 by the subtractor 222, and the prediction error is converted into a frequency coefficient 225 by the frequency analysis unit 224. The converted frequency coefficient is evaluated in the evaluation value generation unit 226 and output as an evaluation value 227.

As described above, the frequency analysis of the prediction error and the determination of the frequency coefficient are equivalent to performing substantially the same process as when transform encoding the prediction error. For example, by generating an evaluation value by weighting the low frequency component of the obtained frequency coefficients, it is possible to predict the amount of code when transform encoding the prediction error with considerable accuracy. By using this code amount as an evaluation value, it can be expected to perform more efficient encoding.
In other words, frequency analysis of the prediction difference is performed to obtain a frequency coefficient, and by calculating an evaluation value from the coefficient, it is possible to predict the coding amount after transform coding, including the prediction error coding, fairly accurately, Efficient encoding is possible.

Reference Example 6
FIG. 6 is a block diagram showing a motion compensation apparatus according to Reference Example 6.
6, the same reference numerals as those in Reference Examples 1 and 5 shown in FIGS. 1 and 5 denote the same parts, and the description thereof is omitted. As a new code, 228 is a differential image encoding unit that differentially encodes the prediction error 223 from the subtractor 222 and outputs a differential image code amount 229. The vector determination unit 209 includes the motion vector 203 and the differential image code. The amount 209 and the vector code amount 208 from the vector value encoding unit 208 are input, and among the plurality of possible motion vectors 203, the code amount obtained by adding the vector code amount 208 to the difference image code amount 209 is the minimum. This motion vector is output as the optimum vector 210.

As shown in FIG. 6, the input image 202 and the predicted image 204 are converted into a prediction error 223 by the subtractor 222, and the prediction error is encoded by the difference image encoding unit 228, and a code amount 229 is output. Differential encoding of the prediction error is equivalent to performing normal encoding processing, and by adding the vector code amount to this, a nearly complete code amount when using the motion vector 203 is calculated. it can. Therefore, by adopting this configuration, it is possible to obtain a vector having the most optimal code amount among all possible motion vectors.
That is, by encoding the prediction error to obtain the encoded information amount, and further using the vector encoding code amount, it becomes possible to obtain a nearly complete code amount in the vector, and the distortion and code amount It is possible to determine whether the code is optimal in the relationship.

Reference Example 7
FIG. 7 is a block diagram showing a motion compensation apparatus according to Reference Example 7.
In FIG. 7, the same parts as those of the reference example 1 shown in FIG. As a new code, 230a is a first luminance / color difference separation unit that separates the input image luminance signal 231 and the input image color difference signal 232 from the input image 202, and 230b is a predicted image luminance signal 233 and a predicted image color difference signal from the predicted image 204. 234a separates the difference between the input image luminance signal 231 from the first luminance color difference separation unit 230a and the predicted image luminance signal 233 from the second luminance color difference separation unit 230b. A second subtractor 235b to be obtained is a first subtractor for obtaining a difference between the input image color difference signal 232 from the first luminance color difference separation unit 230a and the predicted image color difference signal 234 from the second luminance color difference separation unit 230b. 237 is a color difference evaluation value generation unit that generates a color difference evaluation value based on the output from the first subtractor 235b, and 236 is the output from the second subtractor 235a. Based on the color difference evaluation value from the color difference evaluation value generation unit 237 and the luminance evaluation value from the luminance evaluation value generation unit 236, a luminance evaluation value generation unit 238 that generates a luminance evaluation value based on the luminance evaluation value generation unit 238 The vector determination unit 209 inputs the motion vector 203 and the optimum evaluation value for optimal vector determination 239, and determines the optimal vector among a plurality of possible motion vectors. The motion vector having the minimum total evaluation value is output as the optimum vector 210.

  As shown in FIG. 7, the input image 202 and the predicted image 204 are respectively input to the luminance / color difference separation units 230 a and 230 b, and the input image luminance signal 231, the input image color difference signal 232, the predicted image luminance signal 233, and the predicted image color difference signal 234. And separated. Among the separated signals, the difference between the luminance signals or the color difference signals is obtained by the subtracters 235a and 235b, and the evaluation values are calculated by the luminance evaluation value generation unit 236 and the color difference evaluation value generation unit 237, respectively. . The calculated luminance evaluation value and color difference evaluation value are added by an adder 238 to obtain a total evaluation value 239. According to the total evaluation value, the motion vector having the minimum total evaluation value among the plurality of possible motion vectors is the optimal vector 210.

In conventional motion compensation prediction, an evaluation value is generally calculated using only a luminance component. Obtaining an evaluation value by separating into luminance and color differences indicates that a motion that cannot be obtained only by luminance can be obtained by using a color difference image. When the luminance component and the color difference component are separated, there is an image that does not change at all in the luminance signal but clearly changes in the color difference component. In the case of such an image, if motion compensation is performed using only the luminance component, it is impossible to follow the change of the color difference component, and an incorrect vector is obtained. By simultaneously evaluating the color difference component, it is possible to obtain an optimum vector in the above case.
In this way, by adding not only the luminance signal but also the color difference signal to the evaluation value of the motion compensation prediction, it is possible to improve the reproducibility of not only the luminance signal but also the color movement that cannot be evaluated.

  Although not described here, it is possible to obtain a more optimal evaluation value by performing weighted addition in the addition of the luminance evaluation value and the color difference evaluation value by the adder 238.

Reference Example 8
FIG. 8 is a block diagram showing a motion compensation apparatus according to Reference Example 8.
8, the same parts as those of the reference example 7 shown in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted. As a new code, 230c is a third luminance color difference separation unit that separates the motion vector luminance signal 242 and the motion vector color difference signal 243 from the motion vector 203, and 244 is a luminance evaluation value and color difference evaluation from the luminance evaluation value generation unit 237. The luminance color difference evaluation value comparison unit as an evaluation value calculation unit that compares the color difference evaluation value from the value generation unit 237 and outputs an evaluation value comparison difference. The motion compensation processing unit 200 includes an input image 202 and a reference image 201. And the predicted image luminance signal 204 and the motion vector color difference extracted from the reference image 201 according to the motion vector luminance signal with respect to the motion vector between the input image 202 and the reference image 201 and the luminance component of the input image 202. The predicted image color difference signal 241 extracted according to the signal is output, and the vector determination unit 209 outputs the motion vector luminance signal 242 and the motion image. Enter the vector chrominance signal 243 and the evaluation value comparison difference, among the motion vectors more can be taken, and outputs a motion vector the evaluation value comparison difference becomes minimum as an optimum vector.

  As shown in FIG. 8, with respect to the luminance component of the input image 202, a predicted image luminance signal 240 according to the motion vector luminance signal 242 from the reference image 201 and a predicted image color difference signal 241 according to the motion vector color difference signal 243 are obtained. Each is required. The input image 202 is separated into a luminance signal 231 and a color difference signal 232 by the luminance / color difference separation unit 230a. The difference between the input image of the luminance signal and the predicted image is taken between the input image of the color difference signal and the predicted image, and the luminance evaluation value and the color difference evaluation value are calculated and input to the luminance / color difference evaluation value comparison unit 244. The The comparison unit 244 performs weighted comparison and outputs an evaluation value determined to be more appropriate to the optimum vector determination unit 209. The optimum vector determination unit 209 receives the luminance and chrominance vectors, and outputs the selected one of the luminance and chrominance vectors as the optimum vector 210 based on the evaluation value that is the previous input.

Form state of implementation.
Figure 9 is a block diagram showing a motion compensation device according to the shape condition of the embodiment.
The motion compensation apparatus shown in FIG. 9 is a motion compensation apparatus that predicts motion from data of the previous frame in units of blocks and performs information amount reduction when performing encoding for moving image encoding. The minimum distortion for calculating the motion vector 155 and the minimum distortion 154 that perform the pattern matching from the specific areas of the previous frame memory unit 104 to be stored, the block data 150 of the current frame, and the search range data 151 of the previous frame A calculation unit 101; a specific vector distortion calculation unit 102 for calculating distortion 156 between block data 150 of the current frame and block data 152 of the previous frame corresponding to one or more specific vectors 153 input from the outside; Optimum based on the distortion output from the distortion calculation unit 101 and the distortion output from the specific vector distortion calculation unit 102 And an optimal vector output unit 103 for outputting can vector 157.

Next, the operation of FIG. 9 will be described.
The minimum distortion calculation unit 101 calculates and outputs a minimum distortion 154 and a motion vector 155 that give the minimum distortion from the search range data 151 of the previous frame for the block data 150 of the current frame. On the other hand, the specific vector distortion calculation unit 102 extracts the data 152 corresponding to the specific vector 153 given from the outside from the previous frame memory unit 104, calculates the distortion 156 between the data and the block data 150 of the current frame, and outputs it. To do.
Based on the minimum distortion 154 from the minimum distortion calculation unit 101 and the distortion 156 from the specific vector distortion calculation unit 102, the optimal vector output unit 103 calculates an optimal vector.

  As an optimal vector calculation method of the optimal vector output unit 103, for example, FIG. That is, the offset value 110 is added to the minimum distortion 154 from the minimum distortion calculation unit 101, the result is compared with the distortion 156 from the specific vector distortion calculation unit 102 by the comparator 112, and a vector giving a small value is determined as the optimum motion vector. It outputs as 156.

  Further, as shown in FIG. 10, the optimum motion vector 157 output from the optimum vector output unit 103 is input as the input of the specific vector distortion calculation unit 102 and the specific vector 153, thereby specifying the optimal vector of the previous block. By inputting as a vector, when the optimal vector of the previous block is selected as the optimal vector of the current block, the amount of information in variable length coding of the motion vector can be significantly reduced.

Further, as shown in FIG. 12, the optimum vector output unit 103 adapts the weighting offset value added to the distortion from the minimum distortion calculation unit 101 according to the magnitude of the difference between the motion vector 155 giving the minimum distortion and the specific vector 153. The offset value calculation unit 114 may be further changed and given, and the offset value may be determined based on the difference value. For example, when the vector difference is small, a small offset is given. When the difference is large, a large offset is given. As a result, when the vector difference is large to some extent,
Specific vector value ≤ ((minimum distortion) + offset)
If so, the specific vector is selected as the optimum motion vector.
As a result, the amount of information generated at the time of variable length coding can be greatly reduced.

Reference Example 9
Next, in the following reference example , a motion vector that has a feasibility and versatility with a simple configuration and is considered not to contribute to the reduction of coding distortion in consideration of the balance of the total code amount and distortion. Are encoded into a prediction vector that minimizes the code amount, and a moving picture coding apparatus and method for reducing the code amount of a motion vector while suppressing an increase in overall distortion are provided. This method can be applied to various international standard video encoding methods as described in the prior art, and each reference example described above is used for the purpose of effectively reducing the code amount in the entire encoding system. and consistent with the embodiment.

First, in the moving picture coding apparatus according to the reference example 9 , in the moving picture coding apparatus based on motion compensated prediction, a memory that stores reference picture data used for motion compensated prediction, an input macroblock, and the memory A motion detection unit for detecting a motion vector that gives a minimum prediction error based on reference image data from the image, and a prediction vector for deriving a prediction vector used for encoding a motion vector used for motion compensated prediction of an input macroblock A derivation unit, a motion compensation unit that extracts image data at a position corresponding to the reference image data in the memory based on the given motion vector as a predicted image corresponding to the motion vector, and a motion vector obtained by the motion detection unit Based on the prediction image output from the motion compensation unit, the prediction error amount is obtained, and the prediction error amount is calculated. When the threshold value is determined by using the threshold value, and the prediction result indicates that the prediction error amount is larger than the first threshold value, the prediction vector obtained by the prediction vector deriving unit is output to the motion compensation unit, and the prediction error amount is When the threshold value is smaller than 1, a threshold value processing unit that outputs the motion vector obtained by the motion detection unit to the motion compensation unit, and a prediction error signal based on a prediction image corresponding to the prediction vector are generated. A threshold determination is performed on the generated code amount estimated value using the second threshold. If the generated code amount estimated value of the prediction error signal is larger than the second threshold, the motion vector obtained by the motion detecting unit is determined. In addition to outputting as the final motion vector, if the estimated code amount estimate value of the prediction error signal is smaller than the second threshold, the prediction vector is output as the final motion vector. And a predictive image determination unit that outputs a predictive image corresponding to the final motion vector as a final predictive image, and a low bit that increases the ratio of the code amount of the motion parameter (motion vector) to the code amount of the image data Also in rate coding, coding is performed with a good balance between the overall code amount and coding distortion.

The present reference example 9 is an example in which the technical elements according to the present reference example are incorporated into a moving picture coding apparatus based on general motion compensation prediction, discrete cosine transform (hereinafter also referred to simply as DCT) / quantization. Indicates.
FIG. 13 is an internal configuration diagram of the moving picture coding apparatus according to the ninth embodiment .
The moving picture coding apparatus based on motion compensated prediction using a motion vector in Reference Example 9 shown in FIG. 13 has, as a main configuration, a frame memory 34 that stores reference picture data used for motion compensated prediction, Based on the input macroblock 3 and the reference image data from the memory 34, a motion detector 20 that detects a motion vector 21 that gives a minimum prediction error, and a motion vector used for motion compensated prediction of the input macroblock 3 are encoded. A prediction vector derivation unit 27 for deriving a prediction vector 28 to be used for the prediction, and a prediction image 23 corresponding to the motion vector from the image data corresponding to the reference image data in the memory 34 based on the given motion vector. 29 based on the motion compensation unit 22 extracted as 29 and the motion vector 21 obtained by the motion detection unit 20 The prediction error amount is obtained using the prediction image 23 output from the motion compensation unit 22, the threshold value is determined using the first threshold value, and the determination result is that the prediction error amount is greater than the first threshold value. In this case, the prediction vector 28 obtained by the prediction vector deriving unit 27 is output to the motion compensation unit 22, and when the prediction error amount is smaller than the first threshold, the motion vector 21 obtained by the motion detection unit 20 is output. Are generated by the minimum distortion threshold processing unit 24 that outputs the motion compensation unit 22 and the prediction image 29 corresponding to the prediction vector 28, and the generated code amount estimated value of the prediction error signal is set as the second threshold value. Threshold value determination is performed, and if the generated code amount estimated value of the prediction error signal is larger than the second threshold value, the motion vector 21 obtained by the motion detection unit 20 is finally determined. The prediction vector 28 is output as the final motion vector 33 when the generated code amount estimation value of the prediction error signal is smaller than the second threshold, and the prediction image corresponding to the final motion vector 33 is output. The motion compensated prediction unit 19 includes a final predicted image determination unit 30 that outputs 37 as a final predicted image.

  As other codes, 2 is a macroblock generation unit that generates an input macroblock 3 based on the input video signal 1, 5 is a mode determination based on the input of the input macroblock 3 and the prediction error signal 4, and mode A mode determination unit that outputs the selection information 6 and the encoding target image signal 7, 8 is a DCT unit that performs discrete cosine transform on the encoding target image signal 7 and outputs DCT coefficient data, and 10 quantizes the DCT coefficient data 9. A quantization unit that outputs quantized DCT coefficient data, 12 an inverse quantization unit that inversely quantizes the quantized DCT coefficient data 11 and outputs inversely quantized DCT coefficient data 13, and 14 an inverse quantized DCT coefficient data 13 , The inverse DCT unit for restoring the decoded image data 15, 16 the decoded reproduction image, 17 the switch controlled according to the mode selection information 6 from the mode determination unit 5 H, 26 and 32 are switches in the motion compensation prediction unit 19, 38 is quantized DCT coefficient data 11, mode selection information 6, and motion vector 33 in the form of a bit stream 39 in a predetermined syntax and variable length coding method. This is a variable length coding / multiplexing unit that multiplexes and outputs.

FIG. 14 is a flowchart showing a motion vector detection process including the motion compensation prediction unit 19 of FIG. 13 for explaining the moving picture coding apparatus and method according to the ninth embodiment.
Hereinafter, a description will be given with reference to FIGS. 13 and 14.
(1) Overall Operation of Encoding Device First, the overall operation of the encoding device in FIG. 13 will be briefly described. Data corresponding to one frame is input to the input video signal 1, and this frame data is divided into input macroblocks 3 by the macroblock generator 2. For each input macroblock, the mode determination unit 5 determines whether to perform intra (intraframe) encoding or inter (interframe) encoding. What is used for the determination is a prediction error signal 4 which is a difference signal between the input macroblock 3 and the predicted image 37 obtained by the processing of the motion compensation prediction unit 19 and the input macroblock 3, and is encoded by a predetermined determination method. A signal determined to have high encoding efficiency is selected as the encoding target image signal 7. Various determination methods can be used at this time, but the details are omitted because they do not correspond to the technical elements of the present invention. Details of the process of the motion compensation prediction unit 19 that generates the predicted image 37 and the motion vector 33 will be described later.

  The encoding target image signal 7 is DCTed for each block of 8 × 8 pixels in the DCT unit 8 and converted to DCT coefficient data 9. The DCT coefficient data 9 is quantized by the quantization unit 10 by a predetermined method and sent to the variable length encoding / multiplexing unit 38 as the quantized DCT coefficient data 11. Also, the quantized DCT coefficient data 11 is returned to the inverse quantized DCT coefficient data 13 by the inverse quantization unit 12 and restored to the decoded image data 15 by the inverse DCT unit 14. Here, the switch 17 is controlled according to the mode selection information 6 from the mode determination unit 5, and “0” is added in the intra coding mode, and the predicted image 37 is added in the inter coding mode, and the decoded reproduced image 16 is added. Get.

The decoded reproduction image 16 is stored in the frame memory 34 for use in later motion compensation prediction. In addition to outputting the predicted image 37, the motion compensated prediction unit 19 outputs a motion vector 33 for obtaining the predicted image 37 to the variable length encoding / multiplexing unit 38.
The variable length coding / multiplexing unit 38 multiplexes the quantized DCT coefficient data 11, the motion vector 33, the mode selection information 6 and the like into a bit stream 39 in a predetermined syntax and variable length coding method, and outputs the multiplexed data. . Here, the predetermined syntax means a data multiplexing rule defined by the aforementioned international standard moving picture coding standard.

(2) Operation of Motion Compensation Prediction Unit 19 The internal configuration of the motion compensation prediction unit 19 is as shown in FIG. The operation will be described in detail with the use of the flowchart of FIG.
(2-1) Motion detection process (step S1)
First, as shown in step S1, a motion vector 21 that gives a minimum prediction error (minimum matching distortion) to the input macroblock 3 is obtained. This process is performed in the motion detector 20. Examples of the prediction error include the sum of absolute differences (SAD) described in the prior art and the sum of squares of differences in which the squares of pixel differences are summed for all the pixels of the macroblock luminance component, but in the following, SAD is used. And The motion detection unit 20 obtains a reference image data position that minimizes SAD between the input macroblock 3 and image data included in the motion vector search range to which the reference image in the frame memory 34 is given. The deviation of the input macroblock 3 from the position in the frame is output as the motion vector 21. The reference image data in the frame memory 34 used for calculation of the prediction error is sent to the motion detection unit 20 via the image data bus 35.

  Next, the switch 26 passes this motion vector 21 to the motion compensation unit 22 through. The motion compensation unit 22 extracts and outputs the image data at the position corresponding to the reference image data in the frame memory 34 as the predicted image 23 via the image data bus 36 based on the motion vector 21. The predicted image 23 is first sent to the final predicted image determination unit 30. At this time, the final predicted image determination unit 30 outputs the predicted image 23 as it is as the final predicted image 37 as it is.

(2-2) Mode determination (steps S2, S3)
Next, in step S2, intra / inter determination is performed. This process is performed not by the motion compensation prediction unit 19 but by the mode determination unit 5 as described in (1). Next, in step S3, it is determined whether or not the mode selection information 6 indicates “inter mode”. If it is an intra mode, that is, a mode in which the input macroblock 3 is a direct encoding target signal, the process of the motion compensation prediction unit 19 ends here.

(2-3) Motion vector efficiency determination (step S4)
In the inter mode, that is, in a mode in which a difference image (prediction error signal) between the input macroblock 3 and the final predicted image 37 is an encoding target signal, the code amount and the encoding distortion are determined according to the procedure and operation described below. A total and efficient motion vector is determined again in consideration of balance. First, in the case of the inter mode, the minimum matching distortion threshold value processing unit 24 is activated, and threshold determination is performed on the SAD based on the predicted image 23 based on a predetermined threshold value TH1 (step S4). Here, if TH1 is experimentally or empirically determined and set as a value for achieving prediction efficiency, when a SAD larger than TH1 is generated, the motion-compensated prediction using the motion vector 21 is not so good in prediction efficiency. I can judge. That is, even if a prediction error signal is obtained using the motion vector 21 in this macroblock, it can be inferred to some extent that a large amount of code is required to encode the prediction error signal. Therefore, it is determined that the motion vector 21 at this time is not worth performing motion compensation prediction until the code amount of the motion vector 21 itself is consumed, and the code amount of the motion vector is reduced.

On the contrary, when SAD becomes a value smaller than TH1, the prediction efficiency of the motion vector 21 is recognized, and it is used as it is as a final motion vector for encoding.
(2-4) Motion vector replacement processing (steps S5 to S9)
If it is determined in step S4 that SAD is greater than TH1, it is considered to replace the motion vector 21 with a prediction vector that is actually used for differential encoding. In general, in the existing moving picture coding standard method, a motion vector is often predicted by a nearby motion vector, and the difference value is often coded. Since the motion vector often takes a value similar to that of the neighborhood, it is common to code by assigning a code having a shorter code length as the prediction difference is closer to zero. Therefore, by making the motion vector coincide with the prediction vector, the code amount of the motion vector is minimized. Therefore, if the code amount of the prediction error signal does not become extremely large by this replacement, the efficiency of reducing the total code amount can be improved. Good encoding can be performed.

  If the minimum matching distortion threshold value processing unit 24 determines that SAD is larger than TH1 in step S4, the control signal 25 switches the switch 26 and provides the motion vector 21 to the prediction vector deriving unit 27. The prediction vector deriving unit 27 performs the process of step S5 and derives a motion vector encoding prediction vector 28 used when the motion vector 21 is actually encoded. For example, as a prediction method, a difference is obtained using a motion vector of the immediately preceding macroblock as a prediction vector (this is adopted in MPEG-1 and MPEG-2). Then, when the motion vector 21 is (−2, 8) and the motion vector of the immediately preceding macroblock is (0, 4), the motion vector data to be encoded is (2, 4) as the difference. Here, the motion vector (x, y) means x pixels from the pixel position of the predicted macroblock to the horizontal method (right direction is positive), and y pixels in the vertical direction (down direction is positive). This means that a pixel in the frame memory at a position shifted by only a predicted pixel is used as a predicted pixel. The prediction vector deriving unit 27 outputs (0, 4) corresponding to the prediction vector as a replacement candidate vector.

The motion compensation unit 22 receives the prediction vector 28 and outputs a prediction image 29 corresponding to the prediction vector 28 in the same procedure as the acquisition of the prediction image 23 described above.
The predicted image 29 is input to the final predicted image determination unit 30 together with the predicted image 23. Here, the final predicted image determination unit 30 generates a prediction error signal based on the predicted image 29 (obtained by taking the difference from the input macroblock 3; corresponding to step S6), and determines the encoding efficiency of the prediction error signal. To check. This is because when the motion vector 21 is replaced with the prediction vector 28 and the code amount of the prediction error signal is greatly increased, the replacement of the motion vector has an adverse effect. The purpose is to measure the degree of the amount of codes to some extent.

The check method is based on threshold determination using the variance value of the luminance component as the generated code amount estimation value of the prediction error signal (steps S7 and S8). The basis for this determination method is as follows.
The prediction error signal generally has a Laplace-Gaussian distribution with a peak near zero. In the encoding, the prediction error signal is converted into a frequency component by DCT, and the redundancy is reduced by utilizing the fact that the coefficient distribution is biased to a low frequency. It becomes the measure of. The content rate of the high frequency component is represented to some extent by the variance of the signal distribution of the prediction error signal. The greater the variance, the broader the signal distribution, and the higher the frequency of the high frequency component of the DCT coefficient.

  From this point of view, when the variance value σ2 of the luminance component that occupies most of the code amount is used for threshold determination, and this variance value is smaller than the threshold TH2, the code amount increase of the prediction error signal due to motion vector replacement is to some extent. Motion vector replacement is accepted as a suppression (step S9). On the contrary, when the variance value of the luminance component is larger than TH2, it is determined that the motion vector replacement causes an increase in the code amount of the prediction error signal, and the motion vector that gives the minimum SAD without allowing the motion vector replacement is recognized. Let 21 be the final motion vector.

  The final predicted image determination unit 30 determines whether or not replacement is possible based on the above luminance signal variance value. As a result, the switch 32 is switched by the control signal 31 to output the final motion vector 33 and to the final motion vector 33. The corresponding final predicted image 37 is selected from either the predicted image 23 or the predicted image 29 and output. That is, in the inter mode, inter coding is performed using the final predicted image 37 after the above processing, and the final motion vector 33 is sent to the variable length coding / multiplexing unit 38 for coding.

The processing steps in the moving picture coding method based on the motion compensation prediction using the motion vector shown in FIG. 14 are summarized as follows.
That is, the moving picture coding method shown in FIG. 14 includes a motion vector detection step S1 for detecting a motion vector that gives a minimum prediction error (minimum matching distortion) based on an input macroblock and reference picture data, A first threshold determination step S4 for determining a prediction error signal based on a prediction image corresponding to a motion vector using a first threshold, a first threshold determination result, and the prediction error signal is greater than the first threshold In this case, a prediction vector derivation step S5 for deriving a prediction vector to be used when coding a motion vector, a prediction error signal generation step S6 for generating a prediction error signal based on a prediction image corresponding to the prediction vector, and a prediction error signal A calculation step S7 for calculating the generated code amount estimated value, and the calculated generated code amount estimated value of the prediction error signal are set to a second value. A second threshold determination step S8 for determining a threshold using a value, and the second threshold determination result, and when the generated code amount estimated value of the prediction error signal is smaller than the second threshold, the motion vector is replaced with the prediction vector. A replacement step S9, and performs motion prediction encoding using a motion vector.

  With the coding apparatus and method having the above configuration, a motion vector that does not contribute to the reduction of the code amount of the prediction error signal is specified by threshold determination, and the code amount of the motion vector is reduced while suppressing an increase in the code amount of the prediction error signal. Therefore, it is possible to select an optimal motion vector for the total encoding efficiency of motion information and image information.

In this reference example 9 , the specific determination method of the thresholds TH1 and TH2 has not been described, but these values vary depending on the specific encoding bit rate, frame rate, etc. It can be used as a tuning parameter according to the situation. It is possible to increase the encoding efficiency by setting these values optimally.

Compared to the conventional motion vector search, the encoding apparatus of Reference Example 9 calculates SAD threshold processing, derivation of a prediction error signal using a prediction vector and a prediction vector, calculation of a variance value of a luminance signal of the prediction error signal using the prediction vector, and A threshold process is added. However, SAD and variance threshold processing can be realized with only one conditional branch, and prediction vector derivation is not a special additional mechanism because it is necessary for conventional motion vector coding.

  Also, the prediction error signal can be derived from the prediction vector in the process of detecting the motion vector that gives the minimum SAD (motion detection unit 20) by buffering the macroblock and temporarily storing the prediction vector. Since the variance calculation is also performed only once per macroblock (and only for the macroblock that is a candidate for motion vector replacement in step S4), it can be realized with an extremely small amount of calculation. When it is configured by hardware, it is necessary to add some internal components, but when the encoding device is configured by software, it can be realized with a slight increase in the amount of calculation.

In the ninth embodiment , the motion vector is used as a parameter for expressing the amount of motion. However, the present embodiment is applicable to a case where prediction is performed using a more complicated motion parameter such as an affine parameter or a perspective transformation parameter. For these complex motion parameters, the predictive vector deriving unit 27 may calculate parameters that can be encoded with the least redundancy at the time of parameter encoding.

Reference Example 10
In Reference Example 9 described above, the motion vector that provides the minimum SAD may be selected from several modes. For example, according to the final draft of MPEG-4 (ISO / IEC JTC1 / SC29WG11 / N2202) and Advanced Prediction Mode which is one of the options of ITU-T recommendation H.263, one motion vector per macroblock is used. A “1MV mode” for obtaining a vector and a “4MV mode” for obtaining one motion vector per block of four 8 pixels × 8 lines as a unit of DCT can be selected. In this case, the “1MV mode” and the “4MV mode” are compared to select a mode that gives the minimum SAD.

This processing can be configured to be performed inside the motion detection unit 20 in FIG. 13, and can be positioned as a sub-step of step S1 in FIG.
In this case, the prediction vector deriving unit 27 is configured to calculate a prediction vector using the median value of the neighboring motion vector.

  FIG. 15 shows a motion vector prediction mechanism based on the MPEG-4 final draft. In FIG. 15, MV indicates a motion vector to be predicted, MV1 to MV3 indicate motion vectors necessary for calculating a prediction vector, (a) indicates a motion vector of the upper left block in the macroblock, and (b) indicates a macro. In the case of the motion vector of the upper right block in the block, (c) is the motion vector of the lower left block in the macro block, (d) is the motion vector MV1 to MV3 in the case of the motion vector of the lower right block in the macro block. Each position is determined.

The prediction vector (Px, Py) is obtained by the following equation.
Px = Median (MV1x, MV2x, MV3x)
Py = Median (MV1y, MV2y, MV3y)
However, MV1 = (MV1x, MV1y)
MV2 = (MV2x, MV2y)
MV3 = (MV3x, MV3y)
The function Median () outputs the median value of the three arguments.

FIG. 15 is an explanatory diagram of the “4MV mode”, but the “1MV mode” can be described in the case of FIG.
In the case of “1MV mode”, a prediction vector is obtained using (a) of FIG. 3 while regarding MV as a motion vector of the entire macroblock. When the motion vector mode of the macroblock to which each of the MV1 to MV3 belongs is “1MV mode”, all the four blocks are regarded as the same motion vector.
According to the tenth reference example , the present invention can be utilized in an encoding device that targets low bit rate encoding such as MPEG-4 and H.263.

Reference Example 11
In the present reference example 11 , an encoding apparatus having a configuration for switching ON / OFF of the motion vector replacement process described in the reference example 9 in units of frames will be described. For frames with very high efficiency of motion compensation prediction, motion vector replacement may have an adverse effect, and it is possible to prevent this, and to reduce the amount of computation in frames where replacement processing is not performed by turning ON / OFF the entire frame. And so on.

That is, in the reference example 11 , the value of the frame activity is calculated based on the input of the motion vector and the minimum prediction error amount from the motion detection unit 20, and the minimum matching distortion threshold processing unit 24 and the final value are calculated based on the calculated value. A frame activity calculation unit 40 that switches and controls motion vector replacement processing using a prediction vector by the predicted image determination unit 30 in units of frames, so that ON / OFF of motion parameter replacement processing can be controlled for each frame. Thus, encoding with a good balance between the overall code amount and encoding distortion is performed more flexibly.

FIG. 16 shows an internal configuration of the motion compensation prediction unit 19 in the present reference example 11 .
In the figure, the same parts as those of the reference example 9 shown in FIG. As a new code, 40 calculates the value of the frame activity based on the input of the motion vector 21 and the minimum prediction error 41 from the motion detection unit 20, and based on the calculated value, the minimum matching distortion threshold processing unit 24 and the final value. A frame activity calculation unit that outputs a control signal 42 that controls switching of motion vectors using predicted vectors by the predicted image determination unit 30 in units of frames. 43 and 44 are motion vector replacements based on the control signals. This is a switch for performing processing.

FIG. 17 is a flowchart showing a motion compensation prediction processing procedure when the motion compensation prediction unit 19 in FIG. 16 is used. Hereinafter, the operation of the motion compensation prediction unit 19 in Reference Example 11 will be described in detail with reference to FIGS. 16 and 17. The overall configuration of the encoding apparatus is equivalent to that of FIG. 13 described in Reference Example 9 , and only the motion compensation prediction unit 19 is replaced. The operation of the mode determination unit 5 is also the same as that of Reference Example 9 .

(1) Motion detection process (step S1)
First, as in Reference Example 9 , as shown in step S1, a motion vector 21 that gives a minimum prediction error is obtained for the input macroblock 3. All the motion vector detection processes are performed over one frame, and the motion vector 21 and the minimum SAD 41 of each macroblock are input to the frame activity calculation unit 40. The frame activity calculation unit 40 calculates a frame activity that is a criterion for determining whether or not to perform motion vector replacement processing in the frame (step S10). If it is determined from this frame activity value that motion vector replacement is to be performed, all the switches 26, 43, 44 are operated in the direction in which the processing of the reference example 9 is performed by the control signal 42. Otherwise, the switches 26, 43 and 44 are forced to operate so as to perform motion compensation prediction based on the motion vector 21 which gives the minimum SAD. Switching of these switches corresponds to step S11 in FIG. 17, and in FIG. 17, switching is performed in units of macroblocks. Although this may be performed in units of frames, the following description will be made assuming that selection is performed in units of macroblocks based on the value of frame activity.

  As the value of the frame activity, the size and complexity of the movement of the entire frame can be considered. Specific amounts may include the sum of the minimum SAD over the entire frame, motion vector variance indicating the degree of variation in the motion vector of the entire frame, and the like. For example, when the motion between frames is complicated, there is a high possibility that sufficient prediction efficiency cannot be obtained with a motion vector that captures the motion only by parallel movement. In this case, since it is expected that there are many places where the motion vector does not act effectively, the motion vector replacement mechanism is turned on. Conversely, when the motion is monotonous and sufficient prediction efficiency is obtained, it is possible to consider a control such as turning off the motion vector replacement mechanism and always using the motion vector of the minimum SAD. In a frame where the sum of the minimum SAD is large, there may be a case where the motion from the previous frame is complicated and the motion cannot be captured, and even when the motion vector variance is large, the motion complexity can be regarded as large. However, there are various definitions of frame activity.

The following description will be continued on the assumption that the most optimal activity is used for ON / OFF switching of motion vector replacement.
(2) Mode determination (steps S2, S3)
Next, in step S2, intra / inter determination is performed. This process is performed not by the motion compensation prediction unit 19 but by the mode determination unit 5 as described in (1). Next, in step S3, it is determined whether or not the mode selection information 6 indicates “inter mode”. If it is an intra mode, that is, a mode in which the input macroblock 3 is a direct encoding target signal, the process of the motion compensation prediction unit 19 ends here.

(3) ON / OFF determination of motion vector replacement processing based on frame activity (step S11)
As described in (1), in step S11, based on the frame activity obtained in step S10, the process proceeds to the motion vector replacement process (steps S4 to S9), or the motion vector that gives the minimum SAD is used as it is. It is determined whether to end the motion compensation prediction process.

(4) Motion vector replacement processing (steps S4 to S9)
If it is determined in (3) that the motion vector replacement process is to be performed, a total and efficient motion vector is calculated in the inter mode in accordance with the procedure described in Reference Example 1 in consideration of the balance between the code amount and the coding distortion. Determine again. At this time, the switches 43 and 44 are controlled to perform the operation of the reference example 1. Since the processing procedure in this case is exactly the same as the method described in the reference example 9 , the description here is omitted.

  As described above, in the processing step in the moving picture coding method based on motion compensated prediction using the motion vector shown in FIG. 17, the frame activity of the frame activity based on the motion vector and the minimum prediction error is different from the processing step shown in FIG. A motion prediction encoding is performed by further comprising a control step of calculating a value and performing switching control as to whether or not to replace the motion vector and the prediction vector in units of frames based on the calculated value.

  By the coding apparatus and method having the above-described configuration, a motion vector that does not contribute to the reduction of the code amount of the prediction error signal is identified by threshold determination in a frame unit and a macro block unit, and the motion vector is suppressed while suppressing an increase in the code amount of the prediction error signal. Therefore, it is possible to select the motion vector optimal for the total encoding efficiency of motion information and image information more flexibly. As a result, for a frame for which motion vector replacement is not effective, the replacement process can be turned off from the beginning to prevent a reduction in efficiency. Further, when the processing is OFF, the calculation amount of the replacement processing can be reduced in units of frames.

In the encoding apparatus and method of the present reference example 11 , compared with the encoding apparatus and method of the reference example 9 , a process process for calculating the frame activity is further added. However, since this is performed only once per frame, Does not significantly affect the increase in the amount of computation.

In this reference example 11 , the frame activity is calculated between the image data encoded in the past in the frame memory 34 and the input original image. The activity is calculated between the image including the original image and the original image, and the exact inter-frame activity cannot be calculated. Therefore, by providing a frame memory for buffering the original image of the past frame, it is possible to calculate the activity between the frames of the original images.

Further, the encoding apparatus according to the present reference example 11 is configured to be compatible with the “1MV mode” and the “4MV mode” as described in the reference example 10 , so that a low bit rate such as MPEG-4 or H.263 is achieved. Application to an encoding device is possible.

It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 1 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 2 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 3 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 4 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 5 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 6 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 7 of this invention. It is a block block diagram which shows the motion compensation apparatus which concerns on the reference example 8 of this invention. It is a block diagram showing a motion compensation device according to the shape condition of the present invention. It is a block diagram showing a modified example of the motion compensator according to the shape condition of the present invention. It is a block diagram showing another modified example of the motion compensator according to the shape condition of the present invention. It is a further block diagram showing another modified example of the motion compensator according to the shape condition of the present invention. It is a block block diagram which shows the moving image encoder which concerns on the reference example 9 of this invention. It is a flowchart which shows the flow of a process of the motion compensation estimation part 19 in the encoding apparatus of FIG. MPEG-4 or ITU-T Recommendation H.264 according to Reference Example 10 of the present invention. 263 is an explanatory diagram illustrating a method for calculating a motion vector prediction value in H.263. FIG. It is a block block diagram which shows the moving image encoder which concerns on the reference example 11 of this invention. FIG. 17 is a flowchart showing a processing flow of a motion compensation prediction unit 19 in the encoding device of FIG. 16. FIG. It is a block block diagram which shows the conventional motion compensation apparatus. It is explanatory drawing which calculates | requires an optimal image (predicted image).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 Minimum distortion detection part, 102 Specific vector distortion calculation part, 103 Optimal vector output part, 104 Previous frame memory, 110 Offset value, 111 Adder, 112 Comparator, 113 Selector, 114 Offset calculation part, 152 Previous frame with respect to specific vector Data, 153 specific vector, 154 minimum distortion data, 155 motion vector giving minimum distortion, 156 distortion data for specific vector, 157 optimal motion vector, 201 reference image, 202 input image, 203 motion vector, 204 prediction image, 205 absolute difference Value sum operation unit, 206 Difference absolute value sum, 207 Motion vector value encoding unit, 208 Motion vector code amount, 209 Optimal vector determination unit, 210 Optimal vector, 211 Difference square sum operation unit, 212 Difference square sum, 213 Extender, 214 Difference calculator, 215 Difference vector encoder, 216a, 216b Average value separation unit, 217 Average value separation predicted image, 218 Average value separation input image, 219 Distortion amount calculation unit (evaluation value calculator), 220 Evaluation value, 222 differentiator, 223 prediction error, 224 frequency analysis unit, 225 frequency coefficient, 226 evaluation value generation unit, 227 evaluation value, 228 encoding unit, 229 encoded information amount, 230a, 230b, 230c luminance color difference separation unit 231 predicted image luminance signal, 232 predicted image color difference signal, 233 input image luminance signal, 234 input image color difference signal, 235a, 235b differentiator, 236 luminance evaluation value generation unit, 237 color difference evaluation value generation unit, 238 adder, 239 Total evaluation value, 240 predicted image of only luminance component, 241 predicted image of only color difference component, 24 2 motion vector luminance signal, 243 motion vector color difference signal, 244 luminance color difference evaluation value comparison unit, 20 motion detection unit, 22 motion compensation unit, 24 minimum matching threshold processing unit, 27 prediction vector derivation unit, 30 final prediction image determination unit, 34 frame memory, 40 frame activity calculator.

Claims (1)

In the motion compensation device that predicts the motion from the data of the previous frame in units of blocks and performs the information amount reduction when performing the coding for moving image coding,
A previous frame memory unit for storing data of the previous frame;
A minimum distortion calculation unit that performs pattern matching from the block data of the current frame and the partial image region of the search range data of the previous frame stored in the frame memory unit, and calculates a motion vector and a distortion value that give the minimum distortion;
Enter a specific vector is a motion vector to be compared reference information for obtaining the current optimal motion vectors, it calculates the distortion of the block data of the previous frame corresponding to the block data and the specific vectors of the current frame, the specific A specific vector distortion calculation unit for outputting a vector and the calculated distortion;
An optimal vector output unit that outputs an optimal motion vector based on the distortion output from the minimum distortion calculation unit and the distortion output from the specific vector distortion calculation unit ;
The specific vector distortion calculation unit inputs the optimal motion vector output from the optimal vector output means as a specific vector,
The optimum vector output unit has a small offset if the difference value is small according to the difference value between the motion vector giving the minimum distortion from the minimum distortion calculation unit and the specific vector output from the specific vector distortion calculation unit. An offset value calculator that adaptively changes the offset value to give a large offset value if the difference value is large, and an offset from the offset value calculator to the distortion value from the minimum distortion calculator An adder that adds a value to weight the distortion from the minimum distortion calculation unit, a comparator that compares the weight from the weighted minimum distortion calculation unit and the distortion from the specific vector distortion calculation unit, and A selector that selects either a motion vector that gives the minimum distortion or a specific vector based on the comparison result of the comparator and outputs it as an optimal motion vector; Motion compensation unit for the butterflies.

Images