JP4448115B2 - Encoded video playback device - Google Patents

Description

  The present invention relates to an encoded moving image reproducing apparatus, and more particularly to an encoded moving image reproducing apparatus capable of reducing a frame memory size when reproducing a moving image encoded by MPEG2 or the like.

  So far, a method for reducing the frame memory size in the decoding process of encoded video data has been shown in Iwahashi et al.'S "Scalable decoder without low-frequency drift" (Science Technical Report DSP94-108). There is such a scalable decoder (FIG. 2). Originally, a scalable decoder is a device that decodes using a part of encoded data, but in the case of the figure, the frame memory size is reduced by using an inverse DCT (Inverse Discrete Cosine Transform) with reduced order. be able to.

  Video data encoded by MPEG2 or the like is variable-length decoded by the variable-length decoder 1, and the quantized DCT coefficients obtained there are inverse-quantized by the inverse quantizer 2 to obtain 8 × 8 DCT coefficients. . When 2D inverse DCT processing is performed with 4 × 4 IDCT using only 4 × 4 coefficients among 8 × 8 DCT coefficients, 4 × 4 pixel data that is 1/2 in both the vertical and horizontal directions of the original image is obtained. When the block to be decoded is a motion compensation block, the motion compensation block from the motion compensation unit 8 is added to perform image restoration. Since the restored image is reduced to 1/2 in both the vertical and horizontal directions compared to the original image, the enlargement unit 11 enlarges the screen and sequentially stores and displays it in a display memory (not shown).

Further, the restored image data is stored in the frame memory 6. Also in this case, since the restored image is 1/2 in both length and width, the required frame memory size is also the original 1/4 size. In addition, since motion compensation is also performed in a reduced space, the motion vector converter 12 converts the motion vector into 1/2 in the vertical and horizontal directions and extracts the corresponding block from the frame memory 6. In addition, when 2 × IDCT uses 4 × 4 IDCT only in the horizontal direction and 8 × 8 IDCT in the vertical direction, the image size and the frame memory size are ½ of the original.
Iwahashi et al. “Scalable decoder without low-frequency drift” (Science Tech. DSP94-108)

  In the conventional method as described above, it is possible to reduce the frame memory size by changing the IDCT size, but drift noise occurs due to the mismatch of the predicted image on the encoding side and the decoding side, and the image quality There was a problem of causing deterioration. In addition, since the resolution of the restored image is also 1/2 or 1/4 of the original, there is a problem that the quality of the decoded image is deteriorated.

  The present invention solves the above-mentioned problems, suppresses drift noise when restoring encoded moving image data, and can reduce only the frame memory size without reducing the resolution to be decoded as much as possible. It is an object of the present invention to provide an encoded moving image reproduction apparatus.

According to the present invention, there is provided means for inputting encoded moving image data frequency-transform encoded with a block size of N × N pixels (N is a positive integer), and among the inputted N × N frequency conversion coefficients. , K × M of the low-frequency component (K, M is a positive integer smaller than N) only the inverse transform processing in the DCT basis of K × K in the horizontal direction, the inverse transform processing in the DCT basis of M × M in the vertical direction A means for decoding the encoded moving image data, and a compression means for recompressing the decoded moving image data by predictive encoding, so as to obtain K × M pixel block data ; Accumulating means for accumulating data outputted from the compressing means, decompressing means for decompressing data outputted from the accumulating means, and motion compensated prediction for performing motion compensation prediction on moving image data outputted from the decompressing means Means for compensating for movement. Prediction is characterized respectively horizontal and vertical direction, K / N, in that to use a scaled motion vector to M / N.

  According to the present invention, the pixel data input to the frame memory is compressed without being degraded or slightly degraded, and the frame memory size is reduced by combining with the reduced block size IDCT. Compared with the frame memory reduction method, there is an effect that drift noise and resolution degradation can be suppressed.

  As an example, using video data that is compressed from 2Mbit / s to 6Mbit / s in MPEG2 format for test video sequence (Flower Garden) used in ISO, decoding characteristics under the condition of halving the frame memory size in the horizontal direction In the case of the second embodiment, a gain of 1.5 dB to 4.5 dB could be obtained as compared with the conventional method. In addition, it was confirmed subjectively that resolution degradation and drift noise were greatly improved.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a first embodiment of the encoded moving image reproduction apparatus of the present invention. The moving image data encoded by MPEG2 or the like is variable-length decoded by the variable-length decoder 1, and the quantized DCT coefficient obtained there is inversely quantized by the inverse quantizer 2 to obtain N × NDCT coefficients. In the case of the MPEG system, N = 8. Then, the inverse DCT converter 3 obtains N × N difference pixel data by inverse DCT processing. The adder 4 adds the difference pixel data and the image data read from the frame memory 6 and outputs reproduced image data. When the block to be decoded is a motion compensation block, the motion compensation unit 8 reads out a block necessary for motion compensation from the frame memory 6 and performs image restoration. The output data is sequentially stored and displayed in a display memory (not shown) in the subsequent stage.

  The restored image data is further compressed again by the compressor 5 and stored in the frame memory 6. When a block necessary for motion compensation is extracted from the compressed pixel data, all the pixels in the motion compensation block are decoded by the decompressor 7 and subjected to motion compensation processing by the motion compensation unit 8. As described above, the present invention is characterized in that the frame memory size is reduced by compressing the pixel data input to the frame memory.

FIG. 3 is a block diagram showing the configuration of the compressor 5. In the compressor 5, the input pixel data is input to the subtracter 20, and the prediction value output from the predictor 24 is subtracted to obtain a prediction error Δ, and the quantizer 21 performs quantization processing. The quantized data is decoded by a local decoder including an inverse quantizer 25, an adder 23, and a predictor 24. If the input pixel data is X and the predicted pixel data is PX,
Δ = X-PX
It becomes.

  PX can be determined in several ways. FIG. 5 is an explanatory diagram showing various prediction methods for the predicted value PX. In FIG. 5, the horizontal axis indicates the horizontal direction of the pixel. Here, description will be made by taking prediction in the horizontal direction as an example. Also, the hatched pixels are pixels that encode the pixel values themselves without using prediction, and the white circle pixels indicate pixels that are predicted and compressed and encoded. In order to perform motion compensation, random access to read a block from an arbitrary position in the frame memory is necessary, but even if differential encoding is adopted by storing the pixel value itself at a predetermined cycle, random access is performed. Is possible.

FIG. 5A shows a prediction method using a pixel on the left side. For example, if the predicted pixel for X1 is PX1,
PX1 = X0, Δ1 = X1-X0
.DELTA.1 is encoded as follows. X2 is predicted by using encoded and locally restored X1 (referred to as RX1). Therefore, for X2, if the predicted pixel is PX2,
PX2 = RX1, Δ2 = X2-RX1
.DELTA.2 is encoded as follows. However, the pixels for each PN pixel are encoded without using prediction. PN is an integer greater than or equal to 2, and when it is infinite, only the first pixel is encoded without using prediction.

FIG. 5B shows an example in which predictive encoding is performed using only pixels to be encoded without using predictive encoding. In FIG. 5, X1, X2, and X3 are all predicted from X0. For example, for X3, if the predicted pixel is PX3,
PX3 = X0, Δ3 = X3-X0
.DELTA.3 is encoded as follows. However, the pixels for each PN pixel are encoded without using prediction.

FIG. 5C shows an example of a prediction method using a plurality of decoded pixels as prediction pixels. In the figure, for example, X1 is predicted using X0 and X4 adjacent in the horizontal direction. In this case, the predicted pixel PX1 is
PX1 = αX0 + βX4
It becomes. α and β are weighting factors, and α = β = 1/2 can be used as a simple average, or a weighted average can be taken according to the distance between pixels, and α = 3/4 and β = 1/4 can be set. It is.

  The prediction method can be processed not only in the horizontal direction but also in the vertical direction. Further, prediction is possible not only in the vertical direction but also in the diagonal direction. FIG. 6 is an explanatory diagram showing an example in which the prediction of FIG. 5C is extended to a two-dimensional space. Here, X00, X02, X20, and X22 are pixels to be encoded without prediction. As for the prediction method, X01 is predicted using pixels X00 and X02 on the same horizontal. X10 can be predicted using X00 and X20 positioned in the vertical direction, and X11 can be predicted using X00 and X22 positioned in the diagonal direction.

  FIG. 7 is a graph illustrating an example of characteristics of the quantizer 21. In FIG. 7, the horizontal axis represents the input value and the vertical axis represents the output value. Q is the quantization step size. A characteristic (a) indicated by a dotted line in FIG. 7 indicates a characteristic of the linear quantizer. For example, assuming that the input value is X and the quantized representative value that is the output value is Y, Y = Q when (Q / 2) ≦ X <(3Q / 2). The characteristic (b) indicated by the solid line is a linear quantizer with a dead zone used in MPEG or the like, and 0 is output while the input value is ± Q. The quantizer 21 can use a nonlinear quantizer in addition to the linear quantizer including them. It is also possible to change the quantization method depending on the pixel. For example, for pixels to be encoded without using the prediction shown in FIG. 4, it is also possible to output the input value directly without performing any quantization processing.

  As the encoder 22, a fixed length encoder or a variable length encoder such as that used in MPEG can be used. In the case of a fixed-length encoder, for example, when 64 quantized representative values are obtained, each data can be represented by 6 bits. It is also possible to reduce the amount of encoded information using 2's complement representation. For example, when the pixel data is represented by 8 bits, the prediction error Δ is −255 to +255, and the quantization step Q is 4, −252, −248... 0 to 248 using the quantizer of FIG. , 252 of 127 kinds of quantized representative values are obtained. 7 bits are required to represent 127 data.

  However, if the prediction error Δ always falls within the 256 dynamic range for a certain pixel X, the quantized representative value can be expressed by 64 data and can be expressed by 6 bits. For example, when P = 0, the range of X is from 0 to 255, so the range of Δ = X−P is from 0 to 255. In the case of P = 255, in the case of Δ, the range is from −255 to 0. As described above, Δ always falls within 256 dynamic ranges. Therefore, when the quantization step Q is 4, the quantization representative value can also be expressed by 64 data.

  FIG. 4 is a block diagram showing the configuration of the decompressor 7. The compressed pixel data stored in the frame memory is first subjected to the reverse processing of the encoder 22 in FIG. 3 in the decoder 30 to obtain quantized data from a code such as a fixed length code or a variable length code. The quantized data is subjected to an inverse quantization process in the inverse quantizer 31 to restore the prediction error Δ. For example, when the quantization characteristic of FIG. 7A is used, the representative quantization values are 0, Q, 2Q,..., The quantization step Q is 4, and the quantized data is 0, 1,. Becomes 0, 4, 8, 12,.

  Further, the prediction error Δ is added to the predicted value P by the adder 32, and the pixel X is decoded. That is, X = P + Δ. For the predictor 33, the same device as the predictor 24 of FIG. 3 can be used. As the inverse quantization process, the inverse process of the linear quantization process or the nonlinear quantization process of FIG. 7 can be used.

  Next, a second embodiment will be described. FIG. 8 is a block diagram showing the configuration of the second embodiment of the decoding apparatus of the present invention. Moving picture data encoded by MPEG2 or the like is variable-length decoded by the variable-length decoder 1, and the quantized DCT coefficient obtained there is inverse-quantized by the inverse quantizer 2 to obtain N × N DCT coefficients. . In the K × M inverse DCT converter 40, only the K × M portion (K, M ≦ N) of the low frequency component in the N × NDCT coefficient is subjected to K × K inverse DCT processing in the horizontal direction and M × in the vertical direction. M × DCM processing is performed to obtain K × M pixel data. When the block to be decoded is a motion compensation block, the motion compensation block from the motion compensation unit 8 is added by the adder 4 to restore the image.

  Further, the restored image data is compressed again in the compressor 5 by the compressor 5 and stored in the frame memory 6. When a block necessary for motion compensation is extracted from the compressed pixel data, all the pixels in the motion compensation block are decoded by the decompressor 7 and subjected to motion compensation processing by the motion compensation unit 8. However, for the motion vector, the motion vector converter 12 performs motion compensation using the motion vector MV ′ obtained by scaling the motion vector MV obtained from the variable length decoder 1 to K / N and M / N respectively in the horizontal and vertical directions. Do. The restored image is converted into a necessary image size by the enlarger 41. For example, in order to make it the same as the encoded image size, the scale is enlarged and scaled to N / K and N / M times respectively in the vertical and horizontal directions.

  As described above, the present invention is characterized in that the frame memory size is reduced by using both the reduced size IDCT 40 and the compression of the pixel data input to the frame memory.

  Here, K × MIDCT will be described with reference to FIG. FIG. 10A shows N × NDCT coefficients. K × MIDCT performs IDCT processing using K × M DCT coefficients in the region of M lines in the vertical direction and K × M DCT coefficients in the horizontal direction from the low frequency component among the N × N DCT coefficients. This IDCT processing is performed using K × KIDCT in the horizontal direction and M × M IDCT in the vertical direction to obtain K × M pixel block data.

The motion vector converter 12 multiplies the motion vector MV horizontally and vertically by K / N and M / N, respectively, and outputs it as MV ′. Therefore, if the horizontal and vertical MV and MV ′ are MVx, MVx ′, MVy and MVy ′, respectively,
MVx '= MVx × K / N
MVy '＝ MVy × M / N
It becomes. However, the calculation accuracy can be obtained with integer accuracy as well as decimal point accuracy, and motion compensation with decimal point accuracy as used in MPEG can be applied for motion compensation.

The enlarger 41 converts the restored image into a necessary image size. For example, in order to make it the same as the encoded image size, the scale is enlarged and scaled to N / K and N / M times respectively in the vertical and horizontal directions. This process can be realized by linear interpolation. For example, when converting 3 pixels to 4 pixels, if the original continuous pixel values are R0, R1, R2 and the pixel values to be calculated are Q0, Q1, Q2, Q3, Q0, Q1, Q2, Q3 are R0, It can be obtained as follows according to the distance ratio of the pixel positions of R1 and R2.
Q0 = R0, Q1 = 1/3 R0 + 2/3 R1
Q2 = 2/3 R1 + 1/3 R2, Q3 = R2

  FIG. 9 is a block diagram showing the configuration of the third embodiment of the reproducing apparatus of the present invention. Video data encoded by MPEG2 or the like is variable-length decoded by the variable-length decoder 1, and the quantized DCT coefficient obtained there is inverse-quantized by the inverse quantizer 2 to obtain N × N DCT coefficients. . In the zero pad IDCT (0 Pad IDCT) 50, the N × N DCT coefficient output from the inverse quantizer 2 is filled with 0 except for the K × M portion of the low frequency component, and the N × N inverse DCT processing is performed. N × N pixel data is obtained. FIG. 10B is an explanatory diagram of the zero pad IDCT. The zero pad IDCT 50 sets N × N DCT coefficients other than K × M coefficients from the low frequency side to 0, and performs N × N IDCT to obtain N × N pixel data. When the block to be decoded is a motion compensation block, the motion compensation block from the motion compensation unit 8 is added by the adder 4 to restore the image.

  Further, the restored image data is compressed again in the compressor 5 by the compressor 5 and stored in the frame memory 6. When a block necessary for motion compensation is extracted from the compressed pixel data, all the pixels in the motion compensation block are decoded by the decompressor 7 and subjected to motion compensation processing by the motion compensation unit 8. As described above, this embodiment is characterized in that the frame memory size is reduced by performing IDCT by filling a certain area with 0 and compressing the pixel data input to the frame memory. With such a configuration, by reducing the high-frequency component of the difference image data output from the inverse DCT converter, compatibility with the accuracy of the image data stored in the frame memory 6 is improved.

  FIG. 11 is a block diagram showing the configuration of the fourth embodiment of the reproducing apparatus of the present invention. Video data encoded by MPEG2 or the like is variable-length decoded by the variable-length decoder 1, and the quantized DCT coefficient obtained there is inverse-quantized by the inverse quantizer 2 to obtain N × N DCT coefficients. . N × N pixel data is obtained by inverse DCT processing in IDCT3. If the block to be decoded is a motion compensation block, the motion compensation block from the motion compensation unit 8 is added to perform image restoration.

  Further, the restored image data is stored in the frame memory 6 after the downsampler 60 reduces the number of pixel data. When the reduced pixel data is used to extract a block necessary for motion compensation, all the pixels in the block are restored by the upsampler 61 and subjected to motion compensation processing by the motion compensation unit 8.

For example, the down sampler 60 samples only pixels for each SN pixel (SN> 0, SN is an integer) and discards the other pixels, thereby reducing the number of pixel data at a ratio of 1: N. As a result, the number of pixel data recorded in the frame memory is 1 / SN. The upsampler 61 restores pixel data discarded by, for example, linear interpolation. For example, assuming that X0 and X3 are pixel data recorded in the frame memory 6, X1 and X2 can be obtained as follows using the distance ratio between X0 and X3.
X1 = 2/3 X0 + 1/3 X3, X2 = 1/3 X0 + 2/3 X3

  FIGS. 12 and 13 are block diagrams showing the main configuration of the fifth embodiment of the reproducing apparatus of the present invention. In the fifth embodiment, the configuration shown in FIGS. 12 and 13 is adopted for the compressor 5 and the expander 7 in the first embodiment shown in FIG. FIG. 12 is a block diagram showing the configuration of the compressor 5 in the fifth embodiment. The S × T Hadamard transformer 70 performs Hadamard transform on the pixel data for each S × T block (S and T are positive integers), and converts the data into frequency domain data. This output data is quantized by the quantizer 21 and encoded by the encoder 22. The quantizer 21 and the encoder 22 can use the same system as in the first embodiment, and the quantizer 21 changes the quantization step Q according to the position and size of the coefficient in the block. It is also possible to make it.

  FIG. 13 is a block diagram showing the configuration of the decompressor 7 in the fifth embodiment. The compressed image data stored in the frame memory 6 is first restored to quantized data by the decoder 30, and the quantized data is restored to frequency domain data for each S × T block by the inverse quantizer 31. Then, the frequency domain data is restored to image data by the S × T inverse Hadamard transformer 71. For the decoder 30 and the inverse quantizer 31, the same method as in the first embodiment can be used. In the fifth embodiment, an example using Hadamard transform is disclosed, but other orthogonal transform such as DCT can be adopted instead of Hadamard transform.

  Although the embodiments have been disclosed above, the following modifications are also conceivable. The transform coding can be applied not only to DCT but also to any transform coding such as wavelet transform.

  In this embodiment, an example using a quantizer has been disclosed. However, a configuration that does not use a quantizer is possible, and in this case, lossless (completely recoverable) encoding can be performed.

  Each functional block of the present invention can be processed by software by a computer such as a personal computer or a workstation, and can also be processed by a dedicated LSI or the like.

It is a block diagram which shows the structure of 1st Example of the reproducing | regenerating apparatus of this invention. It is a block diagram which shows the structure of the conventional reproducing | regenerating apparatus. 3 is a block diagram showing a configuration of a compressor 5. FIG. 3 is a block diagram showing a configuration of an expander 7. FIG. It is explanatory drawing which shows the various prediction methods of the predicted value PX. It is explanatory drawing which shows the example which expanded the prediction of FIG. 5c to two-dimensional space. 3 is a graph showing an example of characteristics of a quantizer 21. It is a block diagram which shows the structure of 2nd Example of the reproducing | regenerating apparatus of this invention. It is a block diagram which shows the structure of 3rd Example of the reproducing | regenerating apparatus of this invention. It is explanatory drawing which shows the example of a system of IDCT regarding a 2nd, 3rd Example. It is a block diagram which shows the structure of 4th Example of a reproducing | regenerating apparatus. It is a block diagram which shows the structure of the compressor 5 of 5th Example. It is a block diagram which shows the structure of the expander 7 of 5th Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Variable length decoder, 2 ... Inverse quantizer, 3 ... Inverse DCT converter, 4 ... Adder, 5 ... Compressor, 6 ... Frame memory, 7 ... Decompressor, 8 ... Motion compensation part, 10 ... 4 X4 inverse DCT converter, 11, 41 ... enlarger, 12 ... motion vector converter, 20 ... subtractor, 21 ... quantizer, 22 ... encoder, 23, 32 ... adder, 24, 33 ... predictor 30 ... Decoder, 31 ... Inverse quantizer, 40 ... KxM inverse DCT converter, 50 ... Zero pad inverse DCT converter, 60 ... Downsampler, 61 ... Upsampler, 70 ... Hadamard transformer, 71 ... Inverse Hadamard transformer

Claims (2)

Means for inputting encoded moving image data frequency-transform encoded with a block size of N × N pixels (N is a positive integer);
Of N × N number of frequency transform coefficients the input, K × M of the low-frequency component (K, M is a positive integer smaller than N) inverse transform processing in the DCT basis of K × K only in the horizontal direction Means for decoding the encoded moving image data , wherein K × M pixel block data is obtained by performing an inverse transformation process on the M × M DCT base in the vertical direction;
Compression means for recompressing the decoded moving image data by predictive encoding;
Storage means for storing data output from the compression means;
Decompression means for decompressing data output from the storage means;
Motion compensation prediction means for performing motion compensation prediction on the moving image data output from the decompression means,
The motion compensated prediction uses a motion vector scaled to K / N and M / N in the horizontal and vertical directions, respectively. In the encoded moving image reproduction device according to claim 1,
The encoded moving image characterized in that the compression means for recompressing keeps periodically sampled image data uncompressed and compresses other image data by predictive encoding using the uncompressed image data. Image reproduction device.

Images