JP4871305B2 - Video compression encoding apparatus - Google Patents

Description

  The present invention relates to a moving image compression coding apparatus, and more particularly to a compression coding apparatus suitable for compression coding of a moving image in which a partial image such as a logo is inserted.

  As a technique for inserting a partial image such as a logo into a moving image, Patent Documents 1 to 8 disclose techniques for partially inserting an image into a moving image that has been compression-encoded by an encoding method such as MPEG.

  These methods are techniques that insert partial images after partially decoding a moving image and returning it to an uncompressed image, or on a conversion coefficient, both of which are essential in terms of reusing moving images. Although there is no technology, it is difficult to perform insertion control that minimizes the degradation of the image quality of the partial image to be inserted because the restriction caused by the amount of generated code in the encoded video before inserting the partial image is large. Met.

  In addition, in the invention described in the above-mentioned patent document, when encoded by an encoding method using inter-frame prediction such as MPEG including a logo image indicating an advertisement or a copyright holder or important information, especially for mobile etc. However, there is a problem that images such as important advertisements and logos often deteriorate due to the influence of limited encoding parameters.

Therefore, in order to cope with this problem, the present applicant has applied for a patent by inventing an encoding of a higher quality partial image by performing temporal and spatial control of the quantization parameter ( No. 2007-572, hereinafter referred to as “application invention”).
JP 2004-304231 A Special table 2003-532316 Special Table 2002-531019 Special Table 2002-531019 Japanese Patent Laid-Open No. 11-261966 JP 2001-268572 A JP 2000-78471 A Japanese Unexamined Patent Publication No. 2000-78468

  However, in the above-mentioned application invention, this is referred to as MPEG-4 AVC / H. When applied to H.264, the influence of the smoothing of pixels between macroblocks by a deblocking filter (a filter for reducing block distortion that occurs at the time of image encoding) propagates in time by inter-frame prediction. In particular, it has been found that there is a problem that the quality deterioration at the edge of the partial image (logo) becomes gradually remarkable. In particular, in the content for mobile, there is a problem that this phenomenon appears remarkably because the appearance interval of the intra picture that becomes a randomly accessible key frame becomes large.

  The present invention has been made in view of the above-described problems of the prior art, and its purpose is to avoid the phenomenon in which the quality of partial images gradually deteriorates over time due to the effects of the deblocking filter and inter-frame prediction. It is an object of the present invention to provide a moving image compression encoding apparatus which can be used.

In order to achieve the above-described object, the present invention provides a compression encoding apparatus that compresses and encodes an uncompressed moving image in which a partial image is inserted into a partial region on an image by inter-frame prediction or intra-frame prediction. Means for deblocking the local decoded signal, an image memory for storing the deblocked local decoded signal, and storing the first frame including a partial image in the deblocked local decoded signal in the image memory Control means for allocating a memory area to be used as a long-term reference memory, and the long- term reference memory is intra-coded by intra-coded image data immediately after all of the image memory data is discarded. There is a feature in that the image data is updated .

  According to the present invention, the memory area for storing the first frame including the partial image of the deblocked image to be encoded is allocated as the long-term reference memory and becomes the reference image in the inter-frame prediction. It is possible to prevent image degradation from propagating in time.

  In addition, even when the appearance interval of an intra picture, which is a randomly accessible key frame, is increased, such as mobile content, it is possible to prevent the degradation of partial images from being propagated in time. A mobile-oriented image including a good partial image can be provided.

  Hereinafter, the present invention will be described in detail with reference to the drawings. First, the invention of the application as a premise of the present invention will be briefly described.

  FIG. 1 is a block diagram showing a configuration of a main part of a moving image compression encoding apparatus according to the present invention. Here, illustrations of configurations unnecessary for the description of the present invention are omitted.

  The moving image compression encoding apparatus 10 of the present invention includes an uncompressed moving image Dm, an uncompressed partial image Dp such as a logo image, a mark, and an important character string inserted into the uncompressed moving image Dm, and this portion. As the attribute information Da of the image Dp, the shape and size of the partial image Dp, spatial and temporal arrangement information of the partial image Dp, and the like are input. The arrangement information of the partial image Dp includes an insertion position and an insertion timing on the reproduced image of the partial image Dp. However, the input of the attribute information Da of the partial image Dp is arbitrary, and when the attribute information Da is not input, the attribute information Da of the partial image Dp can be automatically extracted by an existing method such as Hough transform or template matching. Hereinafter, a case where the attribute information Da is input from the outside will be described.

  In the moving image compression coding apparatus 10, the arrangement control unit 1 spatially displays the partial image Dp on the reproduced image based on the attribute information Da of the partial image Dp and the boundary information of each control block of the moving image Dm. In addition to correcting the correct insertion position, block attribute information indicating whether or not the partial image Dp is superimposed on each control block of the moving image Dm is extracted. The partial image insertion unit 2 inserts and outputs the partial image Dp at the insertion position of the uncompressed moving image Dm designated by the arrangement control unit 1.

  The encoding control unit 3 determines the encoding mode and motion vector MV of each control block based on the attribute information Da of the partial image Dp provided from the arrangement control unit 1 and the block attribute information of each control block. At the same time, the quantization parameter QP is determined based on the image parameters such as the encoding rate and the difficulty of encoding. The compression encoding unit 4 controls the non-compressed video Dm + p in which the partial image Dp is inserted based on the encoding mode, the quantization parameter QP, the motion vector MV, and the like notified from the encoding control unit 3. Compression encoding is performed for each block.

  The configuration when the attribute information Da is not input from the outside is as shown in FIG. The partial image information extraction unit 5 extracts a partial image in the block by a method such as Hough transform or template matching, and extracts information on the partial image Dp and the attribute information Da.

  FIG. 3 is a diagram schematically showing the control block of the uncompressed moving image Dm + p into which the partial image Dp is inserted and the boundary information of the partial image. Here, the control block is a macro block (MB) and a motion. A case where the compensation block (MC) is used will be described as an example.

  In the present embodiment, among MBs of moving images, as indicated by hatching in the figure, MBs that at least partially overlap the partial image Dp are expressed as MBlogo in particular. The insertion position of the partial image Dp is defined as an offset value from the reference point (in this embodiment, the upper left corner of the reproduced image). It is assumed that the inserted partial image (logo image or the like) is arranged with a certain fixed offset value and does not change with time.

  FIG. 4 is a diagram schematically representing the operation of the arrangement control unit 1. In this embodiment, the partial image Dp has a rectangular shape including a square, and its vertical and horizontal size is an integer of MB or MC. It is prescribed in advance twice. The partial image Dp is arranged so that its outline coincides with the MB boundary or the MC boundary.

  The arrangement control unit 1 first determines whether or not the contour of the partial image Dp matches the MB boundary. For the contour that does not match, the MC is changed from 16 × 16 pixels to 16 × 8 pixels, and further to 8 × 16. The pixel, 8 × 8 pixel, 8 × 4 pixel, 4 × 8 pixel, and 4 × 4 pixel are subdivided until the contour of the partial image Dp matches the MC boundary. Alternatively, the arrangement coordinates of the partial image Dp are corrected so that the contour of the partial image Dp matches the MC boundary. The amount of generated codes can be suppressed by matching the contour of the partial image with a larger block boundary.

  FIG. 5 is a block diagram showing the configuration of the main parts of the encoding control unit 3 and the compression encoding unit 4, and the compression encoding unit 4 responds to an instruction from the encoding control unit 3 with a moving image Dm. + p compresses and encodes.

In the encoding control unit 3, the quantization parameter deriving unit 301 derives the quantization parameter QP by a known method based on the image parameters such as the encoding rate and the difficulty of encoding. In the correction coefficient storage unit 302, a correction coefficient α for reducing the quantization parameter QP when applying intra-frame prediction to MBlogo to make the quantization step finer, and when applying inter-frame prediction to MBlogo A correction coefficient β for increasing the quantization parameter QP to roughen the quantization step is stored. The MV setting unit 303 sets all motion vectors for MBlogo prediction between frames to zero. In the present embodiment, the correction coefficients α and β are set such that the quantization parameter (QP2) that has been increased and corrected is always larger than the quantization parameter (QP1) that has been corrected and decreased.

The motion prediction unit 401 generates a motion vector MV based on the uncompressed moving image Dm + p on which the partial image is superimposed and the reference image D7 stored in the image memory unit 410, and uses the motion vector MV to generate a frame The encoding cost Cinter when performing inter prediction is output. The motion compensation unit 402 performs inter-frame prediction based on the motion vector MV and the reference image D7 stored in the image memory unit 410, and outputs a prediction result as inter-frame prediction data D1. The memory area of the image memory unit 410 is controlled by the encoding control unit 3 as described in detail later.

  The subtractor 412 subtracts the prediction signal D8 selected by the selector 407 from the uncompressed moving image Dm + p to generate a prediction residual D2. The prediction residual D2 is orthogonally transformed in units of MB in the orthogonal transformation unit 403 to become an orthogonal transformation coefficient D3. The orthogonal transform coefficient D3 is quantized by the quantization unit 404 using the quantization parameter QP instructed from the encoding control unit 3.

  The orthogonal transform coefficient D4 quantized by the quantization unit 404 is output after being variable-length encoded by the entropy encoding unit 405 and dequantized by the inverse quantization unit 407. The output of the inverse quantization unit 407 is orthogonally inverse transformed by an orthogonal inverse transform unit 408. That is, the inverse quantization unit 407 and the orthogonal inverse transform unit 408 perform processing reverse to that of the quantization unit 404 and the orthogonal transform unit 403, and the output of the orthogonal inverse transform unit 408 approximates the prediction residual D2. A signal is obtained.

  The output of the orthogonal inverse transform unit 408 is added to the prediction signal D8 by the adder circuit 406 to generate a local decoded signal D5. The local decoded signal D5 is stored in the image memory 410 as a reference image signal D7 after block noise is removed by the deblocking filter unit 409.

  The intra-frame prediction unit 411 generates an intra-frame prediction signal D6 based on the local decoded signal D5, and outputs an encoding cost Cintra when intra-frame prediction is performed. The selector 407 selects one of the inter-frame prediction signal D1 and the intra-frame prediction signal D6 for each control block based on the comparison result between the coding cost of the intra-frame prediction and the coding cost of the inter-frame prediction. Is output as a prediction signal.

  FIG. 6 is a flowchart showing the setting procedure of the predictive coding method and the quantization parameter QP in the present embodiment, and mainly shows the operation of the coding control unit 3. Since the description of the setting procedure has been described in detail in the specification of the invention, the description is omitted.

  The above is the outline of the invention of the application. In the invention of the application, as described above, in the problem that the quality of the partial image gradually deteriorates due to the influence of the deblocking filter unit 409, and in the content for mobile, There has been a problem that this phenomenon appears prominently because the appearance interval of intra pictures as key frames that can be randomly accessed increases.

  The present invention solves such problems, and its configuration and operation will be described below.

  FIG. 7 is a diagram illustrating an example of a frame sequence when a partial image Dp such as a logo image is inserted into a moving image. The partial image Dp is inserted into (T + 1) image sections from frame numbers M to M + T. Shows the case.

  Now, assuming that the number of images that can be stored in the image memory unit (frame memory) 410 is R ′, encoding by the procedure shown in FIG. 8 avoids the propagation of noise due to the deblocking filter. Or it can be reduced. This procedure can be executed by control by the encoding control unit 3, for example.

  In step S11, it is determined whether or not a deblocked frame including the partial image Dp has been input. If this determination is negative, the process proceeds to step S18, where all R ′ frame memories are stored. It is used as a short-term reference (hereinafter abbreviated as “STR”) memory. The STR memory has a FIFO structure, and when the memory capacity is exceeded, the oldest data is discarded.

  If step S11 is positive, the process proceeds to step S12 to determine whether or not the number of frames including the partial image is equal to or greater than the R′th frame. If this determination is negative, the process proceeds to step S18. If the determination is positive, the process proceeds to step S13. In step S13, it is determined whether or not the frame including the partial image is exactly the R'th frame. If this determination is affirmative, the process proceeds to step S14, and if negative, the process proceeds to step S15.

  In step S14, a memory area for storing the first frame (frame number M) including the partial image is allocated as a long-term reference (hereinafter referred to as “LTR”) memory. It is obvious that the image data having the partial image Dp region with the smallest image quality degradation is stored in the memory corresponding to the frame number M.

In step S15, whether or not the frame is intra-coded image data (for example, IDR (Instantaneous Decoding Refresh) picture in H.264) immediately after all the data in the image memory unit 410 is discarded. Judgment is made. If it is not an IDR picture, the process proceeds to step S16 to refer to the LTR memory allocated in step S14. On the other hand, if an IDR picture appears (Yes in step S15) , the process proceeds to step S17, and the IDR picture is again assigned as the LTR memory.

  When the above operation is repeated and the determination in step S11 is negative, that is, when the frame (M + T) +1 in FIG. 7 is reached, the process proceeds to step S18 to release the LTR frame area that is no longer referenced, and again for R ′ frames. Used as a STR memory.

The procedure of FIG. 8 can also be expressed as the following (1) to (4). The frame number is represented by n.
(1) When 0 ≦ n ≦ (M + R−1), all R frame memories are used as STR memories.
(2) When n = (M + R), a memory corresponding to the frame number M is allocated as an LTR memory.
(3) When (M + R) ≦ n ≦ (M + T), the partial image encoding always refers to the LTR memory allocated in (2). When an IDR picture appears, the IDR picture is assigned again as an LTR.
(4) When n ≧ (M + T + 1), the LTR area that is no longer referred to is released and used again as R STR memories.

  FIG. 9 is a diagram for explaining a specific example of the procedure when the logo insertion start frame M = 10, the logo insertion period T = 30, and the reference frame number R = 3.

  Up to the processing of frame number n = 11, the frame memory for three frames is STR, but at frame number n = 12, n = 10 area having the logo area inserted first is assigned to the LTR. The logo area included in n = 10 is intra-coded by the compression coding unit 4 in FIG. 5 and has the highest quality logo image.

  In the section from frame number n = 10 to 39, the memory area where the image of frame number n = 10 is kept as an LTR, or when an IDR picture occurs after n = 10, the IDR picture is refreshed ( Remains as an updated LTR. For this reason, the inter-frame prediction of the logo area in the section from frame number n = 10 to 39 is predicted by referring to the clear logo area of frame number 10 or the IDR picture, so that the logo area deteriorates. It is possible to encode without any loss.

  After the encoding of n = 39 including the last logo image is completed, the frame number 10 or the LTR area of the IDR picture that is unnecessary is released, and thereafter, it is used as a STR memory for three FIFOs. In addition, the arrow in FIG. 9 has shown the reference image.

  H. When the LTR mechanism defined in H.264 is used, the above functions can be realized as follows. H. LTR allocation and release in H.264 are performed by memory_management_control_operation (MMCO) defined in ISO / IEC 14496-10. For example, the LTR allocation at n = 12 in FIG. 9 is performed using MMCO = 3, and the release of the LTR can be realized using MMCO = 2. Also, since the LTR is refreshed by the appearance of the IDR picture, each time the IDR picture is encoded, the IDR picture is reset as the LTR.

It is the block diagram which showed the structure of the principal part of the moving image compression encoding apparatus which concerns on this invention. It is the block diagram which showed the other structure of the principal part of the moving image compression encoding apparatus which concerns on this invention. It is the figure which showed typically the frame structure of the uncompressed moving image in which the partial image was inserted. It is the figure which expressed typically operation | movement of the arrangement | positioning control part. It is the block diagram which showed the structure of the principal part of a compression encoding part. It is the flowchart which showed the setting procedure of the prediction encoding system and a quantization parameter. It is explanatory drawing of the frame row | line | column in which a partial image is inserted. It is a flowchart which shows the use or allocation procedure of the memory area of an image memory part. It is explanatory drawing which shows a specific example of the use or allocation procedure of the memory area of an image memory part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Arrangement | positioning control part, 2 ... Partial image insertion part, 3 ... Coding control part, 4 ... Compression encoding part, 5 ... Partial image information extraction part, 10 ... Movie Image compression encoding device, 301... Quantization parameter derivation unit, 302... Correction coefficient storage unit, 303... MV setting unit, 409.

Claims (4)

In a compression encoding apparatus that compresses and encodes an uncompressed moving image in which a partial image is inserted into a partial region on an image by inter-frame prediction or intra-frame prediction,
Means for deblocking the locally decoded signal;
An image memory for storing the deblocked local decoded signal;
Control means for allocating a memory area for storing a first frame including a partial image in the deblocked local decoded signal as a long-term reference memory in the image memory ;
The compression encoding apparatus according to claim 1, wherein the long-term reference memory is updated to the intra-encoded image data by the intra-encoded image data immediately after all the data in the image memory is discarded . The compression encoding apparatus according to claim 1,
A compression coding apparatus, wherein the partial image area in the long-term reference memory is forcibly referred to for predictive coding of a partial image area appearing in the second and subsequent sections of the section in which the partial image is inserted. The compression encoding apparatus according to claim 1 or 2 ,
The compression coding apparatus according to claim 1, wherein the long-term reference memory is released after the coding of the last image in the section in which the partial image is inserted. The compression encoding device according to any one of claims 1 to 3 ,
The intra-coded image data is H.264. The compression coding apparatus is an IDR picture in H.264.