JP6590412B2 - Video encoding apparatus, method and program, and video decoding apparatus, method and program - Google Patents

Description

  The present invention relates to a video encoding device and a video decoding device that can reduce the amount of transmitted data.

  As a video encoding method, H.264 / AVC (Advanced Video Coding) recommended by ITU-T (International Telecommunication Union Telecommunication Standardization Sector) is widely used. Further, H.265 / HEVC (High Efficiency Video Coding) is a subsequent standard of H.264 / AVC. In the video coding system based on H.265, each frame of a digitized video is divided into coding tree units (CTUs), and each CTU is coded in raster scan order. Each CTU has a quad-tree structure and is encoded by being divided into coding units (CU: Coding Unit). Each CU is predicted by being divided into prediction units (PUs). Further, the prediction error of each CU is divided into transform units (TU: Transform Unit) in a quad-tree structure, and is subjected to frequency conversion. Hereinafter, the largest CU is referred to as the largest CU (LCU: Largest Coding Unit), and the smallest CU is referred to as the smallest CU (SCU: Smallest Coding Unit). The LCU size and CTU size are the same.

  The CU is predictively encoded by intra prediction or inter-frame prediction. Intra prediction (intra-screen prediction) is prediction in which a prediction signal is generated from a reconstructed image of an encoding target frame. In Non-Patent Document 1, 33 types of angle intra prediction shown in FIG. 11 are defined. In the angle intra prediction, an intra prediction signal is generated by extrapolating the reconstructed pixels around the encoding target block in any one of the 33 types of directions shown in FIG.

  As intra prediction, DC prediction and Planar prediction are defined in addition to angle intra prediction. In DC prediction, the average value of the reference image is used as the prediction value for all pixels of the prediction target TU. In Planar prediction, a predicted image is generated by linear interpolation from pixels in a reference image.

  Inter-frame prediction is prediction based on an image of a reconstructed frame (reference picture) having a display time different from that of an encoding target frame. Inter-frame prediction is also called inter prediction. In inter prediction, an inter prediction signal is generated based on a reconstructed image block of a reference picture (using pixel interpolation if necessary). Hereinafter, inter-frame prediction is also referred to as inter prediction.

  A frame encoded only by the intra CU is called an I frame (or I picture). A frame encoded including not only an intra CU but also an inter CU is called a P frame (or P picture). A frame encoded by including not only one reference picture for inter prediction of a block but also an inter CU using two reference pictures at the same time is called a B frame (or B picture).

  With reference to FIG. 12, the configuration and operation of a general video encoding apparatus will be described. Note that the video encoding apparatus shown in FIG. 12 uses a video encoding system based on H.265.

  12 includes a buffer 100, a subtractor 101, a converter 102, a quantizer 103, an arithmetic encoder (CABAC: context-based adaptive binary arithmetic coder) 304, an inverse quantizer 105, and an inverse quantizer. A converter 106, an adder 114, a first frame buffer 107, a deblocking filter 308, a second frame buffer 109, an intra predictor 310, an inter predictor 311, a switch 113, and an encoding control unit 312 are provided.

  FIG. 13 shows an example of CTU division of frame t when the spatial resolution of the frame is CIF (CIF: Common Intermediate Format) and a CTU size of 64, and an example of CU division of the eighth CTU (CTU8) included in frame t It is explanatory drawing which shows. As shown in FIG. 13, the size of the CU is 64 × 64, 32 × 32, 16 × 16, or 8 × 8. FIG. 14 is an explanatory diagram showing a quadtree structure corresponding to the CU partitioning example of CTU8.

  For each CTU, the encoding control unit 312 determines the CU quadtree structure / PU partition shape / TU quadtree structure so as to increase the encoding efficiency in accordance with the characteristics of the image. Also, the encoding control unit 312 determines an intra prediction direction, a motion vector, and the like that minimize the encoding cost for each PU.

  The buffer 100 stores an input image (pixel value of the input image frame). An input image (specifically, CU) is supplied from the buffer 100 to the subtractor 101 via the switch 113. The subtractor 101 subtracts the prediction signal supplied from the intra predictor 310 or the inter predictor 311 from the input image.

  The converter 102 performs frequency conversion on the prediction error image obtained by subtracting the prediction signal from the input image signal based on the TU division shape determined by the encoding control unit 312. The quantizer 103 quantizes the prediction error image (frequency conversion coefficient) subjected to frequency conversion. Hereinafter, the frequency transform coefficient is simply referred to as a transform coefficient, and the quantized transform coefficient is referred to as a quantized coefficient.

  The arithmetic encoder 304 entropy indicates the split_cu_flag syntax value, pred_mode_flag syntax value, part_mode syntax value, split_tu_flag syntax value, intra prediction direction difference information, motion vector difference information, quantization coefficient, and the like determined by the encoding control unit 312. Encoding (here binary arithmetic encoding) is performed. The arithmetic encoder 304 outputs a bit stream.

  The inverse quantizer 105 inversely quantizes the quantization coefficient. The inverse transformer 106 performs inverse frequency transform on the inversely quantized transform coefficient. The adder 114 adds a prediction signal to the reconstructed prediction error image that has been subjected to inverse frequency conversion, and generates a reconstructed image. The reconstructed image is supplied to the first frame buffer 107.

  The deblocking filter 308 performs filter processing for removing block distortion on the reconstructed image stored in the first frame buffer 107. The output of the deblocking filter 308 is stored in the second frame buffer 109.

  The second frame buffer 109 stores the reconstructed image frame from which block distortion has been removed as a reference image frame. Note that the image of the reference image frame is used as a reference image for generating an inter-frame prediction signal.

  The intra predictor 310 generates an intra prediction signal using the reconstructed image stored in the first frame buffer 107 and having the same display time as the current frame. The inter predictor 311 generates an inter-frame prediction signal using a reference image stored in the second frame buffer 109 having a display time different from that of the current frame.

  When the input image is an HDTV (High Definition Television) video signal, for example, one frame is formed with a resolution of 1920 pixels × 1080 lines. In H.264 / AVC, encoding is performed in units of 16 × 16 macroblocks (MB). However, since the number of pixels in the vertical direction of the HDTV image signal is not a multiple of 16, the buffer 100 is used for encoding. 8, 8 lines of redundant pixels are added (see, for example, Patent Document 1). Hereinafter, in the present specification, processing including processing for adding redundant pixels (pixel padding processing) and encoding processing based on an image with redundant pixels added (including frequency conversion processing, quantization processing, and the like) is extended. This is called image processing. An image composed of 8 lines of redundant pixels is called an additional image or a redundant image.

  Then, the fact that the extended image processing has been performed is signaled by, for example, conformance_window_flag syntax. When recognizing that the extended image processing has been executed in the video encoding device, the video decoding device removes the additional image from the video decoded based on the received bitstream.

JP 2012-195702 A

ITU-T recommendation H.265 High efficiency video coding, April 2013

  Even when the video encoding method based on H.265 is used, if the number of pixels in the horizontal or vertical direction of the input image is not a multiple of 8 corresponding to the SCU size, extended image processing is performed during encoding. Is required. If the number of pixels in the horizontal and vertical directions of the input image becomes a multiple of 8 due to the addition of the redundant image, at least the SCU can perform the encoding process. Even if the number of pixels in the horizontal and vertical directions of the input image is a multiple of 8, if it is not a multiple of 16, an area where the use of a 16 × 16 CU is restricted appears. Even if the number of pixels in the horizontal and vertical directions of the input image is a multiple of 16, if it is not a multiple of 32, an area where the use of a 32 × 32 CU is restricted appears. Furthermore, even if the number of pixels in the horizontal direction and the vertical direction of the input image is a multiple of 32, if it is not a multiple of 64, an area where the use of a 64 × 64 CU corresponding to the LCU size is restricted appears. In the case where there is no redundant image, it is necessary to add an exceptional control flow in such a case, and the processing complexity increases. By adding redundant images so as to be a multiple of the LCU size, a single control flow that always performs processing for the LCU size (64 × 64) can be obtained, which is preferable. That is, in order to enable the application of a general video encoding method based on H.265, when the number of pixels in the horizontal or vertical direction of the input image is not a multiple of 64 (corresponding to the LCU size), a multiple of 64 Therefore, it is preferable to perform extended image processing.

  FIG. 15 is an explanatory diagram showing the concept of extended image processing. As shown in FIG. 15, it is assumed that the number of pixels of an input image to the video encoding device 32 is 1920 (horizontal direction) × 1080 (vertical direction). The video encoding device 32 adds a 1920 × 8 image so that the number of pixels in the vertical direction is a multiple of 64 (in the case of LCU size). Then, the video encoding device 32 performs an encoding process on an image of 1920 × 1088 pixels. The encoding result is transmitted to the video decoding device 42.

  The video decoding device 42 obtains a frame of 1920 × 1088 pixels by decoding processing. Next, the video decoding device 42 removes the additional image at the bottom of the frame. Then, the video decoding device 42 outputs a 1920 × 1080 pixel frame to a display device (not shown) or the like.

  As described above, if the number of pixels in the horizontal or vertical direction of the input image is not a multiple of the CU size in H.265 (in the preferred example, it is not a multiple of the LCU size), The amount of data to be increased.

  Note that in a system that cannot perform extended image processing, that is, a system that does not use conformance_window_flag, an image whose horizontal or vertical number of pixels is not a multiple of 8 cannot be encoded and decoded and displayed on a display device or the like.

  An object of the present invention is to reduce the amount of data transmitted after performing extended image processing.

The video encoding apparatus according to the present invention includes an image adding unit that adds a redundant image to an input image to make the size of the image an integer multiple of the encoding unit, and a prediction process that generates a prediction signal for each encoding unit. And an entropy encoding unit that entropy-encodes a parameter based on the prediction signal and the input image. The entropy encoding unit includes a parameter related to the redundant image when the redundant image is added to the input image. Are encoded, parameters relating to the input image are encoded, and the predicting means includes an inter prediction means and an intra prediction means , and the inter prediction means determines that a block in the reference frame is added when a redundant image is added to the input image. When the block includes a redundant image, the pixel in the input image adjacent to the redundant image is replaced with the redundant image. Run the prediction processing is regarded as the pixel, the intra prediction unit, when redundant image is added to the input image, and executes the predictive processing block as a target included in the redundant image.

The video encoding method according to the present invention increases the size of an image by an integral multiple of the encoding unit by adding a redundant image to the input image, and executes a prediction process for generating a prediction signal for each encoding unit, If the input image is entropy-encoded and a redundant image is added to the input image, when the entropy encoding is performed, the parameter related to the input image is encoded without encoding the parameter related to the redundant image. The prediction process includes an inter prediction process and an intra prediction process. When a redundant image is added to the input image when the inter prediction process is executed, the block in the reference frame is a block including the redundant image. sometimes, adjacent to the redundant image, the pixels in the input image by executing the predictive processing is regarded as a pixel of the redundant image, Lee When performing tigers prediction processing, when a redundant picture is added to the input image, and executes the predictive processing block as a target included in the redundant image.

A video encoding program according to the present invention includes a process for adding a redundant image to an input image to a computer to make the image size an integer multiple of the encoding unit, and a prediction process for generating a prediction signal for each encoding unit. And a process for entropy encoding a parameter based on the prediction signal and the input image, and when a redundant image is added to the input image, The process for encoding the parameters relating to the input image is performed without encoding the parameters, and the prediction process includes an inter prediction process and an intra prediction process . When the computer performs the inter prediction process, the input image When a redundant image is added to the reference frame, if the block in the reference frame is a block including the redundant image, Adjacent to the image, the pixels in the input image to execute the prediction process is regarded as a pixel of the redundant image, when performing intra prediction processing, when a redundant picture is added to the input image, it is included in the redundant image block also to execute the prediction process as a target, characterized in Rukoto.

The video decoding apparatus according to the present invention includes an entropy decoding unit that entropy decodes a parameter included in a bitstream, and a prediction unit that executes a prediction process for generating a prediction signal for each coding unit based on the parameter. The prediction means includes an intra prediction means and an inter prediction means, and the intra prediction means considers that a redundant image for making the size of the image an integral multiple of the encoding unit is added to the decoded image, and the decoding target is decoded. When the block is included in the image, the prediction process is performed by excluding the pixels of the redundant image from the reference pixel candidates, and the inter prediction unit performs the prediction process assuming that the redundant image is added to the decoded image. The inter prediction means is a block in which a block in a reference frame includes a redundant image when a redundant image is added to the decoded image. The prediction process is performed by regarding the pixels in the decoded image that are adjacent to the redundant image as pixels of the redundant image, and the intra prediction unit converts the redundant image into a redundant image when the redundant image is added to the decoded image. The prediction process is executed on the included block as a target.

The video decoding method according to the present invention entropy-decodes a parameter included in a bitstream, executes an intra prediction process or an inter prediction process for generating a prediction signal for each coding unit based on the parameter, and performs an intra prediction process. When executing, when it is considered that a redundant image for making the size of the image an integer multiple of the encoding unit is added to the decoded image, and the decoding target is a block included in the decoded image, the pixel of the redundant image is When it is excluded from the reference pixel candidates and the redundant image is added to the decoded image and the inter prediction process is executed, and the redundant image is added to the decoded image when the inter prediction process is executed, When the block includes a redundant image, the pixel in the decoded image adjacent to the redundant image is replaced with the redundant image. When the prediction process is performed by assuming that the pixel is an image and the intra prediction process is performed, if a redundant image is added to the decoded image, the prediction process is also performed on the blocks included in the redundant image. Features.

The video decoding program according to the present invention performs, on a computer, processing for entropy decoding parameters included in a bitstream, and intra prediction processing or inter prediction processing for generating a prediction signal for each coding unit based on the parameters. When the intra prediction process is executed, it is considered that a redundant image for making the size of the image an integer multiple of the encoding unit is added to the decoded image, and the decoding target is included in the decoded image When the block is a block, the pixel of the redundant image is excluded from the reference pixel candidates, the decoded image is assumed to have been added with the redundant image, the inter prediction process is executed, and the computer performs the inter prediction process. When a redundant image is added to the block, the block in the reference frame is a block including the redundant image. Sometimes, when a redundant image is added to the decoded image when the prediction process is executed by regarding the pixel in the decoded image adjacent to the redundant image as a pixel of the redundant image and the intra prediction process is executed, The prediction process is executed for blocks included in the redundant image.

  According to the present invention, the amount of data to be transmitted can be reduced.

It is a block diagram which shows the structural example of a video coding apparatus. It is explanatory drawing for demonstrating an effect | action of an intra predictor when a partial expansion image process is performed. It is explanatory drawing for demonstrating an effect | action of the inter predictor when a partial expansion image process is performed. It is explanatory drawing which shows the concept of a partial expansion image process. It is a flowchart which shows operation | movement of a video coding apparatus. It is a block diagram which shows the structural example of a video decoding apparatus. It is a flowchart which shows operation | movement of a video decoding apparatus. It is a block diagram which shows the structural example of the information processing system which can implement | achieve the function of a video coding apparatus and a video decoding apparatus. It is a block diagram which shows the principal part of the video coding apparatus by this invention. It is a block diagram which shows the principal part of the video decoding apparatus by this invention. It is explanatory drawing which shows the example of 33 types of angle intra prediction. It is explanatory drawing which shows the structure of a general video coding apparatus. FIG. 6 is an explanatory diagram showing an example of CTU partitioning for frame t and an example of CU partitioning for CTU8 in frame t. It is explanatory drawing which shows the quadtree structure corresponding to the CU division | segmentation example of CTU8. It is explanatory drawing which shows the concept of an extended image process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
In the first embodiment, the video encoding apparatus can selectively execute the extended image processing and the partial extended image processing as shown in FIG. In the partial extended image processing, an image including an additional image (for example, an image of 1920 × 1088 pixels) is processed in a part of the process, and an image not including the additional image in the other process (for example, 1920 × 1080). This is a process for processing a pixel image).

  FIG. 1 is a block diagram illustrating a configuration example of a video encoding device. The video encoding apparatus shown in FIG. 1 performs encoding processing based on the H.265 standard (H.265 / HEVC standard).

  The video encoding apparatus includes a buffer 100, a subtractor 101, a converter 102, a quantizer 103, an arithmetic encoder 104, an inverse quantizer 105, an inverse transformer 106, an adder 114, a first frame buffer 107, a demultiplexer. A blocking filter 108, a second frame buffer 109, an intra predictor 110, an inter predictor 111, a switch 113, and an encoding control unit 112 are provided.

  The functions of the subtractor 101, the converter 102, the quantizer 103, the inverse quantizer 105, the inverse transformer 106, the switch 113, and the adder 114 are the same as those shown in FIG.

  In the present embodiment, the encoding control unit 112 performs extended image processing or partial extended image processing on each unit in the video encoding device in addition to the function of the encoding control unit 312 shown in FIG. It has a function of issuing an instruction to execute.

  In addition, when the CTU is divided into CUs, the encoding control unit 112 does not generate a CU that straddles the input image and the additional image. That is, the encoding control unit 112 generates a CU that includes only a region included in the input image, and generates a CU that includes only a region included in the additional image.

  The arithmetic encoder 104, the deblocking filter 108, the intra predictor 110, and the inter predictor 111 are functions of the arithmetic encoder 304, the deblocking filter 308, the intra predictor 310, and the inter predictor 311 illustrated in FIG. In addition to (function for executing extended image processing), the following functions for partial extended image processing are provided.

  The deblocking filter 108 does not execute a filter process on an area including an area exceeding the original size of the input image (an additional image area) when the partial extended image processing is executed.

  As shown in FIG. 2, when executing the partial extended image processing, the intra predictor 110, when the encoding target block 51 is a block included in the original image (input image), adds the additional image 60. These pixels are not set as reference pixel candidates (see FIG. 2A). When the encoding target block 51 is a block included in the additional image 60, the pixel of the additional image 60 can be used as a reference pixel (see FIG. 2B). Note that the reason why the blocks included in the additional image 60 that are not originally required are to be encoded is to reduce functional differences from the existing intra predictor 310. Also, the intra prediction signal of the encoding target block 52 of the additional image 60 is substantially removed by a subsequent unit in the video encoding device.

  Note that the intra predictor 110 is a block in which the encoding target block 51 is included in the original image and the pixel of the additional image 60 does not become a reference pixel when the partial extended image processing is executed. Performs intra prediction according to the H.265 standard.

  The inter predictor 111 performs inter prediction according to the H.265 standard when executing partial extended image processing. However, as illustrated in FIG. 3, the inter predictor 111 performs off-screen processing when the block 62 of the reference frame for the prediction source PU (current PU) 61 includes the pixel of the additional image 60 (FIG. 3A )reference). The off-screen processing is processing for preventing an additional image 60, that is, an image that is not an original image (image displayed on the receiving side) from being referred to. As a specific example, the inter predictor 111 replaces the pixel of the portion 621 of the additional image 60 in the PU 62 with the pixel of the portion 622 at the screen end.

  When the prediction source PU is a block included in the additional image 60, the inter predictor 111 performs inter prediction according to the H.265 standard. This is because, with respect to the additional image 60, since the generated prediction signal is substantially removed by the subsequent unit, any prediction signal may be generated.

  Specifically, the arithmetic encoder 104 does not perform arithmetic encoding on data (such as an inter prediction signal) related to the additional image 60 when executing the partial extended image processing.

  FIG. 4 is an explanatory diagram showing the concept of partial extended image processing. As illustrated in FIG. 4, it is assumed that the number of pixels of an input image to the video encoding device 31 is 1920 × 1080. The video encoding device 31 adds a 1920 × 8 image so that the number of pixels in the vertical direction is a multiple of 64.

  In the video encoding device 31, units other than the arithmetic encoder 104 and the deblocking filter 108 perform encoding processing (including frequency conversion processing and quantization processing) on an image of 1920 × 1088 pixels. However, the intra predictor 110 and the inter predictor 111 perform processes as illustrated in FIGS. 2 and 3 for the pixels of the additional image 60. The encoding result is transmitted to the video decoding device 41.

  The video decoding device 41 obtains a frame of 1920 × 1080 pixels by decoding processing. Then, the video decoding device 41 outputs a 1920 × 1080 pixel frame to a display device (not shown) or the like. However, in the present embodiment, the video decoding device 41 can execute processing similar to the partial extended image processing described above when the partial extended image processing is executed in the video encoding device 31. Hereinafter, processing similar to the partial extended image processing executed in the video decoding device 41 is also referred to as partial extended image processing.

  Next, with reference to the flowchart of FIG. 5, operations of the encoding control unit 112 and the arithmetic encoder 104 when executing partial extended image processing will be described.

  At the time of executing the partial extended image processing, the encoding control unit 112 performs control for adding an 8-line redundant image below the input image (for example, an image of 1920 × 1080 pixels) in the buffer 100 ( Step S11). The redundant image is, for example, a black image or a gray image, but the redundant image may be any image when executing the partial extended image processing. Note that an image at the boundary with the redundant image in the input image may be a redundant image.

  Then, the encoding control unit 112 instructs the deblocking filter 108, the intra predictor 110, the inter predictor 111, and the arithmetic encoder 104 to execute the partial extended image processing (Step S12).

  When the execution of the partial extended image processing is instructed, the deblocking filter 108, the intra predictor 110, and the inter predictor 111 operate for the partial extended image processing as described above.

  The arithmetic encoder 104 arithmetically encodes only the parameters relating to the input image (for example, an area of 1920 × 1080 pixels) (step S13). In other words, the input image is 1920 × 1080 and there is no redundant image, and arithmetic coding is performed according to the H.265 standard. For example, the arithmetic encoder 104 does not encode split_cu_flag that is a parameter indicating whether or not to divide a CU with respect to a redundant image region. Also, other parameters relating to the redundant image area are not encoded. Other parameters include, for example, cu_trasquant_bypass_flag, cu_skip_flag, pred_mode_flag, and part_mode related to the CU.

  When performing extended image processing instead of partial extended image processing, the encoding control unit 112 performs the extended image processing on the arithmetic encoder 104, the deblocking filter 108, the intra predictor 110, and the inter predictor 111. Instruct.

  The arithmetic encoder 104, the deblocking filter 108, the intra predictor 110, and the inter predictor 111, which are instructed to execute the extended image processing, execute the process as illustrated in FIG. That is, for an extended image (for example, an image of 1920 × 1088 pixels), the intra predictor 110 and the inter predictor 111 generate a prediction signal. The arithmetic encoder 104 arithmetically encodes parameters related to the extended image (input image + additional image). That is, each unit of the video encoding device executes processing according to the H.265 standard.

  When extended image processing is executed in the video encoding device, the video decoding device obtains an extended image frame by decoding processing, but removes an additional image at the bottom of the frame. Then, the video decoding device outputs an original image frame (for example, a 1920 × 1080 pixel frame) to a display device or the like.

  The arithmetic encoder 104 includes conformance_window_flag having a value of “1” in the bitstream when the extended image processing is executed, and “0” when the partial extended image processing is executed. Conformance_window_flag with the value of is included in the bitstream.

  As described above, in the present embodiment, when partial extended image processing is executed in the video encoding device, some units process an extended image obtained by adding a redundant image to an input image. I do. However, parameters related to redundant images are not output from the video encoding device. Therefore, the amount of data transmitted is reduced.

  Further, when a function for executing partial extended image processing is implemented, a redundant image is added so that the number of pixels in the horizontal direction and the vertical direction is a multiple of 64, so that the converter 102, the quantizer 103, As the inverse quantizer 105 and the inverse transformer 106, commonly used units can be applied as they are. The intra predictor 110 also has reference pixel restrictions (see FIG. 2A), but generally used units can be applied as they are. That is, in order to realize a configuration in which the amount of transmitted data is reduced, the amount of change in hardware or software of each unit is not large.

  Even if there is no function for executing the extended image processing, the partial extended image processing in the present embodiment displays the image after encoding and decoding an image of a pixel whose horizontal or vertical number of pixels is not a multiple of 8. It can be displayed on a device or the like.

Embodiment 2. FIG.
FIG. 6 is a block diagram illustrating a configuration example of a video decoding apparatus that can selectively execute the extended image processing and the partial extended image processing. The video decoding apparatus shown in FIG. 6 includes an arithmetic decoder (CABAD: context-based adaptive binary arithmetic decoder) 201, an inverse quantizer 202, an inverse transformer 203, an adder 211, a first frame buffer 204, and a deblocking filter 205. , A second frame buffer 206, an intra predictor 207, an inter predictor 208, a decoding control unit 209, and a switch 210.

  The arithmetic decoder 201 arithmetically decodes the bitstream and outputs parameters, quantization coefficients, and the like related to the prediction signal of the decoding target CU.

  The intra predictor 207 generates an intra prediction signal using a reconstructed image stored in the first frame buffer 204 having the same display time as the currently decoded frame. The inter predictor 208 generates an inter-frame prediction signal using a reference image stored in the second frame buffer 206 having a display time different from that of the currently decoded frame.

  The decoding control unit 209 controls the switch 210 so that the intra prediction signal or the inter frame prediction signal is supplied to the adder 211 based on the type of inter frame prediction subjected to entropy decoding (specifically, arithmetic decoding). To do.

  The inverse quantizer 202 inversely quantizes the quantization coefficient supplied from the arithmetic decoder 201 in the same manner as the inverse quantizer 105 shown in FIG. That is, the inverse transformer 203 performs inverse frequency transformation on the quantized representative value and returns it to the original spatial domain.

  In the first frame buffer 204, a reconstructed image block obtained by adding a prediction signal to the reconstructed prediction error image block returned to the original spatial domain until all CUs included in the currently decoded frame are decoded. Is stored.

  In this embodiment, when it is determined that the partial extended image processing has been executed in the video encoding device, the partial extended image processing is also executed in the video decoding device. When the partial extended image processing is executed, the inverse quantizer 202 and the inverse transformer 203 consider that the redundant image is added to the decoded image (the image generated based on the received bit stream), and the redundant image Also, the inverse quantization process and the inverse frequency conversion process are executed.

  At the time of executing the partial extended image processing, the deblocking filter 205 performs a filtering process on an area including an area exceeding the original size of the input image (additional image area), like the deblocking filter 108 in the video encoding device. Do not execute.

  At the time of executing the partial extended image processing, the intra predictor 207 performs intra prediction after assuming that a redundant image is added to the decoded image. However, similarly to the intra predictor 110 in the video encoding device, when the encoding target block is a block included in the original image (composite image), the pixel of the additional image is not used as the reference pixel (see FIG. 2 (A)). When the encoding target block is a block included in the additional image, the pixel of the additional image can be used as a reference pixel (see FIG. 2B).

  When executing the partial extended image processing, the inter predictor 208, when the PU of the reference frame for the prediction source PU (current PU) includes the pixel of the additional image, like the inter predictor 111 in the video encoding device. Out-of-screen processing is performed (see FIG. 3A). As already described, the off-screen processing is processing for preventing the additional image 60, that is, an image that is not an original image (decoded image) from being referred to.

  Next, the operation at the time of execution of the partial extended image processing of the decoding control unit 209 will be described with reference to the flowchart of FIG.

  The decoding control unit 209 confirms that the partial extended image processing should be executed (step S21). For example, when the arithmetic decoder 201 extracts conformance_window_flag having a value of “0” from the bitstream, the arithmetic decoder 201 notifies the decoding control unit 209 to that effect. When receiving the notification, the decoding control unit 209 determines that the partial extended image processing should be executed.

  The decoding control unit 209 notifies the inverse quantizer 202 and the inverse transformer 203 that the partial extended image processing is executed (step S22). When receiving the notification, the inverse quantizer 202 and the inverse transformer 203 perform the operation at the time of executing the partial extended image processing as described above.

  Furthermore, the decoding control unit 209 instructs the deblocking filter 205, the intra predictor 207, and the inter predictor 208 to execute partial extended image processing (step S23). When the deblocking filter 205, the intra predictor 207, and the inter predictor 208 are instructed to execute the partial extended image processing, the deblocking filter 205, the intra predictor 207, and the inter predictor 208 execute the operation for the partial extended image processing as described above.

  When the partial extended image processing as described above is executed in the video encoding device, an extended image frame (for example, a frame of 1920 × 1088 pixels) is stored in the first frame buffer 204. After the additional image at the bottom of the frame stored in the one-frame buffer 204 is removed, only the original image frame (for example, a 1920 × 1080 pixel frame) is output to the display device or the like.

  Further, when extended image processing is executed in the video encoding device, the video decoding device executes decoding processing according to the H.265 standard. In that case, the video decoding apparatus obtains an extended image frame by decoding processing, but after the additional image at the bottom of the frame stored in the first frame buffer 204 is removed, the original image frame (eg, 1920 × 1080) is obtained. Only the pixel frame) is output to a display device or the like.

  Moreover, although each said embodiment can also be comprised with a hardware, it is also possible to implement | achieve by a computer program.

  The information processing system illustrated in FIG. 8 includes a processor 1001, a program memory 1002, a storage medium 1003, and a storage medium 1004. The storage medium 1003 and the storage medium 1004 may be separate storage media, or may be storage areas composed of the same storage medium. A magnetic storage medium such as a hard disk can be used as the storage medium.

  In the information processing system shown in FIG. 8, the program memory 1002 stores a program for realizing the functions of the units (except for the buffers) shown in FIGS. The processor 1001 implements the functions of the video encoding device or the video decoding device shown in FIGS. 1 and 6 by executing processing according to the program stored in the program memory 1002.

  FIG. 9 is a block diagram showing the main part of the video encoding apparatus according to the present invention. As shown in FIG. 9, a video encoding apparatus 10 according to the present invention adds an image adding unit 11 (for example, a buffer 100 and a code) that adds a redundant image to an input image to make the size of the image an integral multiple of the encoding unit. And a prediction unit 12 (for example, an intra predictor 110 and an inter predictor 111) that executes a prediction process for generating a prediction signal for each coding unit (for example, CU). And an entropy encoding unit 13 (for example, realized by the arithmetic encoder 104) that entropy encodes a parameter based on the prediction signal and the input image. The entropy encoding unit 13 includes: When the redundant image is added to the input image, the parameter regarding the input image is encoded without encoding the parameter regarding the redundant image.

  It is preferable that the entropy encoding unit 13 also has a function (for example, a function for executing extended image processing) for encoding a parameter related to an extended image in which a redundant image is added to the input image.

  When the prediction unit 12 includes an intra prediction unit, when the redundant image is added to the input image and the encoding target is a block included in the input image, the intra prediction unit uses the pixel of the redundant image as a reference pixel. It is preferable to execute the prediction process by excluding the candidates (see FIG. 2A).

  When the prediction unit 12 includes an inter prediction unit, the inter prediction unit, when a redundant image is added to the input image and the block in the reference frame is a block included in the redundant image, It is preferable to execute the prediction process by regarding the pixel as a block (see FIG. 3A).

  When a deblocking filter (for example, the deblocking filter 108) that performs a filter process for removing block distortion is provided, the deblocking filter performs a redundant image when a redundant image is added to the input image. It is preferable not to execute the filtering process for the region.

  FIG. 10 is a block diagram showing the main part of the video decoding apparatus according to the present invention. As shown in FIG. 10, the video decoding apparatus 20 according to the present invention is based on an entropy decoding unit 21 (for example, realized by an arithmetic decoder 201) that entropy decodes a parameter included in a bitstream, and based on the parameter. A prediction unit 22 that executes a prediction process for generating a prediction signal for each coding unit, and the prediction unit 22 includes an intra prediction unit 23 (for example, an intra predictor 207) and an inter prediction unit 24 (for example, an inter prediction). 208), the intra prediction unit 23 considers that the redundant image is added to the decoded image, and excludes the pixel of the redundant image from the reference pixel candidates when the decoding target is a block included in the decoded image. The inter prediction unit 24 executes the prediction process by regarding that the redundant image is added to the decoded image.

  When the block in the reference frame is a block included in the redundant image, the intra prediction unit 23 preferably performs the prediction process by regarding the pixel in the decoded image as the pixel of the block.

  In the case where a deblocking filter (for example, the deblocking filter 205) that performs a filter process for removing block distortion is provided, the deblocking filter performs a redundant image when a redundant image is added to the decoded image. It is preferable not to execute the filtering process for the region.

  In each of the above embodiments, the input image that is the target of the partial extended image processing is assumed to be an image mainly having 1920 pixels in the horizontal direction and 1080 pixels in the vertical direction. The number) is not limited to 1920 × 1080 pixels. Other images in which the number of pixels does not become an integral multiple of the coding unit (particularly LCU) can be the target of the partial extended image processing.

  In each of the above embodiments, an input image in which the number of pixels in the vertical direction is not an integral multiple of the encoding unit is taken as an example. However, an image in which the number of pixels in the horizontal direction is not an integral multiple of the encoding unit is input. Even in this case, the concept of each of the above embodiments can be applied.

  While the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  This application claims the priority on the basis of the JP Patent application 2014-077600 for which it applied on April 4, 2014, and takes in those the indications of all here.

DESCRIPTION OF SYMBOLS 10 Video coding apparatus 11 Image addition part 12 Prediction part 13 Entropy coding part 20 Video decoding apparatus 21 Entropy decoding part 22 Prediction part 23 Intra prediction part 24 Inter prediction part 31 Video coding apparatus 41 Video decoding apparatus 100 Buffer 101 Subtractor 102 Transformer 103 Quantizer 104 Arithmetic Encoder (CABAC)
105 Inverse Quantizer 106 Inverse Transformer 107 First Frame Buffer 108 Deblocking Filter 109 Second Frame Buffer 110 Intra Predictor 111 Inter Predictor 112 Coding Control Unit 113 Switch 114 Adder 201 Arithmetic Decoder (CABAD)
202 Inverse Quantizer 203 Inverse Transformer 204 First Frame Buffer 205 Deblocking Filter 206 Second Frame Buffer 207 Intra Predictor 208 Inter Predictor 209 Decoding Control Unit 210 Switch 211 Adder 1001 Processor 1002 Program Memory 1003 Storage Medium 1004 Storage Medium

Claims (15)

Image adding means for adding a redundant image to the input image to make the image size an integral multiple of the encoding unit;
Prediction means for performing a prediction process for generating a prediction signal for each coding unit;
Entropy encoding means for entropy encoding parameters based on the prediction signal and the input image,
The entropy encoding means encodes the parameter related to the input image without encoding the parameter related to the redundant image when the redundant image is added to the input image.
The prediction means includes an inter prediction means and an intra prediction means,
When the redundant image is added to the input image and the block in the reference frame is a block including the redundant image, the inter prediction unit determines that the pixel in the input image adjacent to the redundant image is the redundant image. The prediction process is performed by regarding the pixel as an image,
The intra-coding means, when the redundant image is added to the input image, executes the prediction process for blocks included in the redundant image. The video encoding apparatus according to claim 1, wherein the entropy encoding unit also has a function of encoding a parameter related to an extended image in which the redundant image is added to the input image. When the redundant image is added to the input image and the encoding target is a block included in the input image, the intra prediction unit excludes the pixels of the previous redundant image from the reference pixel candidates, and The video encoding apparatus according to claim 1 or 2, wherein prediction processing is executed. A deblocking filter that performs filtering to remove block distortion;
The video code according to any one of claims 1 to 3, wherein the deblocking filter does not perform a filtering process on a region of the redundant image when the redundant image is added to the input image. Device. The video encoding device according to any one of claims 1 to 4, wherein encoding processing is executed based on the H.265 / HEVC standard. By adding a redundant image to the input image, the image size is made an integral multiple of the encoding unit,
For each coding unit, execute a prediction process for generating a prediction signal,
Entropy encoding parameters based on the prediction signal and the input image,
When the redundant image is added to the input image, when performing the entropy encoding, the parameter regarding the input image is encoded without encoding the parameter regarding the redundant image,
The prediction process includes an inter prediction process and an intra prediction process,
When the redundant image is added to the input image when the inter prediction process is performed, if the block in the reference frame is a block including the redundant image, the input image is adjacent to the redundant image. Performing the prediction process by regarding the pixel as a pixel of the redundant image,
A video encoding method, wherein, when the intra prediction process is performed, if the redundant image is added to the input image, the prediction process is also performed on a block included in the redundant image. The video encoding method according to claim 6, wherein when performing the entropy encoding, a process for encoding a parameter related to an extended image in which the redundant image is added to the input image can be executed. On the computer,
Processing to make the size of the image an integral multiple of the encoding unit by adding a redundant image to the input image;
A process of performing a prediction process for generating a prediction signal for each coding unit;
A process of entropy encoding a parameter based on the prediction signal and the input image;
When the redundant image is added to the input image, when performing the entropy encoding, do not encode the parameter related to the redundant image, but execute the process of encoding the parameter related to the input image,
The prediction process includes an inter prediction process and an intra prediction process,
On the computer,
When the redundant image is added to the input image when the inter prediction process is performed, if the block in the reference frame is a block including the redundant image, the input image is adjacent to the redundant image. The pixel is regarded as a pixel of the redundant image, and the prediction process is executed.
A video encoding program for executing the prediction process on a block included in the redundant image when the redundant image is added to the input image when the intra prediction process is executed. On the computer,
The video encoding program according to claim 8, wherein when performing the entropy encoding, a process for encoding a parameter related to an extended image in which the redundant image is added to the input image is also executed. Entropy decoding means for entropy decoding parameters included in the bitstream;
A prediction unit that executes a prediction process for generating a prediction signal for each coding unit based on the parameter;
The prediction means includes an intra prediction means and an inter prediction means,
The intra prediction means assumes that a redundant image for making the size of the image an integral multiple of the coding unit is added to the decoded image, and when the decoding target is a block included in the decoded image, Performing the prediction process by excluding pixels from candidate reference pixels;
The inter prediction means performs the prediction process by regarding the redundant image as being added to the decoded image,
When the redundant image is added to the decoded image and the block in the reference frame is a block including the redundant image, the inter prediction unit determines a pixel in the decoded image adjacent to the redundant image as the redundant image. The prediction process is executed assuming that
The intra-prediction unit, when the redundant image is added to the decoded image, performs the prediction process on a block included in the redundant image. A deblocking filter that performs filtering to remove block distortion;
The video decoding device according to claim 10, wherein the deblocking filter does not perform a filtering process on a region of the redundant image when the redundant image is added to the decoded image. The video decoding device according to claim 10 or 11, wherein decoding processing is executed based on the H.265 / HEVC standard. Entropy decodes the parameters included in the bitstream,
Based on the parameters, for each coding unit, execute an intra prediction process or an inter prediction process for generating a prediction signal,
When executing the intra prediction process, when it is considered that a redundant image for making the size of the image an integral multiple of the encoding unit is added to the decoded image, and the decoding target is a block included in the decoded image , Excluding the redundant image pixels from the candidate reference pixels;
The inter prediction process is performed assuming that the redundant image is added to the decoded image,
When the redundant image is added to the decoded image when the inter prediction process is performed, if the block in the reference frame is a block including the redundant image, the pixel in the decoded image adjacent to the redundant image And performing the prediction process with regard to the pixels of the redundant image,
A video decoding method, wherein, when the intra prediction process is performed, if the redundant image is added to the decoded image, the prediction process is also performed on a block included in the redundant image. The video decoding method according to claim 13, wherein the decoding process is executed based on the H.265 / HEVC standard. On the computer,
A process of entropy decoding the parameters included in the bitstream;
Based on the parameter, for each coding unit, to perform a prediction signal generating intra prediction process or inter prediction process is executed,
When executing the intra prediction process, when it is considered that a redundant image for making the size of the image an integral multiple of the encoding unit is added to the decoded image, and the decoding target is a block included in the decoded image , Excluding the redundant image pixels from the candidate reference pixels;
The inter prediction process is executed assuming that the redundant image is added to the decoded image,
On the computer,
When the redundant image is added to the decoded image when the inter prediction process is performed, if the block in the reference frame is a block including the redundant image, the pixel in the decoded image adjacent to the redundant image Are considered as pixels of the redundant image, and the prediction process is executed,
A video decoding program for executing the prediction process on a block included in the redundant image when the redundant image is added to the decoded image when the intra prediction process is performed.

Images