JP7595145B2 - Video encoding device and video encoding method - Google Patents

Description

This disclosure relates to a video encoding device and a video encoding method that divide an input image into blocks and encodes the blocks.

In recent years, with the development of multimedia applications, it has become common to handle all media information, including images, audio, and text, in a unified manner. In addition, since digitized images contain huge amounts of data, image information compression technology is essential for storage and transmission. On the other hand, standardization of compression technology is also important for interoperability of compressed image data. For example, video compression technology standards include H.261, H.263, and H.264 from the International Telecommunication Union Telecommunication Standardization Sector (ITU-T), and MPEG-1, MPEG-3, MPEG-4, and MPEG-4AVC from the International Organization for Standardization (ISO/IEC). Currently, ITU-T and ISO/IEC are working together to standardize a next-generation video coding method called High Efficiency Video Coding (HEVC).

In this type of video coding, each picture to be coded is divided into blocks, which are coding units, and the amount of information is compressed by reducing the temporal and spatial redundancy for each block. In intra-frame predictive coding, which aims to reduce spatial redundancy, a predicted image is generated from pixel information from surrounding coded blocks, and a difference image is obtained between the obtained predicted image and the block to be coded. In inter-frame predictive coding, which aims to reduce temporal redundancy, motion is detected and a predicted image is generated on a block-by-block basis by referencing previous or subsequent pictures that have already been coded, and a difference image is obtained between the obtained predicted image and the block to be coded. The amount of information is compressed by performing orthogonal transform processing such as discrete cosine transform and quantization processing on these obtained difference images, and generating code strings using variable-length coding and arithmetic coding.

Figure 1 is a conceptual diagram for explaining the combinations of each block size defined in the HEVC standard. In HEVC (Non-Patent Document 1), any size can be selected and used as a coding unit (hereinafter referred to as "Coding Unit: CU") from four block sizes of 64 x 64 pixels, 32 x 32 pixels, 16 x 16 pixels, and 8 x 8 pixels, as shown in Figure 1.

Furthermore, when the CU size is 32x32 pixels as a prediction unit (hereinafter referred to as "Prediction Unit: PU"), which is a unit into which a CU is divided and which generates a predicted image in intra-frame predictive coding and inter-frame predictive coding, any size can be selected from eight block sizes, such as 32x32 pixels, 16x32 pixels, and 16x16 pixels, as shown in Figure 1. For example, high coding efficiency is achieved by using a small block size for an image in which the object movement is complex, and a large block size for an image in which the object movement is simple.

Furthermore, as shown in FIG. 1, if the CU size is 32x32 pixels, any block size can be selected from four types of block sizes: 32x32 pixels, 16x16 pixels, 8x8 pixels, and 4x4 pixels, as the orthogonal transform unit (hereinafter referred to as "Transform Unit: TU"), which is a unit into which a CU is divided and used for the above-mentioned orthogonal transform processing and quantization processing. For example, high coding efficiency is achieved by using small block sizes for images with characteristics that differ over a small range, and large block sizes for images with characteristics that are similar over a wide range.

ITU-T H. 265: High efficiency video coding (04/2013)

In HEVC, four different block sizes can be selected for a CU. However, the more blocks in a CU are added during the encoding process, the more code is required for header information in the CU layer. This can result in poor encoding efficiency.

This disclosure has been made in consideration of the above problems, and provides a video encoding device that reduces the amount of coding for header information in the CU layer and PU layer, improving the coding efficiency without increasing the amount of processing.

The video encoding device according to the present disclosure is a video encoding device that generates a code string by encoding a target picture, and includes a block division unit that, in operation, (i) divides the target picture into a plurality of coding units and (ii) divides each of the plurality of coding units into at least one prediction unit, and a residual encoding unit that, in operation, generates a residual component by calculating a difference between a predicted image generated for each prediction unit and the target picture. When a first size of a first coding unit of the plurality of coding units is 2N×2N, which is the same as a basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N, and when a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N.

The video coding method according to the present disclosure is a video coding method for coding a target picture to generate a code string, which divides the target picture into a plurality of coding units, divides each of the plurality of coding units into at least one prediction unit, and generates a residual component by calculating a difference between a predicted image generated for each prediction unit and the target picture. When a first size of a first coding unit of the plurality of coding units is 2N×2N, which is the same as a basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N, and when a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N.

As a result, the video encoding device of the present invention is able to reduce the amount of coding required for header information in the CU layer and PU layer, thereby improving encoding efficiency without increasing the amount of processing.

The present disclosure can be realized not only as such a video encoding device, but also as a program or integrated circuit that performs processing equivalent to each of the means included in such a video encoding device.

FIG. 1 is a conceptual diagram for explaining combinations of block sizes defined in the HEVC standard. 1 is a block diagram showing a configuration of a moving image coding device 100 according to a first embodiment. FIG. 11 is a conceptual diagram for explaining combinations of block sizes according to the first embodiment. 11 is a flowchart showing an integrated determination process according to the first embodiment; FIG. 1 is a conceptual diagram showing an integrated determination process according to the first embodiment; FIG. 1 is a conceptual diagram showing an integrated determination process according to the first embodiment; 10 is a conceptual diagram for explaining combinations of block sizes that appear by performing the integrated determination process according to the first embodiment; FIG. 13 is a flowchart showing an integrated determination process according to the second embodiment; FIG. 13 is a conceptual diagram showing an integrated determination process according to the second embodiment; FIG. 13 is a conceptual diagram showing an integrated determination process according to the second embodiment; FIG. 11 is a conceptual diagram for explaining conditions for the integrated determination process according to the second embodiment. FIG. 11 is a conceptual diagram for explaining combinations of block sizes that appear by performing the integrated judgment process according to the second embodiment. 13 is a flowchart showing an integrated determination process according to the third embodiment; FIG. 13 is a conceptual diagram showing an integrated judgment process according to the third embodiment. FIG. 13 is a conceptual diagram showing an integrated judgment process according to the third embodiment. FIG. 13 is a conceptual diagram showing an integrated judgment process according to the third embodiment. FIG. 13 is a conceptual diagram for explaining combinations of block sizes that appear by performing the integrated judgment process according to the third embodiment.

Below, the embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to make it easier for those skilled in the art to understand.

The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
The video coding device 100 according to the first embodiment is realized by a microprocessor mounted on, for example, a video camera, a digital camera, a video recorder, a mobile phone, a mobile information terminal, and a personal computer. The video coding device 100 performs a coding process of video data according to HEVC, which is a video compression standard. The video coding device 100 according to the present disclosure integrates a plurality of CUs into one CU based on the coding information of the CU (coding unit) and the PU (prediction unit), and assigns one header information to the integrated CU. As a result, it is possible to suppress the coding amount of the header information of the CU layer and the PU layer, and it is possible to improve the coding efficiency without increasing the processing amount.

The configuration and operation of the video encoding device 100 according to the first embodiment will be described below with reference to the drawings.

[1-1. Configuration of video encoding device]
A description will be given of the configuration of the video coding device 100. Fig. 2 is a block diagram showing the configuration of the video coding device 100 according to the first embodiment.

As shown in FIG. 2, the video coding device 100 includes a picture memory 101, a basic block division unit 102, a basic block unit processing loop unit 111, a unification judgment unit 107, and a code string generation unit 108. The video coding device 100 also includes, as the basic block unit processing loop unit 111, a prediction residual coding unit 103, a prediction residual decoding unit 104, a picture buffer 105, a prediction processing unit 106, a difference calculation unit 109, and an addition calculation unit 110. With this configuration, the video coding device 100 divides an image input in picture units into basic blocks, performs coding processing in units of these divided basic blocks, and outputs a code string.

Next, we will provide a detailed explanation of each component of the video encoding device 100.

The picture memory 101 inputs image signals on a picture-by-picture basis in the order in which they are to be displayed on the display device. The picture memory 101 rearranges and stores the input image signals on a picture-by-picture basis in the order in which they are to be encoded. When the picture memory 101 receives a read command from the basic block division unit 102, it outputs to the basic block division unit 102 the picture to be encoded, which is the input image signal related to the read command.

The basic block division unit 102 as a division unit divides the picture to be coded, which is sequentially input from the picture memory 101, into coding units and outputs the divided pictures. The basic block division unit 102 divides the picture into basic blocks, which are basic units of processing in the basic block unit processing loop unit 111. The video coding device 100 limits the size of the basic block to 16×16 pixels, which is smaller than 64×64 pixels and 32×32 pixels specified in HEVC. The basic block includes one or more CUs, which are coding units defined in the HEVC standard. Figure 3 is a conceptual diagram for explaining the combination of each block size according to the first embodiment. As shown in the "Basic CU Size" column in Figure 3, the basic CU size can be 16×16 pixels or 8×8 pixels for a basic block of 16×16 pixels. One CU of 16×16 pixels is included in a basic block of 16×16 pixels. Four CUs of 8x8 pixels in size are included in a basic block of 16x16 pixels. Hereinafter, these CUs included in the basic block are referred to as "basic CUs". The basic block division unit 102 divides the input picture into basic blocks and selects the size of the basic CU. In general, the basic block division unit 102 selects a small basic CU when the pixel configuration of the input picture is complex, and selects a large basic CU when the pixel configuration of the input picture is simple. Note that the same explanation can be applied even if some of the sizes of the basic CUs described in the example of FIG. 3 are not used, or if a basic CU that is not described in the example of FIG. 3 and is smaller than the basic block is used.

The basic block division unit 102 sequentially outputs the picture divided into basic blocks (the size of the basic CU has been selected) to the prediction processing unit 106 and the difference calculation unit 109.

The prediction processing unit 106 performs prediction processing for each basic block using either intra-screen prediction or inter-screen prediction based on pictures divided into basic blocks sequentially input from the basic block division unit 102. The prediction processing unit 106 performs prediction processing for each PU (hereinafter referred to as "basic PU"), which is a prediction unit obtained by further dividing a basic CU. Specifically, as shown in the "basic PU size" column in FIG. 3, when the size of a basic CU is 16×16 pixels, it is divided into one basic PU of 16×16 pixels, two basic PUs of 16×8 pixels, or two basic PUs of 8×16 pixels. However, the basic PU sizes of 16×8 pixels and 8×16 pixels are used only when inter-screen prediction is selected. On the other hand, when the size of the basic CU is 8×8 pixels, the basic PU is divided into one basic PU of 8×8 pixels, two basic PUs of 8×4 pixels, two basic PUs of 4×8 pixels, or four basic PUs of 4×4 pixels. However, the basic PU sizes of 8×4 pixels and 4×8 pixels are used only when inter-screen prediction is selected. The basic PU size of 4×4 pixels is used only when intra-screen prediction is selected. In general, when the pixel configuration of the input basic CU is complex, the prediction processing unit 106 selects a basic PU of a small size, whereas when the pixel configuration of the input basic CU is simple, the prediction processing unit 106 selects a basic PU of a large size. Note that the same explanation can be applied even when some of the sizes of the basic PUs described in the example of FIG. 3 are not used, or when a basic PU of a size not described in the example of FIG. 3 and smaller than the basic CU is used.

When intra-screen prediction is used, the prediction processing unit 106 performs prediction processing for a block in a picture to be coded using reconstructed image signals of surrounding blocks that have already been coded in the same picture. The reconstructed image signal is a signal generated by the addition calculation unit 110, which will be described later. The prediction processing unit 106 performs prediction processing by selecting one intra-screen prediction mode from among multiple intra-screen prediction methods (intra-screen prediction modes) that generates a predicted image that is most similar to the pixel configuration of the block to be coded.

On the other hand, when inter-picture prediction is used, the prediction processing unit 106 performs prediction processing using a reconstructed image signal of another picture that has already been encoded and is stored in the picture buffer 105. Specifically, the prediction processing unit 106 searches for an area having a pixel configuration that is most similar to the pixel configuration of the block to be encoded from among the reconstructed images of the other picture that have already been encoded. The prediction processing unit 106 then generates a predicted image by determining which picture's reconstructed image to refer to (hereinafter, the information on the picture to be referred to is referred to as "reference picture information") and how far the reconstructed image in the referenced picture is displaced from the position corresponding to the block to be encoded (hereinafter, the information indicating the amount of displacement is referred to as "motion vector information").

The difference calculation unit 109 generates a difference image signal that is a difference value between an input image signal in units of basic PUs selected based on a basic CU in a basic block input from the basic block division unit 102 and a prediction image signal in units of basic PUs input from the prediction processing unit 106. The difference calculation unit 109 outputs the generated difference image signal to the prediction residual coding unit 103.

The predictive residual coding unit 103, which is a residual coding unit, performs an orthogonal transform process on the difference image signal input from the difference calculation unit 109, and performs a quantization process on the orthogonal transform coefficients of each frequency component obtained. As a result, the predictive residual coding unit 103 generates a residual coefficient signal. The predictive residual coding unit 103 performs an orthogonal transform process and a quantization process for each TU (Transform Unit) (hereinafter referred to as a "basic TU"), which is an orthogonal transform unit obtained by further dividing a basic CU. Specifically, as shown in the "basic TU size" column in FIG. 3, when the size of the basic CU is 16×16 pixels, it can be divided into one basic TU of 16×16 pixels or four basic TUs of 8×8 pixels. On the other hand, when the size of the basic CU is 8×8 pixels, it can be divided into one basic TU of 8×8 pixels or four basic TUs of 4×4 pixels. In the example shown in FIG. 3, when a basic PU is determined, a basic TU is uniquely assigned, so there is no need to select the size of the basic TU. This makes it possible to significantly reduce the amount of processing. Note that the same explanation can be applied even if some of the sizes of the basic TUs described in the example of FIG. 3 are not used, or if a basic TU that is a size not described in the example of FIG. 3 and is smaller than the basic PU is used.

The prediction residual decoding unit 104 uses the basic TU processed by the prediction residual coding unit 103 as a processing unit. The prediction residual decoding unit 104 performs inverse quantization processing on the residual coefficient signal input from the prediction residual coding unit 103, and further performs inverse orthogonal transform processing to generate a reconstructed differential image signal.

The addition calculation unit 110 generates a reconstructed image signal by adding the reconstructed differential image signal input from the prediction residual decoding unit 104 and the predicted image signal input from the prediction processing unit 106 on a basic PU basis.

The picture buffer 105 stores the reconstructed image signal input from the addition calculation unit 110. The reconstructed image signal stored in the picture buffer 105 is referenced in the inter-frame prediction process of a picture to be coded temporally after the current coding target picture.

The integration determination unit 107, which functions as an integration unit, determines whether to integrate basic CUs belonging to a plurality of basic blocks to be integrated into one integrated coding unit (hereinafter referred to as "integrated CU") at the time when a series of processes of the basic block unit processing loop unit 111 is completed for the plurality of basic blocks to be integrated, and generates an integration determination result signal. In other words, the integration determination unit 107 determines whether to integrate a plurality of basic CUs belonging to an N×N pixel (e.g., 32×32 pixel) region composed of a plurality of basic blocks into one integrated CU. The integration determination unit 107 has a first operation mode in which the basic CUs are not integrated and are output to the code string generation unit 108 as they are, and a second operation mode in which a plurality of basic CUs belonging to the N×N pixel region are integrated into one new CU and output to the code string generation unit 108. The integration determination unit 107 switches between the first operation mode and the second operation mode according to the integration determination result. That is, the integration determination unit 107 integrates multiple basic CUs included in an N×N pixel region into one new CU if (1) multiple basic CUs and basic PUs belonging to an N×N pixel region all have the same block size, and (2) the prediction information of multiple basic PUs belonging to the N×N pixel region is the same.

The code sequence generation unit 108 generates a code sequence by performing variable length coding and arithmetic coding on the residual coefficient signal input from the predictive residual coding unit 103 and other coding information signals required during the decoding process, in units of integrated CUs or basic CUs, according to the integration judgment result signal input from the integration judgment unit 107. That is, the code sequence generation unit 108 performs variable length coding and arithmetic coding on the coding information set for the new CU after integration and the residual coefficients related to the multiple basic CUs before integration that belonged to the new CU after integration, to generate a code sequence for the new CU after integration. When the integrated CU is output from the integration judgment unit 107, the coding information set for the integrated CU and the residual coefficients related to the multiple basic CUs before integration that belonged to the integrated CU are variable length coded and arithmetic coded to generate a code sequence for the integrated CU.

As described above, in the video encoding device 100, the basic block size is limited to 16x16 pixels. This makes it possible to limit the selectable basic CU/basic PU/basic TU combinations to only three pairs when intra prediction is selected, and to only six pairs when inter prediction is selected, as shown in FIG. 3. This makes it possible to significantly reduce the amount of processing required to select the optimal combination in a series of processes in the basic block unit processing loop unit 111.

In the above embodiment, the basic block size is limited to 16×16 pixels, but the present disclosure is not limited to this. That is, the basic block size may not be limited, and an optimal combination may be selected from the combinations in FIG. 1 defined in the HEVC standard. However, in this case, the amount of processing will be enormous compared to when the basic block size is limited. Alternatively, the basic block size may not be limited to 16×16 pixels, but may be limited to other block sizes such as 32×32 pixels. For example, when the block size is limited to 32×32 pixels, the number of options for the block sizes of CUs and PUs belonging to the basic block size increases, and therefore the coding efficiency can be improved. On the other hand, the amount of processing for selecting the optimal block size of CUs and PUs increases. Therefore, a large block size may be used as the basic block size in a video coding device that can tolerate an increase in the amount of processing, while a small block size may be used as the basic block size in a video coding device that cannot tolerate an increase in the amount of processing.

[1-2. Operation of the Integrated Judgment Unit]
The integration determination unit 107 according to the first embodiment defines an integration area including a plurality of basic blocks. The integration determination unit 107 performs integration determination processing for all basic blocks included in the integration area when a series of processes of the basic block unit processing loop unit 111 is completed.

A method for determining whether or not to merge basic CUs belonging to multiple basic blocks into one merged CU in the merger determination unit 107 will be specifically described with reference to Figs. 4, 5A, and 5B. Fig. 4 is a flowchart showing the merger determination process according to the first embodiment. Figs. 5A and 5B are conceptual diagrams showing the merger determination process according to the first embodiment. Fig. 4 shows the process when the basic block size is 16x16 pixels and the merged area size is 32x32 pixels. In this case, the merged area contains four basic blocks. Note that the size of the merged area may be a size other than 32x32 pixels depending on the size of the basic block, as long as it is larger than the size of the basic block.

First, the integration determination unit 107 determines whether the four basic blocks included in the integration area are all composed of 16x16 pixel basic CUs and 16x16 pixel basic PUs (S301).

If the condition of S301 is not met (No in S301), the basic CUs in the merged area are not merged, as shown in FIG. 5A.

On the other hand, when the condition of S301 is satisfied (Yes in S301), the integrated determination unit 107 determines whether the prediction information of the four basic PUs in the integrated area is all the same (S302). Specifically, in the case of intra-screen prediction, the integrated determination unit 107 determines whether at least the intra-screen prediction modes of the four basic PUs in the integrated area are all the same. That is, when all basic PUs included in the integrated area are intra-screen prediction, the integrated determination unit 107 switches between the first operation mode and the second operation mode using at least whether the intra-screen prediction modes are all the same as the prediction information. On the other hand, in the case of inter-screen prediction, the integrated determination unit 107 determines whether at least the motion vector information and the reference picture information of the four basic PUs in the integrated area are all the same. That is, when all basic PUs included in the integrated area are inter-screen prediction, the integrated determination unit 107 switches between the first operation mode and the second operation mode using at least whether the motion vector information and the reference picture information are all the same as the prediction information.

If the condition of S302 is not met (No in S301), the basic CUs in the merged area are not merged, as shown in FIG. 5A.

On the other hand, if the condition of S302 is met (Yes in S301), the integration determination unit 107 integrates the four 16x16 pixel basic CUs into one integrated CU of 32x32 pixels, as shown in FIG. 5B (S303).

Figure 6 is a conceptual diagram for explaining the combinations of block sizes that emerge as a result of performing the integration determination process. These block sizes are the subject of variable-length coding and arithmetic coding in the code string generation unit 108. Comparing with Figure 3, it can be seen that the integration determination process adds an integrated CU with a CU size of 32 x 32 pixels, a PU size of 32 x 32 pixels, and a TU size of 16 x 16 pixels.

As described above, in the video encoding device 100 according to the first embodiment, if multiple basic CUs and basic PUs belonging to an integrated region (a region of N×N pixels consisting of multiple basic blocks) all have the same block size and the prediction information of all basic PUs included in the integrated region is the same, the integration determination unit 107 integrates multiple basic CUs included in the integrated region into one new CU. Then, a code string is generated based on the new CU after integration.

If the integration process shown in FIG. 4 is not performed, even if the basic PUs have the same prediction information, it is necessary to describe the prediction information of each basic PU individually in the code string, which results in unnecessary coding of the header information of the CU layer and the PU layer. On the other hand, if the integration process is performed, it is only necessary to describe one integrated prediction information in the code string, which makes it possible to suppress the coding amount of the header information of the CU layer and the PU layer, and makes it possible to improve the coding efficiency without increasing the amount of processing. Note that the integration determination unit 107 integrates only the CU and PU in the integrated CU, while leaving the TU as it is before integration. This makes it possible to convert to an integrated CU by only changing the header information of the CU layer and the PU layer, without the need to reconstruct the residual coefficient signal after integration.

(Embodiment 2)
Next, a video coding device 100 according to the second embodiment will be described with reference to the drawings. Note that the configuration of the video coding device 100 is similar to that described in the first embodiment, and therefore the description will be omitted.

The video encoding device 100 according to the second embodiment differs from the video encoding device 100 according to the first embodiment in the integrated determination process performed by the integrated determination unit 107.

Figure 7 is a flowchart showing the integration determination process according to the second embodiment. Figures 8A and 8B are conceptual diagrams showing the integration determination process according to the second embodiment. Figure 7 shows the process when the size of the basic block is 16 x 16 pixels and the size of the integrated area is 32 x 32 pixels. In this case, the integrated area contains four basic blocks.

First, the integration determination unit 107 determines whether or not all four basic blocks included in the integration region are composed of 16×16 pixel basic CUs and 16×16 pixel basic PUs (S301). If the condition of S301 is not met (No in S301), the basic CUs in the integration region are not integrated, as shown in FIG. 8A. On the other hand, if the condition of S301 is met (Yes in S301), the integration determination unit 107 determines whether or not the prediction information of only the basic PUs that make up the integrable combinations for the four basic PUs in the integration region is the same (S502 in FIG. 7). Details of the determination in S502 in FIG. 7 will be described with reference to FIG. 9.

9 is a conceptual diagram for explaining the conditions of the integration determination process according to the second embodiment. As shown in FIG. 9, when the prediction information of the four basic PUs is all the same, the integration determination unit 107 integrates the four basic PUs into one integrated PU of 32×32 pixels. Also, even if the prediction information of the four basic PUs is not all the same, when the prediction information of two sets of basic PUs adjacent to each other on the left and right is the same, the integration determination unit 107 integrates the two basic PUs on the top into two integrated PUs, a PU of 32×16 pixels, and a PU of 32×16 pixels, when the prediction information of the two basic PUs adjacent to each other on the bottom is the same. Also, even if the prediction information of the four basic PUs is not all the same, when the prediction information of the two sets of basic PUs adjacent to each other on the top and bottom is the same, the integration determination unit 107 integrates the two basic PUs on the left into two integrated PUs, a PU of 16×32 pixels, and a basic PU of 16×32 pixels, when the prediction information of the two PUs on the right is the same. Note that merging of 32x16 pixel merged PUs and 16x32 pixel merged PUs can only be selected in the case of inter-screen prediction, and in the case of intra-screen prediction, only merging to a 32x32 pixel merged PU can be selected.

In short, in the integration determination process according to the second embodiment, when multiple basic PUs belonging to an integration region (region of N×N pixels) are divided into a group of two adjacent basic PUs, and when the prediction information of the basic PUs belonging to each group is the same, the multiple basic PUs included in the integration region are integrated into one new PU. In this case, the integration determination unit 107 integrates four 16×16 pixel basic CUs into one 32×32 pixel integrated CU, as shown in FIG. 8B (S303 in FIG. 7).

Figure 10 is a conceptual diagram for explaining the combinations of block sizes that appear as a result of performing the integration determination process. These block sizes are the subject of variable length coding and arithmetic coding in the code string generation unit 108. Comparing with Figure 3, it can be seen that the integration determination process adds an integrated CU with a CU size of 32x32 pixels, a PU size of 32x32 pixels, and a TU size of 16x16 pixels, as well as an integrated CU with a CU size of 32x32 pixels, a PU size of 32x16 pixels or 16x32 pixels, and a TU size of 16x16 pixels.

As described above, in the video encoding device 100 according to the second embodiment, when multiple basic PUs belonging to an integrated region (region of N×N pixels) are divided into a group of two adjacent basic PUs, if the prediction information of the basic PUs belonging to each divided group is the same, the integration determination unit 107 integrates multiple basic CUs included in the integrated region into one new CU. Then, a code string is generated based on the new CU after integration.

For example, if the prediction information of all basic PUs included in the upper half of the integrated region (N×N pixel region) is the same, and the prediction information of all basic PUs included in the lower half of the integrated region is the same, the integration determination unit 107 integrates multiple basic CUs included in the integrated region into one new CU. Alternatively, if the prediction information of all basic PUs included in the left half of the integrated region (N×N pixel region) is the same, and the prediction information of all basic PUs included in the right half of the integrated region is the same, the integration determination unit 107 integrates multiple basic CUs included in the integrated region into one new CU.

In the merging process according to the first embodiment, the four basic PUs are merged into a merged CU only when the prediction information of the four basic PUs is the same. In contrast, in the merging process according to the second embodiment, two sets of basic PUs are merged into a merged CU even when the prediction information of the two sets of basic PUs is the same, so that it is possible to merge more CUs. This makes it possible to further reduce the amount of coding for the header information of the CU layer and the PU layer, and to improve the coding efficiency without increasing the amount of processing.

Note that in the integrated CU, the integration determination unit 107 integrates only the CU and PU, while leaving the TU in its pre-integration state. This eliminates the need to reconstruct the residual coefficient signal after integration, and makes it possible to convert to an integrated CU by simply changing the header information of the CU layer and the PU layer.

(Embodiment 3)
Next, a video coding device 100 according to a third embodiment will be described with reference to the drawings. Note that the configuration of the video coding device 100 is similar to that described in the first embodiment, and therefore the description thereof will be omitted.

The video encoding device 100 according to the third embodiment differs from the video encoding device 100 according to the first and second embodiments in the integrated determination process performed by the integrated determination unit 107.

In the integration determination unit 107 according to the third embodiment, two integration areas, integration area 1 and integration area 2, are defined as integration areas that include multiple basic blocks. The integration determination unit 107 performs integration determination processing for all basic blocks included in the integration area when a series of processes in the basic block unit processing loop unit 111 is completed.

11, 12A, 12B, and 12C, a method for determining whether or not to integrate basic CUs belonging to multiple basic blocks into one integrated CU in the integration determination unit 107 will be specifically described. FIG. 11 is a flowchart showing the integration determination process according to the third embodiment. FIGS. 12A, 12B, and 12C are conceptual diagrams showing the integration determination process according to the third embodiment. FIG. 11 shows the process when the basic block is 16×16 pixels, the integrated area 1 is 32×32 pixels, and the integrated area 2 is 64×64 pixels. The integrated area 2 includes the integrated area 1. In this case, the integrated area 1 includes four basic blocks. The integrated area 2 includes four integrated areas 1. In other words, the integrated area 2 includes 16 basic blocks. Note that the size of the integrated area may be a size other than 32×32 pixels and 64×64 pixels depending on the size of the basic block, as long as it is larger than the size of the basic block.

First, the integration determination unit 107 determines whether the four basic blocks contained in the integrated region 1 are all composed of 16x16 pixel basic CUs and 16x16 pixel basic PUs (S801).

If the condition of S801 is not met (No in S801), the basic CUs in merged area 1 are not merged, as shown in FIG. 12A.

On the other hand, if the condition of S801 is satisfied (Yes in S801), the integration determination unit 107 determines whether the prediction information of the four basic PUs in integrated region 1 is all the same (S802). Specifically, in the case of intra-prediction, it determines whether at least the intra-prediction modes of the four basic PUs in integrated region 1 are all the same. Also, in the case of inter-prediction, it determines whether at least the motion vector information and reference picture information of the four basic PUs in integrated region 1 are all the same.

If the condition of S802 is not met (No in S802), the basic CUs in the merged area 1 are not merged, as shown in FIG. 12A.

On the other hand, if the condition of S802 is met (Yes in S802), the integration determination unit 107 integrates the four 16x16 pixel basic CUs into one integrated CU1 of 32x32 pixels (S803).

The integration determination unit 107 performs a series of processes from S801 to S803 for all four integrated regions 1 belonging to the integrated region 2 (S804). That is, if the series of processes from S801 to S803 have not been completed for all integrated regions 1 in the integrated region 2 (No in S804), the integration determination unit 107 performs a series of operations from S801 to S803 for the integrated region 1 for which the processes have not been completed. As a result, if integration processing has not been performed for any integrated region 1, as shown in FIG. 12A, none of the 16 basic CUs in the integrated region 2 are integrated. On the other hand, if only some of the four integrated regions 1 are integrated, only some of the basic CUs in the integrated region 2 are integrated into an integrated CU 1 of 32×32 pixels, as shown in FIG. 12B.

Next, when the series of processes from S801 to S803 is completed for all integrated areas 1 within integrated area 2 (YES in S804), the integration determination unit 107 determines whether all CUs for the four integrated areas 1 within integrated area 2 have been integrated into integrated CU 1 (S805).

If the condition of S805 is not met (No in S805), the integration determination unit 107 terminates the integration determination process.

If the condition of S805 is met (Yes in S805), the integration determination unit 107 determines whether the prediction information of the four integrated PU1s in the integrated region 2 is all the same (S806).

If the condition of S806 is not met (No in S806), the integration determination unit 107 terminates the integration determination process.

If the condition of S806 is satisfied (Yes in S806), the integration determination unit 107 further integrates the four 32x32 pixel integrated CU1s into one 64x64 pixel integrated CU2 (S807), as shown in FIG. 12C. The video encoding device 100 limits the size of the basic blocks to 16x16 pixels, which is smaller than the 64x64 pixels and 32x32 pixels specified in HEVC. This makes it possible to integrate the blocks into an integrated CU1 of 32x32 pixels and an integrated CU2 of 64x64 pixels.

Figure 13 is a conceptual diagram for explaining the combinations of block sizes that emerge as a result of performing the integration determination process. These block sizes are the subject of variable length coding and arithmetic coding in the code string generation unit 108. Comparing with Figure 3, it can be seen that the integration determination process adds an integrated CU1 with a CU size of 32x32 pixels, a PU size of 32x32 pixels, and a TU size of 16x16 pixels, and an integrated CU2 with a CU size of 64x64 pixels, a PU size of 64x64 pixels, and a TU size of 16x16 pixels.

As described above, in the video encoding device 100 according to the third embodiment, the integration determination unit 107 integrates the multiple basic CUs included in the integrated region 1 into a single new integrated CU1 when the multiple basic CUs and basic PUs belonging to the integrated region 1 (a region of N×N pixels consisting of multiple basic blocks) all have the same block size and the prediction information of all the basic PUs included in the integrated region 1 is the same. Furthermore, when all the CUs in the integrated region 2 (a region consisting of multiple integrated regions 1) are integrated into the integrated CU1 and the prediction information of all the integrated PUs 1 in the integrated region 2 is the same, the integration determination unit 107 integrates the multiple basic CUs included in the integrated region 2 into a single new integrated CU2. Then, a code string is generated based on the new CU after integration.

In the merging process according to the first embodiment, a maximum of four basic CUs are merged. In contrast, in the merging process according to the third embodiment, a maximum of 16 basic CUs are merged, making it possible to merge more CUs into one CU. This makes it possible to further reduce the amount of coding for the header information of the CU layer and the PU layer, thereby improving the coding efficiency without increasing the amount of processing.

Note that the integration determination unit 107 integrates only the CUs and PUs in integrated CU1 and integrated CU2, while leaving the TUs in their pre-integration state. This eliminates the need to reconstruct the residual coefficient signal after integration, and makes it possible to convert to integrated CU1 and integrated CU2 simply by changing the header information of the CU layer and the PU layer.

Other Embodiments
As described above, the first to third embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the first to third embodiments to create new embodiments.

Therefore, other embodiments are given below as examples.

The integrated determination processes described in the first to third embodiments are not limited to being used individually. In other words, any combination of two or more of the integrated determination processes described in the first to third embodiments may be used. For example, in the flowchart of the third embodiment shown in FIG. 11, both or either of step S802 and step S805 may be replaced with step S502 in the flowchart of the second embodiment shown in FIG. 7.

Furthermore, a program having functions equivalent to those of each means included in the video encoding device 100 shown in the above embodiments 1 to 3 may be recorded on a recording medium such as a flexible disk. This makes it possible to easily implement the processes shown in the above embodiments in an independent computer system. Note that the recording medium is not limited to a flexible disk, and can be an optical disk, an IC card, a ROM cassette, or any other device capable of recording a program.

Functions equivalent to those of the means included in the video encoding device 100 shown in the first to third embodiments above may be realized as an LSI, which is an integrated circuit. These may be integrated into a single chip to include some or all of the means. Depending on the level of integration, LSIs may also be referred to as ICs, system LSIs, super LSIs, or ultra LSIs.

In addition, the method of integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. It is also possible to use a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI.

Furthermore, if an integrated circuit technology that can replace LSIs and the like emerges due to advances in semiconductor technology or other derived technologies, it is of course possible to use that technology to integrate functional blocks.

The above-described embodiments are intended to illustrate the technology disclosed herein, and various modifications, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents.

This disclosure can be applied to a video encoding device that encodes each picture constituting an input image and outputs the encoded video data. For example, this disclosure can be applied to video cameras, digital cameras, video recorders, mobile phones, mobile information terminals, and personal computers.

REFERENCE SIGNS LIST 100 Video encoding device 101 Picture memory 102 Basic block division unit 103 Prediction residual encoding unit 104 Prediction residual decoding unit 105 Picture buffer 106 Prediction processing unit 107 Integrated determination unit 108 Code stream generation unit 109 Difference calculation unit 110 Addition calculation unit 111 Basic block unit processing loop unit

Claims (2)

A video encoding device that encodes a target picture to generate a code string, comprising:
In operation, a block division unit that (i) divides the current picture into a plurality of coding units, and (ii) divides each of the plurality of coding units into at least one prediction unit;
A residual coding unit that generates a residual component by calculating a difference between a prediction image generated for each prediction unit and the target picture in operation,
When a first size of a first coding unit of the plurality of coding units is 2N×2N, which is the same as a basic block size, a second size of a prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N;
When a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N.
A video encoding device. 1. A video encoding method for encoding a target picture to generate a code string, comprising:
Dividing the current picture into a plurality of coding units, and dividing each of the plurality of coding units into at least one prediction unit;
generating a residual component by calculating a difference between a predicted image generated for each prediction unit and the target picture;
When a first size of a first coding unit of the plurality of coding units is 2N×2N, which is the same as a basic block size, a second size of a prediction unit in the first coding unit is any one of 2N×2N, 2N×N, and N×2N;
When a first size of the first coding unit of the plurality of coding units is 2N×2N, which is different from the basic block size, the second size of the prediction unit in the first coding unit is any one of 2N×2N, 2N×N, N×2N, and N×N.
A video coding method.

Images