JP7415058B2 - Determination device, encoding device, decoding device and program - Google Patents

Description

The present invention relates to a determining device, an encoding device, a decoding device, and a program.

Research is being conducted on encoding methods for compressing the amount of data when transmitting or storing still images and moving images. In recent years, with regard to video (moving image) encoding technology, ultra-high resolution videos such as 8K-SHV (Super High Vision) are becoming more and more popular, and huge amounts of moving images can be transmitted over broadcast waves or IP networks. Coding methods such as AVC (Advanced Video Coding)/H.264 and HEVC (High Efficiency Video Coding)/H.265 are known as methods for transmission.

The above-mentioned encoding method uses hierarchical quadtree partitioning for block division, and the encoding device (encoder) selects the optimal block division shape according to the texture contained in the original image. Information regarding the block division shape thus obtained is transmitted to the decoding device (decoder) side (for example, see Non-Patent Document 1).

The performance evaluation software (JEM) for next-generation video encoding methods jointly researched by MPEG (Moving Picture Experts Group) and ITU (International Telecommunication Union) uses hierarchical quadtree decomposition as an extension of HEVC. Hierarchical binary tree partitioning (QTBT: Quad Tree plus Binary Tree) is introduced, allowing more flexible block partitioning shapes to be selected (see Figure 6).

As shown in FIG. 7, there are 67 intra prediction modes in JEM: Planar mode (mode 0), DC mode (mode 1), and directional prediction modes (modes 2 to 66). One optimal intra prediction mode is selected from these 67 intra prediction modes, and the selected intra prediction mode is transmitted to the decoding device side.

When performing intra prediction, decoded pixels adjacent to the current block to be encoded are used as reference pixels.

However, depending on the block to be encoded, the adjacent blocks at the lower left or upper right may not have been decoded. In such a case, as shown in FIG. 8A, reference pixels are created by successively zero-order extrapolating the decoded pixels closest to the non-decoded pixels among the reference pixels. In this way, it is estimated that intra prediction using undecoded reference pixels has lower prediction accuracy than intra prediction using decoded reference pixels, as shown in FIG. 8(b).

Further, as shown in FIG. 9, the tendency for prediction accuracy to decrease in intra prediction using reference pixels that have not been decoded becomes more pronounced in blocks subjected to binary tree partitioning.

Therefore, if the lower left or upper right adjacent block has not been decoded, the probability that the intra prediction mode using the reference pixels in the adjacent block will be used will be low. Nevertheless, there is a problem in that if the amount of code is allocated equally to all intra prediction modes, it will lead to a reduction in encoding efficiency.

Next, a method of allocating code amounts to existing intra prediction modes to solve this problem will be described.

(MPM)
First, as a method of allocating code amount to intra prediction modes, a mechanism is used in which a small amount of code is allocated to intra prediction modes used in specific surrounding blocks of each block that are considered to have a high probability of being used. ing. Such an intra prediction mode is called MPM (Most Probable Mode).

In JEM, six intra prediction modes can be selected as MPM. The amount of code allocated to these six intra prediction modes is set to be smaller than the amount of code allocated to the other 61 (67-6) intra prediction modes.

(Selected Mode)
Second, for intra prediction modes that are not selected as MPM, Selected Mode (16 intra prediction modes) is first specified. The Selected Modes are evenly selected from 61 intra prediction modes at intervals of one intra prediction mode for every four intra prediction modes, and a fixed length of 4 bits is assigned to each intra prediction mode.

(Non-Selected Mode)
Thirdly, for the remaining 45 (67-6-16) intra prediction modes, fixed length 5 bits or fixed length 6 bits are allocated as Non-Selected Modes by Truncated Binary Coding. At this time, 5 bits are assigned in ascending order from the lowest mode number of the intra prediction mode, and 6 bits are assigned to the intra prediction mode of the higher mode number (see FIG. 7).

As an example of how to allocate code amount to each intra prediction mode other than MPM (Selected Mode and Non-Selected Mode), MPM is an intra prediction mode whose mode number is {0, 1, 2, 18, 34, 50}. Consider the case where .

First, from the 61 intra prediction modes other than MPM, the mode numbers are {3, 7, 11, 15, 20, 24, 28, 32, 37, 41, 45, 49, 54, 58, 62, 66 } is selected as the Selected Mode. In addition to the MPM identification flag and the Selected Mode identification flag, a fixed length 4-bit code is assigned to each of the intra prediction modes.

Codes are assigned to intra prediction modes other than MPM and Selected Mode as Non-Selected Mode. The Non-Selected Mode is assigned an MPM identification flag and a Selected Mode identification flag in the same way as the Selected Mode, and in addition, up to 19 intra prediction modes are assigned a fixed-length 5-bit code in ascending order of mode number. is assigned, and a fixed length 6-bit code is assigned to the other 26 intra prediction modes.

An example of Non-Selected Mode when MPM and Selected Mode are selected as described above is shown below.
- Non-Selected Mode (5 bits): {4, 5, 6, 8, 9, 10, 12, 13, 14, 16, 17, 19, 21, 22, 23, 25, 26, 27, 29}
- Non-Selected Mode (6 bits): {Other intra prediction modes}

Supervised by Sakae Okubo, “Impress Standard Textbook Series H.265/HEVC Textbook”, Impress Japan Co., Ltd., October 21, 2013.

However, in the above-mentioned technology, it is specified that Selected Modes are equally selected from the entire range of intra prediction modes each time, and a small amount of code is assigned to a fixed intra prediction mode among non-selected modes each time. , there was a problem that it led to a reduction in encoding efficiency.

Therefore, the present invention has been made to solve the above-mentioned problems, and provides a determining device, an encoding device, a decoding device, and a program that can reduce the amount of code in intra prediction and improve encoding efficiency. The purpose is to provide

A first feature of the present invention is a determination device that determines an intra prediction mode to be applied to a block obtained by dividing an original image in units of frames constituting a moving image, The candidate selection means selects a candidate for the intra prediction mode, and the candidate selection means selects an intra prediction mode different from a priority prediction mode that is an intra prediction mode used in a specific peripheral block of the block. The gist of the present invention is to change the code amount allocation method for the selected prediction mode based on the division shape of the block.

A second feature of the present invention is an encoding device configured to divide an original image in units of frames constituting a moving image into blocks and encode the blocks, the encoding device according to the first feature described above. The gist is to have a decision device.

A third feature of the present invention is a decoding device configured to decode an original image in units of frames constituting a moving image by dividing it into blocks, the determining device according to the first feature above. The main point is to have the following.

A fourth feature of the present invention is a program for causing a computer to function as the encoding device according to the second feature.

A fifth feature of the present invention is a program for causing a computer to function as the decoding device according to the third feature.

According to the present invention, it is possible to provide an encoding device, a decoding device, and a program that can reduce the amount of code in intra prediction and improve encoding efficiency.

FIG. 1 is a diagram showing an example of functional blocks of an encoding device 1 according to the first embodiment. FIG. 2 is a diagram for explaining an example of a method of changing the code amount allocation in Selected Mode by the determining device 1 according to the first embodiment. FIG. 3 is a diagram for explaining an example of a method of changing the code amount allocation in Non-Selected Mode by the determining device 1 according to the first embodiment. FIG. 4 is a diagram illustrating an example of a block division shape when changing the code amount allocation in Selected Mode and Non-Selected Mode by the determining device 1 according to the first embodiment. FIG. 5 is a diagram showing an example of functional blocks of the decoding device 3 according to the first embodiment. FIG. 6 is a diagram for explaining the prior art. FIG. 7 is a diagram for explaining the prior art. FIG. 8 is a diagram for explaining the prior art. FIG. 9 is a diagram for explaining the prior art.

(First embodiment)
Hereinafter, an encoding device 1 and a decoding device 3 including a determining device A according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 5. Here, the determining device A is configured to divide an original image in units of frames constituting a moving image into blocks and determine an intra prediction mode for each block.

The encoding device 1 and the decoding device 3 according to this embodiment are configured to support the intra prediction mode (see FIG. 7) in the above-mentioned JEM. Furthermore, the encoding device 1 and the decoding device 3 according to this embodiment are configured to support QTBT.

Note that the encoding device 1 and the decoding device 3 according to this embodiment are configured to be compatible with any video encoding method as long as it performs intra prediction.

The encoding device 1 according to the present embodiment is configured to divide an original image in units of frames constituting a moving image into blocks and encode the blocks. Hereinafter, such a block may be a CU (Coding Unit), a PU (Prediction Unit), or a TU (Transform Unit).

As shown in FIG. 1, the encoding device 1 according to the present embodiment includes a block division section 11, a residual signal generation section 12, a transformation section 13, a quantization section 14, an entropy encoding section 15, It includes an inverse quantization section 16, an inverse transformation section 17, a decoded image generation section 18, a memory 19, a code amount allocation determination section 20, and an intra prediction section 21.

The block division unit 11 is configured to perform quadtree division or binary tree division for each block, and output a flag indicating the division shape of the block to the residual signal generation unit 12 or the like.

The residual signal generation unit 12 generates a residual signal (prediction difference image) indicating the difference between the original image of each block divided by the block division unit 11 and the predicted image generated by the intra prediction unit 21. It is configured.

The orthogonal transform unit 13 is configured to perform orthogonal transform processing on the residual signal generated by the residual signal generator 12 to obtain orthogonal transform coefficients.

The quantization unit 14 is configured to perform quantization processing on the orthogonal transformation coefficients generated by the orthogonal transformation unit 13 to obtain quantized orthogonal transformation coefficients.

The entropy encoding unit 15 is configured to perform entropy encoding processing on the quantized orthogonal transform coefficients and the intra prediction mode of each block.

The dequantization unit 16 is configured to perform dequantization processing on the quantized transform coefficients generated by the quantization unit 14 to generate orthogonal transform coefficients.

The inverse orthogonal transform section 17 is configured to perform inverse orthogonal transform processing on the orthogonal transform coefficients generated by the inverse quantization section 16 to generate a residual signal.

The decoded image generation unit 18 is configured to generate a decoded image by adding the predicted image generated by the intra prediction unit 21 to the residual signal generated by the inverse orthogonal transformation unit 17.

The memory 19 is configured to hold the decoded image generated by the decoded image generation unit 18 so that it can be used as a reference image.

The code amount allocation determination unit 20 can change the code amount allocation method for intra prediction modes other than MPM (priority prediction mode) based on the division shape of each block divided by the block division unit 11. It is configured.

Here, the code amount allocation determination unit 20 can change the code amount allocation method for intra prediction modes other than MPM based on whether reference pixels in adjacent blocks of each block have been decoded. It may be configured as follows.

Hereinafter, in this embodiment, the total number of intra prediction modes is 67, 6 intra prediction modes can be selected as MPM, and 16 intra prediction modes can be selected as Selected Mode. This is possible, and a case where 45 intra prediction modes can be selected as Non-Selected Modes will be described as an example.

In addition, in the existing mechanism, the Selected Mode is set at predetermined intervals (4 intra prediction modes) in ascending order from the smallest mode number, as shown in FIG. It is specified that the intra prediction modes are selected evenly at intervals of one intra prediction mode for each intra prediction mode.

Here, the code amount allocation determination unit 20 is configured to change the code amount allocation method for intra prediction modes other than MPM by changing at least one of the above-mentioned predetermined interval and the range that can be selected as the Selected Mode. may have been done.

For example, as shown in FIG. 2B, the code amount allocation determination unit 20 selects mode number 2 (a small mode The selected mode may be configured to be equally selected at intervals of one intra-prediction mode for every three intra-prediction modes in ascending order from the number).

Alternatively, as shown in FIG. 2C, the code amount allocation determination unit 20 selects mode number 10 (smaller mode) in the range of intra prediction modes with mode numbers 10 to 58 among the intra prediction modes not selected as MPM. The selected mode may be configured to be equally selected at intervals of one intra-prediction mode for every three intra-prediction modes in ascending order from the number).

Alternatively, as shown in FIG. 2(d), the code amount allocation determination unit 20 selects mode number 66 (larger mode) in the range of intra prediction modes with mode numbers 18 to 66 among the intra prediction modes not selected as MPM. The selected modes may be equally selected at intervals of one intra-prediction mode for every three intra-prediction modes in descending order from the number).

In addition, in the existing mechanism, for Non-Selected Mode, as shown in FIG. A code amount of 5 bits (first predetermined length) is assigned to a predetermined number (for example, 19) of intra prediction modes, and a code amount of 6 bits (second predetermined length) is assigned to the remaining intra prediction modes. It is specified that a code amount of (long) is to be allocated.

Here, as shown in FIG. 3B, for Non-Selected Mode, the code amount allocation determination unit 20 selects mode number 66 (larger The code amount allocation method for intra prediction modes other than MPM may be changed by selecting in descending order from the mode number).

For example, as shown in FIG. 4A, the code amount allocation determination unit 20 determines that the divided shape of the target block #3 is vertically long, and the reference pixel in the block #2 adjacent to the lower left of the block #3 is decoded. If the reference pixel in the block adjacent to the upper right of block #3 has not been decoded, as shown in FIG. Even if the Selected Mode is configured to be selected evenly at intervals of one intra prediction mode for every three intra prediction modes in the range of intra prediction modes, in ascending order from mode number 2 (smallest mode number). good.

In such a case, the code amount allocation determination unit 20 selects modes in ascending order starting from mode number 2 (smallest mode number) in the entire range of intra prediction modes that are not selected as MPM and Selected Mode, as shown in FIG. 3(a). The configuration may be such that a code amount of 5 bits is allocated to a predetermined number (for example, 19) of intra prediction modes, and a code amount of 6 bits is allocated to the remaining intra prediction modes. .

In the example of FIG. 4(a), firstly, the binary tree is divided into block #1 and blocks #2/#3/#4, and secondly, the block is divided into two parts: block #2 and blocks #3/#4. Thirdly, it is divided into a binary tree into block #3 and block #4. In this example, it is assumed that encoding processing (decoding processing) is performed in the order of block #1 → block #2 → block #3 → block #4.

Further, as shown in FIG. 4B, the code amount allocation determination unit 20 determines that the divided shape of the target block #3 is vertically long, and the reference pixel in the block #5 adjacent to the lower left of the block #3 is decoded. If the reference pixel in the block adjacent to the upper right of block #3 has not been decoded, as shown in FIG. In the range of intra prediction modes, even if the Selected Mode is configured to be selected evenly at intervals of one intra prediction mode for every three intra prediction modes in ascending order from mode number 10 (smallest mode number). good.

In such a case, the code amount allocation determination unit 20 selects modes in ascending order starting from mode number 2 (smallest mode number) in the entire range of intra prediction modes that are not selected as MPM and Selected Mode, as shown in FIG. 3(a). The configuration may be such that a code amount of 5 bits is allocated to a predetermined number (for example, 19) of intra prediction modes, and a code amount of 6 bits is allocated to the remaining intra prediction modes. .

In the example of FIG. 4(b), firstly, quadtree partitioning is performed into block #1, block #2/#3, block #4, and block #5, and secondly, block #2 and block #3 are divided into block #1, block #2/#3, block #4, and block #5. It is split into a binary tree. In this example, it is assumed that encoding processing (decoding processing) is performed in the order of block #1 → block #2 → block #3 → block #4 → block #5.

Further, as shown in FIG. 4C, the code amount allocation determination unit 20 determines that the divided shape of the target block #4 is square, and that the reference pixels in the block adjacent to the lower left of the block #4 have already been decoded. If the reference pixel in the block adjacent to the upper right of block #4 has not been decoded, as shown in FIG. In the range of modes, the Selected Mode may be configured to be selected evenly at intervals of one intra prediction mode for every three intra prediction modes in ascending order from mode number 10 (smallest mode number).

In such a case, the code amount allocation determination unit 20 selects modes in ascending order starting from mode number 2 (smallest mode number) in the entire range of intra prediction modes that are not selected as MPM and Selected Mode, as shown in FIG. 3(a). The configuration may be such that a code amount of 5 bits is allocated to a predetermined number (for example, 19) of intra prediction modes, and a code amount of 6 bits is allocated to the remaining intra prediction modes. .

In the example of FIG. 4(c), the block is divided into four blocks: block #1, block #2, block #3, and block #4. In this example, it is assumed that encoding processing (decoding processing) is performed in the order of block #1 → block #2 → block #3 → block #4.

Further, as shown in FIG. 4D, the code amount allocation determination unit 20 determines that the divided shape of the target block #4 is horizontally long, and that the reference pixels in the block adjacent to the lower left of the block #4 have already been decoded. If the reference pixel in the block adjacent to the upper right of block #4 has not been decoded, as shown in FIG. In the range of modes, the Selected Mode may be configured to be selected evenly at intervals of one intra prediction mode for every three intra prediction modes in ascending order from mode number 10 (smallest mode number).

In such a case, as shown in FIG. 3B, the code amount allocation determination unit 20 selects modes in descending order from mode number 66 (larger mode number) in the entire range of intra prediction modes that are not selected as MPM and Selected Mode. The configuration may be such that a code amount of 5 bits is allocated to a predetermined number (for example, 19) of intra prediction modes, and a code amount of 6 bits is allocated to the remaining intra prediction modes. .

In the example of FIG. 4(d), firstly, quadtree partitioning is performed into block #1, block #2, block #3/#4, and block #5, and secondly, block #3 and block #4 are divided into block #1, block #2, block #3/#4, and block #5. It is split into a binary tree. In this example, it is assumed that encoding processing (decoding processing) is performed in the order of block #1 → block #2 → block #3 → block #4 → block #5.

Further, as shown in FIG. 4E, the code amount allocation determination unit 20 determines that the divided shape of the target block #3 is horizontally long, and that the reference pixel in the block #5 adjacent to the lower left of the block #3 is decoded. If the reference pixel in the block adjacent to the upper right of block #3 has already been decoded, mode numbers 18 to 66 of the intra prediction modes not selected as MPM are selected as shown in FIG. In the range of intra prediction modes, the Selected Mode is configured to be selected evenly at intervals of one intra prediction mode for every three intra prediction modes in descending order from mode number 66 (larger mode number). Good too.

In such a case, as shown in FIG. 3B, the code amount allocation determination unit 20 selects modes in descending order from mode number 66 (larger mode number) in the entire range of intra prediction modes that are not selected as MPM and Selected Mode. The configuration may be such that a code amount of 5 bits is allocated to a predetermined number (for example, 19) of intra prediction modes, and a code amount of 6 bits is allocated to the remaining intra prediction modes. .

In the example of FIG. 4(e), firstly, the binary tree is partitioned into block #1 and blocks #2/#3/#4, and secondly, the block is partitioned into blocks #2 and blocks #3/#4. Thirdly, it is divided into a binary tree into block #3 and block #4. In this example, it is assumed that encoding processing (decoding processing) is performed in the order of block #1 → block #2 → block #3 → block #4.

The intra prediction unit 21 generates a predicted image of each block based on the intra prediction mode of each block determined by the encoding device 1 (decision device A) and the reference image of each block held in the memory 19. It is configured as follows.

Further, the decoding device 3 according to the present embodiment is configured to divide an original image in units of frames constituting a moving image into blocks and decode the blocks.

As shown in FIG. 5, the decoding device 3 according to the present embodiment includes an entropy decoding section 31, an inverse quantization section 32, an inverse transformation section 33, an intra prediction section 34, a decoded image generation section 35, and a memory. 36, and a code amount allocation determining unit 37.

The entropy decoding unit 31 decodes quantized orthogonal transform coefficients etc. from the stream output from the encoding device 1 by performing entropy decoding processing on the stream output from the encoding device 1. It is configured.

The inverse quantization unit 32 is configured to generate orthogonal transformation coefficients by performing inverse quantization processing on the quantized orthogonal transformation coefficients output by the entropy decoding unit 31.

The inverse transform unit 33 is configured to generate a residual signal by performing an inverse transform process (for example, an inverse orthogonal transform process) on the orthogonal transform coefficients output by the inverse quantizer 32.

The intra prediction unit 34 is configured to generate a predicted image based on the intra prediction mode of each block output by the entropy decoding unit 31 and the reference image of each block held in the memory 36.

The decoded image generation unit 35 is configured to generate a decoded image by adding the predicted image generated by the intra prediction unit 34 and the residual signal generated by the inverse transformation unit 33.

The memory 36 is configured to hold the decoded image generated by the decoded image generation unit 35 so that it can be used as a reference image for intra prediction.

The code amount allocation determination unit 37 has the same function as the code amount allocation unit 20 described above.

According to the determination device A (encoding device 1 and decoding device 3) according to the present embodiment, if a specific adjacent block of each block has not been decoded, depending on the location of the adjacent block and the division shape of each block, By changing the amount of code allocated to each intra prediction mode, it is possible to reduce the amount of code in intra prediction and improve coding efficiency.

(Other embodiments)
Although the present invention has been described with reference to the embodiments described above, the statements and drawings that form part of the disclosure in these embodiments should not be understood as limiting the present invention. Various alternative embodiments, implementations, and operational techniques will be apparent to those skilled in the art from such disclosure.

Further, although not specifically mentioned in the above-described embodiment, a program that causes a computer to execute each process performed by the above-described encoding device 1 and decoding device 3 may be provided. Moreover, such a program may be recorded on a computer-readable medium. Computer readable media allow such programs to be installed on a computer. Here, the computer-readable medium on which such a program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM, for example.

Alternatively, a chip may be provided that includes a memory that stores a program for realizing at least some of the functions in the encoding device 1 and decoding device 3 described above, and a processor that executes the program stored in the memory. good.

A...Determining device 1...Encoding device 11...Block dividing unit 12...Residual signal generating unit 13...Transforming unit 14...Quantizing unit 15...Entropy encoding unit 16...Inverse quantizing unit 17...Inverse transforming unit 18...Decoding Image generation unit 19...memory 20...code amount allocation determination unit 21...intra prediction unit 31...entropy decoding unit 32...inverse quantization unit 33...inverse transformation unit 34...intra prediction unit 35...decoded image generation unit 36...memory 37... Code amount allocation determination unit

Claims (4)

A determining device that determines an intra prediction mode to be applied to blocks obtained by dividing an original image in units of frames constituting a moving image,
As a candidate for the intra prediction mode, a priority prediction mode that is an intra prediction mode used in a specific peripheral block of the block, and a selected prediction mode selected from an intra prediction mode different from the priority prediction mode; candidate selection means for selecting ;
code amount allocation means for changing a code amount allocation method for the selected prediction mode based on a division shape of the block ;
A determining device comprising : An encoding device comprising the determining device according to claim 1. A decoding device comprising the determining device according to claim 1. A program for causing a computer to function as the determining device according to claim 1.

Images