JP7422254B2 - Image decoding device, image decoding method and program - Google Patents

Description

The present invention relates to an image decoding device, an image decoding method, and a program.

Conventionally, techniques called RST (Reduced Secondary Transform) (see Non-Patent Document 2) and LFNST (Low-Frequency Non-Separable Transform) (see Non-Patent Document 1), which perform secondary transformation on coefficients after linear transformation, are known. It is being

Versatile Video Coding (Draft 5), JVET-N1001 CE6: Reduced Secondary Transform (RST) (CE6-3.1), JVET-N0193 Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), JVET-N1002

However, in the above-mentioned conventional technology, when decoding coefficients, a plurality of types of counters are provided to count the number of non-zero coefficients, and it is determined whether quadratic transformation is applied based on the counter values. Therefore, it is necessary to add coefficient counting processing just to determine whether or not quadratic transformation is applied. Coefficient decoding processing requires high throughput, but there is a problem in that the processing load increases when additional processing is performed.

The present invention has been made in view of the above-mentioned problems, and provides an image decoding device, an image decoding method, and an image decoding device that can omit the additional processing for determining whether or not to apply quadratic transformation and can be expected to speed up the processing. The purpose is to provide programs.

A first feature of the present invention is an image decoding device including an inverse transform unit configured to generate a prediction residual signal by inverse transform, the inverse transform unit performing a quadratic transform on a target block. The gist is that the present invention is configured to determine whether or not to apply the transform coefficients, and to control the decoding method of the transform coefficients according to the determination result.

A second feature of the present invention is an image decoding device that includes an inverse transform unit configured to generate a prediction residual signal by inverse transform, and the inverse transform unit is configured to: The gist is that the present invention is configured to control a decoding method of the secondary transformation index of the target block.

A third feature of the present invention is an image decoding device including an inverse transform unit configured to generate a prediction residual signal by inverse transform, the inverse transform unit including non-zero coefficients of a target block. The gist is that the decoding method of the secondary transformation index of the target block is controlled based on information indicating the occurrence position of the target block.

In a third feature of the present invention, the information indicating the occurrence position of the non-zero coefficient of the target block is a coefficient in which the non-zero coefficient cannot occur when the quadratic transformation is applied to the target block. When pointing to a position, the inverse transformer may be configured not to decode the secondary transform index of the target block.

In the third feature of the present invention, when the information indicating the occurrence position of the non-zero coefficient of the target block indicates a coefficient position of a DC component of the target block, the inverse transform unit The secondary conversion index may be configured not to be decoded.

A fourth feature of the present invention is an image decoding device, which includes an inverse transform unit configured to generate a prediction residual signal by inverse transform, and the inverse transform unit includes a non-zero in the target block. The gist of the present invention is that the decoding method of the secondary transform index of the target block is controlled by a flag indicating whether or not a coefficient is generated.

A fifth feature of the present invention is an image decoding device, which includes an inverse transform unit configured to generate a prediction residual signal by inverse transform, and the inverse transform unit includes a sub-block in a target block. The gist is that the decoding method of the secondary transform index of the target block is controlled by a flag indicating the presence or absence of a non-zero coefficient in the target block.

In a fifth feature of the present invention, a flag indicating whether or not the non-zero coefficient occurs in the sub-block indicates that the non-zero coefficient may occur when the quadratic transform is applied to the target block. If it is indicated that the non-zero coefficient occurs in a sub-block that does not exist, the inverse transform unit may be configured not to decode the secondary transform index of the target block.

A sixth feature of the present invention is an image decoding method, which includes a step of generating a prediction residual signal by inverse transform, and in the step, determines whether or not quadratic transform is applied to the target block, and determines whether or not quadratic transform is applied to the target block. The gist is to control the decoding method of transform coefficients according to the results.

A seventh feature of the present invention is a program that causes a computer to function as an image decoding device, wherein the image decoding device includes an inverse transform unit configured to generate a prediction residual signal by inverse transform. The gist is that the inverse transform unit is configured to determine whether or not quadratic transform is applied to the target block, and to control a method of decoding transform coefficients according to the determination result.

According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can omit additional processing for determining whether or not to apply quadratic transformation, and can be expected to speed up processing.

1 is a diagram showing an example of the configuration of an image processing system 1 according to an embodiment. FIG. 1 is a diagram illustrating an example of functional blocks of an image encoding device 100 according to an embodiment. FIG. 2 is a diagram illustrating an example of functional blocks of a transform/quantization unit 131 of an image encoding device 100 according to an embodiment. FIG. 2 is a diagram illustrating an example of functional blocks of an image decoding device 200 according to an embodiment. FIG. 2 is a diagram illustrating an example of functional blocks of an inverse transform/inverse quantization unit 220 of an image decoding device 200 according to an embodiment. FIG. 2 is a diagram for explaining an example of a function of an inverse transform/inverse quantization unit 220 of an image decoding device 200 according to an embodiment. FIG. 2 is a diagram for explaining an example of a function of an inverse transform/inverse quantization unit 220 of an image decoding device 200 according to an embodiment. 2 is a flowchart illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in the image decoding device 200 according to the first embodiment. 2 is a flowchart illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in the image decoding device 200 according to the first embodiment. 7 is a flowchart illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in the image decoding device 200 according to Modification Example 1 of the first embodiment. It is a figure for explaining a 2nd embodiment. 12 is a flowchart illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in an image decoding device 200 according to the third embodiment. 12 is a flowchart illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in an image decoding device 200 according to the third embodiment.

<First embodiment>
An image processing system 10 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 to 9. FIG. 1 is a diagram showing an image processing system 10 according to this embodiment.

As shown in FIG. 1, an image processing system 10 according to this embodiment includes an image encoding device 100 and an image decoding device 200.

Image encoding device 100 is configured to generate encoded data by encoding an input image signal. Image decoding device 200 is configured to generate an output image signal by decoding encoded data.

The encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path. The encoded data may be stored in a storage medium and then provided from the image encoding device 100 to the image decoding device 200.

(Image encoding device 100)
The image encoding device 100 according to this embodiment will be described below with reference to FIG. 2. FIG. 2 is a diagram illustrating an example of functional blocks of the image encoding device 100 according to the present embodiment.

As shown in FIG. 2, the image encoding device 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtracter 121, an adder 122, a transformation/quantization unit 131, and an inverse transformation/inverse quantization unit. 132, an encoding section 140, an in-loop filter processing section 150, and a frame buffer 160.

The inter prediction unit 111 is configured to generate a prediction signal by inter prediction (interframe prediction).

Specifically, the inter prediction unit 111 identifies the reference block included in the reference frame by comparing the frame to be encoded (hereinafter referred to as the target frame) and the reference frame stored in the frame buffer 160, and identifies the reference block included in the reference frame. and is configured to determine a predicted motion vector for the reference block.

Further, the inter prediction unit 111 is configured to generate a prediction signal included in the prediction block for each prediction block based on the reference block and the motion vector. The inter prediction unit 111 is configured to output a prediction signal to a subtracter 121 and an adder 122. Here, the reference frame is a frame different from the target frame.

The intra prediction unit 112 is configured to generate a prediction signal by intra prediction (intraframe prediction).

Specifically, the intra prediction unit 112 is configured to identify a reference block included in a target frame, and generate a prediction signal for each prediction block based on the identified reference block. Further, the intra prediction unit 112 is configured to output a prediction signal to a subtracter 121 and an adder 122.

Here, the reference block is a block that is referenced for a prediction target block (hereinafter referred to as a target block). For example, the reference block is a block adjacent to the target block.

The subtracter 121 is configured to subtract the prediction signal from the input image signal and output a prediction residual signal to the conversion/quantization unit 131. Here, the subtracter 121 is configured to generate a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal.

The adder 122 adds the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 132 to generate a pre-filtered decoded signal, and sends the unfiltered decoded signal to the intra prediction unit 112 and the intra prediction unit 112 and the intra prediction unit 112 and the intra prediction unit 112. The signal is configured to be output to the loop filter processing section 150.

Here, the pre-filtered decoded signal constitutes a reference block used by the intra prediction unit 112.

The transformation/quantization unit 131 is configured to perform transformation processing on the prediction residual signal and acquire coefficient level values. Furthermore, the conversion/quantization unit 131 may be configured to quantize the coefficient level values.

Here, the conversion/quantization unit 131 is configured to output a quantization index when quantizing coefficient level values. Hereinafter, the output of the conversion/quantization unit 131 will be described as a coefficient level value, regardless of whether quantization is applied or not.

Here, the conversion process is a process of converting a prediction residual signal into a frequency component signal. In such a transformation process, a base pattern (transformation matrix) corresponding to a discrete cosine transform (DCT) may be used, and a base pattern (transformation matrix) corresponding to a discrete sine transform (DST) may be used. may be used.

Note that the conversion process may be performed multiple times before quantization is performed. As an example, a secondary conversion that performs a second conversion process will be described later.

The inverse transformation/inverse quantization unit 132 is configured to perform inverse transformation processing on the coefficient level values output from the transformation/quantization unit 131. Here, the inverse transform/inverse quantization unit 132 may be configured to perform inverse quantization of the coefficient level values prior to the inverse transform process.

Here, the inverse transformation processing and inverse quantization are performed in a procedure opposite to the transformation processing and quantization performed by the transformation/quantization section 131.

The encoding unit 140 is configured to encode the coefficient level value output from the conversion/quantization unit 131 and output encoded data.

Here, for example, encoding is entropy encoding that counts the coefficients of a target block (encoded block or transform block) and assigns codes of different lengths based on the probability of occurrence of coefficient level values. The method of counting coefficients will be described later.

Further, the encoding unit 140 is configured to encode control data used in decoding processing in addition to coefficient level values.

Here, the control data may include size data such as a coding block (CU: Coding Unit) size, a prediction block (PU: Prediction Unit) size, a transform block (TU: Transform Unit) size, and the like.

The in-loop filter processing unit 150 is configured to perform filter processing on the unfiltered decoded signal output from the adder 122 and output the filtered decoded signal to the frame buffer 160.

Here, for example, the filter processing is deblocking filter processing that reduces distortion occurring at the boundary portions of blocks (encoding blocks, prediction blocks, or transform blocks).

The frame buffer 160 is configured to accumulate reference frames used by the inter prediction unit 111.

Here, the filtered decoded signal constitutes a reference frame used by the inter prediction unit 111.

(Conversion/quantization unit 131)
The conversion/quantization unit 131 of the image encoding device 110 according to this embodiment will be described below with reference to FIG. 3. FIG. 3 is a diagram illustrating an example of functional blocks of the conversion/quantization unit 131 of the image encoding device 110 according to the present embodiment.

As shown in FIG. 13, the conversion/quantization unit 131 includes a primary conversion unit 131A, a secondary conversion unit 131B, and a quantization unit 131C.

The transformation/quantization unit 131 is an example of a transformation/quantization unit configured to generate coefficient level values from the prediction residual signal by transformation/quantization.

The primary transformation unit 131A is configured to receive the prediction residual signal as input and generate primary transformation coefficients for the target block.

Here, the base pattern (transformation matrix) used in the linear transformation process may be selected from a plurality of base patterns. For example, in Non-Patent Document 1, base patterns corresponding to DCT2, DCT8, and DST7 are used, respectively. Further, as a base pattern selection method, for example, there is a method in which a base pattern with the lowest encoding cost is selected by the image encoding device 100, and the selected base pattern is transmitted to the image decoding device 200 as side information.

The secondary transformation unit 131B is configured to receive the primary transformation coefficients from the primary transformation unit 131A and determine whether or not to apply secondary transformation to the target block.

Here, when it is determined that the secondary transformation is to be applied, the secondary transformation unit 131B generates secondary transformation coefficients by performing a transformation process on the primary transformation coefficients, and converts the secondary transformation coefficients into the quantization unit 131C. is configured to output to .

On the other hand, the secondary transformation unit 131B is configured to output the primary transformation coefficients as they are to the quantization unit 131C when it is determined that the secondary transformation is not applied.

Further, the secondary transformation unit 131B is configured to output information regarding whether or not to apply secondary transformation (hereinafter referred to as a secondary transformation index) to the encoding unit 140.

The quantization unit 131C is configured to input transform coefficients (primary transform coefficients or secondary transform coefficients) and determine whether or not quantization is applied to the transform coefficients of the target block.

Here, the quantization unit 131C is configured to generate a quantization index when it is determined that quantization is applied. Note that a known method can be used as a method for determining whether or not to apply quantization and a method for generating a quantization index, so the details will be omitted.

(Encoding unit 140)
When a quadratic transform is applied, the encoding unit 140 may be configured to encode the coefficient level value using a context optimized for the probability of applying the quadratic transform.

(Inverse transformation/inverse quantization unit 132)
The inverse transform/inverse quantization section 132 operates in the same way as the inverse transform/inverse quantization section 220 in the image decoding device 200, which will be described later. shall be handled in the same way as the inverse transform/inverse quantization unit 220.

(Image decoding device 200)
The image decoding device 200 according to this embodiment will be described below with reference to FIG. 4. FIG. 4 is a diagram illustrating an example of functional blocks of the image decoding device 200 according to the present embodiment.

As shown in FIG. 4, the image decoding device 200 includes a decoding section 210, an inverse transform/inverse quantization section 220, an adder 230, an inter prediction section 241, an intra prediction section 242, and an in-loop filter processing section. 250 and a frame buffer 260.

The decoding unit 210 is configured to decode encoded data generated by the image encoding device 100 and decode coefficient level values.

Here, decoding is, for example, decoding of entropy encoded data that assigns codes of different lengths based on the probability of occurrence of the coefficients.
The decoding is entropy decoding that is a reverse procedure to the entropy encoding performed by the encoding unit 140.

Furthermore, the decoding unit 210 may be configured to obtain control data through decoding processing of encoded data.

Note that, as described above, the control data may include size data such as the encoded block size, predicted block size, and transformed block size.

The inverse transform/inverse quantization unit 220 generates a prediction residual signal by performing inverse quantization of the quantization index output from the decoding unit 210 and inverse transformation processing of the coefficient level values output from the decoding unit 210. is configured to do so.

The adder 230 adds the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 220 to generate a pre-filtered decoded signal, and sends the unfiltered decoded signal to the intra prediction unit 242 and the in-loop. It is configured to output to the filter processing section 250.

Here, the pre-filtered decoded signal constitutes a reference block used by the intra prediction unit 242.

Like the inter prediction unit 111, the inter prediction unit 241 is configured to generate a prediction signal by inter prediction (interframe prediction).

Specifically, the inter prediction unit 241 is configured to generate a prediction signal for each prediction block based on a motion vector decoded from encoded data and a reference signal included in a reference frame. The inter prediction unit 241 is configured to output a prediction signal to the adder 230.

The intra prediction unit 242, like the intra prediction unit 112, is configured to generate a prediction signal by intra prediction (intraframe prediction).

Specifically, the intra prediction unit 242 is configured to identify a reference block included in the target frame, and generate a prediction signal for each prediction block based on the identified reference block. Intra prediction unit 242 is configured to output a prediction signal to adder 230.

Like the in-loop filter processing section 150, the in-loop filter processing section 250 performs filter processing on the unfiltered decoded signal output from the adder 230, and outputs the filtered decoded signal to the frame buffer 260. is configured to do so.

Here, for example, the filter processing is deblocking filter processing that reduces distortion occurring at the boundary portions of blocks (encoded blocks, prediction blocks, transform blocks, or subblocks obtained by dividing them).

The frame buffer 260, like the frame buffer 160, is configured to store reference frames used by the inter prediction unit 241.

Here, the filtered decoded signal constitutes a reference frame used by the inter prediction unit 241.

(Inverse transform/inverse quantization unit 220)
The inverse transform/inverse quantization unit 220 of the image decoding device 200 according to this embodiment will be described below with reference to FIG. 5. FIG. 5 is a diagram illustrating an example of functional blocks of the inverse transform/inverse quantization unit 220 of the image decoding device 200 according to the present embodiment.

As shown in FIG. 5, the inverse transformation/inverse quantization section 220 includes an inverse quantization section 242A, an inverse quadratic transformation section 242B, and an inverse linear transformation section 242C.

The inverse quantization unit 220A is configured to generate transform coefficients of the target block using the quantization index as an input signal. Since a known method can be used to generate coefficient level values from quantization indexes, details will be omitted.

The inverse quadratic transform unit 220B is configured to receive the coefficient level value as input and determine whether or not to apply the inverse quadratic transform to the target block.

Here, when it is determined that the inverse quadratic transform is to be applied, the inverse quadratic transform unit 220B generates primary transform coefficients by performing an inverse transform process on the input coefficient level value, and It is configured to output the coefficients to the linear conversion coefficient section 220C.

On the other hand, inverse quadratic transformation section 220B is configured to output the input coefficient level value as a primary transformation coefficient to primary transformation coefficient section 220C as is when it is determined that quadratic transformation is not applied.

Hereinafter, in order to simplify the description, the quadratic transformation and the inverse quadratic transformation will be collectively referred to as the quadratic transformation, and the quadratic transformation in the inverse quadratic transformation section will be read as the inverse quadratic transformation as appropriate.

Note that a method for determining whether or not to apply quadratic transformation by the inverse quadratic transformation unit 220B will be described later.

The inverse linear transform unit 220C is configured to receive the linear transform coefficients as input and generate a prediction residual signal of the target block.

Here, the base pattern (transformation matrix) used for the inverse linear transformation process may be selected from a plurality of prescribed patterns, as in the case of the image encoding device 100. The selection of the base pattern may be performed using, for example, side information transmitted from the image encoding device 100 (base pattern information selected by the image encoding device 100).

(Explanation of quadratic transformation)
Hereinafter, with reference to FIG. 6, an example of the quadratic transformation performed by the image encoding device 100 and the image decoding device 200 according to this embodiment will be described.

As described above, the secondary transformation is a technique of transforming the primary transformation coefficients of the target block again before quantization. By converting the linear transform coefficients again before quantization, coefficients with non-zero coefficient level values (hereinafter referred to as non-zero coefficients) tend to be concentrated in the low frequency region, and in the coefficient scanning process described later, the coefficient level Since coefficients having a value of zero (hereinafter referred to as zero coefficients) are continuous, further coding improvement effects can be expected due to entropy coding.

Here, the secondary transformation may use a plurality of base patterns (transformation matrices), similarly to the primary transformation. For example, as in Non-Patent Document 2, usable base patterns may be set in advance based on the target block size, intra prediction mode, and secondary transformation index. If the value of the secondary transformation index is 0, it indicates that the secondary transformation is invalid. On the other hand, a value of 1 or more indicates that quadratic transformation is effective. Furthermore, two types of base patterns may be adaptively selected using the quadratic transformation index, for example, as in Non-Patent Document 2.

Further, the application range of the secondary transform may be adaptively set according to the block size of the target block. For example, FIG. 6 shows an example of the application range of quadratic transformation in Non-Patent Document 2.

As shown in FIG. 6(a), when the block size of the target block is 8×8 pixels, the application range R1 of the quadratic transform covers all 64 linear transform coefficients, and The subsequent non-zero coefficient generation region R2 is limited to the low frequency region (4×2 pixels) of the quadratic transform coefficient, and all other regions are made to have zero coefficients (hereinafter referred to as zeroing).

On the other hand, as shown in FIG. 6(b), when the block size of the target block is M×N (where M>8, N>8) pixels, the application range R1 of the quadratic transformation is 48 low It is a linear transformation coefficient in the frequency domain. That is, the secondary transformation is not applied to the primary transformation coefficients other than the 48 low frequency regions, and the primary transformation coefficients are treated as they are. Furthermore, the region where non-zero coefficients occur after the quadratic transformation is limited to the low frequency region (4×4 pixels) of the quadratic transformation coefficients, and the other regions are zeroed.

(Coefficient encoding method)
Hereinafter, a method for encoding coefficient level values will be described with reference to FIG. 7. FIG. 7 is a diagram illustrating a method of scanning coefficient level values of a target block.

As a method for scanning the coefficients, for example, as described in Non-Patent Document 1, there is a method of scanning in the upper left direction from the coefficient C1 of the highest frequency component of the target block to the coefficient Cx of the lowest frequency component (hereinafter referred to as diagonal scan). (See FIG. 7).

In natural images, non-zero coefficients are less likely to occur in high frequency regions. Therefore, by diagonally scanning from high frequency components to low frequency components, zero coefficients can be consecutively arranged, and the effect of improving encoding performance by entropy encoding can be expected.

Further, the diagonal scan may be executed in units of subblocks in which the target block is divided into 4×4 pixels, as in Non-Patent Document 1. In addition, the image encoding device 100 transmits a flag (coded_sub_block_flag) indicating the presence or absence of non-zero coefficients to the image decoding device 200 according to the number of non-zero coefficients generated in sub-block units. You may.

Here, the reason for using such a flag (coded_sub_block_flag) is that, for example, a 1-bit flag (coded_sub_block_flag) is used to encode a sub-block with no non-zero coefficients. The point is that it can be decrypted.

In addition, before encoding the flag (coded_sub_block_flag), the image encoding apparatus 100 determines whether or not a non-zero coefficient occurs starting from the sub-block in which a non-zero coefficient occurs first in the scan order. You may also perform a determination process.

Here, the image encoding device 100 may transmit position information (last_coeff_pos) where a non-zero coefficient first occurs to the image decoding device 200. As a result, it is certain that the sub-block before the first non-zero coefficient has no non-zero coefficients, so the transmission of the above-mentioned flag (coded_sub_block_flag) is unnecessary, and the code amount is can be saved.

Note that, for example, as in Non-Patent Document 1, the image encoding device 100 divides the x-coordinate value and y-coordinate value of position information where a non-zero coefficient occurs first in a target block into a prefix part and a suffix part. It may also be encoded.

(Method for determining whether or not to apply quadratic transformation and method for decoding coefficients)
8 and 9 are flowcharts illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in the image decoding device 200. Hereinafter, an example of a method for determining whether or not to apply quadratic transform will be described with reference to FIG. 8, and an example of a coefficient decoding method will be described with reference to FIG.

As shown in FIG. 8, in step S61, the image decoding device 200 decodes a flag (cbf) indicating the presence or absence of a non-zero coefficient in the target block. Here, the value of cbf is either 0 or 1.

In step S62, the image decoding device 200 determines whether a non-zero coefficient exists based on the cbf.

If it is determined that there are no non-zero coefficients (the number of non-zero coefficients is 0) (that is, if it is determined that cbf=0), this process ends.

On the other hand, if it is determined that a non-zero coefficient exists (one or more non-zero coefficients) (that is, if it is determined that cbf=1), the process proceeds to step S63.

Here, cbf may be decoded as separate flags for each of the luminance signal (Y signal) and color difference signal (Cb signal and Cr signal). In this case, the image decoding device 200 may determine as follows depending on whether the block division tree structure of the luminance signal and color difference signal of the target block is different (Dual Tree) or when both are the same (Single Tree). .

The image decoding device 200 may determine that for the Y signal at the time of Dual Tree, if the cbf of the Y signal is 1, the process proceeds to step S63, and if not, the process ends.

If the cbf of at least one of the Cb/Cr signals is 1 in the Cb/Cr signal at the time of Dual Tree, the image decoding device 200 proceeds to step S63, and otherwise determines to end this process. You may.

If the cbf of at least one of the Y signal, Cb signal, and Cr signal at the time of Single Tree is 1, the image decoding device 200 proceeds to step S63, and otherwise determines to end this process. Good too.

In step S63, the image decoding device 200 decodes the secondary transformation index (lfnst_idx).

In step S64, the image decoding device 200 decodes information (last_coeff_pos) indicating the position of occurrence of the first non-zero coefficient in the target block in the scan order.

In step S65, the image decoding device 200 performs coefficient decoding processing (see FIG. 9). Note that the image decoding device 200 performs the decoding process on the coefficients in subblock units obtained by dividing the above-described target block.

As shown in FIG. 9, in step S641, the image decoding device 200 determines whether the subblock in which a non-zero coefficient first occurs is the first or last subblock in the scan order in the target block. do.

If it is determined that it is either the first or last sub-block, it is obvious that non-zero coefficients exist in this sub-block, so the process proceeds to step S645.

If it is determined that the sub-block is neither the first nor the last sub-block, in step S642, the image decoding device 200 determines whether the quadratic transform is applied to the sub-block based on the quadratic transform index (st_idx). Determine whether or not there are any.

In the case where it is determined that the quadratic transformation is applied in such a sub-block (the quadratic transformation is effective), and it is obvious that all coefficients in such sub-block are zero due to the quadratic transformation. (zeroed), there is no need to decode coded_sub_block_flag, and the process proceeds to step S646.

On the other hand, if it is determined that the quadratic transform is not applied in such a sub-block (the quadratic transform is not valid), or even if the quadratic transform is applied in such a sub-block, a non-zero coefficient will occur. If there remains a possibility, the process proceeds to step S643.

Note that the order of step S641 and step S642 may be changed.

In step S643, the image decoding device 200 decodes a flag (coded_sub_block_flag) indicating the presence or absence of a non-zero coefficient.

In step S644, the image decoding device 200 determines whether a non-zero coefficient exists in the sub-block based on coded_sub_block_flag.

If the value of coded_sub_block_flag is 0, the image decoding device 200 determines that there is no non-zero coefficient, and the process proceeds to step S646.

On the other hand, if the value of coded_sub_block_flag is 1, the image decoding device 200 determines that a non-zero coefficient exists, and the process proceeds to step S645.

In step S645, the image decoding device 200 decodes the level value of each coefficient in the sub-block.

In step S646, the image decoding device 200 determines whether the sub-block to be processed is the final block.

If it is determined that the block is not the final block, the process proceeds to step S641, moves to the next sub-block in the scan order, and restarts the process; if it is determined that the block is the final block, the process proceeds to finish.

By using the method for determining whether to apply inverse quadratic transform and the method for decoding coefficients described above, the following two effects can be expected.

The first effect is that by performing the secondary transformation index before decoding the coefficients of the target block, it is possible to omit the coefficient decoding process that was necessary in the conventional technology to determine whether or not to apply the secondary transformation. It means that it can be done.

Specifically, in the conventional technology, two counters (a counter for measuring the number of non-zero coefficients generated in the target block and a counter for measuring the number of non-zero coefficients generated in the target block and a counter for determining whether or not to apply the quadratic transform) are used to determine whether or not to apply the quadratic transform. However, by introducing the technology according to this embodiment, both can be omitted.

The second effect is that when quadratic transform is effective, in the conventional technology, coded_sub_block_flag is redundantly decoded or coded for subblocks in which all coefficients are obviously zero in the block to be transformed. There was a need to change it. On the other hand, if the technology according to the present embodiment is used, such processing can be omitted, and thus an improvement in encoding performance can be expected.

<Change example 1>
Hereinafter, with reference to FIG. 10, modification example 1 of the above-described first embodiment will be described, focusing on the differences from the above-described first embodiment.

In the flowchart shown in FIG. 10, step S66, which did not exist in the flowchart shown in FIG. 8, is added before step S63.

Here, if the secondary transformation is clearly not effective, the transmission of the secondary transformation index becomes redundant, so in FIG. 10, such step S66 is added.

For example, when the application target of the secondary transformation is limited to intra pictures or intra blocks, a determination as to whether the target block is an intra picture or an intra block may be added to the predetermined conditions.

In addition, when one encoded block is divided into multiple prediction blocks or transform blocks, the number of transform blocks to which quadratic transform is applied increases, and while improvement in encoding performance can be expected, It is possible that delays in processing and decoding may increase.

Therefore, if a predetermined condition is met, for example, when intra prediction by dividing a coded block into multiple parts (intra subdivision prediction) described in Non-Patent Document 1 is valid, unintended processing can be avoided by treating the quadratic transformation as invalid. Increase in delay can be suppressed. Therefore, a determination as to whether intra subdivision prediction is applied to the target block may be added to the predetermined conditions.

<Change example 2>
Modification 2 of the first embodiment described above will be described below, focusing on the differences from the first embodiment and Modification 1 described above.

In the first embodiment described above, as shown in FIG. 8, an example was described in which the secondary transformation index is decoded in units of encoded blocks.

In the present modification example 2, when the encoded block is divided into a plurality of transform blocks, the secondary transform index may be decoded in units of transform blocks.

According to this configuration, since the secondary transform index can be decoded in units of different transform blocks, the number of blocks to which the secondary transform is applied increases, and improvement in encoding performance can be expected.

<Second embodiment>
Hereinafter, a second embodiment of the present invention will be described with reference to FIG. 11, focusing on the differences from the above-described first embodiment.

In this embodiment, as shown in FIG. 11, the configuration is configured to control the decoding method of position information (last_coeff_pos) where a non-zero coefficient occurs at the beginning of the target block depending on whether or not quadratic transform is applied. has been done.

This embodiment differs from the first embodiment 1 described above with respect to the decoding method of last_coeff_pos (see FIG. 8). This embodiment has the same configuration as the above-described first embodiment with respect to other parts, and therefore, description thereof will be omitted.

For example, in Non-Patent Document 2, when a quadratic transform is effective for a block and the target block size is 4x4 or 8x8, the occurrence position and number of non-zero coefficients for the target area of the quadratic transform are determined in the upper leftmost subblock, respectively. Moreover, it was limited to eight pieces.

At this time, the occurrence position of the non-zero coefficient can be specified as shown in FIG. 11 by considering the scan order of the sub-blocks.

In other words, if the quadratic transformation is valid, it is impossible for last_coeff_pos to indicate the scan orders 8 to 15 shown in FIG.

Therefore, last_coeff_pos is usually decoded using the prefix and suffix of the x and y coordinate values, but only when quadratic transformation is effective and the position and number of non-zero coefficients in the target block are limited. In this case, the decoding method of last_coeff_pos may be modified as follows.

For example, last_coeff_pos can be transmitted as a 3-bit index in the scan order of coefficients, with the range 0 to 7 in which non-zero coefficients may occur.

In the conventional technology, since the x-coordinate value and the y-coordinate value are each expressed with 2 bits for each sub-block, the amount of code can be saved accordingly, and as a result, an improvement in encoding performance can be expected.

<Third embodiment>
Hereinafter, a third embodiment of the present invention will be described with reference to FIGS. 12 and 13, focusing on the differences from the above-described first and second embodiments.

This embodiment differs from the first embodiment with respect to the method for determining whether or not to apply quadratic transform and the method for decoding coefficients (see FIG. 8). This embodiment has the same configuration as the above-described first embodiment with respect to other parts, and therefore, description thereof will be omitted.

12 and 13 are flowcharts illustrating an example of a method for determining whether or not to apply quadratic transform and a method for decoding coefficients in the image decoding device 200 according to the present embodiment. An example of the operation of the image decoding device 200 according to this embodiment will be described below using this flowchart.

As shown in FIG. 12, in step S61, the image decoding device 200 decodes a flag (cbf) indicating whether a non-zero coefficient exists in the target block, similarly to step S61 shown in FIG.

In step S62, the image decoding device 200 determines whether a non-zero coefficient exists based on the cbf, similarly to step S62 shown in FIG.

If it is determined that there is no non-zero coefficient (the number of non-zero coefficients is 0) (that is, if it is determined that cbf=0), the process proceeds to step S91.

On the other hand, if it is determined that a non-zero coefficient exists (the number of non-zero coefficients is one or more) (that is, if it is determined that cbf=1), the process proceeds to step S64.

In step S64, similarly to step S64 shown in FIG. 8, the image decoding device 200 decodes information (last_coeff_pos) indicating the position of occurrence of the first non-zero coefficient in the target block in the scanning order.

In step S93, the image decoding device 200 performs the coefficient decoding process shown in FIG. 13. The coefficient decoding process shown in FIG. 13 is similar to the coefficient decoding process shown in FIG. 9, except that step S642 shown in FIG. 9 is not included.

In step S91, the image decoding device 200 determines whether a predetermined condition, which will be described later, is satisfied. If it is determined that the predetermined condition is satisfied, the operation proceeds to step S92, and if it is determined that the predetermined condition is not satisfied, the operation ends.

Here, the predetermined condition may be a condition of whether or not the secondary transformation index is decoded depending on the target block size.

For example, when the image decoding device 200 determines in step S91 that at least either the width or the height of the target encoding block is larger than 64, the image decoding device 200 determines in step S92 not to decode the secondary transformation index. Good too. The effects of this determination are as follows.

In Non-Patent Document 3, a tool for maintaining pipeline processing units of a decoder called virtual pipeline data units (VPDUs) is employed. Here, in Non-Patent Document 3, the VPDU size is set to 64x64 pixels, and the maximum width and height of the conversion block is set to 64 pixels.

Therefore, if either the width or the height of the target encoding block is larger than 64 pixels, the conversion block is divided so that the width and height of the conversion block are 64 pixels or less.

For example, if the encoded block size is 128x128 pixels, the encoded block will include four 64x64 pixel transform blocks. Since the maximum size of a conversion block and the size of a VPDU are specified to be the same size, by performing processing for each conversion block, pipeline processing specified by the VPDU can be maintained.

When a plurality of transform blocks are included in the encoded block, the image decoding device 200 performs step S91 after each of steps S61, S62, S64, and S93 shown in FIG. 12 are performed for all transform blocks. .

Therefore, when at least either the width or the height of the encoded block is larger than 64 pixels, it is necessary to wait until the processing of all VPDUs constituting the target block is completed in order to perform decoding control of the secondary transformation index. Otherwise, the pipeline processing of the decoder will be delayed.

Therefore, as described above, if the decoding of the secondary transformation index is limited, it can be expected that this processing delay can be avoided.

In addition, the predetermined condition is a condition determined using non-zero coefficient occurrence position information (last_coeff_pos) or a flag (coded_sub_block_flag) indicating whether a non-zero coefficient has occurred in the target sub-block. The condition may be whether or not the next conversion index is decoded.

For example, the predetermined condition is such that when the area in which non-zero coefficients occur when applying quadratic transform is limited, such as when the target block shown in Non-Patent Document 2 is 4 x 4 pixels or 8 x 8 pixels, For regions where zero coefficients cannot occur, a condition may be included that last_coeff_pos or coded_sub_block_flag does not indicate the existence of non-zero coefficients.

For example, the predetermined conditions may include a condition that cbf of the target block is not zero or that last_coeff_pos of the target block does not indicate a DC component (direct current component).

Note that the determination based on last_coeff_pos and cbf is performed when the block division tree structure of the luminance signal (Y signal) and color difference signal (Cb signal and Cr signal) of the target block is different (Dual Tree), and when both are the same (Dual Tree). Single Tree), the determination may be made as follows.

If the Y signal is a Dual Tree and last_coeff_pos is a DC component or cbf is 0, the image decoding device 200 does not decode the secondary transformation index.

When last_coeff_pos is a DC component or cbf is 0 in the Cb/Cr signal at the time of Dual Tree and for both the Cb/Cr signals, the image decoding device 200 does not decode the secondary transformation index.

If last_coeff_pos is a DC component or cbf is 0 for all of the Y signal, Cb signal, and Cr signal at the time of Single Tree, the image decoding device 200 does not decode the secondary transformation index.

Alternatively, in the case of Single Tree, if the last_coeff_pos of the Y signal is a DC component or cbf is 0, the image decoding device 200 may not decode the secondary transformation index.

According to this configuration, it is possible to determine whether or not the secondary transformation index is decoded without depending on the values of cbf and last_coeff_pos of the Cb signal and Cr signal.

In the above determination, if the secondary conversion index is not decoded, the secondary conversion index is implicitly treated as 0 (that is, it is determined that the secondary conversion is not applied).

As mentioned above, when the region where non-zero coefficients occur when the quadratic transform is applied is limited, last_coeff_pos or coded_sub_block is used for the region where non-zero coefficients cannot occur. If #flag indicates the presence of non-zero coefficients, it is clear that no quadratic transformation is applied to the target block.

Therefore, for example, when determining a predetermined condition, when the target block is 4 x 4 pixels or 8 x 8 pixels, last_coeff_pos or coded_sub# If block_flag indicates the existence of a non-zero coefficient, by not decoding the secondary transform index, it is possible to save the amount of code required to transmit the secondary transform index, and as a result, the coding performance is improved. You can expect it.

Furthermore, when last_coeff_pos refers to the DC component of the target block, the only non-zero coefficient in the target block is the DC component. On the other hand, the quadratic transform is effective when non-zero linear transform coefficients are also present in high frequency components due to the nature of the quadratic transform, which crowds the non-zero linear transform coefficients. If the linear transform coefficients of are only DC components, it is not necessary to apply the quadratic transform.

Therefore, for example, when last_coeff_pos indicates a DC component in the target block, by determining not to decode the secondary transformation index, the amount of code required for transmitting the secondary transformation index can be saved, resulting in We can expect an improvement in encoding performance.

Next, when the cbf of the target block is 0, as mentioned above, there are no non-zero coefficients in the target block, so it is obvious that the quadratic transformation is not applied to the target block. By determining not to decode the secondary transformation index, the amount of code required for transmitting the secondary transformation index can be saved, and as a result, it is possible to expect an improvement in encoding performance.

In step S92, the image decoding device 200 decodes the secondary transformation index and ends this operation.

<Change example 3>
Modification example 3 will be described below, focusing on the differences from the third embodiment described above. Modification example 3 is configured to use last_coeff_pos and coded_sub_block_flag to determine whether or not secondary conversion index decoding is to be performed.

In the third embodiment described above, regarding whether or not the quadratic transform index of the target block is decoded, last_coeff_pos and coded_sub_block_flag indicate whether non-zero coefficients occur when the quadratic transform is applied. It was determined that the quadratic transformation index would not be decoded if the existence of non-zero coefficients was indicated in the restricted region.

On the other hand, in modification example 3, for example, when quadratic transformation is applied, if all high frequency components of the target block become zero (zeroed) to simplify processing, coded#sub#block# Since the determination based on the flag is clearly unnecessary, the presence or absence of decoding of the quadratic transformation may be determined only from last_coeff_pos.

<Change example 4>
Modification example 4 will be described below, focusing on the differences from the third embodiment described above.

In the third embodiment described above, an example has been described in which the decoding determination of the secondary transformation index is performed in units of encoded blocks.

On the other hand, in modification example 4, when an encoded block is divided into a plurality of transform blocks, such determination may be made for each transform block. According to this configuration, by determining the decoding of the secondary transform index on a transform block basis, the number of target blocks to which the secondary transform may be applied increases, and as a result, it is expected that encoding performance will improve. be able to.

Note that the image encoding device 100 and the image decoding device 200 described above may be implemented as programs that cause a computer to execute each function (each step).

In addition, in the above-mentioned embodiment, although the present invention was explained using the example of application to the image encoding device 100 and the image decoding device 200, the present invention is not limited to such an example, and the present invention is not limited to such an example. The present invention can be similarly applied to an image encoding/decoding system having each function of the image decoding device 100 and the image decoding device 200.

According to the present invention, in the conventional method, additional processing was required to count the number of non-zero coefficients in order to determine whether quadratic transform is applied when decoding coefficients, but the present invention has a configuration like the present invention. This eliminates the need for additional processing, and can be expected to speed up processing or reduce load.

Further, according to the present invention, it is possible to omit the decoding of the secondary transform index for blocks to which the secondary transform is not applied or for which it is determined that the effect is low, so that it is possible to expect improvement in encoding performance.

10...Image processing system 100...Image encoding device 111, 241...Inter prediction unit 112, 242...Intra prediction unit 121...Subtractor 122, 230...Adder 131...Conversion/quantization unit 131A...Primary conversion unit 131B...2 Next transformer 131C...quantizer 132, 220...inverse transform/inverse quantizer 220A...inverse quantizer 220B...inverse quadratic transformer 220C...inverse linear transformer 140...encoder 150, 250...in-loop filter Processing units 160, 260...Frame buffer 200...Image decoding device 210...Decoding unit

Claims (3)

An image decoding device,
an inverse transformer configured to generate a prediction residual signal by inverse transform;
The inverse transform unit determines whether to decode the secondary transform index of the encoded block based on whether the encoded block is an intra block and a flag indicating whether or not a non-zero coefficient occurs in the transformed block. An image decoding device characterized by being configured to make a determination. a step of generating a predicted residual signal by inverse transformation;
In the step, it is determined whether or not to decode the secondary transform index of the encoded block based on whether the encoded block is an intra block and a flag indicating whether or not a non-zero coefficient occurs in the transform block. An image decoding method characterized by: A program that causes a computer to function as an image decoding device,
The image decoding device includes an inverse transform unit configured to generate a prediction residual signal by inverse transform,
The inverse transform unit determines whether to decode the secondary transform index of the encoded block based on whether the encoded block is an intra block and a flag indicating whether or not a non-zero coefficient occurs in the transformed block. A program configured to make a determination.

Images