JP7049434B2 - Image decoder, image decoding method and program - Google Patents

Description

The present invention relates to an image decoding device, an image decoding method and a program.

Conventionally, RST (Reduced Secondary Transfer) (see Non-Patent Document 2) and LFNST (Low-Fre) that perform a secondary conversion on a coefficient after a primary conversion.
A technique called separable Non-Separable Transfer (see Non-Patent Document 1) is known.

Versatile Video Coding (Draft 5), JVET-N1001 CE6: Redened Secondary Transfer (RST) (CE6-3.1), JVET-N0193 Algorithm Description for Versatile Video Coding and Test Model 5 (VTM 5), JVET-N1002

However, in the above-mentioned conventional technique, when decoding a coefficient, a plurality of types of counters for counting the number of non-zero coefficients are provided, and whether or not the secondary transformation is applied is determined based on the value of the counter. Therefore, it is necessary to add a coefficient counting process only for determining whether or not the quadratic transformation is applied. The coefficient decoding process is a process that requires a high throughput, but there is a problem that the processing load increases by performing additional processing there.

Therefore, the present invention has been made in view of the above-mentioned problems, and an image decoding device, an image decoding method, and an image decoding device, which can omit additional processing for determining whether or not the secondary transformation is applied and can be expected to speed up the processing. The purpose is to provide a program.

The first feature of the present invention is an image decoding apparatus, which includes an inverse conversion unit configured to generate a predicted residual signal by inverse conversion, and the inverse conversion unit is a secondary conversion with respect to a target block. It is a gist that it is configured to determine whether or not the application of the above is applied and to control the decoding method of the conversion coefficient according to the determination result.

The second feature of the present invention is an image decoding apparatus, which includes an inverse conversion unit configured to generate a predicted residual signal by inverse conversion, and the inverse conversion unit depends on the size of the target block. The gist is that it is configured to control the decoding method of the secondary transformation index of the target block.

The third feature of the present invention is an image decoding apparatus, which includes an inverse conversion unit configured to generate a predicted residual signal by inverse conversion, and the inverse conversion unit is a non-zero coefficient of the target block. It is a gist that it is configured to control the decoding method of the secondary transformation index of the target block by the information indicating the occurrence position of.

In the third feature of the present invention, the information indicating the occurrence position of the non-zero coefficient of the target block is a coefficient at which the non-zero coefficient cannot occur when the quadratic transformation is applied to the target block. When pointing to a position, the inverse conversion unit may be configured not to decode the secondary conversion index of the target block.

In the third feature of the present invention, when the information indicating the generation position of the non-zero coefficient of the target block points to the coefficient position of the DC component of the target block, the inverse conversion unit is the target block. It may be configured not to decode the secondary transformation index.

A fourth feature of the present invention is an image decoding apparatus, which includes a reverse conversion unit configured to generate a predicted residual signal by reverse conversion, and the reverse conversion unit is a non-zero in a target block. The gist is that the flag indicating the presence or absence of the occurrence of the coefficient is configured to control the decoding method of the secondary transformation index of the target block.

The fifth feature of the present invention is an image decoding device, which includes a reverse conversion unit configured to generate a predicted residual signal by reverse conversion, and the reverse conversion unit is a subblock in a target block. It is gist that it is configured to control the decoding method of the secondary transformation index of the target block by the flag indicating the presence / absence of the non-zero coefficient in.

In the fifth feature of the present invention, the flag indicating the presence or absence of the non-zero coefficient in the sub-block may generate the non-zero coefficient when the secondary transformation is applied to the target block. When it is shown that the non-zero coefficient occurs in a subblock that does not exist, the inverse conversion unit may be configured not to decode the secondary conversion index of the target block.

The sixth feature of the present invention is an image decoding method, which includes a step of generating a predicted residual signal by inverse conversion, and in the step, it is determined whether or not a secondary transformation is applied to a target block, and the determination is made. The gist is to control the decoding method of the conversion coefficient according to the result.

A seventh feature of the present invention is a program that causes a computer to function as an image decoding device, and the image decoding device includes an inverse conversion unit configured to generate a predicted residual signal by inverse conversion. The gist is that the inverse conversion unit is configured to determine whether or not the secondary conversion is applied to the target block and control the decoding method of the conversion coefficient according to the determination result.

According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can omit additional processing for determining whether or not to apply a secondary transformation and can be expected to speed up the processing.

It is a figure which shows an example of the structure of the image processing system 1 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the image coding apparatus 100 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the conversion / quantization unit 131 of the image coding apparatus 100 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the image decoding apparatus 200 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the inverse transformation / inverse quantization unit 220 of the image decoding apparatus 200 which concerns on one Embodiment. It is a figure for demonstrating an example of the function of the inverse transformation / inverse quantization unit 220 of the image decoding apparatus 200 which concerns on one Embodiment. It is a figure for demonstrating an example of the function of the inverse transformation / inverse quantization unit 220 of the image decoding apparatus 200 which concerns on one Embodiment. It is a flowchart which shows an example of the secondary transformation application presence / absence determination method and the coefficient decoding method in the image decoding apparatus 200 which concerns on 1st Embodiment. It is a flowchart which shows an example of the secondary transformation application presence / absence determination method and the coefficient decoding method in the image decoding apparatus 200 which concerns on 1st Embodiment. It is a flowchart which shows an example of the secondary transformation application presence / absence determination method and the coefficient decoding method in the image decoding apparatus 200 which concerns on modification 1 of 1st Embodiment. It is a figure for demonstrating the 2nd Embodiment. It is a flowchart which shows an example of the secondary transformation application presence / absence determination method and the coefficient decoding method in the image decoding apparatus 200 which concerns on 3rd Embodiment. It is a flowchart which shows an example of the secondary transformation application presence / absence determination method and the coefficient decoding method in the image decoding apparatus 200 which concerns on 3rd Embodiment.

<First Embodiment>
Hereinafter, the image processing system 10 according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 9. FIG. 1 is a diagram showing an image processing system 10 according to the present embodiment.

As shown in FIG. 1, the image processing system 10 according to the present embodiment includes an image coding device 100 and an image decoding device 200.

The image coding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding the coded data.

The coded data may be transmitted from the image coding device 100 to the image decoding device 200 via a transmission line. The coded data may be stored in the storage medium and then provided from the image coding device 100 to the image decoding device 200.

(Image Coding Device 100)
Hereinafter, the image coding apparatus 100 according to the present embodiment will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of a functional block of the image coding apparatus 100 according to the present embodiment.

As shown in FIG. 2, the image coding apparatus 100 includes an inter-prediction unit 111, an intra-prediction unit 112, a subtractor 121, an adder 122, a conversion / quantization unit 131, and an inverse conversion / dequantization. It has a unit 132, a coding unit 140, an in-loop filter processing unit 150, and a frame buffer 160.

The inter-prediction unit 111 is configured to generate a prediction signal by inter-prediction (inter-frame prediction).

Specifically, the inter-prediction unit 111 identifies and identifies the reference block included in the reference frame by comparing the frame to be encoded (hereinafter referred to as the target frame) with the reference frame stored in the frame buffer 160. It is configured to determine the predicted motion vector for the referenced block.

Further, the inter-prediction unit 111 is configured to generate a prediction signal included in the prediction block for each prediction block based on the reference block and the motion vector. The inter-prediction unit 111 is configured to output a prediction signal to the subtractor 121 and the adder 122. Here, the reference frame is a frame different from the target frame.

The intra prediction unit 112 is configured to generate a prediction signal by intra prediction (in-frame prediction).

Specifically, the intra prediction unit 112 is configured to specify a reference block included in the target frame and generate a prediction signal for each prediction block based on the specified reference block. Further, the intra prediction unit 112 is configured to output the prediction signal to the subtractor 121 and the adder 122.

Here, the reference block is a block referred to with respect to the block to be predicted (hereinafter referred to as the target block). For example, the reference block is a block adjacent to the target block.

The subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the conversion / quantization unit 131. Here, the subtractor 121 is configured to generate a prediction residual signal which is a difference between the prediction signal generated by the intra prediction or the inter prediction and the input image signal.

The adder 122 adds a prediction signal to the prediction residual signal output from the inverse conversion / inverse quantization unit 132 to generate a pre-filter processing decoding signal, and the pre-filter processing decoding signal is used by the intra prediction unit 112 and the input. It is configured to output to the loop filter processing unit 150.

Here, the pre-filtered decoding signal constitutes a reference block used by the intra prediction unit 112.

The conversion / quantization unit 131 is configured to perform conversion processing of the predicted residual signal and acquire a coefficient level value. Further, the conversion / quantization unit 131 may be configured to quantize the coefficient level value.

Here, the conversion / quantization unit 131 is configured to output a quantization index when the coefficient level value is quantized. Hereinafter, the output of the conversion / quantization unit 131 will be described as a coefficient level value regardless of whether or not quantization is applied.

Here, the conversion process is a process of converting the predicted residual signal into a frequency component signal. In such a conversion process, a base pattern (transformation matrix) corresponding to a discrete cosine transform (DCT) may be used, or a base pattern (transformation matrix) corresponding to a discrete sine transform (DST). May be used.

The conversion process may be performed a plurality of times before the quantization is performed. As an example, a secondary conversion that performs the second conversion process will be described later.

The inverse conversion / inverse quantization unit 132 is configured to perform an inverse conversion process of the coefficient level value output from the conversion / quantization unit 131. Here, the inverse transformation / inverse quantization unit 132 may be configured to perform inverse quantization of the coefficient level value prior to the inverse transformation process.

Here, the inverse conversion process and the inverse quantization are performed in the reverse procedure of the conversion process and the quantization performed by the conversion / quantization unit 131.

The coding unit 140 is configured to encode the coefficient level value output from the conversion / quantization unit 131 and output the coded data.

Here, for example, coding is entropy coding that counts the coefficients of the target block (coded block or converted block) and assigns codes of different lengths based on the probability of occurrence of the coefficient level value. The coefficient counting method will be described later.

Further, the coding unit 140 is configured to encode the control data used in the decoding process in addition to the coefficient level value.

Here, the control data may include size data such as a coding block (CU: Coding Unit) size, a prediction block (PU: Precision Unit) size, and a conversion block (TU: Transfer Unit) size.

The in-loop filter processing unit 150 is configured to perform filter processing on the pre-filter processing decoding signal output from the adder 122 and output the post-filter processing decoding signal to the frame buffer 160.

Here, for example, the filtering process is a deblocking filtering process that reduces the distortion generated at the boundary portion of the block (encoded block, prediction block, or conversion block).

The frame buffer 160 is configured to store reference frames used by the inter-prediction unit 111.

Here, the filtered decoded signal constitutes a reference frame used by the inter-prediction unit 111.

(Conversion / Quantization Unit 131)
Hereinafter, the conversion / quantization unit 131 of the image coding apparatus 110 according to the present embodiment will be described with reference to FIG. FIG. 3 is a diagram showing an example of a functional block of the conversion / quantization unit 131 of the image coding device 110 according to the present embodiment.

As shown in FIG. 13, the conversion / quantization unit 131 includes a primary conversion unit 131A, a secondary conversion unit 131B, and a quantization unit 131C.

The conversion / quantization unit 131 is an example of a conversion / quantization unit configured to generate a coefficient level value from a predicted residual signal by conversion / quantization.

The primary conversion unit 131A is configured to generate the primary conversion coefficient of the target block by using the predicted residual signal as an input.

Here, the basis pattern (transformation matrix) used in the linear transformation process may be selected from a plurality of basis patterns. For example, in Non-Patent Document 1, the base patterns corresponding to DCT2, DCT8, and DST7 are used, respectively. Further, as a method of selecting a base pattern, for example, there is a method in which the one having the lowest coding cost is selected by the image coding device 100 and the selected base pattern is transmitted to the image decoding device 200 as side information.

The secondary conversion unit 131B is configured to receive the primary conversion coefficient from the primary conversion unit 131A as an input and determine whether or not the secondary conversion is applied to the target block.

Here, when the secondary conversion unit 131B determines that the secondary conversion is applied, the secondary conversion coefficient is generated by performing a conversion process on the primary conversion coefficient, and the secondary conversion coefficient is converted into the quantization unit 131C. It is configured to output to.

On the other hand, the secondary conversion unit 131B is configured to output the primary conversion coefficient to the quantization unit 131C as it is when it is determined that the secondary conversion is not applied.

Further, the secondary conversion unit 131B is configured to output information regarding whether or not the secondary conversion is applied (hereinafter, secondary conversion index) to the coding unit 140.

The quantization unit 131C is configured to input the conversion coefficient (first-order conversion coefficient or second-order conversion coefficient) and determine whether or not quantization is applied to the conversion coefficient of the target block.

Here, the quantization unit 131C is configured to generate a quantization index when it is determined that the quantization is applied. As a method for determining whether or not quantization is applied and a method for generating a quantized index, known methods can be used, and details thereof will be omitted.

(Encoding unit 140)
When the quadratic transformation is applied, the coding unit 140 may be configured to encode the coefficient level value using a context optimized for the application probability of the quadratic transformation.

(Inverse transformation / inverse quantization unit 132)
Since the inverse conversion / inverse quantization unit 132 operates in the same manner as the inverse conversion / inverse quantization unit 220 in the image decoding device 200 described later, the details will be described later in the description of the inverse conversion / inverse quantization unit 220. Is treated in the same way as the inverse conversion / inverse quantization unit 220.

(Image Decoding Device 200)
Hereinafter, the image decoding apparatus 200 according to the present embodiment will be described with reference to FIG. FIG. 4 is a diagram showing an example of a functional block of the image decoding apparatus 200 according to the present embodiment.

As shown in FIG. 4, the image decoding device 200 includes a decoding unit 210, an inverse conversion / inverse quantization unit 220, an adder 230, an inter-prediction unit 241 and an intra-prediction unit 242, and an in-loop filter processing unit. It has 250 and a frame buffer 260.

The decoding unit 210 is configured to decode the coded data generated by the image coding device 100 and decode the coefficient level value.

Here, decoding is, for example, decoding of entropy-encoded data to which codes of different lengths are assigned based on the probability of occurrence of coefficients.
Decoding is entropy decoding in the reverse procedure of entropy coding performed by the coding unit 140.

Further, the decoding unit 210 may be configured to acquire control data by decoding processing of encoded data.

As described above, the control data may include size data such as a coded block size, a predicted block size, and a conversion block size.

The inverse conversion / inverse quantization unit 220 generates a predicted residual signal by performing inverse quantization of the quantization index output from the decoding unit 210 and inverse conversion processing of the coefficient level value output from the decoding unit 210. It is configured to do.

The adder 230 adds a prediction signal to the prediction residual signal output from the inverse conversion / inverse quantization unit 220 to generate a pre-filter processing decoding signal, and the pre-filter processing decoding signal is used by the intra prediction unit 242 and the in-loop. It is configured to output to the filter processing unit 250.

Here, the pre-filtered decoding signal constitutes a reference block used by the intra prediction unit 242.

Similar to the inter-prediction unit 111, the inter-prediction unit 241 is configured to generate a prediction signal by inter-prediction (inter-frame prediction).

Specifically, the inter-prediction unit 241 is configured to generate a prediction signal for each prediction block based on the motion vector decoded from the coded data and the reference signal included in the reference frame. The inter-prediction unit 241 is configured to output a prediction signal to the adder 230.

Like the intra prediction unit 112, the intra prediction unit 242 is configured to generate a prediction signal by intra prediction (in-frame prediction).

Specifically, the intra prediction unit 242 is configured to specify a reference block included in the target frame and generate a prediction signal for each prediction block based on the specified reference block. The intra prediction unit 242 is configured to output a prediction signal to the adder 230.

Similar to the in-loop filter processing unit 150, the in-loop filter processing unit 250 performs filter processing on the pre-filter processing decoded signal output from the adder 230, and outputs the post-filter processing decoded signal to the frame buffer 260. It is configured to do.

Here, for example, the filtering process is a deblocking filter process that reduces the distortion that occurs at the boundary portion of a block (encoded block, prediction block, conversion block, or sub-block that divides them).

Like the frame buffer 160, the frame buffer 260 is configured to store reference frames used by the inter-prediction unit 241.

Here, the filtered decoded signal constitutes a reference frame used by the inter-prediction unit 241.

(Inverse conversion / inverse quantization unit 220)
Hereinafter, the inverse conversion / inverse quantization unit 220 of the image decoding apparatus 200 according to the present embodiment will be described with reference to FIG. FIG. 5 is a diagram showing an example of a functional block of the inverse transformation / inverse quantization unit 220 of the image decoding apparatus 200 according to the present embodiment.

As shown in FIG. 5, the inverse conversion / inverse quantization unit 220 includes an inverse quantization unit 242A, an inverse secondary conversion unit 242B, and an inverse primary conversion unit 242C.

The inverse quantization unit 220A is configured to generate a conversion coefficient of the target block using the quantization index as an input signal. As a method for generating the coefficient level value from the quantized index, a known method can be used, and thus details will be omitted.

The inverse quadratic transformation unit 220B is configured to determine whether or not the inverse quadratic transformation of the target block is applied by using the coefficient level value as an input.

Here, when the inverse secondary transformation unit 220B determines that the inverse secondary transformation is applied, the inverse secondary transformation unit 220B generates a primary transformation coefficient by performing an inverse transformation process on the input coefficient level value, and the linear transformation is performed. The coefficient is configured to be output to the linear conversion coefficient unit 220C.

On the other hand, when it is determined that the secondary conversion is not applied, the inverse secondary conversion unit 220B is configured to output the input coefficient level value as the primary conversion coefficient to the primary conversion coefficient unit 220C as it is.

Hereinafter, for the sake of simplification of the description, the secondary transformation and the inverse secondary transformation are collectively referred to as the secondary transformation, and the secondary transformation in the inverse secondary transformation unit is appropriately read as the inverse secondary transformation.

The method of determining whether or not the secondary conversion is applied by the inverse secondary conversion unit 220B will be described later.

The inverse primary conversion unit 220C is configured to generate a predicted residual signal of the target block by using the primary conversion coefficient as an input.

Here, the base pattern (transformation matrix) used for the inverse linear transformation process may be selected from a plurality of defined patterns as in the case of the image coding apparatus 100. The selection of the base pattern may be performed using, for example, the side information (base pattern information selected by the image coding device 100) transmitted from the image coding device 100.

(Explanation of secondary transformation)
Hereinafter, an example of the secondary transformation performed by the image coding device 100 and the image decoding device 200 according to the present embodiment will be described with reference to FIG.

As described above, the secondary transformation is a technique for converting the linear transformation coefficient of the target block again before quantization. By converting the first-order conversion coefficient again before quantization, the coefficient with a non-zero coefficient level value (hereinafter referred to as the non-zero coefficient) becomes more likely to be concentrated in the low frequency region, and the coefficient level in the coefficient scan process described later. Since the coefficient with a value of zero (hereinafter referred to as the zero coefficient) becomes continuous, further coding improvement effect by entropy coding can be expected.

Here, the secondary transformation may use a plurality of basis patterns (transformation matrices) as in the linear transformation. For example, as in Non-Patent Document 2, a base pattern that can be used in advance may be set according to the target block size, the intra prediction mode, and the secondary conversion index. If the value of the secondary transformation index is 0, it indicates that the secondary transformation is invalid. On the other hand, when the value is 1 or more, it indicates that the secondary transformation is effective. Further, two types of base patterns may be adaptively selected by the secondary conversion index, for example, as in Non-Patent Document 2.

Further, the applicable range of the secondary transformation may be adaptively set according to the block size of the target block. For example, FIG. 6 shows an example of the scope of application of the secondary transformation in Non-Patent Document 2.

As shown in FIG. 6A, when the block size of the target block is 8 × 8 pixels, the applicable range R1 of the secondary conversion covers all 64 primary conversion coefficients, and further, the secondary conversion is performed. The later non-zero coefficient generation region R2 is limited to the low frequency region (4 × 2 pixels) of the secondary conversion coefficient, and the other regions are all set to the zero coefficient (hereinafter, zeroing).

On the other hand, as shown in FIG. 6B, when the block size of the target block is M × N (however, M> 8, N> 8) pixels, the applicable range R1 of the secondary transformation is 48 low. It is the first-order conversion coefficient in the frequency region. That is, the second-order conversion is not applied to the first-order conversion coefficients other than the 48 low-frequency regions, and the first-order conversion coefficients are treated as they are. Further, the region where the non-zero coefficient is generated after the secondary conversion is limited to the low frequency region (4 × 4 pixels) of the secondary conversion coefficient, and the other regions are zeroed.

(Coefficient coding method)
Hereinafter, a method for coding the coefficient level value will be described with reference to FIG. 7. FIG. 7 is a diagram showing a method of scanning the coefficient level value of the target block.

As a coefficient scanning method, for example, as described in Non-Patent Document 1, a method of scanning from the coefficient C1 of the highest frequency component of the target block to the coefficient Cx of the lowest frequency component in the upper left direction (hereinafter, diagonal scan) is used. It may be used (see FIG. 7).

In natural images, non-zero coefficients are unlikely to occur in the high frequency region. Therefore, by diagonally scanning from the high frequency component to the low frequency component, the zero coefficients can be arranged continuously, and the effect of improving the coding performance by entropy coding can be expected.

Further, the execution unit of the oblique scan may be a sub-block unit in which the target block is divided into 4 × 4 pixels as in Non-Patent Document 1. Further, the image coding device 100 may transmit a flag (code_sub_block_flag) indicating the presence or absence of the non-zero coefficient to the image decoding device 200 according to the number of occurrences of the non-zero coefficient in the subblock unit.

Here, the reason for using such a flag (coded_sub_block_flag) is
For example, a 1-bit flag (coded_sub_block_flag) can be used to encode and decode subblocks with no nonzero coefficients.

Further, the image encoding device 100 has such a flag (coded_sub_block_fl).
Prior to the coding of ag), a determination process may be performed on whether or not the non-zero coefficient is generated from the subblock in which the non-zero coefficient is first generated in the scan order.

Here, the image coding apparatus 100 first generates position information (last_) in which the non-zero coefficient is generated.
coeff_pos) may be transmitted to the image decoding device 200. As a result, since it is certain that the subblock before the occurrence of the first non-zero coefficient has no non-zero coefficient, the transmission of the above-mentioned flag (coded_sub_block_flag) becomes unnecessary, and the code amount can be saved.

The image coding apparatus 100 divides the x-coordinate value and the y-coordinate value of the position information in which the non-zero coefficient first occurs in the target block into a prefix unit and a suffix unit, as in Non-Patent Document 1, for example. And may be encoded.

(Method of determining whether secondary transformation is applied and how to decode the coefficient)
8 and 9 are flowcharts showing an example of a method for determining whether or not a secondary transformation is applied and a method for decoding a coefficient in the image decoding apparatus 200. Hereinafter, an example of a method for determining whether or not a secondary transformation is applied will be described with reference to FIG. 8, and an example of a coefficient decoding method will be described with reference to FIG. 9.

As shown in FIG. 8, in step S61, the image decoding device 200 decodes a flag (cbf) indicating the presence or absence of the non-zero coefficient of the target block. Here, the value of cbf is either 0 or 1.

In step S62, the image decoding device 200 determines whether or not a nonzero coefficient exists based on cbf.

When it is determined that there is no non-zero coefficient (that is, when it is determined that the non-zero coefficient is 0) (that is, when it is determined that cbf = 0), this process ends.

On the other hand, when it is determined that the non-zero coefficient exists (that is, when it is determined that the non-zero coefficient is one or more) (that is, when it is determined that cbf = 1), this process proceeds to step S63.

Here, cbf may be decoded as different flags for each of the luminance signal (Y signal) and the color difference signal (Cb signal and Cr signal). In this case, the image decoding device 200 may determine as follows when the block division tree structure of the luminance signal and the color difference signal of the target block is different (Dual Tree) and when both are the same (Single Tree). ..

In the Y signal at the time of Dual Tree, the image decoding apparatus 200 may proceed to step S63 when cbf = 1 of the Y signal, and may determine that the present process is terminated otherwise.

The image decoding device 200 determines that if the Cb / Cr signal at the time of Dual Tree and at least one of the Cb / Cr signals has a cbf of 1, the process proceeds to step S63, and if not, the process is terminated. You may.

The image decoding device 200 determines that if at least one of the Y signal, the Cb signal, and the Cr signal at the time of Single Tree has a cbf of 1, the process proceeds to step S63, and if not, the process is terminated. May be good.

In step S63, the image decoding device 200 decodes the secondary conversion index (lfnst_idx).

In step S64, the image decoding device 200 decodes information (last_coeff_pos) indicating the position where the first non-zero coefficient occurs in the target block in the scanning order.

In step S65, the image decoding device 200 performs a coefficient decoding process (see FIG. 9). The image decoding device 200 performs decoding processing of such a coefficient in sub-block units obtained by dividing the above-mentioned target block.

As shown in FIG. 9, in step S641, the image decoding apparatus 200 determines whether or not the subblock in which the non-zero coefficient first occurs is the first or last subblock in the scan order in the target block. do.

If it is determined to be either the first or last subblock, it is obvious that such a subblock has a non-zero coefficient, so this process proceeds to step S645.

If it is determined that it is neither the first nor the last subblock, in step S642, the image decoding apparatus 200 determines whether or not the secondary transformation is applied in such a subblock based on the secondary transformation index (st_idx). Judgment about.

It is obvious that when it is determined that the quadratic transformation is applied (the quadratic transformation is valid) in such a subblock, and the total coefficient is zero due to the quadratic transformation in such a subblock. If (zeroed), coded_sub_block_fl
Since there is no need to decode the ag, this process proceeds to step S646.

On the other hand, if it is determined that the quadratic transformation is not applied in such a subblock (the quadratic transformation is not valid), or if the quadratic transformation is applied in such a subblock, a non-zero coefficient occurs. If the possibility remains, this process proceeds to step S643.

The order of steps S641 and S642 may be interchanged.

In step S643, the image decoding device 200 decodes a flag (coded_sub_block_flag) indicating the presence or absence of the non-zero coefficient.

In step S644, the image decoder 200 has coded_sub_block_.
Based on the flag, it is determined whether or not there is a non-zero coefficient in such a subblock.

When the value of coded_sub_block_flag is 0, the image decoding apparatus 200 determines that the non-zero coefficient does not exist, and the process proceeds to step S646.

On the other hand, when the value of coded_sub_block_flag is 1, the image decoding apparatus 200 determines that the non-zero coefficient exists, and the process proceeds to step S645.

In step S645, the image decoding device 200 decodes the level value of each coefficient in the subblock.

In step S646, the image decoding device 200 determines whether or not the subblock to be processed is the final block.

If it is determined that it is not the final block, the present process proceeds to step S641, moves to the next subblock in the scan order, restarts the present process, and if it is determined to be the final block, the present process is performed. finish.

The following two effects can be expected by using the above-mentioned method for determining whether or not the inverse secondary transformation is applied and the method for decoding the coefficient.

The first effect is that the secondary conversion index is performed before the decoding process of the coefficient of the target block, so that the decoding process of the coefficient required for determining whether or not the secondary conversion is applied in the prior art can be omitted. It means that you can do it.

Specifically, in the prior art, two counters (a counter for measuring the number of non-zero coefficients generated in the target block and a region where the zero coefficient is obvious when the secondary conversion is enabled are used to determine whether or not the secondary conversion is applied. It was necessary to use a non-zero coefficient generation counter), but any of them can be omitted by introducing the technique according to the present embodiment.

The second effect is that when the secondary transformation is effective, in the prior art, it is necessary to redundantly decode or encode the coded_sub_block_flag for the subblock whose all coefficients are obvious to be zero in the secondary transformation target block. .. On the other hand, if the technique according to the present embodiment is used, such processing can be omitted, so that improvement in coding performance can be expected.

<Change example 1>
Hereinafter, modification 1 of the above-mentioned first embodiment will be described with reference to FIG. 10, focusing on the differences from the above-mentioned first embodiment.

In the flowchart shown in FIG. 10, step S66, which did not exist in the flowchart shown in FIG. 8, is added before step S63.

Here, if the secondary transformation is clearly not valid, the transmission of the secondary transformation index becomes redundant, so that step S66 is added in FIG.

For example, when the application target of the secondary transformation is limited to the intra picture or the intra block, a determination as to whether or not the target block is an intra picture or an intra block may be added to a predetermined condition.

Further, when one coding block is divided into a plurality of prediction blocks or conversion blocks, the number of conversion blocks to which the secondary conversion is applied increases, so that the coding performance can be expected to be improved, but the coding is performed. It is conceivable that the delay in processing and decoding processing will increase.

Therefore, for example, when the intra prediction by dividing the coded block described in Non-Patent Document 1 into a plurality of pieces (intra subdivision prediction) is valid under predetermined conditions, the secondary conversion is treated as invalid, which is an unintended process. The increase in delay can be suppressed. Therefore, a determination as to whether or not the intra-subdivision prediction is applied to the target block may be added to the predetermined conditions.

<Change example 2>
Hereinafter, the modified example 2 of the first embodiment described above will be described with a focus on the differences from the first embodiment and the modified example 1 described above.

In the above-mentioned first embodiment, as shown in FIG. 8, an example in which the decoding of the secondary transformation index is performed in units of coded blocks has been described.

In the second modification, when the coded block is divided into a plurality of conversion blocks, it may be configured to decode the secondary conversion index in units of conversion blocks.

According to such a configuration, since the secondary conversion index can be decoded in units of different conversion blocks, the number of blocks to which the secondary conversion is applied can be increased, and improvement in coding performance can be expected.

<Second Embodiment>
Hereinafter, the second embodiment of the present invention will be described with reference to FIG. 11, focusing on the differences from the first embodiment described above.

In the present embodiment, as shown in FIG. 11, it is configured to control the decoding method of the position information (last_coeff_pos) in which the non-zero coefficient is generated at the beginning of the target block depending on whether or not the secondary transformation is applied.

This embodiment is different from the above-mentioned first embodiment 1 in terms of the method of decoding last_coeff_pos (see FIG. 8). Since this embodiment has the same configuration as that of the first embodiment described above for other parts, the description thereof will be omitted.

For example, in Non-Patent Document 2, when the block for which the secondary transformation is effective and the target block size is 4x4 or 8x8, the position and the number of non-zero coefficients with respect to the target region of the secondary transformation are set as the upper left subblocks, respectively. And it was limited to eight.

At this time, the position where the non-zero coefficient is generated can be specified as shown in FIG. 11 in consideration of the scan order of the subblocks.

In other words, if the secondary transformation is enabled, the last_coeff_pos cannot indicate the scan order 8-15 as shown in FIG.

Therefore, last_coeff_pos is usually decoded by the prefix and suffix of the x-coordinate value and the y-coordinate value, but when the quadratic conversion is effective and the non-zero coefficient generation position and the number of the target block are limited, the last_coeff_pos is usually decoded. The decoding method of last_coordinate_pos may be modified as follows.

For example, last_coeff_pos can be transmitted with a 3-bit index in the order of coefficient scanning, with a range of 0 to 7 where a non-zero coefficient may occur.

In the prior art, since the x-coordinate value and the y-coordinate value are each represented by 2 bits for each subblock, the code amount can be saved by that amount, and as a result, the effect of improving the coding performance can be expected.

<Third Embodiment>
Hereinafter, the third embodiment of the present invention will be described with reference to FIGS. 12 and 13, focusing on the differences from the above-mentioned first embodiment and the second embodiment.

This embodiment is different from the first embodiment 1 in terms of the method for determining whether or not the secondary transformation is applied and the method for decoding the coefficients (see FIG. 8). Since this embodiment has the same configuration as that of the first embodiment described above for other parts, the description thereof will be omitted.

12 and 13 are flowcharts showing an example of a method for determining whether or not a secondary transformation is applied and a method for decoding a coefficient in the image decoding apparatus 200 according to the present embodiment. Hereinafter, an example of the operation of the image decoding apparatus 200 according to the present embodiment will be described with reference to such a flowchart.

As shown in FIG. 12, in step S61, the image decoding device 200 decodes a flag (cbf) indicating the presence or absence of the non-zero coefficient of the target block, similarly to step S61 shown in FIG.

In step S62, the image decoding device 200 determines whether or not a non-zero coefficient exists based on cbf, as in step S62 shown in FIG.

When it is determined that the non-zero coefficient does not exist (that is, when it is determined that the non-zero coefficient is 0) (that is, when it is determined that cbf = 0), this process proceeds to step S91.

On the other hand, when it is determined that the non-zero coefficient exists (that is, when it is determined that the non-zero coefficient is one or more) (that is, when it is determined that cbf = 1), this process proceeds to step S64.

In step S64, the image decoding device 200 decodes the information (last_coeff_pos) indicating the position where the first non-zero coefficient occurs in the target block in the scanning order, similarly to step S64 shown in FIG.

In step S93, the image decoding device 200 performs the decoding process of the coefficient shown in FIG. The coefficient decoding process shown in FIG. 13 is the same as the coefficient decoding process shown in FIG. 9, except that step S642 shown in FIG. 9 is not included.

In step S91, the image decoding device 200 determines whether or not the predetermined conditions described later are satisfied. If it is determined that the predetermined condition is satisfied, the operation proceeds to step S92, and if it is determined that the predetermined condition is not satisfied, the operation ends.

Here, the predetermined condition may be a condition of whether or not the secondary conversion index is decoded according to the target block size.

For example, when the image decoding device 200 determines in step S91 that at least one of the widths or heights of the target coded blocks is larger than 64, the image decoding device 200 determines in step S92 that the secondary conversion index is not decoded. May be good. The effects of this determination are as follows.

In Non-Patent Document 3, a tool for maintaining a pipeline processing unit of a decoder called a virtual pipeline data unit (VPDU: Visual Pipeline Data Units) is adopted. Here, in Non-Patent Document 3, the VPDU size is set to 64x64 pixels, and the maximum value of the width and height of the conversion block is set to 64 pixels.

Therefore, if either the width or the height of the target coded block is larger than 64 pixels, the conversion block is divided so that the width and height of the conversion block are 64 pixels or less.

For example, when the coded block size is 128 × 128 pixels, the coded block includes four conversion blocks of 64 × 64 pixels. Since the maximum size of the conversion block and the size of the VPDU are specified to be the same size, the pipeline processing specified by the VPDU can be maintained by performing the processing for each conversion block.

When a plurality of conversion blocks are included in the coding block, the image decoding apparatus 200 performs step S91 after each of steps S61, S62, S64 and S93 shown in FIG. 12 is performed for all the conversion blocks. ..

Therefore, when at least one of the width or height of the coded block is larger than 64 pixels, it is necessary to wait for the processing of all the VPDUs constituting the target block to be completed in order to perform the decoding control of the secondary conversion index. Therefore, the pipeline processing of the decoder is delayed.

Therefore, if the decoding of the secondary conversion index is restricted as described above, the effect of avoiding this processing delay can be expected.

In addition, the predetermined condition is the occurrence position information (last_coeff_pos) of the non-zero coefficient or the flag (coded_sub_block_) indicating whether or not the non-zero coefficient of the target subblock is generated.
It may be a condition of presence / absence of decoding of the secondary transformation index determined by using flag).

For example, the predetermined condition is not when the region where the non-zero coefficient is generated when the secondary transformation is applied is limited, such as when the target block shown in Non-Patent Document 2 is 4 × 4 pixels or 8 × 8 pixels. It may include the condition that the last_coeff_pos or coded_sub_block_flag does not indicate the existence of the non-zero coefficient for the region where the zero coefficient cannot occur.

For example, the predetermined condition may include the condition that the cbf of the target block is not zero or that the last_coeff_pos of the target block does not point to the DC component (DC component).

The determination based on last_coeff_pos and cbf is based on the case where the block division tree structure of the luminance signal (Y signal) and the color difference signal (Cb signal and Cr signal) of the target block is different (Dual Tree) and when both are the same (Single Tree). Then, it may be determined as follows.

When it is a Y signal at the time of Dual Tree and the last_coeff_pos is a DC component or cbf is 0, the image decoding apparatus 200 does not decode the secondary conversion index.

Last_ for both Cb / Cr signal and Cb / Cr signal at the time of Dual Tree
When coeff_pos is a DC component or cbf is 0, the image decoding device 200 does not decode the secondary conversion index.

Last_ for all Y signal, Cb signal and Cr signal at the time of Single Tree
When coeff_pos is a DC component or cbf is 0, the image decoding device 200 does not decode the secondary conversion index.

Alternatively, at the time of Single Tree, if the last_coeff_pos of the Y signal is a DC component or cbf is 0, the image decoding apparatus 200 may not decode the secondary conversion index.

According to such a configuration, it is possible to determine whether or not the secondary conversion index is decoded without depending on the values of cbf and last_coeff_pos of the Cb signal and the Cr signal.

In the above determination, if the secondary transformation index is not decoded, it is implicitly treated as having a secondary transformation index of 0 (that is, it is determined that the secondary transformation is not applied).

As described above, when the region where the non-zero coefficient is generated when the quadratic transformation is applied is limited, the region where the non-zero coefficient cannot occur is the last_coeff_pos or coded_s.
If ub_block_flag indicates the presence of a nonzero coefficient, it is clear that no quadratic transformation is applied to the target block.

Therefore, for example, in the determination of a predetermined condition, when the target block has 4 × 4 pixels or 8 × 8 pixels, the last_coeff_pos or coded_sub_block_flag indicates the existence of the non-zero coefficient in the region where the non-zero coefficient cannot occur. In this case, by not decoding the secondary conversion index, the amount of code required for transmission of the secondary conversion index can be saved, and as a result, the effect of improving the coding performance can be expected.

Further, when last_coeff_pos points to the DC component of the target block, the non-zero coefficient in the target block is only the DC component. On the other hand, the second-order transformation is effective when the non-zero first-order transformation also exists in the high-frequency component due to the nature that the second-order transformation concentrates the non-zero first-order conversion coefficients, so that it is non-zero. When the primary conversion coefficient is only the DC component, the secondary conversion may not be applied.

Therefore, for example, when last_coeff_pos points to the DC component in the target block, it is determined that the secondary conversion index is not decoded, so that the code amount required for transmission of the secondary conversion index can be saved, and as a result, the coding performance can be saved. Can be expected to improve.

Next, when the cbf of the target block is 0, as described above, since there is no non-zero coefficient in the target block, it is obvious that the quadratic transformation is not applied to the target block. By determining that the secondary conversion index is not decoded, the amount of code required for transmission of the secondary conversion index can be saved, and as a result, the effect of improving the coding performance can be expected.

In step S92, the image decoding device 200 decodes the secondary conversion index and ends this operation.

<Change example 3>
Hereinafter, the third modification will be described with a focus on the differences from the third embodiment described above. In the third modification, the last_coeff_pos and coded_sub_block_flag are used to determine whether or not the secondary conversion index is decoded.

In the third embodiment described above, regarding the presence or absence of decoding of the secondary transformation index of the target block, the last_coeff_pos and coded_sub_block_flag are non-zero coefficients for the region where the generation of the non-zero coefficient is restricted when the secondary transformation is applied. When indicating the existence of, it was determined that the secondary transformation index would not be decoded.

On the other hand, in modification 3, for example, when a secondary transformation is applied and the high frequency components of the target block are all zeroed (zeroed) for processing simplification, coded_s
Judgment by ub_block_flag is obviously unnecessary, so last_coef
It may be determined whether or not the secondary transformation is decoded only from f_pos.

<Change example 4>
Hereinafter, the fourth modification will be described with a focus on the differences from the third embodiment described above.

In Example 3 described above, an example in which the decoding determination of the secondary transformation index is performed in units of coded blocks has been described.

On the other hand, in the modification example 4, when the coded block is divided into a plurality of conversion blocks, such determination may be performed for each conversion block. According to this configuration, by determining the decoding of the secondary transformation index in units of transformation blocks, the number of target blocks to which the secondary transformation can be applied increases, and as a result, the coding performance is expected to improve. be able to.

The image coding device 100 and the image decoding device 200 described above may be realized by a program that causes a computer to execute each function (each step).

In the above-described embodiment, the present invention has been described by using the application to the image coding device 100 and the image decoding device 200 as an example, but the present invention is not limited to such an example, and the image coding device is not limited to this. It can be similarly applied to an image coding / decoding system having each function of the 100 and the image decoding device 200.

According to the present invention, in the conventional method, an additional process of counting the number of non-zero coefficients is required in order to determine whether or not the secondary transformation is applied at the time of decoding the coefficients. This eliminates the need for additional processing, and can be expected to speed up processing or reduce the load.

Further, according to the present invention, the decoding of the secondary conversion index can be omitted for the block to which the secondary conversion is not applied or the effect can be determined to be low, so that the coding performance can be expected to be improved.

10 ... Image processing system 100 ... Image coding device 111, 241 ... Inter prediction unit 112, 242 ... Intra prediction unit 121 ... Subtractor 122, 230 ... Adder 131 ... Conversion / quantization unit 131A ... Primary conversion unit 131B ... Second Next conversion unit 131C ... Quantization unit 132, 220 ... Inverse conversion / inverse quantization unit 220A ... Inverse quantization unit 220B ... Inverse secondary conversion unit 220C ... Inverse primary conversion unit 140 ... Coding unit 150, 250 ... In-loop filter Processing unit 160, 260 ... Frame buffer 200 ... Image decoding device 210 ... Decoding unit

Claims (3)

It is an image decoder
It has an inverse converter configured to generate a predicted residual signal by inverse transformation.
The inverse conversion unit is configured not to decode the secondary conversion index of the coded block when at least one of the width or height of the coded block is larger than the maximum value of the width and height of the coded block. Ori,
The image decoding device is characterized in that the secondary transformation index indicates whether or not the secondary transformation is valid for the coded block, and sets a usable transformation matrix . It has a step of generating a predicted residual signal by inverse conversion, and has a process of generating a predicted residual signal.
In the step, if at least one of the width or height of the coded block is greater than the maximum width and height of the converted block, the secondary conversion index of the coded block is not decoded.
The image decoding method , wherein the secondary transformation index indicates whether or not the secondary transformation is valid for the coded block, and sets a usable transformation matrix . A program that makes a computer function as an image decoder
The image decoding device includes an inverse conversion unit configured to generate a predicted residual signal by inverse conversion.
The inverse conversion unit is configured not to decode the secondary conversion index of the coded block when at least one of the width or height of the coded block is larger than the maximum value of the width and height of the coded block. Ori,
The quadratic transformation index is a program characterized by indicating whether or not a quadratic transformation is valid for the coded block and setting a usable transformation matrix .

Images