JP7597880B2 - Decoding device and program - Google Patents

Description

The present invention relates to a decoding device and a program.

Video coding methods such as H.265/HEVC (High Efficiency Video Coding) are configured to generate residual signals by switching between two types of prediction: inter-prediction, which uses temporal correlation between frames, and intra-prediction, which uses spatial correlation within a frame, and then output the resulting stream by performing orthogonal transform processing, loop filter processing, and entropy coding processing.

HEVC provides 35 different intra prediction modes, including planar prediction, DC prediction, and directional prediction, and is configured to perform intra prediction using adjacent decoded reference pixels according to the mode determined by the encoder.

Here, in intra prediction, in a CU (Coding Unit) that has no adjacent decoded reference pixels, such as the top leftmost CU in a frame, the reference pixels used when generating a predicted image are created by filling in a specified value (for a 10-bit video image, this is "512").

In addition, in conventional HEVC, the encoding process is performed in raster scan order from the top left, so there are cases where the reference pixels have not been decoded. In such cases, the system is configured to generate a predicted image using a value obtained by zero-order extrapolation of the closest decoded reference pixel.

A CU is divided into multiple blocks (TUs: Transform Units) and undergoes orthogonal transform processing. In HEVC, the intra prediction mode, which indicates the type of intra prediction to be applied, is common within a CU, and predictions are performed using the same intra prediction mode for multiple TUs.

As shown in Figure 7, in intra prediction, due to the encoding process in raster scan order, the reference pixel N located at the bottom left or top right of CU#X is often not yet decoded. In such cases, if directional prediction is performed from the direction in which the undecoded reference pixel exists, the prediction accuracy decreases, resulting in a problem of reduced encoding efficiency.

The following describes these problems in detail with reference to Figures 8(a) to 8(d). Figure 8 shows an example of intra prediction in conventional HEVC.

In this example, as shown in FIG. 8(a), similar to the example in FIG. 7, all of the reference pixels for CU#A1 (the CU in the top left of the frame) have been decoded. Similarly, as shown in FIG. 8(c), all of the reference pixels for CU#A3 (the CU in the bottom left of the frame) have been decoded.

In contrast, as shown in FIG. 8(b), reference pixels W1 to W3 located in CU#A1 have already been decoded, but reference pixels B1 to B4 located in CU#A3 have not been decoded, and therefore cannot be used as reference pixels when generating a predicted image for CU#A2 (the top right CU in the frame).

For this reason, conventional HEVC specifies that the value of reference pixel W1, which is located at the bottom of decoded reference pixels W1 to W3 in CU#A1, is copied to undecoded reference pixels B1 to B4 located in the same column in CU#A3, as shown in Figure 8(b).

Similarly, in conventional HEVC, as shown in FIG. 8(d), it is specified that the value of reference pixel B1, which is located at the bottom of the decoded reference pixels B1 to B3 located in CU#A3, is copied to undecoded reference pixel P3, which is located in the same column in the CU below CU#A3.

Therefore, when directional prediction is performed from the bottom left to the top right, as in the example of Figure 8, many of the generated predicted images are composed of undecoded reference pixels filled with copies, resulting in a problem of reduced prediction accuracy and reduced coding efficiency.

To solve this problem, in intra prediction where TU division is performed, a technique is known that aims to improve prediction accuracy by allowing freedom in the coding order for multiple TUs in a CU, such as raster scan order (e.g., Z-type), as well as U-type and X-type coding orders (see Non-Patent Document 1).

Mochizuki et al., "An adaptive intra prediction method based on mean coordinates," Information Processing Society of Japan Research Report, vol. 2012-AVM-77, No. 12

However, the technology defined in the above-mentioned non-patent document 1 requires the transmission of a flag indicating which encoding process order to use on a CU-by-CU basis, which increases the amount of information to be transmitted. In addition, in order to select the best encoding process order from among all the encoding process orders, the encoding device needs to try all combinations, which increases the calculation time on the encoding device side.

The present invention has been made to solve the above-mentioned problems, and aims to provide an encoding device, a decoding device, and a program that can improve prediction accuracy and encoding efficiency without increasing the amount of information transmitted by the encoding device and without increasing the calculation time on the encoding device side.

The first feature of the present invention is a decoding device configured to decode encoding target blocks obtained by dividing an original image in units of frames constituting a moving image, the decoding device including a first prediction unit that generates a first predicted image by a first prediction process determined based on a prediction mode, a synthesis area determination unit that is configured to determine a synthesis area corresponding to the encoding target block, a second prediction unit that generates a second predicted image by a second prediction process different from the first prediction process, and a predicted image synthesis unit that generates a predicted image included in the synthesis area by a weighted average of the first predicted image and the second predicted image, and the predicted image synthesis unit determines a weight to be used for the weighted average based on whether or not a block adjacent to the encoding target block has been decoded. The encoding device according to one embodiment is an encoding device configured to divide an original image in units of frames constituting a moving image into encoding target blocks and encode it, the encoding device including a synthesis area determination unit that is configured to determine a synthesis area in which a predicted image is generated from adjacent pixels that have not been decoded in the encoding target block based on an intra prediction mode, and an intra prediction unit that is configured to change a method of generating a predicted image for each area based on whether or not it is included in the synthesis area.

The decoding device according to one embodiment is configured to divide frame-by-frame original images constituting a moving image into blocks to be coded and decode them, and is provided with a synthesis area determination unit configured to determine a synthesis area in which a predicted image is generated from adjacent pixels that have not yet been decoded within the blocks to be coded based on an intra prediction mode, and an intra prediction unit configured to change the method of generating a predicted image for each area based on whether or not it is included in the synthesis area.

The gist of the program according to one embodiment is that it is a program for causing a computer to function as the encoding device described above.

The gist of the program according to one embodiment is that it is a program for causing a computer to function as the above-mentioned decryption device.

The present invention provides an encoding device, a decoding device, and a program that can improve prediction accuracy and encoding efficiency without increasing the amount of information transmitted by the encoding device and without increasing the calculation time on the encoding device side.

FIG. 1 is a functional block diagram of an encoding device 1 according to the first embodiment. FIG. 2 is a diagram showing an example of how a predicted image is generated in the first embodiment. FIG. 3 is a diagram showing an example of how a predicted image is generated in the first embodiment. FIG. 4 is a functional block diagram of the decoding device 3 according to the first embodiment. FIG. 5 is a flowchart showing the operations of the encoding device 1 and the decoding device 3 according to the first embodiment. FIG. 6 is a diagram showing an example of how a predicted image is generated in the second embodiment. FIG. 7 is a diagram showing an example of how a predicted image is generated in conventional HEVC. FIG. 8 is a diagram showing an example of how a predicted image is generated in conventional HEVC.

(First embodiment)
An encoding device 1 and a decoding device 3 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 5. Here, the encoding device 1 and the decoding device 3 according to this embodiment are configured to support intra prediction in HEVC.

The encoding device 1 according to this embodiment is configured to divide an original image in units of frames that constitute a moving image into blocks to be encoded and encode the divided blocks. Note that in this embodiment, it is assumed that all pixels adjacent to the left and above the CU to be encoded have already been decoded.

The encoding device 1 according to this embodiment may be configured to be able to divide a CU into a plurality of TUs. Therefore, in this embodiment, the encoding target block may be a CU or a TU. Hereinafter, this embodiment will be described taking as an example a case where a CU is used as the encoding target block.

In addition, in this embodiment, the coding order and decoding order of CUs are described using as an example a case in which the raster scan order (Z-type as shown in Figures 7 and 8) used in conventional HEVC is used, but other coding orders and decoding orders such as U-type and X-type may also be used as the coding order and decoding order.

As shown in FIG. 1, the encoding device 1 according to this embodiment includes an intra-prediction mode determination unit 11, a synthesis region determination unit 12, an intra-prediction unit 13, a residual signal generation unit 14, an orthogonal transform and quantization unit 15, an inverse orthogonal transform and inverse quantization unit 16, a local decoded image generation unit 17, a memory unit 18, and an entropy encoding unit 19.

The intra prediction mode determination unit 11 is configured to determine the optimal intra prediction mode to apply to the CU.

The synthesis area determination unit 12 is configured to determine a synthesis area X in which a predicted image is generated from adjacent pixels that have not yet been decoded within the CU, based on the intra prediction mode determined by the intra prediction mode determination unit 11.

Specifically, as shown in FIG. 2, the synthesis area determination unit 12 may be configured to determine a synthesis area X in which a predicted image is generated from adjacent pixels B1 to B4 that have not yet been decoded within CU#A2 when the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is from the bottom left to the top right (i.e., when directional prediction is performed from the bottom left to the top right).

The present invention is also applicable to cases where the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is from the top right to the bottom left (i.e., when directional prediction is performed from the top right to the bottom left), but below we will explain an example where the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is from the bottom left to the top right (i.e., when directional prediction is performed from the bottom left to the top right).

Note that in the figures in this specification, the arrow indicating the direction of the intra prediction mode (prediction direction) points from the pixel targeted for intra prediction to the reference pixel, as described in the HEVC standard (same below).

The intra prediction unit 13 is configured to change the method of generating a predicted image for each region based on whether or not it is included in the synthesis region X determined by the synthesis region determination unit 12.

Specifically, as shown in FIG. 3(a), in CU#A2, in an area Y that is not included in synthesis area X (i.e., an area where reference pixels W1 to W3 have already been decoded), the intra prediction unit 13 is configured to generate a predicted image based on the decoded reference pixels W1 to W3 using the intra prediction mode determined by the intra prediction mode determination unit 11.

On the other hand, as shown in FIG. 3(b), the intra prediction unit 13 is configured to generate a predicted image using a predefined prediction method in the area of CU#A2 that is included in synthesis area X (i.e., the area where reference pixels B2 to B4 have not yet been decoded).

Here, examples of the predefined prediction method include intra prediction such as DC prediction and planar prediction, as well as prediction methods that generate a predicted image by averaging the values of adjacent decoded reference pixels.

Note that if the encoding device 1 and the decoding device 3 have previously determined the prediction method, there is no need to transmit a new flag indicating the prediction method from the encoding device 1 to the decoding device 3, regardless of the prediction method used.

In addition, since the synthesis area determined by the synthesis area determination unit 12 can be uniquely determined based on the direction of the intra prediction mode and whether or not the adjacent pixels have already been decoded, there is no need to transmit a flag indicating which area is the new synthesis area from the encoding device 1 to the decoding device 3.

For example, as shown in FIG. 3B, the intra prediction unit 13 may be configured to generate a predicted image by synthesizing pixels W3, D1, D2, and D4 using planar prediction at the position of pixel X1 in an area included in synthesis area X in CU#A2. Here, the value of pixel B4 is copied to pixel D4, and the value of pixel W1 is copied to pixel B4.

The residual signal generation unit 14 is configured to generate a residual signal based on the difference between the predicted image generated by the intra prediction unit 13 and the original image.

The orthogonal transform and quantization unit 15 is configured to perform orthogonal transform processing and quantization processing on the residual signal generated by the residual signal generation unit 14, and generate quantized transform coefficients.

The inverse orthogonal transform and inverse quantization unit 16 is configured to again perform inverse quantization and inverse orthogonal transform processing on the quantized transform coefficients generated by the orthogonal transform and quantization unit 15, thereby generating a quantized residual signal.

The local decoded image generating unit 17 is configured to generate a local decoded image by adding the predicted image generated by the intra prediction unit 13 to the quantized residual signal generated by the inverse orthogonal transform and inverse quantization unit 16.

The memory unit 18 is configured to hold the locally decoded image generated by the locally decoded image generating unit 17 so that it can be used as a reference image.

The entropy coding unit 19 is configured to perform entropy coding processing on flag information including the intra-prediction mode determined by the intra-prediction mode determination unit 11 and quantized transform coefficients, and output the result as a stream.

The decoding device 3 according to this embodiment is configured to divide original images in units of frames that make up a moving image into CUs and decode them. Similarly to the encoding device 1 according to this embodiment, the decoding device 3 according to this embodiment is configured to be able to divide a CU into multiple TUs.

As shown in FIG. 4, the decoding device 3 according to this embodiment includes an entropy decoding unit 31, a synthesis region determination unit 32, an intra prediction unit 33, an inverse quantization and inverse transform unit 34, a local decoded image generation unit 35, and a memory unit 36.

The entropy decoding unit 31 is configured to decode the transform coefficients, flag information, and the like from the stream output from the encoding device 1. Here, the transform coefficients are quantized transform coefficients obtained as signals encoded by the encoding device 1 by dividing an original image in units of frames into CUs. In addition, the flag information includes associated information such as a prediction mode.

The synthesis area determination unit 32 is configured to determine a synthesis area X in which a predicted image is generated from adjacent pixels that have not yet been decoded within the CU, based on the intra prediction mode output by the entropy decoding unit 31.

Specifically, as shown in FIG. 2, the synthesis area determination unit 32 may be configured to determine a synthesis area X in which a predicted image is generated from adjacent pixels B1 to B4 that have not yet been decoded within CU#A2 when the direction of the intra prediction mode output by the entropy decoding unit 31 is from the bottom left to the top right (i.e., when directional prediction is performed from the bottom left to the top right).

The intra prediction unit 33 may be configured to generate a predicted image using the synthesis area X determined by the synthesis area determination unit 32 and the intra prediction mode output by the entropy decoding unit 31.

Specifically, like the intra prediction unit 13, the intra prediction unit 33 is configured to change the method of generating a predicted image for each region based on whether or not it is included in the synthesis region X determined by the synthesis region determination unit 12.

For example, as shown in FIG. 3(a), in CU#A2, in an area Y that is not included in synthesis area X (i.e., an area where reference pixels W1 to W3 have already been decoded), the intra prediction unit 33 is configured to generate a predicted image based on the decoded reference pixels W1 to W3 using the intra prediction mode output by the entropy decoding unit 31.

On the other hand, as shown in FIG. 3(b), the intra prediction unit 33 is configured to generate a predicted image using the above-mentioned predefined prediction method in the area of CU#A2 that is included in synthesis area X (i.e., the area where reference pixels B2 to B4 have not yet been decoded).

The inverse quantization and inverse transform unit 34 is configured to generate a residual signal by performing inverse quantization processing and inverse transform processing (e.g., inverse orthogonal transform processing) on the quantized transform coefficients output by the entropy decoding unit 31.

The local decoded image generating unit 35 is configured to generate a local decoded image by adding the predicted image generated by the intra prediction unit 33 and the residual signal generated by the inverse quantization and inverse transform unit 34.

The memory unit 36 is configured to hold the locally decoded image generated by the locally decoded image generating unit 35 so that it can be used as a reference image for intra prediction and inter prediction.

Figure 5 shows a flowchart for explaining an example of the operation of generating a predicted image by the encoding device 1 and the decoding device 3 according to this embodiment.

First, with reference to FIG. 5, an example of an operation of generating a predicted image by the encoding device 1 according to this embodiment will be described.

As shown in FIG. 5, in step S101, the encoding device 1 determines the optimal intra prediction mode to apply to the CU.

In step S102, the encoding device 1 determines a synthesis area X in which a predicted image is generated from adjacent pixels that have not yet been decoded within the CU, based on the intra prediction mode determined in step S101.

In step S103, the encoding device 1 determines whether or not the area in the CU for which a predicted image is to be generated is included in the synthesis area. If the answer is "No," the operation proceeds to step S104; if the answer is "Yes," the operation proceeds to step S105.

In step S104, the encoding device 1 generates a predicted image based on the decoded reference pixels using the intra prediction mode determined in step S101.

In step S105, the encoding device 1 generates a predicted image using the above-mentioned predefined prediction method.

Secondly, referring to FIG. 5, an example of an operation of generating a predicted image by the decoding device 3 according to this embodiment will be described.

As shown in FIG. 5, in step S101, the decoding device 3 determines the intra prediction mode based on the information obtained by the entropy decoding process.

In step S102, the decoding device 3 determines a synthesis area X in which a predicted image is generated from adjacent pixels that have not yet been decoded within the CU, based on the intra prediction mode determined in step S101.

In step S103, the decoding device 3 determines whether or not the area in the CU for which a predicted image is to be generated is included in the synthesis area. If the answer is "No," the operation proceeds to step S104; if the answer is "Yes," the operation proceeds to step S105.

In step S104, the decoding device 3 generates a predicted image based on the decoded reference pixels using the intra prediction mode determined in step S101.

In step S105, the decoding device 3 generates a predicted image using the above-mentioned predefined prediction method.

According to the encoding device 1 and the decoding device 3 of this embodiment, when an undecoded pixel is included in the reference destination of the direction (prediction direction) of the intra prediction mode, a new predicted image is generated by combining a predicted image generated using reference pixels that have been interpolated by copying adjacent decoded reference pixels with a predicted image generated using a predefined prediction method, thereby suppressing a decrease in prediction accuracy.

Second Embodiment
Hereinafter, with reference to FIG. 6, the encoding device 1 and the decoding device 3 according to the second embodiment of the present invention will be described, focusing on the differences from the encoding device 1 and the decoding device 3 according to the first embodiment described above.

In the encoding device 1 according to this embodiment, the intra prediction unit 13 is configured to apply a smoothing filter to the predicted image of a boundary region between an area included in the synthesis region X and an area Y not included in the synthesis region X when a predetermined condition is satisfied in the boundary region.

Similarly, in the decoding device 3 according to this embodiment, the intra prediction unit 33 is configured to apply a smoothing filter to the predicted image of a boundary region between an area included in the synthesis region X and an area Y not included in the synthesis region X when a predetermined condition is satisfied in the boundary region.

For example, in the example of Figure 6, such a boundary area corresponds to the area in which pixels X1, X2, X3, and X4 in the area included in synthesis area X and pixels Y1, Y2, and Y3 in area Y not included in synthesis area X are located.

Another predetermined condition is that the discontinuity in the boundary region is higher than a predetermined threshold. Here, the discontinuity can be evaluated, for example, by expressing it as a linear combination of horizontal differences, vertical differences, diagonal differences, etc.

In other words, the intra prediction unit 13 and the intra prediction unit 33 are configured to apply a smoothing filter to the predicted image of the boundary region when the evaluation value of the discontinuity in the boundary region is higher than a predetermined threshold value.

For example, in the example of FIG. 6, the intra prediction unit 13 and the intra prediction unit 33 are configured to apply a smoothing filter to the predicted image of the area in which pixels X1, X2, X3, and X4 in the area included in the synthesis area X and pixels Y1, Y2, and Y3 in the area Y not included in the synthesis area X are located when the evaluation value of the discontinuity in the boundary area is higher than a predetermined threshold value.

In the example of FIG. 6, the intra prediction unit 13 and the intra prediction unit 33 are configured to also apply a smoothing filter to the predicted image of the area in which pixels X5, X6, and X7 are located within the area included in the synthesis area X.

Here, by sharing the above-mentioned predetermined threshold between the encoding device 1 and the decoding device 3, there is no need to transmit a new flag from the encoding device 1 to the decoding device 3.

The encoding device 1 and decoding device 3 according to this embodiment can reduce discontinuity in boundary areas caused by changing the method of generating a predicted image for each area based on whether or not it is included in the synthesis area X, thereby improving encoding performance.

Third Embodiment
Below, with reference to Figure 6, the encoding device 1 and decoding device 3 according to the third embodiment of the present invention will be described, focusing on the differences from the encoding device 1 and decoding device 3 according to the first and second embodiments described above.

In the encoding device 1 and the decoding device 3 according to this embodiment, the intra prediction unit 13 and the intra prediction unit 33 are configured to generate a predicted image in an area included in the synthesis area X by taking a weighted average of a predicted image generated using a conventional HEVC prediction method (a predicted image generated based on undecoded reference pixels filled in by copying) and a predicted image generated in the synthesis area X (a predicted image generated using the above-mentioned predefined prediction method).

In the encoding device 1 and the decoding device 3 according to this embodiment, the intra prediction unit 13 and the intra prediction unit 33 may be configured to generate a predicted image in an area included in the synthesis area X by taking a weighted average of a predicted image generated using a prediction method in the conventional HEVC (a predicted image generated based on undecoded reference pixels filled in by copying) and a predicted image generated in the synthesis area X (a predicted image generated using the above-mentioned predefined prediction method) only when the above-mentioned specified condition is satisfied.

In the encoding device 1 and the decoding device 3 according to this embodiment, the intra prediction unit 13 and the intra prediction unit 33 may be configured to calculate weighting coefficients to be used when calculating such a weighted average.

The intra prediction unit 13 and the intra prediction unit 33 may be configured to use a predetermined value as the weighting factor, or may be configured to calculate the weighting factor using the reliability of the decoded reference pixels when generating the predicted image.

In the example of FIG. 3(b), the reliability of the decoded reference pixels W1 to W4 is high, while the reliability of the reference pixels B1 to B4 generated by copying is low. In addition, when comparing the reliability of reference pixel B1 with that of reference pixel B4, the reliability of reference pixel B4, which is closer to the source of the copy, is higher.

Furthermore, the intra prediction unit 13 and the intra prediction unit 33 may be configured to calculate such weighting coefficients depending on the distance from the boundary area between an area included in the synthesis area X and an area Y not included in the synthesis area X.

For example, the intra prediction unit 13 and the intra prediction unit 33 may calculate the weighting coefficient of the predicted image generated using the prediction method in the conventional HEVC (the predicted image generated based on the undecoded reference pixels filled in by copying) to be stronger near such boundary areas, and may calculate the weighting coefficient of the predicted image generated in the synthesis area X (the predicted image generated using the above-mentioned predefined prediction method) to be stronger in areas away from such boundary areas.

Other Embodiments
As described above, the present invention has been described by the above-mentioned embodiment, but the description and drawings forming a part of the disclosure of such embodiment should not be understood as limiting the present invention. From such disclosure, various alternative embodiments, examples and operating techniques will become apparent to those skilled in the art.

Although not specifically mentioned in the above embodiment, a program may be provided that causes a computer to execute each process performed by the encoding device 1 and the decoding device 3 described above. Such a program may be recorded on a computer-readable medium. Using a computer-readable medium, such a program can be installed on a computer. Here, the computer-readable medium on which such a program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

Alternatively, a chip may be provided that is configured with a memory that stores a program for implementing at least some of the functions of the encoding device 1 and the decoding device 3 described above, and a processor that executes the program stored in the memory.

1...Encoding device 11...Intra prediction mode determination unit 12...Synthesis region determination unit 13...Intra prediction unit 14...Residual signal generation unit 15...Orthogonal transformation/quantization unit 16...Inverse quantization unit/inverse orthogonal transformation unit 17...Local decoded image generation unit 18...Memory unit 19...Entropy encoding unit 3...Decoding device 31...Entropy decoding unit 32...Synthesis region determination unit 33...Intra prediction unit 34...Inverse quantization/inverse transform unit 35...Local decoded image generation unit 36...Memory unit

Claims (2)

A decoding device configured to decode encoding target blocks obtained by dividing an original image in units of frames constituting a moving image, comprising:
a first prediction unit that generates a first predicted image by a first prediction process determined based on an intra prediction mode;
a synthesis region determination unit configured to determine a synthesis region in the encoding target block;
a second prediction unit that generates a second predicted image by a second prediction process different from the first prediction process;
a predicted image synthesis unit that generates a predicted image included in the synthesis area by taking a weighted average of the first predicted image and the second predicted image,
The decoding device according to claim 1, wherein the prediction image synthesis unit determines weights to be used for the weighted average based on whether or not blocks adjacent to the current block have been decoded. A program for causing a computer to function as the decoding device described in claim 1.

Images