JP7827614B2 - Image decoding device, image decoding method, and program - Google Patents

Description

The present invention relates to an image decoding device, an image decoding method, and a program.

Non-Patent Document 1 defines an area (i.e., a repeated padding area) in which pixel values at the screen boundary are duplicated (padded) a predetermined number of pixels outward from the reference image boundary, so that motion compensation can be performed even when the motion vector references an area outside the reference image.

Furthermore, Non-Patent Document 1 discloses a technique for translating (clipping) the reference position of a motion vector into the referenceable area when the motion vector references an area outside the referenceable area, including a reference image or repeated padding area.

Non-Patent Document 2 discloses template matching for correcting motion vectors.

ITU-T H.266/VVC Algorithm description of Enhanced Compression Model 6 (ECM 6), JVET-AA2025

When the template matching disclosed in Non-Patent Document 2 is effective and a motion vector references an area outside the referenceable area (referenceable area) that includes the reference image, the search range of template matching with the clipped motion vector as the origin exceeds that referenceable area. The present invention has been made in light of the above-mentioned problem, and aims to provide an image decoding device, image decoding method, and program that can prevent the search range of template matching with the motion vector as the origin from exceeding the referenceable area that includes the reference image after a motion vector that references an area outside the referenceable area is clipped.

A first feature of the present invention is an image decoding device comprising: a decoding unit that decodes control information; an intra prediction unit that generates first predicted pixels based on decoded pixels and the control information; a storage unit that stores the decoded pixels; and a motion compensation unit that generates second predicted pixels based on the stored decoded pixels and the control information, wherein the motion compensation unit controls the clip position of the motion vector depending on whether template matching is valid when the motion vector references a region outside a referenceable area that includes a reference image.

A second feature of the present invention is an image decoding device comprising: a decoding unit that decodes control information; an intra prediction unit that generates first predicted pixels based on decoded pixels and the control information; a storage unit that stores the decoded pixels; and a motion compensation unit that generates second predicted pixels based on the stored decoded pixels and the control information, wherein, when a motion vector references a region outside a referenceable region that includes a reference image, the motion compensation unit limits the search range of the template matching depending on whether template matching is valid and the width or height of the block to be decoded.

A third feature of the present invention is an image decoding method comprising: step A of decoding control information; step B of generating first predicted pixels based on decoded pixels and the control information; step C of accumulating the decoded pixels; and step D of generating second predicted pixels based on the accumulated decoded pixels and the control information, wherein in step D, when a motion vector references a region outside a referenceable area that includes a reference image, the clip position of the motion vector is controlled depending on whether template matching is valid.

A fourth feature of the present invention is a program that causes a computer to function as an image decoding device, the image decoding device comprising: a decoding unit that decodes control information; an intra prediction unit that generates first predicted pixels based on decoded pixels and the control information; a storage unit that stores the decoded pixels; and a motion compensation unit that generates second predicted pixels based on the stored decoded pixels and the control information, and the motion compensation unit controls the clip position of the motion vector depending on whether template matching is valid when the motion vector references a region outside the referenceable area that includes a reference image.

The present invention provides an image decoding device, image decoding method, and program that can prevent the search range of template matching, which has the motion vector as its origin, from exceeding the referenceable area that includes the reference image after the motion vector that references the area outside the referenceable area is clipped to the outside of the referenceable area.

FIG. 1 is a diagram showing an example of functional blocks of an image decoding device 200 according to an embodiment. FIG. 2 is a diagram for explaining template matching (TM). FIG. 3 is a diagram showing an example of a referenceable area when the motion compensation unit 208 disclosed in Non-Patent Documents 1 and 2 derives a motion vector. FIG. 4 is a diagram for explaining a specific example of generation of a motion compensation padding area. FIG. 5 is a diagram showing an example of a clipping method when a motion vector refers to an area outside the referenceable area on the right side of the reference image, as disclosed in Non-Patent Document 1. FIG. 6 is a diagram showing an example of a clipping method when a motion vector refers to an area outside the referenceable area on the left side of the reference image, as disclosed in Non-Patent Document 1. FIG. 7 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. FIG. 8 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. FIG. 9 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. FIG. 10 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. FIG. 11 is a diagram for explaining the solution method 1. In FIG. FIG. 12 is a diagram for explaining the solution method 2. In FIG. FIG. 13 is a diagram for explaining the solution method 2. In FIG.

Embodiments of the present invention will be described below with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations, including combinations with other existing components, are possible. Therefore, the following description of the embodiments does not limit the content of the invention described in the claims.

First Embodiment
An image decoding device 200 according to this embodiment will be described below with reference to Figures 1 to 13. Figure 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

As shown in FIG. 1, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, an intra prediction unit 204, an adder 206, an accumulation unit 207, a motion compensation unit 208, and an image output unit 220.

The code input unit 210 is configured to acquire code information encoded by the image encoding device.

The decoding unit 201 is configured to decode control information and quantized values from the code information input from the code input unit 210. For example, the decoding unit 201 is configured to output control information and quantized values by performing variable-length decoding on the code information.

Here, the quantized values are sent to the inverse quantization unit 202, and the control information is sent to the motion compensation unit 208 and the intra prediction unit 204. Note that this control information includes information necessary for controlling the inverse quantization unit 202, motion compensation unit 208, intra prediction unit 204, etc., and may also include header information such as a sequence parameter set, picture parameter set, picture header, slice header, etc.

The inverse quantization unit 202 is configured to inverse quantize the quantized values sent from the decoding unit 201 to generate decoded transform coefficients. These transform coefficients are sent to the inverse transform unit 203.

The inverse transform unit 203 is configured to inversely transform the transform coefficients sent from the inverse quantization unit 202 to generate decoded prediction residuals. These prediction residuals are sent to the adder 206.

The intra prediction unit 204 is configured to generate a first predicted pixel based on the decoded pixel and control information sent from the decoding unit 201. Here, the decoded pixel is obtained via the adder 206 and stored in the storage unit 207. The first predicted pixel is a predicted pixel to be added to the prediction residual in the adder 206. The first predicted pixel is sent to the adder 206.

The storage unit 207 is configured to cumulatively store the decoded pixels sent from the adder 206. These decoded pixels are referenced by the motion compensation unit 208 via the storage unit 207.

The motion compensation unit 208 is configured to generate second predicted pixels based on the decoded pixels stored in the storage unit 207 and the control information sent from the decoding unit 201. Here, the second predicted pixels are predicted pixels to be added to the prediction residual in the adder 206. The second predicted pixels are sent to the adder 206.

The adder 206 is configured to add the prediction residual sent from the inverse transform unit 203 to either the input first predicted pixel or second predicted pixel to obtain a decoded pixel. This decoded pixel is sent to the image output unit 220, storage unit 207, and intra prediction unit 204.

The image output unit 220 is configured to output the decoded pixels transmitted from the adder 206.

The following describes the motion compensation unit 208, a characteristic component of the image decoding device 200 according to this embodiment.

<Method of deriving motion vectors and method of generating inter-predicted images>
The motion compensation unit 208 derives a combination of a motion vector and a reference image required to generate an inter-predicted pixel (i.e., to perform motion compensation from a reference image).

Here, the combination of motion vector and reference image derived by the motion compensation unit 208 may be composed of up to two different combinations of motion vector and reference image as disclosed in Non-Patent Document 1.

Alternatively, the combination of motion vectors and reference images derived by the motion compensation unit 208 may be composed of a combination of two or more (e.g., three or four) motion vectors and reference images, such as Multiple Hypothesis Prediction (MHP) or Overlapped Block Motion Compensation (OBMC), as disclosed in Non-Patent Document 2.

In addition, the motion compensation unit 208 may use the adaptive motion vector prediction (AMVP) mode or merge mode disclosed in Non-Patent Document 1 as a method for deriving motion vectors and reference images.

When using AMVP mode, the motion compensation unit 208 generates motion vector prediction (MVP) candidates using methods such as spatial AMVP, temporal AMVP, or history-based AMVP disclosed in Non-Patent Document 1 and Non-Patent Document 2 to derive the motion vector of the block to be decoded, and selects MVP candidates according to the control information sent from the decoding unit 201.

In addition, when using AMVP mode, the motion compensation unit 208 adds the motion vector difference (MVD) included in the control information sent from the decoding unit 201 to the selected MVP candidate to derive the final motion vector and reference image.

When using merge mode, the motion compensation unit 208 generates merge candidates (combinations of motion vectors and reference images) from spatial merge, temporal merge, non-adjacent spatial merge, history-based merge, pairwise average merge, etc., as disclosed in Non-Patent Document 1 and Non-Patent Document 2, in order to derive the motion vector of the block to be decoded, and registers them in a merge list.

In addition, when using merge mode, the motion compensation unit 208 uses the merge index included in the control information sent from the decoding unit 201 to derive the final motion vector and reference image from the merge candidates.

Furthermore, the motion compensation unit 208 may apply template matching to correct motion vectors, as disclosed in Non-Patent Document 2, to the motion vectors derived in the above-mentioned AMVP mode or merge mode.

Specifically, as shown in Figure 2, template matching is a technique in which the motion vector (Initial MV) of the block to be decoded is used as the origin, and the reference position (i.e., MVD) that minimizes the template cost within a specified search range is derived, and this is added to the motion vector (Initial MV) of the block to be decoded to generate the final motion vector (Final MV).

Here, the template cost is the sum of absolute differences (SAD) or the sum of absolute Hadamard transformed differences (SATD) between the block to be decoded and one or more lines of decoded pixels adjacent to the left and/or top of the reference block.

In addition, the motion compensation unit 208 may apply the above-mentioned template cost comparison not only to generating the final motion vector, but also to selecting MVP candidates in AMVP mode.

In other words, in AMVP mode, the motion compensation unit 208 may select the MVP with the smallest template cost.

The search range for template matching may be configured as a range of 8 pixels (±8 pixels) in the up, down, left, and right directions, with the motion vector (Initial MV) specified in Figure 2 and Non-Patent Document 2 as the origin.

Alternatively, the template matching search range may be configured as a range smaller than the above range (e.g., ±2, ±4, ±6 pixels) or a range larger than the above range (e.g., ±10, ±12, ±14 pixels).

Alternatively, the template matching search range may be configured as an asymmetric search range in the up/down and left/right directions.

Furthermore, the search accuracy of MVD in template matching may be controlled taking into account the accuracy of motion vectors for AMVP mode and merge mode, as disclosed in Figure 2 and non-patent document 2.

For example, in the AMVP mode, if the motion vector accuracy is AMVR (Adaptive Motion Vector Resolution) disclosed in Non-Patent Document 1, with 4-pixel accuracy or integer pixel accuracy, the motion compensation unit 208 may perform searches only at the respective pixel accuracy levels.

In merge mode, if the alternative interpolation filter for motion compensation (Switchable Interpolation Filter) disclosed in Non-Patent Document 1 is enabled, the motion compensation unit 208 may perform searches only with half-pixel accuracy and integer pixel accuracy, as shown in Figure 2.

Finally, the motion compensation unit 208 generates inter-predicted pixels for the block to be decoded using the motion vector and reference image derived as described above.

<Area where motion vectors can be referenced>
The referenceable area of the motion vector derived by the motion compensation unit 208 will be described below.

Figure 3 shows an example of a referenceable area when deriving a motion vector in the motion compensation unit 208 disclosed in Non-Patent Document 1 and Non-Patent Document 2.

In video scenes with fast movement across the screen (video in which the camera pans quickly left and right or up and down), the person, object, background, etc. being predicted may be outside the reference image (outside the reference image), i.e., the motion vector derived by the motion compensation unit 208 may refer to an area outside the reference image.

To deal with such cases, Non-Patent Documents 1 and 2 define a referenceable area of a certain number of pixels outside the reference image.

Specifically, Non-Patent Document 1 prescribes a repetitive padding area outside the reference image, as shown in Figure 3(a), whose size is the maximum width or height of the block to be decoded (block to be coded) (i.e., the width or height of the coding tree block) (maxCUwidth in the example of Figure 3) plus 16 pixels. The size of this repetitive padding area (the number of pixels from the boundary of the reference image) may be set to a fixed value other than 16 pixels. Specific examples will be described later.

Such repeated padding areas are generated by duplicating (padding) pixel values located on the boundary of the reference image outward from the reference image.

When a motion vector references an area outside the referenceable area, including the reference image or the repeated padding area, the motion compensation unit 208 translates (clips) the reference position of the motion vector within the repeated padding area.

Next, Non-Patent Document 2 defines a motion compensation padding area between the boundary of the reference image and the repeated padding area, as shown in Figure 3(b).

To improve the accuracy of motion compensation in the motion compensation unit 208, the motion compensation padding area assigns reference pixel values from the reference image, rather than duplicating pixel values located on the reference image boundary as in the repeated padding area.

Figure 4 shows a specific example of how motion compensation padding areas are generated.

For example, as shown in Figure 4, if the reference blocks for a block to be decoded (BBlk) within the image to be decoded and an adjacent block (MCP Blk) outside the image to be decoded, which are separated by a boundary between the two, are within the reference image, the reference block for the adjacent block contains pixel values within the reference image, rather than a copy of the boundary pixel values of the reference image.

In such cases, by storing reference pixel values that are not duplicates of pixel values at the reference image boundary as the motion compensation padding area, the accuracy of motion compensation is improved when the motion vector references a pixel outside the reference image but within the motion compensation padding area, compared to when a repeated padding area is defined outside the reference image.

The size of the motion compensation padding area may be 16 pixels, as disclosed in Non-Patent Document 2, or 64 pixels, as shown in Figure 3. Alternatively, the size of the motion compensation padding area may be other than these, such as 4 pixels, 8 pixels, 32 pixels, 128 pixels, or 256 pixels, which is the minimum size, maximum size, or possible size value (number of pixels that is a power of 2) of the block to be decoded.

Note that if a motion vector references an area outside the referenceable area, including the reference image, motion compensation padding area, or repeated padding area, the motion compensation unit 208 translates (clips) the reference position of the motion vector within the repeated padding area.

<Clipping method when a motion vector references an area outside the referenceable area>
The clipping method used by the motion compensation unit 208 when a motion vector refers to a region outside the referenceable region will be described below.

First, using Figures 5 and 6, we will explain how to clip a motion vector when template matching is disabled and the motion vector references an area outside the referenceable area.

Figure 5 shows an example of a clipping method when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) to the right of the reference image, as disclosed in Non-Patent Document 1.

In Non-Patent Document 1, as shown in Figure 5, when a motion vector (Initial MV) references an area outside the referenceable area, i.e., when the reference block (RefBlk) is outside the referenceable area, the motion compensation unit 208 translates (clips) the motion vector indicating the upper left corner of the reference block to a position 8 pixels away from the boundary of the reference image.

This clipping operation also applies when the motion vector references a location outside the reference image in the downward direction of the reference image.

Figure 6 shows an example of a clipping method when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) to the left of the reference image, as disclosed in Non-Patent Document 1.

In Non-Patent Document 1, as shown in Figure 6, when a motion vector references an area outside the referenceable area, i.e., when the reference block (RefBlk) is outside the referenceable area, the motion compensation unit 208 translates (clips) the motion vector (Initial MV) indicating the upper left corner of the reference block to a position that is 8 pixels away from the boundary of the reference image plus the maximum width (maxCUwidth) or maximum height of the block to be decoded.

This clipping operation also applies when the motion vector references an area outside the reference image in the upward direction of the reference image.

In Non-Patent Document 1, the motion vector is clipped to a position 8 pixels away from the boundary of the reference image or the boundary of the referenceable area. Therefore, as shown in Figures 5 and 6, even if the width or height of the reference block (RefBlk) is the maximum width or height of the block to be decoded, the reference block (RefBlk) will be contained within the referenceable area, as shown in Figures 5 and 6.

Furthermore, because eight pixels are secured from the boundary of the referenceable area in the direction of the reference image, even when the motion vector references a position with fractional pixel accuracy, reference pixels are secured to the left, right, and top and bottom of the reference block for the motion compensation interpolation filter (eight taps in Non-Patent Document 1, 12 taps in Non-Patent Document 2) used to generate motion compensation pixels (in the case of eight taps, three pixels are required on the left and top of the block, and four pixels are required on the right and bottom; in the case of 12 taps, five pixels are required on the left and top of the block, and six pixels are required on the right and bottom).

Note that the motion compensation unit 208 may change the number of pixels in the repeated padding area of the referenceable area depending on the number of taps (filter length) of the motion compensation interpolation filter.

Specifically, the motion compensation unit 208 may specify the size of such repeated padding area as a multiple (power) of the minimum width or height of the coding block (block to be decoded), in addition to the above-mentioned 16 pixels.

For example, if the minimum width of the coding block (block to be decoded) is 4 pixels, the motion compensation unit 208 may set the size of the repeated padding area to a fixed value such as 8 pixels, 32 pixels, or 64 pixels.

Alternatively, the number of pixels in the width or height of the repeated padding area may be calculated by adding a number of pixels greater than the number of taps (filter length) T of the motion compensation interpolation filter, i.e., T + α, to the maximum width or height of the block to be decoded.

The number of pixels T+α, which is greater than the number of taps (filter length) T of this motion compensation interpolation filter, may be set to a fixed value between 1.0 and 2.0 times T, such as T+α=kT (e.g., k=1.1, 1.2, 1.3, 1.4, 1.5, 2.0).
In addition, when the motion compensation unit 208 references the area outside the motion vector referenceable area, the motion compensation unit 208 may change the settings in advance depending on the number of taps (filter length) of the motion compensation interpolation filter so that the clip position of the motion vector from the boundary of the reference image or the boundary of the referenceable area (8 pixels in the examples of Figures 5 and 6) can be accommodated by the number of taps of the motion compensation interpolation filter.

Specifically, if the number of pixels in the width or height of the repeated padding area is calculated by adding a number of pixels greater than the number of taps (filter length) of the motion compensation interpolation filter to the maximum width or maximum height of the block to be decoded, as shown in Figures 5 and 6 above, the motion compensation unit 208 may clip the motion vector clip position to a position that is (T + α)/2 pixels away from the boundary of the reference image or the boundary of the referenceable area, respectively.

Secondly, using Figures 7 to 10, we will explain how to clip a motion vector when template matching is enabled and the motion vector references an area outside the referenceable area.

Figures 7 to 10 show an example of clipping a motion vector when template matching is enabled and the motion vector references an area outside the referenceable area.

Figures 7 and 8 show examples of clipping methods when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) to the right of the reference image, as disclosed in Non-Patent Document 1.

In the cases shown in Figures 7 and 8, template matching is enabled in both cases, so unlike the case shown in Figure 5, the reference position of the motion vector (Initial MV) is not translated (clipped) to a position 8 pixels away from the boundary of the reference image, but rather the left edge position of the template matching search range, which has the motion vector as its origin, is translated (clipped) to the same position.

This clipping operation also applies when the motion vector references a location outside the reference image in the downward direction of the reference image.

Here, in the case shown in Figure 7, the template matching search range (8 pixels x 2 + width of RefBlk) is the number of pixels that can fit from the clip position of the repeated padding area to the boundary of the referenceable area. In other words, the template matching search range does not extend beyond the boundary of the referenceable area, as shown in Figure 7, so the template matching search can be performed.

On the other hand, in the case shown in Figure 8, the template matching search range (8 pixels x 2 + width of RefBlk) is a number of pixels that cannot be accommodated from the clip position of the repeated padding area to the boundary of the referenceable area. In other words, the template matching search range extends beyond the boundary of the referenceable area, as shown in Figure 8, so template matching search cannot be performed.

Note that even in the case shown in Figure 8, if the reference position of the motion vector is a decimal pixel accuracy position, the template matching search range may extend outside the boundary of the referenceable area, depending on the number of taps of the motion compensation interpolation filter, just as in the case of Figure 7.

The search range behavior when template matching is enabled is also the same when the motion vector references a location outside the reference image in the downward direction of the reference image.

Figures 9 and 10 show an example of a clipping method when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) to the left of the reference image, as disclosed in Non-Patent Document 1.

In the cases shown in Figures 9 and 10, template matching is enabled in both cases, so unlike the case shown in Figure 6, the reference position of the motion vector is not translated (clipped) to a position 8 pixels away from the boundary of the referenceable area, but rather the left edge position of the template matching search range, which has the motion vector as its origin, is translated (clipped) to the same position.

This clipping operation also applies when the motion vector references an area outside the reference image in the upward direction of the reference image.

Here, in the case shown in Figure 9, the template matching search range (8 pixels x 2 + width of RefBlk) is the number of pixels that can fit from the clip position of the repeated padding area to the boundary of the reference image area. In other words, the template matching search range does not extend inward toward the boundary of the reference image, as shown in Figure 9, so the template matching search can be performed.

On the other hand, in the case shown in Figure 10, the template matching search range (8 pixels x 2 + width of RefBlk) is the number of pixels that cannot be accommodated from the clip position of the repeated padding area to the boundary of the reference image area. In other words, the template matching search range extends beyond the inside of the reference image boundary as shown in Figure 10. Therefore, template matching search can be performed, but only in the area within the template matching search range that extends beyond the inside of the reference image boundary will the pixel values at the reference image boundary have pixel values that are not repeated.

In other words, for pixel values within the template matching search range, non-uniformity in pixel values occurs in the left-right direction inside the boundary of the reference image mentioned above, so a left-right motion vector differential (MVD) is generated and the motion vector is corrected in the left-right direction by template matching (in this case, if the direction does not go beyond the inside boundary of the reference image, only MVD occurs in the up-down direction).

The search range behavior when template matching is enabled is also the same when the motion vector references an area outside the reference image in the upward direction of the reference image.

Figures 6 to 10 explain the issues that arise when template matching is enabled and a motion vector references an area outside the referenceable area, based on the case where there is no motion compensation padding area as shown in Non-Patent Document 2 and explained in Figure 3.

On the other hand, the same problem occurs in the case of the motion compensation padding area described in Non-Patent Document 2. This is because, in Non-Patent Document 2, the position to which the motion vector is clipped when it references an area outside the referenceable area is set on the assumption that a motion compensation padding area exists.

5, the motion vector is clipped not to the boundary of the reference image but to a position 8 pixels away from the boundary of the motion compensation padding area toward the boundary of the referenceable area. Note that this clipping operation is also the same when the motion vector references an area outside the reference image in the upward direction of the reference image.
6, the motion vector is clipped not at the boundary of the reference image but at a position that is 8 pixels away from the boundary of the motion compensation padding area plus the maximum width of the block to be decoded in the direction of the boundary of the referenceable area (i.e., a position that is 8 pixels away from the boundary of the referenceable area in the direction of the reference image). Note that this clipping operation is also the same when the motion vector references an area outside the reference image in the upward direction of the reference image.

To resolve this issue, the motion compensation unit 208 selects at least one of the following solutions 1 to 3.

<Solution 1: Controlling/Limiting the Search Range of Template Matching>
When a motion vector references a region outside the referenceable region, the motion compensation unit 208 controls the search range of template matching depending on whether template matching is valid or not.

Alternatively, when a motion vector references an area outside the referenceable area, the motion compensation unit 208 may control the search range for template matching depending on whether template matching is enabled and the width, height, or size of the reference block.

Specifically, the motion compensation unit 208 may control the template matching search range by comparing the width, height, or size of the reference block and the template matching search range with a predetermined number of pixels.

Here, the specified number of pixels may be, for example, the maximum width, maximum height, or maximum size of the block to be coded (block to be decoded), or it may be a predetermined number of pixels T + α (8 pixels in the examples shown in Figures 5 to 10 above).

Specifically, the motion compensation unit 208 maintains the template matching search range if the following formula is satisfied, and limits the template matching search range if the following formula is not satisfied.

CuWidth+TmPmWidth<maxCUwidth+(T+α)/2
Here, CuWidth is the width of the block to be decoded (reference block), TmPmWidth is the search width of template matching (±8 pixels = 16 pixels in Non-Patent Document 2), and maxCUwidth is the maximum width of the block to be decoded (reference block). Also, as described above, T + α is the value obtained by adding a predetermined number of pixels α to the number of taps (filter length) of the interpolation filter.

When using the template matching search range reduction method, the motion compensation unit 208 prohibits all searches in directions in which the motion vector references areas outside the referenceable area, as shown in Figure 11 (in the example of Figure 11, the search width in the left and right directions is limited).

In other words, the search range for template matching is limited to the vertical direction, so the MVD generated by template matching contains components only in the vertical direction, as shown in Figure 11.

Figure 11 shows an example where the motion vector references the right side outside the referenceable area, but even if the motion vector references the left side outside the referenceable area, by limiting the template matching search range in the same way, it is possible to avoid cases where template matching is performed across the reference image boundary with the motion vector as the origin.

Similarly, the motion compensation unit 208 can apply the same method to cases where the motion vector references the upper or lower side outside the referenceable area.

Alternatively, the motion compensation unit 208 may reduce the number of pixels in the template matching search range, rather than restricting (prohibiting) the template matching search range in a lump, which only targets reference directions outside the referenceable area, as described above.

Furthermore, the motion compensation unit 208 may control the number of pixels reduced in template matching depending on the width, height, or size (number of pixels) of the block to be decoded (reference block).

<Solution 2: Correcting the clip position of the motion vector>
When a motion vector references a region outside the referenceable region, the motion compensation unit 208 controls the clip position of the motion vector depending on whether template matching is valid or not.

Specifically, when template matching is not effective, the motion compensation unit 208 translates (clips) the reference position of the motion vector to the position described above in Figures 5 and 6.

On the other hand, when template matching is enabled, the motion compensation unit 208 translates (clips) the reference position of the motion vector to the positions shown in Figures 12 and 13 (i.e., the boundary of the reference image in Figure 2, and the boundary of the referenceable area in Figure 13).

Here, if each clip position has fractional pixel accuracy, the motion compensation unit 208 may (always) round the clip position to a position with integer pixel accuracy. This eliminates the need for a motion compensation interpolation filter, and the template search range is accommodated within the repeated padding area, as shown in Figures 12 and 13.

In the examples of Figures 12 and 13, the template matching search range is 16 pixels (±8 pixels), and the number of pixels in the repeated padding area excluding the maximum width of the block to be coded (block to be decoded) is 16 pixels (repeated padding reserve area). However, if the template matching search range is more than 16 pixels, the same issue can be addressed by setting the repeated padding reserve area to 16 pixels or more to match the template matching search range.

As a modification, the motion compensation unit 208 may clip the motion vector to the above-mentioned position, regardless of whether template matching is valid or not.

<Solution 3: Expanding the repeat padding area>
The motion compensation unit 208 extends the repeated padding area if template matching is effective.

Specifically, the motion compensation unit 208 expands the repeated padding area by the number of pixels in the template matching search range relative to the specified area when template matching is disabled, depending on the template matching search range.

The image decoding device 200 according to this embodiment can prevent template matching when the motion vector goes beyond the referenceable area that includes the reference image.

The motion compensation unit 208 may determine whether template matching is effective based on control information decoded or estimated by the decoding unit 201.

Specifically, the control information may include a flag for each block to be coded (block to be decoded) that specifies whether template matching is enabled or disabled, and the motion compensation unit 208 can make a determination based on the value of this flag.

For example, the motion compensation unit 208 may determine that template matching is enabled when the value of such a flag is 1, and that template matching is disabled when the value of such a flag is 0.

The control information may also include a flag for specifying whether template matching is enabled or disabled for each sequence, picture, or slice before the current block to be coded (current block to be decoded), and the decoding unit 201 may control whether to decode based on the decoding result of the flag for the previous layer (slice for blocks, picture for slices).

The above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each process).

Note that, according to this embodiment, it is possible to achieve an improvement in overall service quality in video communication, for example, and therefore contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation."

200... Image decoding device 201... Decoding unit 202... Inverse quantization unit 203... Inverse transform unit 204... Intra prediction unit 206... Adder 207... Storage unit 208... Motion compensation unit 210... Code input unit 220... Image output unit

Claims (5)

An image decoding device,
a decoding unit that decodes the control information;
an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the control information;
a storage unit that stores the decoded pixels;
a motion compensation unit that generates a second predicted pixel based on the stored decoded pixel and the control information,
An image decoding device characterized in that the motion compensation unit controls the clip position of the motion vector depending on whether template matching is valid or not when the motion vector references a reference area outside the referenceable area including the reference image. An image decoding device,
a decoding unit that decodes the control information;
an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the control information;
a storage unit that stores the decoded pixels;
a motion compensation unit that generates a second predicted pixel based on the stored decoded pixel and the control information,
The image decoding device is characterized in that, when a motion vector references a region outside a referenceable region including a reference image, the motion compensation unit limits the search range of the template matching depending on whether template matching is valid and the width or height of the block to be decoded. The image decoding device of claim 1, wherein the motion compensation unit always rounds the reference position of the motion vector to an integer pixel precision position when controlling the clip position of the motion vector depending on whether the template matching is valid. An image decoding method, comprising:
a step A of decoding control information;
a step B of generating a first predicted pixel based on the decoded pixel and the control information;
a step C of storing the decoded pixels;
a step D of generating a second predicted pixel based on the stored decoded pixel and the control information,
An image decoding method characterized in that in step D, when a motion vector references a region outside a referenceable area including a reference image, the clip position of the motion vector is controlled depending on whether template matching is valid or not. A program that causes a computer to function as an image decoding device,
The image decoding device comprises:
a decoding unit that decodes the control information;
an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the control information;
a storage unit that stores the decoded pixels;
a motion compensation unit that generates a second predicted pixel based on the stored decoded pixel and the control information,
A program characterized in that the motion compensation unit controls the clip position of the motion vector depending on whether template matching is valid or not when the motion vector references a region outside the referenceable region including the reference image.