JP7783715B2 - Video decoding device and video encoding device - Google Patents

Description

Embodiments of the present invention relate to a video decoding device and a video encoding device.

To efficiently transmit or record video, video encoding devices are used that encode video to generate coded data, and video decoding devices that decode the coded data to generate decoded images.

Specific video encoding methods include H.264/AVC and HEVC (High-Efficiency Video Coding).

In such video coding methods, the images (pictures) that make up a video are managed in a hierarchical structure consisting of slices obtained by dividing the images, coding tree units (CTUs) obtained by dividing the slices, coding units (sometimes called coding units: CUs) obtained by dividing the coding tree units, and transform units (TUs) obtained by dividing the coding units, and are coded/decoded for each CU.

In addition, in such video coding methods, a predicted image is typically generated based on a locally decoded image obtained by encoding/decoding an input image, and the prediction error (sometimes called a "difference image" or "residual image") obtained by subtracting this predicted image from the input image (original image) is coded. Methods for generating predicted images include inter-frame prediction (inter-prediction) and intra-frame prediction (intra-prediction).

Non-Patent Document 1 also discloses a GPM (Geometric Partition Mode) mode in which the target block is divided into shapes other than rectangular and different inter-predictions are performed for each region. By dividing the target block into shapes other than rectangular in this way, more accurate predictions can be made even for complex textures, improving coding efficiency.

Patent Document 1 describes a method of identifying a line segment using a point through which the line segment passes and its slope, stating that "in the division method decoding step, if the division method for the block is division along the arbitrary line segment, and the division line segment information describes a point through which the line segment passes and the slope of the line segment, based on a division point on the line where the block meets a peripheral block, the line segment information is identified from the point through which the line segment passes and the slope of the line segment, and the line segment information is notified to the division step."

Patent No. 4313710

ITU-T Rec. H.266

The method described in Non-Patent Document 1 encodes GPM information for each block, which has the problem of not being able to use information from surrounding blocks.

Furthermore, the method described in Patent Document 1 has the problem of not being able to describe the single point through which the line segment passes in the GPM mode described in Non-Patent Document 1.

In order to solve the above problem, a video decoding device according to one aspect of the present invention includes:
An image decoding device comprising: a parameter decoding unit that decodes GPM parameters of a current block from encoded data; and a prediction unit that derives a predicted image from two predicted images and weights,
The parameter decoding unit decodes different parameters depending on whether or not the prediction division line of the adjacent block and the current block have an intersection point, and derives the intersection point using a table that associates the block size of the adjacent block, the prediction mode number of the adjacent block, and a coordinate position of the intersection point;
When the prediction division line of the adjacent block and the target block have an intersection point, the prediction unit derives a partition index of the prediction mode of the target block using a table that associates the division angle derived from the index indicating the intersection point and the decoded division angle, the block size of the adjacent block, the prediction mode number of the adjacent block, and the coordinate position of the intersection point, and a table that associates the block size of the target block, the prediction mode number of the target block, and the coordinate position of the intersection point;
the prediction unit derives weights for making the prediction based at least on the derived partition index values and a table that associates the partition index with pairs of angle indexes and distance indexes;
The weights are used to derive the predicted image.

According to this aspect of the present invention, it is possible to improve the coding efficiency in video coding and decoding processes.

1 is a schematic diagram showing the configuration of an image transmission system according to an embodiment of the present invention. FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. FIG. 1 is a schematic diagram illustrating a configuration of a video decoding device. FIG. 10 is a schematic diagram showing the configuration of an inter-prediction image generation unit. FIG. 10 is a schematic diagram showing the configuration of an inter-prediction parameter derivation unit. 10 is a flowchart illustrating a schematic operation of the video decoding device. FIG. 10 is a syntax diagram illustrating coding parameters for GPM prediction. FIG. 1 is a diagram illustrating GPM prediction. 10 is a diagram showing the structure of a table indicating the correspondence between merge_gpm_partition_idx (mergeGpmPartitionIdx) and angleIdx and distanceIdx in GPM prediction. 11 is a diagram showing the structure of a table indicating the correspondence between idx and DisLut[idx] used in the weighting coefficient derivation process and the motion vector storage process in GPM prediction. 10 is a flowchart showing the flow of processing for GPM prediction. FIG. 10 is a syntax diagram illustrating coding parameters in mode number prediction of GPM prediction. This is a table for deriving the x-coordinate of the intersection between the GPM division line of the upper adjacent block and the target block in mode number prediction of GPM prediction. This is a table for deriving the y coordinate of the intersection between the GPM division line of the left adjacent block and the target block in mode number prediction of GPM prediction. This is a table for deriving the x-coordinate of the intersection between the GPM division line of the target block and the upper adjacent block in mode number prediction of GPM prediction. This is a table for deriving the y coordinate of the intersection between the GPM division line of the target block and the adjacent block on the left side in mode number prediction of GPM prediction. FIG. 10 is a diagram showing the structure of a table indicating the correspondence between merge_gpm_angle_idx and angleIdx in mode number prediction of GPM prediction. 10 is a flowchart showing the flow of processing for mode number prediction of GPM prediction. 10 is another flowchart showing the flow of processing for mode number prediction of GPM prediction. 10A and 10B are diagrams illustrating GPM division lines of adjacent blocks and line segments that can become GPM division lines in a target block. FIG. 1 is a block diagram showing a configuration of a video encoding device. FIG. 10 is a schematic diagram showing the configuration of an inter-prediction parameter encoding unit.

(First embodiment)
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

Figure 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

Image transmission system 1 is a system that transmits an encoded stream obtained by encoding a target image, decodes the transmitted encoded stream, and displays the image. Image transmission system 1 is composed of a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41.

Image T is input to video encoding device 11.

The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

The video decoding device 31 decodes each of the coded streams Te transmitted over the network 21 and generates one or more decoded images Td.

The video display device 41 displays all or part of one or more decoded images Td generated by the video decoding device 31. The video display device 41 is equipped with a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. Display types include stationary, mobile, and HMD. Furthermore, if the video decoding device 31 has high processing power, it displays high-quality images, and if it has lower processing power, it displays images that do not require high processing or display power.

<Operator>
The operators used in this specification are listed below.

>> indicates a right bit shift, << indicates a left bit shift, & indicates a bitwise AND, | indicates a bitwise OR, and |= indicates the OR assignment operator.

x?y:z is a ternary operator that takes y if x is true (non-zero) and z if x is false (0).

Clip3(a,b,c) is a function that clips c to a value greater than or equal to a and less than or equal to b. If c<a, it returns a; if c>b, it returns b; otherwise, it returns c (where a<=b).

abs(a) is a function that returns the absolute value of a.

Int(a) is a function that returns the integer value of a.

floor(a) is a function that returns the largest integer less than or equal to a.

ceil(a) is a function that returns the smallest integer greater than or equal to a.

a/d represents the division of a by d (rounded down to the nearest integer).

a % d represents the remainder when a is divided by d.

x||y indicates the OR operation of x and y. || can also be x+y.

x&&y indicates the AND operation of x and y. && can also be written as x*y.

x++ increments x by 1. When you write Tbl[x++], it indicates that after using the value of Tbl[x], x is incremented by 1.

<Structure of the coded stream Te>
Before proceeding to a detailed description of the video encoding device 11 and video decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 will be described.

Figure 2 shows the hierarchical structure of data in the coded stream Te. The coded stream Te illustratively includes a sequence and multiple pictures that make up the sequence. (a) to (f) of Figure 2 show, respectively, a coded video sequence that defines the sequence SEQ, a coded picture that defines the picture PICT, a coded slice that defines the slice S, coded slice data that defines the slice data, a coded tree unit included in the coded slice data, and a coded unit included in the coded tree unit.

(encoded video sequence)
The coded video sequence defines a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in Fig. 2, the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptation parameter set (APS), a picture PICT, and supplemental enhancement information SEI (Supplemental Enhancement Information).

A video parameter set (VPS) specifies a set of coding parameters common to multiple videos composed of multiple layers, as well as a set of coding parameters related to the multiple layers and individual layers contained in the video.

The sequence parameter set SPS defines a set of coding parameters that the video decoding device 31 references to decode the target sequence. For example, the width and height of a picture are defined. Note that there may be multiple SPSs. In this case, one of the multiple SPSs is selected from the PPS.

The picture parameter set PPS defines a set of coding parameters that the video decoding device 31 references to decode each picture in the target sequence. For example, it includes the reference value of the quantization width used in decoding the picture (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag). Note that there may be multiple PPSs. In that case, one of the multiple PPSs is selected for each picture in the target sequence.

(encoded picture)
A coded picture defines a set of data that the video decoding device 31 references in order to decode a picture PICT to be processed. As shown in Figure 2, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

Note that, hereinafter, when there is no need to distinguish between slices 0 to NS-1, the subscripts of the symbols may be omitted. The same applies to other data with subscripts included in the coded stream Te described below.

(Coding Slice)
A coded slice defines a set of data that the video decoding device 31 refers to in order to decode a target slice S. As shown in Fig. 2, a slice includes a slice header and slice data.

The slice header includes a set of coding parameters that the video decoding device 31 references to determine the decoding method for the current slice. The slice type specification information (slice_type), which specifies the slice type, is an example of a coding parameter included in the slice header.

Slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction during encoding, (2) a P slice that uses unidirectional or intra prediction during encoding, and (3) a B slice that uses unidirectional, bidirectional, or intra prediction during encoding. Note that inter prediction is not limited to unidirectional or bidirectional prediction, and a predicted image may be generated using more reference pictures. Hereinafter, P and B slices refer to slices that include blocks that can use inter prediction.

Note that the slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id).

(encoded slice data)
The coded slice data defines a set of data that the video decoding device 31 references in order to decode the slice data to be processed. As shown in Fig. 2(d), the slice data includes a CTU. A CTU is a block of a fixed size (e.g., 64x64) that constitutes a slice, and is also called a Largest Coding Unit (LCU).

(coding tree unit)
2 specifies a set of data that the video decoding device 31 refers to in order to decode the CTU to be processed. The CTU is divided into coding units (CU), which are the basic units of the encoding process, by recursive quad tree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division), or ternary tree division (TT (Ternary Tree) division). BT division and TT division are collectively called multi tree division (MT (Multi Tree) division). A node of the tree structure obtained by recursive quad tree division is called a coding node. Quad tree, binary tree, and ternary tree division are
The intermediate nodes of the bintree are coding nodes, with the CTU itself being defined as the top coding node, and the bottom coding node being defined as the coding unit.

(Encoding Unit)
2 defines a set of data that the video decoding device 31 references in order to decode a coding unit to be processed. Specifically, a CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header defines a prediction mode, etc.

Prediction processing can be performed on a CU basis, or on sub-CU basis, which is a further division of a CU. If the size of the CU and sub-CU are equal, there will be one sub-CU in the CU. If the size of the CU is larger than the size of the sub-CU, the CU will be divided into sub-CUs. For example, if the CU is 8x8 and the sub-CU is 4x4, the CU will be divided into four sub-CUs, divided horizontally in half and vertically in half.

Prediction types (prediction modes) include intra prediction (MODE_INTRA), inter prediction (MODE_INTER), and intra block copy (MODE_IBC). Intra prediction is prediction within the same picture, while inter prediction refers to prediction processing performed between different pictures (for example, between display times or between layer images).

Transformation and quantization processing is performed in units of CU, but quantized transform coefficients may be entropy coded in units of subblocks such as 4x4.

(Prediction parameters)
The predicted image is derived from prediction parameters associated with the block, which include intra-prediction and inter-prediction parameters.

(Prediction parameters for inter prediction)
The prediction parameters of inter prediction will be described. The inter prediction parameters are composed of prediction list usage flags predFlagL0 and predFlagL1, reference picture indices refIdxL0 and refIdxL1, and motion vectors mvL0 and mvL1. predFlagL0 and predFlagL1 are flags indicating whether a reference picture list (L0 list, L1 list) is used, and when the value is 1, the corresponding reference picture list is used. Note that in this specification, when the term "flag indicating whether XX is true" is used, a flag other than 0 (e.g., 1) is considered to be XX, and 0 is considered to be not XX, and in logical negation, logical product, etc., 1 is treated as true and 0 is treated as false (the same applies below). However, in actual devices and methods, other values may be used as true and false values.

Syntax elements for deriving inter prediction parameters include, for example, the merge flag merge_flag (general_merge_flag), the merge index merge_idx, merge_subblock_flag indicating whether to use subblock-based inter prediction, regulare_merge_flag, the intra-inter flag ciip_flag, GPM_flag indicating whether to use GPM mode (Geometric partitioning merge mode), merge_gpm_partition_idx indicating the partition shape for GPM mode, merge_gpm_idx0 and merge_gpm_idx1 indicating GPM merge indices, the inter prediction identifier inter_pred_idc for selecting the reference picture to be used in AMVP mode, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx for deriving a motion vector, the difference vector mvdLX, and the motion vector precision mode amvr_mode. The partition shape of the target block is not limited to the syntax element merge_gpm_partition_idx, and may also use mergeGpmPartitionIdx obtained by prediction from surrounding blocks.

(Reference Picture List)
The reference picture list is a list of reference pictures stored in the reference picture memory 306. In each CU, refIdxLX specifies which picture in the reference picture list RefPicListX (X=0 or 1) to actually reference. Note that LX is a notation method used when there is no distinction between L0 prediction and L1 prediction; hereinafter, parameters for the L0 list and parameters for the L1 list will be distinguished by replacing LX with L0 or L1.

(Merge prediction and AMVP prediction)
Prediction parameter decoding (encoding) methods include merge prediction mode (merge mode) and AMVP (Advanced Motion Vector Prediction, adaptive motion vector prediction) mode, and merge_flag is a flag for distinguishing between them. Merge mode is a prediction mode that omits some or all of the motion vector difference, and derives it from prediction parameters of already processed neighboring blocks without including the prediction list usage flag predFlagLX, reference picture index refIdxLX, and motion vector mvLX in the encoded data. AMVP mode is a mode that includes inter_pred_idc, refIdxLX, and mvLX in the encoded data. Note that mvLX is encoded as mvp_LX_idx, which identifies the prediction vector mvpLX, and the difference vector mvdLX. The general term for prediction modes that omit or simplify the motion vector difference is called general merge mode, and general merge mode and AMVP prediction may be selected using general_merge_flag.
Furthermore, the regular_merge_flag may be transmitted, and if the regular_merge_flag is 1, the normal merge mode or MMVD may be selected, and otherwise the CIIP mode or GPM mode may be selected.

inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of the values PRED_L0, PRED_L1, or PRED_BI. PRED_L0 and PRED_L1 indicate uni-prediction using one reference picture managed in the L0 list and L1 list, respectively. PRED_BI indicates bi-prediction using two reference pictures managed in the L0 list and L1 list.

merge_idx is an index that indicates which prediction parameter from the prediction parameter candidates (merge candidates) derived from the processed block will be used as the prediction parameter for the target block.

(motion vector)
mvLX indicates the amount of shift between blocks on two different pictures. The predicted vector and differential vector related to mvLX are called mvpLX and mvdLX, respectively.

(Inter prediction identifier inter_pred_idc and prediction list usage flag predFlagLX)
The relationship between inter_pred_idc, predFlagL0, and predFlagL1 is as follows, and they can be converted to each other.

inter_pred_idc = (predFlagL1<<1)+predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
Note that the inter prediction parameters may use a prediction list usage flag or an inter prediction identifier. Furthermore, the determination using the prediction list usage flag may be replaced with a determination using the inter prediction identifier. Conversely, the determination using the inter prediction identifier may be replaced with a determination using the prediction list usage flag.

(Bi-prediction biPred determination)
The flag biPred indicating whether or not the prediction is bi-predictive can be derived based on whether or not two prediction list usage flags are both 1.

Alternatively, biPred can be derived based on whether the inter-prediction identifier is a value indicating the use of two prediction lists (reference pictures).

(Configuration of video decoding device)
The configuration of a video decoding device 31 (FIG. 3) according to this embodiment will be described.

The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse quantization and inverse transform unit 311, an addition unit 312, and a prediction parameter derivation unit 320. Note that the video decoding device 31 may also be configured without the loop filter 305, in line with the video encoding device 11 described below.

The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, and APS, and slice headers (slice information) from the encoded data. The CT information decoding unit 3021 decodes the CT from the encoded data. The CU decoding unit 3022 decodes the CU from the encoded data. If a prediction error is included in the TU, the TU decoding unit 3024 decodes QP update information (quantization correction value) and quantized prediction error (residual_coding) from the encoded data.

sps_gpm_enabled_flag is coded and decoded using SPS and indicates whether GPM prediction mode is used for the target sequence. sps_max_num_merge_cand_minus_max_num_gpm_cand is coded and decoded in SPS. sps_max_num_merge_cand_minus_max_num_gpm_cand is a parameter used to derive MaxNumGpmMergeCand, the maximum number of merge candidates for GPM prediction. When MaxNumGpmMergeCand is 0, GPM prediction is disabled.

The predicted image generation unit 308 is composed of an inter predicted image generation unit 309 (Figure 4) and an intra predicted image generation unit 310.

The prediction parameter derivation unit 320 is composed of an inter prediction parameter derivation unit 303 (Figure 5) and an intra prediction parameter derivation unit.

Furthermore, although the following describes an example in which CTUs and CUs are used as processing units, this is not limiting and processing may be performed in sub-CU units. Alternatively, CTUs and CUs may be read as blocks, and sub-CUs as sub-blocks, and processing may be performed in block or sub-block units.

The entropy decoding unit 301 performs entropy decoding on the externally input coded stream Te to decode individual codes (syntax elements). Entropy coding can be performed in two ways: one is to perform variable-length coding of syntax elements using a context (probability model) adaptively selected according to the type of syntax element and the surrounding circumstances, and the other is to perform variable-length coding of syntax elements using a predetermined table or formula.

The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. The decoded code may be, for example, the prediction mode predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, etc. The code to be decoded is controlled based on instructions from the parameter decoding unit 302.

(Basic flow)
FIG. 6 is a flowchart illustrating the general operation of the video decoding device 31.

(S1100: Decode parameter set information) The header decoder 3020 decodes the VPS and SP from the coded data.
Decode parameter set information such as S and PPS.

(S1200: Slice information decoding) The header decoding unit 3020 decodes the slice header (slice information) from the encoded data.

The video decoding device 31 then repeats the processes from S1300 to S5000 for each CTU included in the target picture to derive a decoded image for each CTU.

(S1300: Decode CTU information) The CT information decoding unit 3021 decodes the CTU from the encoded data.

(S1400: CT information decoding) The CT information decoding unit 3021 decodes the CT from the encoded data.

(S1500: CU decoding) The CU decoding unit 3022 performs S1510 and S1520 to decode the CU from the encoded data.

(S1510: CU information decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU division flags, CU residual flags, etc. from the encoded data.

(S1520: TU information decoding) If a TU contains a prediction error, the TU decoding unit 3024 decodes the quantized prediction error, transform index lfnst_idx, etc. from the encoded data.

(S2000: Predicted image generation) The predicted image generation unit 308 generates a predicted image for each block included in the target CU based on the prediction information.

(S3000: Inverse quantization and inverse transform) The inverse quantization and inverse transform unit 311 performs inverse quantization and inverse transform processing on each TU included in the target CU.

(S4000: Decoded image generation) The adder 312 generates a decoded image of the target CU by adding the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization and inverse transform unit 311.

(S5000: Loop filter) The loop filter 305 applies a loop filter such as a deblocking filter, SAO, or ALF to the decoded image to generate a decoded image.

The loop filter 305 is a filter installed within the encoding loop that removes block distortion and ringing distortion and improves image quality. The loop filter 305 applies filters such as deblocking, sample adaptive offset (SAO), and adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312.

The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 in a predetermined location for each target picture and target CU.

The prediction parameter memory 307 stores prediction parameters at a predetermined location for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

The prediction image generation unit 308 receives input of the prediction mode predMode, prediction parameters, etc. The prediction image generation unit 308 also reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image for a block or sub-block using the prediction parameters and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, a reference picture block is a set of pixels on the reference picture (usually rectangular, so called a block), and is the area referenced to generate a prediction image.

(Configuration of inter-prediction parameter derivation unit)
As shown in Fig. 5, the inter prediction parameter derivation unit 303 derives inter prediction parameters by referring to prediction parameters stored in the prediction parameter memory 307, based on the syntax elements input from the parameter decoding unit 302. The inter prediction parameter derivation unit 303 also outputs the inter prediction parameters to the inter prediction image generation unit 309 and the prediction parameter memory 307. The inter prediction parameter derivation unit 303 and its internal elements, the AMVP prediction parameter derivation unit 3032, the merge prediction parameter derivation unit 3036, the GPM prediction unit 3037, and the MV addition unit 3038, are means common to the video encoding device and the video decoding device, and therefore may be collectively referred to as a motion vector derivation unit (motion vector derivation device).

If merge_flag is 1, i.e., indicates merge prediction mode, merge_idx is derived and output to the merge prediction parameter derivation unit 3036.

When merge_flag is 0, i.e., indicates AMVP prediction mode, the AMVP prediction parameter derivation unit 3032 derives mvpLX from inter_pred_idc, refIdxLX, or mvp_LX_idx.

If GPM_flag is 1, i.e., indicates that the current block is coded in GPM prediction mode, the GPM prediction unit 30377 derives GPM prediction parameters. The derivation of GPM_flag will be described later.

(MV addition section)
The MV adder 3038 adds the derived mvpLX and mvdLX to derive mvLX.

(Merge Prediction)
The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361 and a merge candidate selection unit 30362. Note that merge candidates are configured to include prediction parameters (predFlagLX, mvLX, refIdxLX) and are stored in a merge candidate list. Merge candidates stored in the merge candidate list are assigned indices according to predetermined rules.

The merge candidate derivation unit 30361 derives merge candidates directly using the motion vectors and refIdxLX of decoded adjacent blocks. Alternatively, the merge candidate derivation unit 30361 may apply spatial merge candidate derivation processing, temporal merge candidate derivation processing, etc., as described below.

As part of the spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads prediction parameters stored in the prediction parameter memory 307 according to predetermined rules and sets them as merge candidates. For example, it reads prediction parameters for the following positions: A1, B1, B0, A0, B2.

A1: (xCb-1, yCb+cbHeight-1)
B1: (xCb+cbWidth-1, yCb-1)
B0: (xCb+cbWidth, yCb-1)
A0: (xCb-1, yCb+cbHeight)
B2: (xCb-1, yCb-1)
The upper left coordinates of the target block are (xCb, yCb), the width is cbWidth, and the height is cbHeight.

As a temporal merge derivation process, the merge candidate derivation unit 30361 reads the prediction parameters of the lower right CBR of the target block or the block C in the reference image including the center coordinates from the prediction parameter memory 307, sets it as a merge candidate Col, and stores it in the merge candidate list mergeCandList[ ].

The order in which the merge candidates are stored in mergeCandList[] is, for example, spatial merge candidates (B1, A1, B0, A0, B2) and then temporal merge candidates Col. Note that reference blocks that are unavailable (for example, blocks that are intra-predicted) are not stored in the merge candidate list.
i = 0
if(availableFlagB1)
mergeCandList[i++] = B1
if(availableFlagA1)
mergeCandList[i++] = A1
if(availableFlagB0)
mergeCandList[i++] = B0
if(availableFlagA0)
mergeCandList[i++] = A0
if(availableFlagB2)
mergeCandList[i++] = B2
if(availableFlagCol)
mergeCandList[i++] = Col
Furthermore, the history merge candidate HmvpCand, the average candidate avgCand, and the zero merge candidate zeroCandm may be added to mergeCandList[] and used.

The merge candidate selection unit 30362 selects merge candidate N indicated by merge_idx from the merge candidates included in the merge candidate list using the following formula:

N = mergeCandList[merge_idx]
Here, N is a label indicating a merge candidate, and can be A1, B1, B0, A0, B2, Col, etc. The motion information of the merge candidate indicated by label N is indicated by (mvLXN[0], mvLXN[1]), predFlagLXN, and refIdxLXN.

The selected (mvLXN[0], mvLXN[0]), predFlagLXN, and refIdxLXN are selected as the inter prediction parameters for the current block. The merge candidate selection unit 30362 stores the inter prediction parameters of the selected merge candidate in the prediction parameter memory 307 and outputs them to the inter prediction image generation unit 309.

(GPM forecast)
GPM prediction is used in GPM mode, and generates a predicted image using two non-rectangular prediction units that divide the target CU by a line segment. An example of a line segment is shown in FIG. 8(a). The line segment spanning the target CU is defined by the angle index angleIdx (angle variable) and distance index distanceIdx (distance variable) shown in FIG. 8(b). angleIdx indicates the angle φ between a vertical line and the line segment. distanceIdx indicates the distance ρ from the center of the target CU to the line segment. angleIdx is an integer value assigned to the angle of the line segment. For example, if 360 degrees are roughly divided into 32 equal parts, the value shown in FIG. 8(c) is assigned. Note that in this example, as mentioned "roughly," 360 degrees are not divided into perfect equal parts, but are divided into 32 using a square scale (coordinates). Note that a configuration in which 360 degrees are divided into 64 parts and even finer angle indexes are sent may also be used.

In generating a predicted image for GPM prediction, two "rectangular" predicted images (temporary predicted images) including a non-rectangular prediction unit may be derived, and the two rectangular regions may be derived by weighting according to the shape of the prediction unit. The motion compensation unit 3091 derives two temporary predicted images for the target CU, and the GPM synthesis unit 30952 derives a predicted image by applying weighting processing to each pixel of the two temporary predicted images according to the pixel's position. This processing is called GPM synthesis processing. Processing other than prediction (e.g., transformation (inverse transformation) and quantization (inverse quantization)) is applied to the entire target CU.

The GPM prediction unit 30377 derives prediction parameters for two non-rectangular regions and supplies them to the inter-prediction image generation unit 309. To simplify the processing, GPM prediction may be configured not to use bi-prediction. In this case, unidirectional inter-prediction parameters are derived for the non-rectangular regions.

(Decoding syntax elements in GPM prediction)
7 is a diagram showing the syntax configuration notified by merge_data(). The parameter decoding unit 302 decodes syntax elements in the encoded data, and the GPM prediction unit 30377 (inter-prediction parameter derivation unit 303) derives parameters for GPM prediction.

general_merge_flag is a flag that indicates whether merge mode is used and is notified when the current block is not in skip mode. If it is in skip mode, the inter prediction parameter derivation unit 303 sets general_merge_flag=1.

When ciip_flag is 0, the parameter decoding unit 302 (CU decoding unit 3022) decodes the GPM prediction syntax elements merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1. Alternatively, merge_gpm_partition_idx and merge_gpm_idx0 may be decoded, and merge_gpm_idx1 may also be decoded if MaxNumGpmMergeCand>2. merge_gpm_partition_idx (mergeGpmPartitionIdx) is an index (partition index) that indicates the division pattern (mode number) of the GPM prediction mode. The partition index indicates the combination of angleIdx and distanceIdx that identify the line segment spanning the target block to divide the target block into two regions. The motion information used to generate a predicted image for GPM prediction uses merge candidates. merge_gpm_idx0 and merge_gpm_idx1 are indices of merge candidates that indicate the motion information of two regions. The number of partition index options (number of partition patterns, number of GPM modes) is NumGPMFull, and merge_gpm_partition_idx takes an integer value between 0 and NumGPMFull-1 (=63). Hereinafter, parameters that indicate the shape used to derive a predicted image for GPM prediction will be referred to as GPM parameters. GPM parameters may be merge_gpm_partition_idx (mergeGpmPartitionIdx) or a pair of angleIdx and distanceIdx.

MergeGpmFlag is a flag that indicates whether or not to perform GPM prediction on the target block. If at least the following conditions (GPM determination conditions) are met, the GPM prediction unit 30377 sets MergeGpmFlag=1 (GPM prediction on); otherwise, the GPM prediction unit 30377 sets MergeGpmFlag=0.
(GPM judgment conditions)
・sps_gpm_enabled_flag=1
・sps_gpm_lfnst_enabled_flag=1
・slice_type is B slice ・general_merge_flag=1
・cbWidth>=8 and cbHeight>=8
・cbWidth<8*cbHeight and cbHeight<8*cbWidth
・regular_merge_flag=0
・merge_subblock_flag=0
・ciip_flag=0
The GPM prediction unit 30377 outputs the derived parameters to the GPM synthesis unit 30952.

(Movement information derivation process in GPM prediction)
The GPM prediction unit 30377 derives merge indexes m and n from the syntax elements merge_gpm_idx0 and merge_gpm_idx1 that indicate the motion information of two non-rectangular regions, as follows:

m = merge_gpm_idx0
n = merge_gpm_idx1 + (merge_gpm_idx1 >= m) ? 1 : 0
In the following, the merge candidate pointed to by merge index m is denoted as M, and the merge candidate pointed to by merge index n is denoted as N.

The merge prediction parameter derivation unit 3036 derives motion information (mvLXM, mvLXN, refIdxLXM, refIdxLXN, predFlagLXM, predFlagLXN, bcwIdx, mergeCandList, etc.) for merge candidates M and N using the method described in (Merge Prediction). The GPM prediction unit 30377 uses this motion information to set the motion vectors mvA and mvB, reference indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB for merge_gpm_idx0 and merge_gpm_idx1 as follows:

mvA[0] = mvLXM[0]
mvA[1] = mvLXM[1]
refIdxA = refIdxLXM
predListFlagA = X
Here, the GPM prediction unit 30377 sets the lowest 1 bit of m to X (m & 0x01). Note that if predFlagLXM is 0, the GPM prediction unit 30377 sets X to (1-X).

mvB[0] = mvLXN[0]
mvB[1] = mvLXN[1]
refIdxB = refIdxLXN
predListFlagB = X
Here, the GPM prediction unit 30377 sets the lowest 1 bit of n to X (n & 0x01). Note that if predFlagLXN is 0, the GPM prediction unit 30377 sets X to (1-X).

This motion information is referenced to generate temporary prediction images for two non-rectangular regions.

The GPM prediction unit 30377 derives angleIdx and distanceIdx corresponding to merge_gpm_partition_idx (mergeGpmPartitionIdx) according to the table shown in Figure 9.

(Motion vector storage process in GPM prediction)
The GPM prediction unit 30377 calculates the motion vectors (mvA, mvB) of the non-rectangular areas A and B and the reference picture information (predFlagA, prefFlagB, refIdxLA,
refIdxLB) are stored in memory in 4x4 sub-block units.

Below, we will explain the processing common to the first and second examples.

numSbX and numSbY are the number of 4x4 sub-blocks in the horizontal and vertical directions of the current block, respectively. The GPM prediction unit 30377 sets numSbX = cbWidth >> 2 and numSbY = cbHeight >> 2. The GPM prediction unit 30377 derives the variables displacementX, displacementY, isFlip, and shiftHor as follows:

displacementX = angleIdx
displacementY = (angleIdx + 8) % 32
isFlip = (angleIdx>=13 &&angleIdx<=27) ? 1 : 0
shiftHor = (angleIdx%16==8 || (angleIdx%16!=0 &&cbHeight>=cbWidth)) ? 0 : 1
If shiftHor is 0, the GPM prediction unit 30377 derives offsetX and offsetY as follows:

offsetX = (-cbWidth) >> 1
offsetY = ((-cbHeight) >> 1) + ((angleIdx<16) ? (distanceIdx * cbHeight) >> 3 : -((distanceIdx * cbHeight) >> 3))
When shiftHor is 1, the GPM prediction unit 30377 derives offsetX and offsetY as follows:

offsetX = ((-cbWidth) >> 1) + ((angleIdx<16) ? (distanceIdx * cbWidth) >> 3 : -((distanceIdx * cbWidth) >> 3))
offsetY = (-cbHeight) >> 1
The GPM prediction unit 30377 executes the following process for each position (xSbIdx, ySbIdx) of a 4x4 sub-block where xSbIdx = 0 .. numSbX-1 and ySbIdx = 0 .. numSbY-1.

The GPM prediction unit 30377 uses the disLut shown in Figure 9 to calculate motionIdx as follows:

motionIdx = (((4 * xSbIdx + offsetX) << 1) + 5) * disLut[displacementX] + (((4 * ySbIdx + offsetY) << 1) + 5) * disLut[displacementY]
The GPM prediction unit 30377 derives sType as follows.

sType = (abs(motionIdx)<32) ? 2 : ((motionIdx<=0) ? (1-isFlip) : isFlip)
If sType is 0, the GPM prediction unit 30377 performs the following:

If A's prediction list flag is 0 (predListFlagA == 0), the GPM predictor 30377 stores A's motion vector in L0 as unidirectional prediction. If A's prediction list flag is not 0 (predListFlagA != 0), the GPM predictor 30377 stores A's motion vector in L1 as unidirectional prediction.

predFlagL0 = (predListFlagA==0) ? 1 : 0
predFlagL1 = (predListFlagA==0) ? 0 : 1
refIdxL0 = (predListFlagA==0) ? refIdxA : -1
refIdxL1 = (predListFlagA==0) ? -1 : refIdxA
mvL0[0] = (predListFlagA==0) ? mvA[0] : 0
mvL0[1] = (predListFlagA==0) ? mvA[1] : 0
mvL1[0] = (predListFlagA==0) ? 0 : mvA[0]
mvL1[1] = (predListFlagA==0) ? 0 : mvA[1]
Otherwise, if sType is 1, or if sType is 2 and predListFlagA+predListFlagB is not 1, then the GPM predictor 30377 performs the following: where predListFlagA+predListFlagB is not 1 indicates that the reference picture lists of A and B are the same.

If the prediction list flag for B is 0 (predListFlagB == 0), the GPM predictor 30377 stores the motion vector for B in L0 as unidirectional prediction. If the prediction list flag for B is not 0 (predListFlagB != 0), the GPM predictor 30377 stores the motion vector for B in L1 as unidirectional prediction.

predFlagL0 = (predListFlagB==0) ? 1 : 0
predFlagL1 = (predListFlagB==0) ? 0 : 1
refIdxL0 = (predListFlagB==0) ? refIdxB : -1
refIdxL1 = (predListFlagB==0) ? -1 : refIdxB
mvL0[0] = (predListFlagB==0) ? mvB[0] : 0
mvL0[1] = (predListFlagB==0) ? mvB[1] : 0
mvL1[0] = (predListFlagB==0) ? 0 : mvB[0]
mvL1[1] = (predListFlagB==0) ? 0 : mvB[1]
Otherwise (if sType is 2 and predListFlagA+predListFlagB is 1), the GPM predictor 30377 performs the following: where predListFlagA+predListFlagB being 1 indicates that the reference picture lists of A and B are different.

If A's prediction list flag is 0 (predListFlagA == 0), the GPM prediction unit 30377 stores A's motion vector in L0 and sets up bidirectional prediction with B's motion vector in L1. If A's prediction list flag is not 0 (predListFlagA != 0), the GPM prediction unit 30377 stores B's motion vector in L0 and sets up bidirectional prediction with A's motion vector in L1.

predFlagL0 = 1
predFlagL1 = 1
refIdxL0 = (predListFlagA==0) ? refIdxA : refIdxB
refIdxL1 = (predListFlagA==0) ? refIdxB : refIdxA
mvL0[0] = (predListFlagA==0) ? mvA[0] : mvB[0]
mvL0[1] = (predListFlagA==0) ? mvA[1] : mvB[1]
mvL1[0] = (predListFlagA==0) ? mvB[0] : mvA[0]
mvL1[1] = (predListFlagA==0) ? mvB[1] : mvA[1]
(GPM prediction processing, GPM synthesis flow)
10 is a flowchart showing the flow of GPM prediction and GPM merging processes. The following describes the flow of the processes on the assumption that all the conditions for setting MergeGpmFlag=1 (GPM prediction on) as described above are met.

In S3501, the parameter decoding unit 302 decodes syntax elements related to the GPM notified by merge_data(), etc.

In S3502, the GPM prediction unit 30377 derives MergeGpmFlag and determines whether MergeGpmFlag is 1.

If MergeGpmFlag is not 1 in S3502, the GPM prediction unit 30377 turns off GPM prediction in S3506 and terminates processing.

In S3502, if MergeGpmFlag is 1, the GPM prediction unit 30377 performs GPM prediction processing, and the GPM synthesis unit 30952 (described below) performs GPM synthesis processing.

In S3503, the GPM prediction unit 30377 derives prediction parameters such as motion information. The motion compensation unit 3091, described below, uses the motion information to generate two temporary predicted images, predSamplesLA and predSamplesLB.

In S3504, the GPM synthesis unit 30952 derives the weighting coefficient wValue and generates a predicted image.

In S3505, the GPM prediction unit 30377 stores the motion vector in memory.

In the above GPM prediction, a predicted image is generated for two predicted images with different motion parameters by using weights that indicate the shape of the prediction unit.

(Example of GPM mode number prediction)
Hereinafter, a method for GPM prediction of a target block using GPM parameters of a block adjacent to the target block (e.g., a block above or to the left) will be described.
19A and 19B are diagrams showing PM division lines and candidate GPM division lines for a target block. Because object boundaries (edges) are continuous, when a GPM division line of an adjacent block intersects with a target block, the target block is likely to have an edge passing through that intersection. In other words, this is the case when the edges intersect on the target block, making the edges continuous, as shown in FIG. 19A. Alternatively, edges may be continuous across blocks, as shown in FIG. 19B. In this embodiment, a configuration is described in which the GPM division pattern (GPM mode number) of a target block is predicted from the intersection of the division lines of surrounding GPM predictions and the target block.

Below, we explain a method for predicting the GPM mode number of a target block from the GPM parameters of neighboring blocks. This allows GPM prediction to be performed while reducing the amount of coding required for the GPM parameters, thereby improving coding efficiency.

Decoding Syntax Elements in an Example of GPM Mode Number Prediction
(Basic syntax structure)
Figures 11(a) to 11(c) show examples of syntax configurations that transmit GPM parameters of a target block using GPM parameters of adjacent blocks. As in this example, when merge mode is selected and ciip_flag is 0, GPM prediction is used, and GPM parameters may be transmitted. Figure 11(a) shows an example of a syntax configuration that switches between transmitting merge_gpm_angle_idx (hereinafter referred to as small range syntax) or merge_gpm_partition_idx (hereinafter referred to as full range syntax) depending on the state of the GPM parameters of the adjacent block. merge_gpm_angle_idx is a shortened syntax with a smaller value range than the full range. merge_gpm_partition_idx is a full range syntax.

As shown in the figure, the parameter decoding unit 302 derives the variable mergeGpmAvailableNeighbor, which indicates whether GPM information of a neighboring block is available, and decodes merge_gpm_angle_idx if mergeGpmAvailableNeighbor is enabled (e.g., 1). Otherwise (if mergeGpmAvailableNeighbor is disabled (e.g., 0)), the parameter decoding unit 302 decodes merge_gpm_partition_idx. Here, merge_gpm_angle_idx is a syntax element that indicates the angle of the partition line. merge_gpm_angle_idx takes an integer value between 0 and 9, but is not limited to this. As another example, it may take a value between 0 and 7. merge_gpm_partition_idx is an index that indicates the GPM parameters (angle and distance). The subsequent syntax elements decoded by the parameter decoding unit 302 have already been explained and will be omitted. The maximum value cMax (e.g., 9) that merge_gpm_angle_idx can take is smaller than the cMax (e.g., 63) of merge_gpm_partition_idx. The binarization of merge_gpm_angle_idx may use TB binary using cMax (see below). The binarization of merge_gpm_partition_idx may also use FB binary (see below).

An image decoding device comprising a parameter decoding unit that decodes GPM parameters of a current block from encoded data, and a prediction unit that derives a predicted image from two predicted images and weights. The parameter decoding unit decodes different parameters depending on the GPM parameters of adjacent blocks, and the prediction unit includes a process for deriving GPM parameters of the current block from the GPM parameters of the adjacent blocks. The prediction unit derives weights based on the GPM parameters and uses the weights to derive a predicted image.

As described above, the amount of code can be reduced by switching the syntax elements of the GPM parameters to be coded or decoded depending on the parameter status of the surrounding GPM modes. In addition to the basic configuration described above, the following variations can also be used.

(Modification 1: Configuration using merge_gpm_mpm_flag)
FIG. 11(b) shows an example of a syntax configuration in which the additional flag merge_gpm_mpm_flag is further coded and decoded, and whether to transmit the small-range syntax merge_gpm_angle_idx or the full-range syntax merge_gpm_partition_idx is switched depending on the state of the GPM parameters of the adjacent block.

As shown in the figure, the parameter decoding unit 302 decodes merge_gpm_mpm_flag when the variable mergeGpmAvailableNeighbor is 1. merge_gpm_mpm_flag is a syntax element that indicates whether or not GPM mode number prediction is to be performed on the current block. When merge_gpm_mpm_flag is 1, it indicates that GPM mode number prediction is to be performed, and when it is 0, it indicates that GPM mode number prediction is not to be performed. When mergeGpmAvailableNeighbor is 0, merge_gpm_mpm_flag is not decoded and is set to 0. When merge_gpm_mpm_flag is 1, the parameter decoding unit 302 decodes merge_gpm_angle_idx, which is a syntax element that indicates the angle of the partition line. On the other hand, when merge_gpm_mpm_flag is 0, the parameter decoding unit 302 decodes merge_gpm_partition_idx, as in the case when mergeGpmAvailableNeighbor==0.

This configuration switches the syntax elements to be coded and decoded depending on the parameter state of the surrounding GPM modes, and also uses the additional flag merge_gpm_mpm_flag to switch between a GPM mode with limited possible patterns and a GPM mode with a wide range of possible patterns. This achieves both code size reduction and diverse prediction.

(Modification 2: Configuration 2 using merge_gpm_mpm_flag)
In Modification 2, an example will be described in which a list of predicted values of GPM mode numbers (candGPMList) is derived and used. Fig. 11(c) shows an example of a syntax configuration that switches between transmitting the small-range syntax merge_gpm_mpm_flag or the full-range syntax merge_gpm_partition_idx depending on the state of the GPM parameters of adjacent blocks.

As shown in the figure, the parameter decoding unit 302 decodes merge_gpm_mpm_flag when the variable mergeGpmAvailableNeighbor is 1. When merge_gpm_mpm_flag is 1, the parameter decoding unit 302 decodes mpm_merge_gpm_partition_idx. mpm_merge_gpm_partition_idx is a syntax element that indicates which candidate to use from a list (candGPMList) that holds multiple GPM mode numbers that are likely to be used. On the other hand, when merge_gpm_mpm_flag is 0, the parameter decoding unit 302 decodes remainder_merge_gpm_partition_idx. remainder_merge_gpm_partition_idx is a syntax element that indicates which mode number to use from among the mode numbers other than those included in candGPMList. The method for deriving the partition mode using mpm_merge_gpm_partition_idx and remainder_merge_gpm_partition_idx is described in (GPM mode number prediction using candGPMList).

If mergeGpmAvailableNeighbor is 0, the parameter decoding unit 302 directly decodes merge_gpm_partition_idx.

In the following embodiment, the GPM mode number of the target block is derived as the variable mergeGpmPartitionIdx, and angleIdx and distanceIdx are derived using mergeGpmPartitionIdx. When decoding merge_gpm_partition_idx, the parameter decoding unit 302 may set it to mergeGpmPartitionIdx.

mergeGpmPartitionIdx = merge_gpm_partition_idx
Conversely, when the parameter decoding unit 302 does not decode merge_gpm_partition_idx, it derives mergeGpmPartitionIdx (described later).

The parameter decoding unit 302 (CU decoding unit 3022) may binarize (represent in binary) syntax elements such as merge_gpm_angle_idx and remainder_merge_gpm_partition_idx using, for example, truncate binary (TB) (TB binary). TB is a binarization method in which the upper limit cMax of the value that a syntax element can take is determined.

(TB binary example)
The TB binary may be derived as follows:

n = cMax + 1
k = Floor(Log2(n))
u = (1 << (k + 1)) - n
If the value of the syntax element synVal (here, merge_gpm_partition_idx) is less than u, the parameter decoding unit 302 derives the TB binary using a Fixed Length Binary (hereinafter referred to as FL Binary) using cMax = (1<<k)-1. Otherwise (if synVal is greater than or equal to u), cMax = (1<<(k+1))-1 is set.

When deriving the FL binary, the parameter decoding unit 302 may derive the BIN length fixedLength of the syntax element and represent synVal in binary using fixedLength bits.

fixedLength = Ceil(Log2(cMax + 1))
Furthermore, the parameter decoding unit 302 may perform binarization of mpm_merge_gpm_partition_idx (TR binary) using Truncated Rice (TR), in which cMax is determined and the Rice parameter is set to 0. For example, if mpm_merge_gpm_partition_idx takes a value from 0 to 5 and one is selected from six candidates, the value of mpm_merge_gpm_partition_idx is coded as a coded bit string of up to 5 bits (binary values: 0, 10, 110, 1110, 11110, 11111).

(Example of derivation of mergeGpmAvailableNeighbor)
The above mergeGpmAvailableNeighbor is a variable derived from the GPM parameters of the neighboring block, and indicates whether or not it is possible to use the GPM information of the neighboring block.

The parameter decoding unit 302 may derive mergeGpmAvailableNeighbor using the GPM mode number GpmModeA of the upper adjacent block, the GPM mode number GpmModeL of the left adjacent block, and a predetermined table. mergeGpmAvailableNeighbor may also be determined by whether the GPM dividing line in the adjacent block intersects with the target block.

For example, the parameter decoding unit 302 derives mergeGpmAvailableNeighbor using the crstblNeighborForAbove table in Figure 12 and the crstblNeighborForLeft table in Figure 13. crstblNeighborForAbove is a table that derives the x-coordinate of the intersection between the dividing line and the target block for the upper adjacent block from the block size and GPM mode number. Also, crstblNeighborForLeft is a table that derives the y-coordinate of the intersection for the left adjacent block from the block size and GPM mode number.

The following process may be used to determine whether the upper adjacent block NA can be used. The top left coordinates of NA are represented as (xNA, yNA), the width of NA as widthA, the height as heightA, and the GPM mode number as GpmModeA. In this case, the parameter decoding unit 302 derives the width index wIdxA and the height index hIdxA as follows:

wIdxA = log2(widthA) - 2
hIdxA = log2(heightA) - 2
Note that wIdxA and hIdxA may be the difference between the cbWidth and cbHeight of the target block.

wIdxA = log2(widthA) - log2(cbWidth)
hIdxA = log2(heightA) - log2(cbHeight)
To prevent the relative size from being less than 0, you can do the following:

wIdxA = log2(widthA) - log2(cbWidth) + Log2GPMBlkSize
hIdxA = log2(heightA) - log2(cbHeight) + Log2GPMBlkSize
Furthermore, it may be clipped to a value between 0 and a predetermined maximum value.

Here, Log2GPMBlkSize is a predetermined constant (4, 5, 6, 7, 8, etc.). For example, if the maximum block size used in GPM is 128, then Log2BlkSize = log2(128) = 7. The direction of the difference may also be reversed.

wIdxA = log2(cbWidth) - log2(widthA) + Log2GPMBlkSize
hIdxA = log2(cbHeight) - log2(heightA) + Log2GPMBlkSize
The parameter decoding unit 302 derives the x-coordinate (xTmp) of the intersection point using crstblNeighborForAbove[hIdxA][wIdxA][GpmModeA]. In this case, if the top-left coordinate of NA is (xNA, yNA), the coordinate of the intersection point is (xNA+xTmp, yNA+heightA). xTmp is the distance from the left end of NA. If the top-left coordinate of NA (xNA, yNA) is taken as the reference (0,0), the coordinate of the intersection point may be written as (xTmp, heightA). Furthermore, if the top-left coordinate of the target block is (xCb, yCb), the coordinate of the intersection point can be derived as (xNA-xCb+xTmp, yCb). If the top-left coordinate (xCb, yCb) is taken as the reference (0,0), the coordinate of the intersection point may be written as (xNA+xTmp, 0). From the above explanation, when the upper left coordinate (xCb, yCb) is taken as the reference (0,0) and the x coordinate of this intersection is expressed as x0, whether or not it is possible to use the GPM information of the upper adjacent block may be derived using the following (Condition 1).

(Condition 1) The upper adjacent block is available, and x0 is equal to or greater than 0 and equal to or less than the width of the target block.
In other words,
mergeGpmAvailableAbove = availableAbove && x0 >= 0 && x0 <= cbWidth
availableAbove is a flag indicating whether the upper block is available or not; if it is available, it is 1 (true); if it is not available, it is 0 (false).

The parameter decoding unit 302 may derive mergeGpmAvailableAbove directly from the relative sizes of the current block and the upper adjacent block, or the sizes hIdxA and wIdxA of the upper adjacent block, GpmModeA, and the table gpmNeighborForAbove.
mergeGpmAvailableAbove = gpmNeighborForAbove[hIdxA][wIdxA][GpmModeA]
The table may be derived as follows:

gpmNeighborForAbove[hIdxA][wIdxA][GpmModeA] = crstblNeighborForAbove[hIdx][wIdx][GpmModeA] >= 0 && crstblNeighborForAbove[hIdxA][wIdxA][GpmModeA] <= TH
where TH is a predetermined value. Alternatively, a table may be derived as follows:

gpmNeighborForAbove[hIdxA][wIdxA][GpmModeA] = crstblNeighborForAbove[hIdxA][wIdxA][GpmModeA] + 1
Similarly, it may be determined whether the left adjacent block NL can be used. The upper left coordinates of NL are represented as (xNL, yNL), the width of NL is represented as widthL, the height is represented as heightL, and the GPM mode number is represented as GpmModeL. In this case, the parameter decoding unit 302 derives the width index wIdx and the height index hIdxL as follows:

wIdxL = log2(widthL) - 2
hIdxL = log2(heightL) - 2
Note that wIdx and hIdx may be the difference between the cbWidth and cbHeight of the target block.

wIdxL = log2(widthL) - log2(cbWidth)
hIdxL = log2(heightL) - log2(cbHeight)
To prevent the relative size from becoming less than 0, you can do the following: You can also clip as described above.

wIdxL = log2(widthL) - log2(cbWidth) + Log2GPMBlkSize
hIdxL = log2(heightL) - log2(cbHeight) + Log2GPMBlkSize
The parameter decoding unit 302 derives the y-coordinate (yTmp) of the intersection point using crstblNeighborForLeft[hIdxL][wIdxL][GpmModeL]. In this case, if the top left coordinate of NL is (xNL, yNL), the coordinate of the intersection point is (xNL+widthL, yNL+yTmp). yTmp is the distance from the top end of NL. If the top left coordinate of NL (xNA, yNA) is taken as the reference (0,0), the coordinate of the intersection point may be written as (widthL, yTmp). Furthermore, if the top left coordinate of the target block is (xCb, yCb), the coordinate of the intersection point can be derived as (xCb, yNL-yCb+yTmp). If the top left coordinate (xCb, yCb) is taken as the reference (0,0), the coordinate of the intersection point may be written as (0, yNL+yTmp). From the above explanation, when the upper left coordinate (xCb, yCb) is taken as the reference (0,0) and the y coordinate of this intersection is expressed as y0, whether or not it is possible to use the GPM information of the left adjacent block may be derived from the following (Condition 2).

(Condition 2) The adjacent block on the left is available, y0 is greater than or equal to 0, and y0 is less than or equal to the height of the target block.

mergeGpmAvailableLeft = availableLeft && y0 >= 0 && y0 <= cbHeight
availableLeft is a flag indicating whether the left block is available or not; if it is available, it is 1 (true); if it is not available, it is 0 (false).

The parameter decoding unit 302 may derive mergeGpmAvailableLeft directly from the relative sizes of the current block and the left adjacent block, or the sizes hIdxL, wIdxL and GpmModeL of the left adjacent block, and the table gpmNeighborForLeft.
mergeGpmAvailableLeft = gpmNeighborForLeft[hIdxL][wIdxL][GpmModeL]
Here, the table may be derived as follows:

gpmNeighborForLeft[hIdxL][wIdxL][GpmModeL] = crstblNeighborForLeft[hIdxL][wIdxL][GpmModeL] >= 0 && crstblNeighborForLeft[hIdxL][wIdxL][GpmModeL] <= TH
where TH is a predetermined value. Alternatively, a table may be derived as follows:

gpmNeighborForLeft[hIdxL][wIdxL][GpmModeL] = crstblNeighborForLeft[hIdxL][wIdxL][GpmModeL] + 1
If (Condition 1) or (Condition 2) is met, the parameter decoding unit 302 may set mergeGpmAvailableNeighbor to 1. This can be expressed as the following equation.

mergeGpmAvailableNeighbor = mergeGpmAvailableAbove || mergeGpmAvailableLeft
Note that the above || can also be +.

As described above, the intersection point may be derived using a table that associates the size of the adjacent block, the GPM prediction mode number of the adjacent block, and the coordinate position of the intersection point. Furthermore, as added, a table that also includes the determination of the coordinate position of the intersection point may be used.

(Summary of derivation examples of mergeGpmAvailableNeighbor)
To summarize what has already been described, the parameter decoding unit 302 may derive mergeGpmAvailableNeighbor as follows.

mergeGpmAvailableNeighbor = gpmNeighborForAbove[hIdxA][wIdxA][GpmModeA] || gpmNeighborForLeft[hIdxL][wIdxL][GpmModeL]
(Process flow for predicting GPM mode numbers for target blocks)
17 and 18 are flowcharts showing the flow of processing for predicting the GPM mode number of the target block.

In S3501, the parameter decoding unit 302 decodes syntax elements related to the GPM notified in merge data, etc.

In S3502, the GPM prediction unit 30377 derives MergeGpmFlag and determines whether MergeGpmFlag is 1.

If MergeGpmFlag is not 1 in S3502, the GPM prediction unit 30377 turns off GPM prediction in S3506 and terminates processing.

If MergeGpmFlag is 1 in S3502, it is determined in S3507 whether to perform GPM mode number prediction for the target block. As a condition for this determination, for the syntax element shown in Figure 11(a), mergeGpmAvailableNeighbor is derived as shown in Figure 17, and it is determined whether mergeGpmAvailableNeighbor is 1. For the syntax element shown in Figure 11(b), it is determined whether merge_gpm_mpm_flag is 1 as shown in Figure 18.

If GPM mode number prediction for the target block is not performed in S3507, the process transitions to S3503.

If GPM mode number prediction is performed for the target block in S3507, GPM mode prediction processing for the target block is performed in S3508.

The subsequent processing is the same as in Figure 10, so explanation will be omitted.

In this embodiment, the GPM prediction unit 30377 may derive angleIdx and distanceIdx using the predicted GPM mode number mergeGpmPartitionIdx instead of gpm_partition_idx decoded from the encoded data, and generate a predicted image from weighting corresponding to the predicted shape. In this case, the GPM prediction unit 30377 may derive angleIdx and distanceIdx using the table shown in Figure 9(c) that associates mergeGpmPartitionIdx with angleIdx and distanceIdx.

Below, we explain how to derive the GPM mode number of the target block from merge_gpm_angle_idx decoded in the syntax structure of Figure 11(a) or Figure 11(b). We also explain how to derive the GPM mode number of the target block from mpm_merge_gpm_partition_idx decoded in the syntax structure of Figure 11(c).

(GPM mode number prediction using merge_gpm_angle_idx)
The GPM prediction unit 30377 of this embodiment derives the GPM mode number of the target block by the following process.

First, the GPM prediction unit 30377 derives mergeGpmAvailableAbove and mergeGpmAvailableLeft for the intersection using a method similar to that described above.

When mergeGpmAvailableAbove is 1, the GPM prediction unit 30377 predicts the GPM mode number of the target block using intersection x0. When mergeGpmAvailableAbove is 0 and mergeGpmAvailableLeft is 1, the GPM prediction unit 30377 predicts the GPM mode number of the target block using intersection y0. Note that in the above example, when mergeGpmAvailableAbove and mergeGpmAvailableLeft are both 1, priority is given to the upper adjacent block, but priority may also be given to the left adjacent block.

(When using the upper adjacent block (intersection x0))
The GPM prediction unit 30377 derives angleIdx using merge_gpm_angle_idx and the table shown in Fig. 16. Fig. 16(a) is a table for deriving angleIdx when merge_gpm_angle_idx takes a value from 0 to 9. Fig. 16(b) is a table for deriving angleIdx when merge_gpm_angle_idx takes a value from 0 to 7.

The GPM prediction unit 30377 sequentially searches through the possible values for the GPM mode number (0 to NumGPMFull-1 (=63)). First, it extracts the mode number i from the GPM modes whose angle matches angleIdx or angleIdx+16. If there are multiple i's, it selects the i's that minimizes the distance between the intersection points x1 and x0 of the target block's partition line with the upper adjacent block. Here, crstblCurrForAbove in Figure 14 is a table that derives the x-coordinate of the intersection point between the partition line and the upper adjacent block for the target block from the block size and GPM mode number. Pseudocode for the above process is shown below. cbWidth and cbHeight are the width and height of the target block. Furthermore, g_GeoParams is a table like the one shown in Figure 9, and is used to derive angleIdx and distanceIdx from the GPM mode number. MAX_INT is the maximum value of an integer (for example, 1<<31-1 for a 32-bit signed integer).
(start)
Derive angleIdx from merge_gpm_angle_idx and the table in Figure 16
wIdxA = log2(cbWidth) - 2
hIdxA = log2(cbHeight) - 2
minDis = MAX_INT
finalCurrGpmMode = -1
for (i = 0; i <NumGPMFull; i++)
tmpAngleIdx = g_GeoParams[i][0]
if (tmpAngleIdx == angleIdx || tmpAngleIdx == angleIdx + 16)
x1 = crstblCurrForAbove[hIdxA][wIdxA][i]
if (Abs(x0-x1) < minDis)
minDis = Abs(x0-x1)
finalCurrGpmMode = i
mergeGpmPartitionIdx = finalCurrGpmMode
(end)
Finally, the value stored in finalCurrGpmMode is set as the GPM mode number (mergeGpmPartitionIdx) of the target block.

(When using the left adjacent block (intersection y0))
The GPM mode number of the target block is predicted using the same procedure as for the upper adjacent block.

The GPM prediction unit 30377 derives angleIdx using merge_gpm_angle_idx and the table shown in Figure 16.

The GPM prediction unit 30377 sequentially searches through the possible values for the GPM mode number (0 to NumGPMFull-1 (=63)). First, it extracts the mode number i from the GPM modes whose angle matches angleIdx or angleIdx+16. If there are multiple i's, it determines the i that minimizes the distance between the intersection points y1 and y0 of the target block's left-side adjacent block obtained by the target block's division line as the GPM mode of the target block. Here, crstblCurrForLeft in Figure 15 is a table that derives the y-coordinate of the intersection point between the division line and the left-side adjacent block for the target block from the block size and GPM mode number. Pseudocode for the above process is shown below.
(start)
Derive angleIdx from merge_gpm_angle_idx and the table in Figure 18
wIdxL = log2(cbWidth) - 2
hIdxL = log2(cbHeight) - 2
minDis = MAX_INT
finalCurrGpmMode = -1
for (i = 0; i <NumGPMFull; i++)
tmpAngleIdx = g_GeoParams[i][0]
if (tmpAngleIdx == angleIdx || tmpAngleIdx == angleIdx + 16)
y1 = crstblCurrForLeft[hIdxL][wIdxL][i]
if (Abs(y0-y1) < minDis)
minDis = Abs(y0-y1)
finalCurrGpmMode = i
mergeGpmPartitionIdx = finalCurrGpmMode
(end)
Finally, the value stored in finalCurrGpmMode is set as the GPM mode number (mergeGpmPartitionIdx) of the target block.

As described above, when the GPM partition line of an adjacent block and the target block intersect at a point, the partition index (mode number) of the GPM prediction mode of the target block may be derived using the intersection point, the partition angle derived from the index indicating the decoded partition angle, a table correlating the size and GPM prediction mode number of the adjacent block with the coordinate position of the intersection point, and a table correlating the size and GPM prediction mode number of the target block with the coordinate position of the intersection point.

(GPM mode number prediction using candGPMList)
In this embodiment, the GPM prediction unit 30377 derives the GPM mode number of the target block using mpm_merge_gpm_partition_idx, remainder_merge_gpm_partition_idx, and candGPMList.

First, the GPM prediction unit 30377 derives mergeGpmAvailableAbove and mergeGpmAvailableLeft for the intersection using a method similar to that described above.

When mergeGpmAvailableAbove is 1, the GPM prediction unit 30377 constructs candGPMList using the GPM information of the upper adjacent block. When mergeGpmAvailableAbove is 0 and mergeGpmAvailableLeft is 1, the GPM prediction unit 30377 constructs candGPMList using the GPM information of the left adjacent block. Note that in the above example, when mergeGpmAvailableAbove and mergeGpmAvailableLeft are both 1, the upper adjacent block is prioritized, but it may also be configured to prioritize the left adjacent block.

When merge_gpm_mpm_flag is 1, the GPM prediction unit 30377 derives the GPM mode number of the target block using candGPMList and the decoded mpm_merge_gpm_partition_idx.

mergeGpmPartitionIdx = candGPMlist[mpm_merge_gpm_partition_idx]
On the other hand, when merge_gpm_mpm_flag is 0, the GPM prediction unit 30377 derives the GPM mode number of the current block from candGPMList and the decoded remainder_merge_gpm_partition_idx. Here, MaxGPMList is the maximum index value of candGPMList. In other words, when the number of MPMs in the GPM is NumGPMCand, MaxGPMList=NumGPMCand-1. In this case, the parameter decoding unit 302 derives mergeGpmPartitionIdx in the following steps.
1) Sort the list, i.e., if candGPMList[i] > candGPMList[j] for j = (i + 1)..MaxGPMList(i = 0..MaxGPMList-1), swap the i-th and j-th values in the list.

(candGPMList[i], candGPMList[j]) = Swap(candGPMList[i], candGPMList[j])
2) Derive mergeGpmPartitionIdx in the following steps:
i. mergeGpmPartitionIdx = remainder_merge_gpm_partition_idx
ii. Increment the value of mergeGpmPartitionIdx by 1.
iii. For i = 0..MaxGPMList, if mergeGpmPartitionIdx >= candGPMList[i], increment the value of mergeGpmPartitionIdx by 1.

The above steps derive the GPM mode number for the target block. The structure of candGPMList is explained below.

(candGPMList configuration)
As the mode numbers that make up candGPMList, GPM mode numbers that pass through the intersection may be preferentially selected. In particular, among those that pass through the intersection, GPM mode numbers that have an angle close to the angle of the dividing line of the adjacent block may be preferentially selected. For example, candGPMList may be derived using the following steps.

1) The GPM prediction unit 30377 searches for modes where the difference between the intersection coordinate x0 or y0 and the intersection coordinate x1 or y1 derived using crstblNeighborForAbove and crstblNeighborForLeft is within a specific value (e.g., 1), and stores the GPM division modes that satisfy the conditions in the array tmpCandList[].

2) Among the GPM division modes stored in tmpCandList[], search for the mode whose angleIdx matches the angle angleIdxN of the division line of the adjacent block, and memorize its number (let's say it's the jth mode).
3) Derive candGPMList as follows:

candGPMList[i++] = tmpCandList[j]
candGPMList[i++] = tmpCandList[j-1]
candGPMList[i++] = tmpCandList[j+1]
candGPMList[i++] = tmpCandList[j-2]
candGPMList[i++] = tmpCandList[j+2]
Alternatively, the mode numbers constituting candGPMList may be preferentially selected from GPM mode numbers having angles close to the angle of the dividing line of the adjacent block, regardless of whether they pass through an intersection. In other words, they may be derived as follows:

candGPMList[i++] = GpmModeA + 1
candGPMList[i++] = GpmModeA - 1
candGPMList[i++] = GpmModeL + 1
candGPMList[i++] = GpmModeL - 1
Furthermore, a modulus operation (mod, where the symbol is %) may be used so that the value falls within a predetermined range as follows:

candGPMList[i++] = (GpmModeA + 1) % NumGPMFull
candGPMList[i++] = (GpmModeA - 1) % NumGPMFull
candGPMList[i++] = (GpmModeL + 1) % NumGPMFull
candGPMList[i++] = (GpmModeL - 1) % NumGPMFull
For example, if the number of partition index choices, NumGPMFull, is 64, it can be written as follows:

candGPMList[i++] = (GpmModeA + 1) % 64
candGPMList[i++] = (GpmModeA - 1) % 64
candGPMList[i++] = (GpmModeL + 1) % 64
candGPMList[i++] = (GpmModeL - 1) % 64
The processing may vary depending on whether the GPM of the neighboring block is available.

if (mergeGpmAvailableAbove) {
candGPMList[i++] = (GpmModeA + 1) % NumGPMFull
candGPMList[i++] = (GpmModeA - 1) % NumGPMFull
}
if (mergeGpmAvailableLeft) {
candGPMList[i++] = (GpmModeL + 1) % NumGPMFull
candGPMList[i++] = (GpmModeL - 1) % NumGPMFull
}
(Another process flow for predicting the GPM mode number of the target block)
In the above, the process of predicting the GPM mode number of the current block is performed by the GPM prediction unit 30377. Here, the process flow of predicting the GPM mode number of the current block may be performed by the parameter decoding unit 302.

(Example in which the intersection point is not on the target block)
(Example of deriving mergeGpmAvailableNeighbor)
When the intersection point is not on the target block as shown in FIG. 19(b), mergeGpmAvailableNeighbor may be derived as follows.

The parameter decoding unit 302 may derive the intersection point using the method already described. That is, for the upper adjacent block, it may derive the x-coordinate x0 of the intersection point using crstblNeighborForAbove, and for the left adjacent block, it may derive the y-coordinate y0 of the intersection point using crstblNeighborForLeft.

Here, whether or not it is possible to use the GPM information of the upper adjacent block and the left adjacent block may be derived from the following (Condition 3) and (Condition 4).
(Condition 3) The upper adjacent block is available, and x0 is equal to or greater than 0 and equal to or less than the width of the upper adjacent block.
(Condition 4) The left adjacent block is available, and y0 is equal to or greater than 0, and equal to or less than the height of the left adjacent block.
In other words,
mergeGpmAvailableAbove = availableAbove && x0 >= 0 && x0 <= widthA
mergeGpmAvailableLeft = availableLeft && y0 >= 0 && y0 <= height
Note that the parameter decoding unit 302 may derive mergeGpmAvailableAbove directly from the relative size between the target block and the upper adjacent block, or the size of the upper adjacent block, hIdxA, wIdxA and GpmModeA, and the table gpmNeighborForAbove, or may derive mergeGpmAvailableLeft directly from the relative size between the target block and the left adjacent block, or the size of the left adjacent block, hIdxL, wIdxL and GpmModeL, and the table gpmNeighborForLeft.

mergeGpmAvailableAbove = gpmNeighborForAbove[hIdxA][wIdxA][GpmModeA]
mergeGpmAvailableLeft = gpmNeighborForLeft[hIdxL][wIdxL][GpmModeL]
Then, when (Condition 3) or (Condition 4) is satisfied, the parameter decoding unit 302 decodes mer
geGpmAvailableNeighbor may be set to 1.

mergeGpmAvailableNeighbor = mergeGpmAvailableAbove || mergeGpmAvailableLeft
(GPM mode number prediction using merge_gpm_angle_idx)
Next, we will explain how to derive the GPM mode number of the target block when the intersection point is not on the target block, as shown in Figure 19(b). In this case, the GPM prediction unit 30377 uses a table different from the crstblCurrForAbove or crstblCurrForLeft described above.

Specifically, when using GPM information for the adjacent block above, the GPM prediction unit 30377 may use a table that associates the size and GPM prediction mode number of the target block with the y-coordinate position of the intersection point on the right side of the target block. Furthermore, when using GPM information for the adjacent block to the left, the GPM prediction unit 30377 may use a table that associates the size and GPM prediction mode number of the target block with the x-coordinate position of the intersection point on the bottom side of the block.

(Inter-prediction image generation unit 309)
When predMode indicates inter prediction mode, the inter prediction image generation unit 309 generates a prediction image of a block or sub-block by inter prediction using the inter prediction parameters input from the inter prediction parameter derivation unit 303 and the reference picture.

Figure 4 is a schematic diagram showing the configuration of the inter-prediction image generation unit 309 included in the prediction image generation unit 308 according to this embodiment. The inter-prediction image generation unit 309 includes a motion compensation unit (prediction image generation device) 3091 and a synthesis unit 3095. The synthesis unit 3095 includes an intra-inter synthesis unit 30951, a GPM synthesis unit 30952, a BIO unit 30954, and a weight prediction unit 3094.

(Motion Compensation)
The motion compensation unit 3091 (interpolated image generation unit 3091) generates an interpolated image (motion-compensated image) by reading a reference block from the reference picture memory 306 based on the inter-prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter-prediction parameter derivation unit 303. The reference block is a block located at a position shifted by mvLX from the position of the current block on the reference picture RefPicLX specified by refIdxLX. Here, if mvLX does not have integer precision, an interpolated image is generated by applying a filter called a motion compensation filter, which is used to generate pixels at decimal positions.

The motion compensation unit 3091 first derives the integer position (xInt, yInt) and phase (xFrac, yFrac) corresponding to the coordinates (x, y) in the prediction block using the following formula:

xInt = xPb+(mvLX[0]>>(log2(MVPREC)))+x
xFrac = mvLX[0]&(MVPREC-1)
yInt = yPb+(mvLX[1]>>(log2(MVPREC)))+y
yFrac = mvLX[1]&(MVPREC-1)
Here, (xPb, yPb) are the upper left coordinates of a bW*bH size block, where x = 0...bW-1 and y = 0...bH-1, and MVPREC indicates the precision of mvLX (1/MVPREC pixel precision), e.g., MVPREC = 16.

The motion compensation unit 3091 derives the temporary image temp[][] by performing horizontal interpolation on the reference picture refImg using an interpolation filter. In the following, Σ is the sum over k, where k=0..NTAP-1, shift1 is a normalization parameter that adjusts the value range, and offset1=1<<(shift1-1).

temp[x][y] = (ΣmcFilter[xFrac][k]*refImg[xInt+k-NTAP/2+1][yInt]+offset1)>>shift1
Next, the motion compensation unit 3091 performs vertical interpolation on the temporary image temp[][] to generate an interpolated image Pred
Derive [][]. Below, Σ is the sum over k=0..NTAP-1, shift2 is a normalization parameter that adjusts the range of values, and offset2=1<<(shift2-1).

Pred[x][y] = (ΣmcFilter[yFrac][k]*temp[x][y+k-NTAP/2+1]+offset2)>>shift2
In the case of bi-prediction, the above Pred[][] is used for the L0 list, and interpolated images PredL0[][] and PredL1[][] are derived for each L1 list, and an interpolated image Pred[][] is generated from PredL0[][] and PredL1[][].

The synthesis unit 3095 includes an IntraInter synthesis unit 30951, a GPM synthesis unit 30952, a weighted prediction unit 3094, and a BIO unit 30954.

(GPM synthesis section 30952)
The GPM synthesis unit 30952 derives a predicted image using temporary predicted images predSamplesLA and predSamplesLB and weighting parameters derived from angle variables. The temporary predicted images predSamplesLA and predSamplesLB are predicted images generated by the motion compensation unit 3091 using the motion information (mvX, refIdxX, predListFlagX, X=A or B). The angle variables are angleIdx and distanceIdx derived by the GPM prediction unit 30377.

The GPM synthesis unit 30952 derives the predicted pixel value pbSample using the following procedure, where nCbW = cbWidth and nCbH = cbHeight. First, the GPM synthesis unit 30952 derives bitDepth as follows for the weighting coefficient derivation process in GPM prediction and the motion vector storage process described below.

If cIdx is 0, the GPM synthesis unit 30952 sets bitDepth to the luminance pixel bit count BitDepthY.

If cIdx is 0, the GPM synthesis unit 30952 sets nW and nH to nCbW and nCbH, respectively. If cIdx is not 0, the GPM synthesis unit 30952 sets nW and nH to nCbW*SubWidthC and nCbH*SubHeightC, respectively. Here, SubWidthC and SubHeightC are values predetermined according to the chrominance format.

If cIdx is 0, the GPM synthesis unit 30952 sets both subW and subH to 1. If cIdx is not 0, the GPM synthesis unit 30952 sets subW and subH to SubWidthC and SubHeightC, respectively.

If cIdx is not 0, the GPM synthesis unit 30952 sets bitDepth to the chrominance pixel bit count, BitDepthC.

Next, the GPM synthesis unit 30952 derives the variables nW, nH, shift1, offset1, displacementX, displacementY, partFlip, and shiftHor as follows:

nW = (cIdx==0) ? nCbW : nCbW * SubWidthC
nH = (cIdx==0) ? nCbH : nCbH * SubHeightC
shift1 = Max(5, 17 - BitDepth)
offset1 = 1 << (shift1 - 1)
displacementX = angleIdx
displacementY = (angleIdx + 8) % 32
partFlip = (angleIdx>=13 &&angleIdx<=27) ? 0 : 1
shiftHor = (angleIdx%16==8 || (angleIdx%16!=0 &&nH>=nW)) ? 0 : 1
When shiftHor is 0, the GPM synthesis unit 30952 derives offsetX and offsetY as follows:

offsetX = (-nW) >> 1
offsetY = ((-nH) >> 1) + (angleIdx<16? (distanceIdx * nH) >> 3 : -((distanceIdx * nH) >> 3))
When shiftHor is 1, the GPM synthesis unit 30952 derives offsetX and offsetY as follows:

offsetX = ((-nW) >> 1) + (angleIdx<16? (distanceIdx * nW) >> 3 : -((distanceIdx * nW) >> 3))
offsetY = (-nH) >> 1
The GPM synthesis unit 30952 derives the predicted pixels pbSamples according to the following steps.

1) The GPM synthesis unit 30952 derives generalized coordinates (xL, yL) from coordinates (x, y) within the CU.

xL = (cIdx==0) ? x : x * SubWidthC
yL = (cIdx==0) ? y : y * SubHeightC
2) The GPM synthesis unit 30952 uses disLut shown in FIG. 9(b) to calculate the weighting coefficient wValue according to the coordinates within the CU as follows:

weightIdx = (((xL + offsetX) << 1) + 1) * disLut[displacementX] + (((yL + offsetY) << 1) + 1) * disLut[displacementY]
weightIdxL = partFlip ? 32 + weightIdx : 32 - weightIdx
wValue = Clip3(0, 8, (weightIdxL + 4) >> 3)
3) The GPM synthesis unit 30952 derives the value of pbSample as follows:

pbSamples[x][y] = Clip3(0, (1 << BitDepth) - 1, (predSamplesLA[x][y] * wValue + predSamplesLB[x][y] * (8 - wValue) + offset1) >> shift1)
(IntraInter compositing processing)
When ciip_mode is 1, the IntraInter synthesis unit 30951 generates a predicted image in CIIP (Combined intra prediction) mode by weighting the inter predicted image and the intra predicted image.

(BIO forecast)
In bi-prediction mode, the BIO unit 30954 generates a predicted image by referring to two predicted images (a first predicted image and a second predicted image) and a gradient correction term.

(Weighted Prediction)
The weighted prediction unit 3094 generates a predicted image of the block by multiplying the interpolated image PredLX by a weighting factor.

The inter-prediction image generation unit 309 outputs the generated prediction image of the block to the addition unit 312.

(Intra-prediction image generation unit)
When predMode indicates an intra prediction mode, the intra prediction image generation unit performs intra prediction using the intra prediction parameters input from the intra prediction parameter derivation unit and reference pixels read from the reference picture memory 306 .

The inverse quantization and inverse transform unit 311 inverse quantizes the quantized transform coefficients input from the parameter decoding unit 302 to obtain transform coefficients.

The adder 312 generates a decoded image for the block by adding, for each pixel, the predicted image for the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization and inverse transform unit 311. The adder 312 stores the decoded image for the block in the reference picture memory 306 and also outputs it to the loop filter 305.

(Configuration of video encoding device)
Next, the configuration of the video encoding device 11 according to this embodiment will be described. Fig. 20 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a transformation/quantization unit 103, an inverse quantization/inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, a coding parameter determination unit 110, a parameter encoding unit 111, a prediction parameter derivation unit 120, and an entropy encoding unit 104.

The predicted image generation unit 101 generates a predicted image for each CU. The predicted image generation unit 101 includes the inter predicted image generation unit 309 and intra predicted image generation unit already explained, so further explanation will be omitted.

The subtraction unit 102 subtracts the pixel values of the predicted image of the block input from the predicted image generation unit 101 from the pixel values of image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the transformation and quantization unit 103.

The transform/quantization unit 103 calculates transform coefficients by frequency transforming the prediction errors input from the subtraction unit 102, and derives quantized transform coefficients by quantizing them. The transform/quantization unit 103 outputs the quantized transform coefficients to the parameter coding unit 111 and the inverse quantization/inverse transform unit 105.

The inverse quantization and inverse transform unit 105 is the same as the inverse quantization and inverse transform unit 311 in the video decoding device 31, and so a detailed description will be omitted. The calculated prediction error is output to the adder 106.

The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes the general operation of each module.

The header encoding unit 1110 performs encoding processing on parameters such as header information, division information, prediction information, and quantized transform coefficients.

The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information, etc.

The CU encoding unit 1112 encodes CU information, prediction information, segmentation information, etc.

When a TU contains a prediction error, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.

The CT information encoding unit 1111 and the CU encoding unit 1112 supply syntax elements such as inter-prediction parameters (predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra-prediction parameters, and quantized transform coefficients to the parameter encoding unit 111.

The entropy coding unit 104 receives quantized transform coefficients and coding parameters (split information, prediction parameters) from the parameter coding unit 111. The entropy coding unit 104 entropy codes these to generate and output the coded stream Te.

The prediction parameter derivation unit 120 is a means including the inter-prediction parameter coding unit 112 and an intra-prediction parameter coding unit, and derives intra-prediction parameters and intra-prediction parameters from the parameters input from the coding parameter determination unit 110. The derived intra-prediction parameters and intra-prediction parameters are output to the parameter coding unit 111.

(Configuration of inter-prediction parameter encoding unit)
21 , the inter prediction parameter coding unit 112 includes a parameter coding control unit 1121 and an inter prediction parameter derivation unit 303. The inter prediction parameter derivation unit 303 has the same configuration as the video decoding device. The parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212.

The merge index derivation unit 11211 derives merge candidates, etc., and outputs them to the inter prediction parameter derivation unit 303. The vector candidate index derivation unit 11212 derives prediction vector candidates, etc., and outputs them to the inter prediction parameter derivation unit 303 and the parameter coding unit 111.

(Configuration of intra prediction parameter encoding unit)
The intra-prediction parameter coding unit includes a parameter coding control unit and an intra-prediction parameter derivation unit, which has the same configuration as the video decoding device.

However, unlike a video decoding device, the inputs to the inter prediction parameter derivation unit 303 and intra prediction parameter derivation unit are the coding parameter determination unit 110 and prediction parameter memory 108, and they are output to the parameter coding unit 111.

The adder 106 generates a decoded image by adding, pixel by pixel, the pixel values of the predicted block input from the predicted image generator 101 and the prediction error input from the inverse quantization and inverse transformer 105. The adder 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies deblocking filtering, SAO, and ALF to the decoded image generated by the adder 106. Note that the loop filter 107 does not necessarily have to include the above three types of filters; for example, it may be configured with only a deblocking filter.

The prediction parameter memory 108 stores the prediction parameters generated by the encoding parameter determination unit 110 in a predetermined location for each target picture and CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 in a predetermined location for each target picture and CU.

The coding parameter determination unit 110 selects one set of coding parameters from multiple sets. The coding parameters are the above-mentioned QT, BT, or TT division information, prediction parameters, or parameters generated in relation to these to be coded. The predicted image generation unit 101 generates a predicted image using these coding parameters.

The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by the coefficient λ. The code amount is the information amount of the coded stream Te obtained by entropy coding the quantization error and coding parameters. The squared error is the sum of the squares of the prediction errors calculated by the subtraction unit 102. The coefficient λ is a preset real number greater than zero. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated cost value. The coding parameter determination unit 110 outputs the determined coding parameters to the parameter coding unit 111 and the prediction parameter derivation unit 120.

Note that portions of the video encoding device 11 and video decoding device 31 in the above-described embodiments, such as the entropy decoding unit 301, parameter decoding unit 302, loop filter 305, predicted image generation unit 308, inverse quantization and inverse transform unit 311, adder unit 312, prediction parameter derivation unit 320, predicted image generation unit 101, subtractor unit 102, transform and quantization unit 103, entropy encoding unit 104, inverse quantization and inverse transform unit 105, loop filter 107, encoding parameter determination unit 110, parameter encoding unit 111, and prediction parameter derivation unit 120, may be implemented by a computer. In this case, a program for implementing these control functions may be recorded on a computer-readable recording medium, and the program may be read into and executed by a computer system. Note that the term "computer system" as used herein refers to a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. Additionally, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording media" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over networks like the Internet or communication lines like telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within computer systems that serve as servers or clients in such cases. Furthermore, the above-mentioned programs may be programs that implement some of the functions described above, or may be programs that can implement the functions described above in combination with programs already stored in the computer system.

Furthermore, part or all of the video encoding device 11 and video decoding device 31 in the above-described embodiments may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the video encoding device 11 and video decoding device 31 may be individually implemented as a processor, or part or all of them may be integrated into a processor. Furthermore, the integrated circuit implementation method is not limited to LSI, and may also be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit implementation technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may also be used.

One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design modifications can be made without departing from the spirit of the present invention.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means that are appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

Embodiments of the present invention can be suitably applied to video decoding devices that decode coded data in which image data has been coded, and video coding devices that generate coded data in which image data has been coded. They can also be suitably applied to the data structure of coded data generated by a video coding device and referenced by the video decoding device.

31 Image decoding device
301 Entropy Decoding Unit
302 Parameter Decoding Unit
303 Inter-prediction parameter derivation unit
30377 GPM Forecasting Department
305, 107 Loop filter
306, 109 Reference Picture Memory
307, 108 Prediction parameter memory
308, 101 Prediction image generation unit
309 Inter-prediction image generation unit
30952 GPM synthesis section
311, 105 Inverse quantization and inverse transform unit
312, 106 Addition section
320 Prediction parameter derivation part
11 Image encoding device
102 Subtraction section
103 Transformation and Quantization Unit
104 Entropy coding unit
110 Encoding parameter determination unit
111 Parameter Encoding Unit
112 Inter-prediction parameter coding unit
120 Prediction parameter derivation part

Claims (6)

A video decoding device comprising: a parameter decoding unit that decodes a GPM parameter indicating a partition shape for deriving a predicted image of a current block from encoded data; and a prediction unit that derives the predicted image from two predicted images and weights,
the parameter decoding unit determines whether to decode the GPM parameters of the target block according to the GPM parameters of the neighboring blocks, and determines whether to decode a first parameter or a second parameter according to the GPM parameters of the neighboring blocks;
the prediction unit includes a process of deriving a GPM mode number of the target block from the GPM parameters of the neighboring blocks, deriving the weights based on the GPM mode numbers, and deriving the predicted image using the weights ;
10. A video decoding device , comprising: a first parameter indicating a division pattern; and a second parameter indicating an angle of a division line . The video decoding device according to claim 1, characterized in that, when the prediction dividing line of the adjacent block and the target block do not have an intersection, the parameter decoding unit derives the GPM mode number of the target block from the first parameter, and when the prediction dividing line of the adjacent block and the target block have an intersection, the parameter decoding unit derives the GPM mode number of the target block from the GPM mode number of the adjacent block, the intersection, and the second parameter. 3. The video decoding device according to claim 2 , wherein the intersection is derived using a table that associates the block size of the adjacent block, the GPM mode number of the adjacent block, and the coordinate position of the intersection. 4. The video decoding device according to claim 2, wherein when the prediction division line of the adjacent block and the target block have the intersection, the prediction unit derives the GPM mode number of the target block using the intersection and a division angle derived from the second parameter. 5. The video decoding device according to claim 4, wherein the prediction unit derives the GPM mode number of the target block using a table that associates the block sizes of the adjacent blocks, the GPM mode numbers of the adjacent blocks, and coordinate positions of the intersections, and a table that associates the block size of the target block, the GPM mode number of the target block , and the coordinate positions of the intersections. A video encoding device comprising: a parameter encoding unit that encodes a GPM parameter indicating a partition shape for deriving a predicted image of a current block from encoded data; and a prediction unit that derives the predicted image from two predicted images and weights,
the parameter encoding unit determines whether to encode the GPM parameters of the current block according to the GPM parameters of the neighboring blocks, and determines whether to decode a first parameter or a second parameter according to the GPM parameters of the neighboring blocks;
the prediction unit includes a process of deriving a GPM mode number of the target block from the GPM parameters of the neighboring blocks, deriving the weights based on the GPM mode numbers, and deriving the predicted image using the weights ;
10. A video encoding device , comprising: a first parameter indicating a division pattern; and a second parameter indicating an angle of a division line .