JP7835330B2 - Video encoding device, video encoding method, and video encoding program - Google Patents

Description

This invention relates to image coding and decoding techniques that divide an image into blocks for prediction.

In image encoding and decoding, the image to be processed is divided into blocks, which are sets of a predetermined number of pixels, and processed in block units. By dividing the image into appropriate blocks and appropriately setting intra-frame prediction and inter-frame prediction, encoding efficiency can be improved.

In video encoding and decoding, encoding efficiency is improved by interpretation, which predicts from encoded and decoded pictures. Patent Document 1 describes a technique for applying affine transformations during interpretation. In video, it is not uncommon for objects to undergo deformations such as scaling and rotation, and by applying the technique in Patent Document 1, efficient encoding becomes possible.

Japanese Patent Application Publication No. 9-172644

However, the technology described in Patent Document 1 involves image conversion, which results in a significant processing load. In view of the above problems, the present invention provides a low-load and efficient encoding technology.

In one aspect of the present invention that solves the above problems, the system comprises: a merge candidate list construction unit that constructs a merge candidate list including spatial merge candidates; a normal merge candidate selection unit that selects normal merge candidates that are single or double predictions from the merge candidate list; a triangular merge candidate selection unit that selects a first triangular merge candidate that is a single prediction and a second triangular merge candidate that is a single prediction from the merge candidate list; and an encoding unit that encodes a first index that identifies the first triangular merge candidate and a second index that identifies the second triangular merge candidate. The triangular merge candidate selection unit uses the first and second indices to first determine whether the candidates in the merge candidate list contain L0 motion information, and if so, to designate the L0 motion information as a triangular merge candidate; then, it determines whether the candidates in the merge candidate list contain L1 motion information, and if so, to designate the L1 motion information as a triangular merge candidate. By doing so, the system selects the first and second triangular merge candidates, and the merge candidate list construction unit adds two or more merge candidates to the merge candidate list.

According to this invention, highly efficient image encoding and decoding can be achieved with low overhead.

This is a block diagram of an image encoding device according to an embodiment of the present invention. This is a block diagram of an image decoding device according to an embodiment of the present invention. This is a flowchart illustrating the process of splitting a tree block. This diagram shows how an input image is divided into tree blocks. This is a diagram explaining the z-scan. This diagram shows the division shape of the block. This is a flowchart to explain the process of dividing a block into four parts. This is a flowchart to explain the process of dividing a block into two or three parts. This is syntax for representing the shape of block divisions. This diagram illustrates intranet prediction. This is a diagram illustrating the reference block of interpretation. This is a syntax for representing the coding block prediction mode. This figure shows the correspondence between syntax elements and modes for interpretation. This diagram illustrates affine transformation motion compensation for two control points. This diagram illustrates affine transformation motion compensation for three control points. Figure 1 is a block diagram showing the detailed configuration of the interpretation unit 102. Figure 16 is a block diagram showing the detailed configuration of the normal predicted motion vector mode derivation unit 301. Figure 16 is a block diagram showing the detailed configuration of the normal merge mode derivation unit 302. Figure 16 is a flowchart illustrating the normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 301. This is a flowchart illustrating the processing steps for the normal predicted motion vector mode derivation process. This is a flowchart illustrating the procedure for deriving the normal merge mode. Figure 2 is a block diagram showing the detailed configuration of the interpretation prediction unit 203. Figure 22 is a block diagram showing the detailed configuration of the normal predicted motion vector mode derivation unit 401. Figure 22 is a block diagram showing the detailed configuration of the normal merge mode derivation unit 402. Figure 22 is a flowchart illustrating the normal predicted motion vector mode derivation process of the normal predicted motion vector mode derivation unit 401. Figure 16 is a block diagram showing the detailed configuration of the subblock predictive motion vector mode derivation unit 303. Figure 22 is a block diagram showing the detailed configuration of the subblock predictive motion vector mode derivation unit 403. Figure 16 is a block diagram showing the detailed configuration of the subblock merge mode derivation unit 304. Figure 22 is a block diagram showing the detailed configuration of the subblock merge mode derivation unit 404. This figure illustrates the derivation of candidate motion vectors for affine inheritance prediction. This figure illustrates the derivation of candidate motion vectors for affine construction prediction. This diagram illustrates the derivation of affine inheritance merge candidates. This diagram illustrates the derivation of affine construction merge candidates. This is a flowchart for deriving candidate motion vectors for affine inheritance prediction. This is a flowchart for deriving candidate motion vectors for affine construction prediction. This is a flowchart for deriving candidates for affine inheritance merge. This is a flowchart for deriving candidates for affine construction merge. This flowchart explains the initialization and update process for the historical prediction motion vector candidate list. This is a flowchart of the identical element verification process in the historical prediction motion vector candidate derivation process. This is a flowchart of the element shifting procedure in the process for deriving candidate motion vectors based on historical prediction. This is a flowchart illustrating the procedure for deriving candidate motion vectors based on historical predictions. This is a flowchart illustrating the procedure for deriving candidates for history merge. This diagram illustrates an example of the process for updating the list of candidate motion vectors predicted in the history. This is a flowchart illustrating the operation of the subblock time merge candidate derivation unit 381. This is a flowchart illustrating the process of deriving information about the adjacent movement of blocks. This is a flowchart illustrating the process of deriving temporal motion vectors. This is a flowchart to explain the derivation of interpretation prediction information. This is a flowchart illustrating the process of deriving subblock movement information. This is a diagram to explain the temporal relationships between the pictures. This is a flowchart illustrating the process of deriving time-predicted motion vector candidates in the normal prediction motion vector mode derivation unit 301. This flowchart explains the derivation process of ColPic in the derivation process of time-predicted motion vector candidates in the normal prediction motion vector mode derivation unit 301. This flowchart illustrates the process of deriving the encoding information of ColPic in the derivation process of time-predicted motion vector candidates in the normal prediction motion vector mode derivation unit 301. This is a flowchart to explain the process of deriving interpretation prediction information. This flowchart shows the procedure for deriving inter-prediction information for an encoded block when the inter-prediction mode of the encoded block colCb is bi-prediction (Pred_BI). This is a flowchart illustrating the procedure for scaling motion vectors. This is a flowchart explaining the process of deriving time merge candidates. This diagram illustrates motion compensation prediction in the case of L0 prediction, where the reference picture for L0 (RefL0Pic) is at a time earlier than the picture to be processed (CurPic). This diagram illustrates motion compensation prediction in the case of L0 prediction, where the reference picture for the L0 prediction is at a later time than the picture being processed. This diagram illustrates motion-compensated prediction in a dual prediction system where the reference picture for L0 prediction is at a time earlier than the picture to be processed, and the reference picture for L1 prediction is at a time later than the picture to be processed. This diagram illustrates the prediction direction of motion-compensated prediction in a dual prediction system where the reference picture for L0 prediction and the reference picture for L1 prediction are at a time earlier than the picture being processed. This diagram illustrates the prediction direction of motion-compensated prediction in a dual prediction system where the reference picture for L0 prediction and the reference picture for L1 prediction are at a later time than the picture being processed. This is a flowchart illustrating the procedure for deriving the average merge candidate. This is a table showing information about merge difference motion vectors. This diagram illustrates the derivation of the merge difference motion vector. This diagram illustrates the prediction of the triangular merge mode. This is a flowchart explaining the derivation of triangular merge candidates. This is a flowchart explaining the derivation of single-prediction motion information for merged triangular partitions. This diagram illustrates merged triangular indexes and merged triangular partitions.

This section defines the technologies and technical terms used in this embodiment.

<Tree Block>
In this embodiment, the image to be encoded and decoded is divided equally into units of a predetermined size. This unit is defined as a tree block. As shown in Figure 4, in this embodiment, the size of the tree block is set to 128 x 128 pixels, but the size of the tree block is not limited to this and can be set to any size. The tree blocks of the processing target (corresponding to the encoding target in encoding processing and the decoding target in decoding processing) are arranged in raster scan order.
In other words, the order of switching is from left to right and from top to bottom. The interior of each tree block can be further recursively divided. The block that is to be encoded and decoded after tree block division is defined as the encoded block. Tree blocks and encoded blocks are collectively defined as blocks. Efficient encoding is possible by performing appropriate block division. The size of the tree block can be a fixed value predetermined by the encoding and decoding devices, or the encoding device can transmit the determined tree block size to the decoding device.

<Prediction Mode>
Intra prediction (MO) is performed on a unit-by-unit basis of the target coding block, using the surrounding image signals to make predictions from the processed image (used in coding for decoded images, image signals, tree blocks, blocks, coding blocks, etc. in coding processing, and used in decoding for decoded images, image signals, tree blocks, blocks, coding blocks, etc.).
It switches between intra prediction (MODE_INTRA) and inter-prediction (MODE_INTER), which makes predictions from the image signal of the processed image. The mode that distinguishes between intra prediction (MODE_INTRA) and inter-prediction (MODE_INTER) is defined as the prediction mode (PredMode). The prediction mode (PredMode) is intra prediction (
It has either MODE_INTRA or MODE_INTER as its value.

<Interface Prediction>
Interpretation, which performs predictions from the image signals of processed images, allows the use of multiple processed images as reference pictures. To manage multiple reference pictures, two types of reference lists, L0 (reference list 0) and L1 (reference list 1), are defined, and each reference picture is identified using a reference index. L0 prediction (Pred_L0) is available for P slices. For B slices, L0 prediction (Pred_L0), L1 prediction (Pred_L1), and dual prediction (Pred_BI) are available.
The following are available: L0 prediction (Pred_L0) is an interpretation that references reference pictures managed in L0, and L1 prediction (Pred_L1) is an interpretation that references reference pictures managed in L1. Dual prediction (Pred_BI) performs both L0 and L1 predictions.
This is an interpretation that references one reference picture each managed by L0 and L1. The information that identifies L0 prediction, L1 prediction, and dual prediction is defined as the interpretation mode. For constants and variables with the subscript LX in the output in subsequent processing, L0, L1
It is assumed that processing is performed for each step.

<Predictive motion vector mode>
The predicted motion vector mode transmits an index for identifying the predicted motion vector, a differential motion vector, an inter-prediction mode, and a reference index to determine the inter-prediction information of the block to be processed. The predicted motion vector is derived from candidate predicted motion vectors derived from processed blocks adjacent to the block to be processed, or from blocks belonging to the processed image that are located at the same position as or near the block to be processed, and from an index for identifying the predicted motion vector.

<Merge Mode>
The merge mode is a mode in which, without transmitting differential motion vectors or reference indices, interpretation information for the target block is derived from interpretation information of processed blocks adjacent to the target block, or blocks belonging to the processed image that are located at the same position or near the target block.

Processed blocks adjacent to the block to be processed, and the inter-prediction information of those processed blocks, are defined as spatial merge candidates. Blocks belonging to the processed image that are located at the same position as or near the block to be processed, and the inter-prediction information derived from the inter-prediction information of those blocks, are defined as temporal merge candidates. Each merge candidate is registered in the merge candidate list, and the merge index identifies the merge candidate to be used for predicting the block to be processed.

<Adjacent Blocks>
Figure 11 illustrates the reference blocks used to derive interprediction information in the predictive motion vector mode and merge mode. A0, A1, A2, B0, B1, B2,
B3 is a processed block adjacent to the block to be processed. T0 is a block belonging to the processed image that is located at the same position as or near (neighboring) the encoded block to be processed in the image to be processed.

Blocks A1 and A2 are located to the left of the coding block to be processed and are adjacent to it. Blocks B1 and B3 are located above the coding block to be processed and are adjacent to it. Blocks A0, B0, and B2 are located to the lower left, upper right, and upper left of the coding block to be processed, respectively.

Details on how adjacent blocks are handled in predictive motion vector mode and merge mode will be described later.

<Affine transformation motion compensation>
Affine transform motion compensation divides an encoded block into predefined subblocks and performs motion compensation by setting a motion vector individually for each subblock. The motion vector for each subblock is derived based on one or more control points derived from interpretation information of processed blocks adjacent to the block to be processed, or blocks belonging to the processed image that are located at the same position as or near the block to be processed. In this embodiment, the size of the subblock is set to 4x4 pixels, but the size of the subblock is not limited to this, and the motion vector may be derived on a pixel-by-pixel basis.

Figure 14 shows an example of affine transform motion compensation with two control points. In this case, since the two control points have two parameters, a horizontal component and a vertical component, the affine transform with two control points is called a four-parameter affine transform. CP1, CP2 in Figure 14
These are the control points. Figure 15 shows an example of affine transform motion compensation when there are three control points. In this case, since the three control points have two parameters, a horizontal component and a vertical component,
An affine transformation with three control points is called a six-parameter affine transformation. CP1, CP2, and CP3 in Figure 15 are the control points.

Affine transformation motion compensation is available in both predictive motion vector mode and merge mode. The mode in which affine transformation motion compensation is applied in predictive motion vector mode is defined as subblock predictive motion vector mode, and the mode in which affine transformation motion compensation is applied in merge mode is defined as subblock merge mode.

<Syntax of coded blocks>
The syntax for representing the prediction mode of an encoded block is explained using Figures 12(a), 12(b), and 13. In Figure 12(a), pred_mode_flag is a flag indicating whether or not it is interpretation. If pred_mode_flag is 0, it is interpretation, and pred_mo
If de_flag is 1, it is an intranet prediction. In the case of an intranet prediction, the intranet prediction information i
Send `ntra_pred_mode`, and if inter-prediction is used, send `merge_flag`. `merge_flag` is a flag indicating whether to use merge mode or predictive motion vector mode. If predictive motion vector mode is used (merge_flag=0), send the `inter_affine_flag` flag indicating whether to apply sub-block predictive motion vector mode. If sub-block predictive motion vector mode is applied (inter_affine_flag=1), send `cu_affine_type_flag`.
g is a flag used to determine the number of control points in the subblock predictive motion vector mode.

On the other hand, in merge mode (merge_flag=1), the merge_subblock_flag shown in Figure 12(b) is sent. merge_subblock_flag is a flag that indicates whether or not to apply subblock merge mode. In subblock merge mode (merge_subblock_flag=1), the merge index merge_subblock_idx is sent. On the other hand, if subblock merge mode is not applied (merge_
subblock_flag=0), merge_triangle_fl flag indicating whether or not to apply triangular merge mode.
Send ag. If triangular merge mode is applied (merge_triangle_flag=1), send merge_triangle_split_dir, the direction to split the block, and merge triangular indices merge_triangle_idx0 and merge_triangle_idx1 for each of the two split partitions. On the other hand, if triangular merge mode is not applied (merge_triangle_flag=0), send merge index merge_idx
Send.

Figure 13 shows the values of each syntax element and their corresponding prediction modes.
flag=0, inter_affine_flag=0 corresponds to the normal predictive motion vector mode (Inter Pred Mode). merge_flag=0, inter_affine_flag=1 corresponds to the subblock predictive motion vector mode (
Supports Inter Affine Mode. merge_flag=1,merge_subblock_flag=0,merge_trianlge
_flag=0 corresponds to normal merge mode. merge_flag=1, merge_subblock
_flag=0,merge_trianlge_flag=1 corresponds to Triangle Merge Mode. merge_flag=1,merge_subblock_flag=1 corresponds to Affine Merge Mode.
(Mode) corresponds to

<POC>
POC (Picture Order Count) is a variable associated with the picture being encoded.
A value is set that increases by 1 according to the output order of the pictures. The value of POC can be used to determine if two pictures are the same, to determine the order in which pictures are output, and to derive the distance between pictures. For example, if two pictures have the same POC value, they can be determined to be the same picture. If two pictures have different POC values, the picture with the smaller POC value will be output first, and so on.
The difference in POC between the two pictures indicates the distance between the pictures in the time axis direction.

(First Embodiment)
An image encoding device 100 and an image decoding device 200 according to a first embodiment of the present invention will be described.

Figure 1 is a block diagram of an image encoding device 100 according to the first embodiment. The video encoding device of this embodiment comprises an image encoding device 100, a block division unit 101, an inter-prediction unit 102, an intra-prediction unit 103, a decoded image memory 104, a prediction method determination unit 105, a residual signal generation unit 106, an orthogonal transformation/quantization unit 107, a bit string encoding unit 108, an inverse quantization/inverse orthogonal transformation unit 109, a decoded image signal superposition unit 110, and an encoded information storage memory 111.

The block division unit 101 recursively divides the input image to generate encoded blocks. The block division unit 101 includes a 4-division unit that divides the block to be divided horizontally and vertically, and a 2-3 division unit that divides the block to be divided horizontally or vertically. The image signals of the generated encoded blocks to be processed are supplied to the inter-prediction unit 102, the intra-prediction unit 103, and the residual signal generation unit 106. Information indicating the determined recursive division structure is also supplied to the bit string encoding unit 108.
The detailed operation of item 1 will be described later.

The interpretation unit 102 performs interpretation of the encoding block to be processed. It derives multiple candidate interpretation information from the interpretation prediction information stored in the encoding information storage memory and the decoded image signal stored in the decoded image memory 104, selects a suitable interpretation prediction mode from among the multiple candidates, and supplies the selected interpretation prediction mode and the predicted image signal corresponding to the selected interpretation prediction mode to the prediction method determination unit 105. The detailed configuration and operation of the interpretation prediction unit 102 will be described later.

The intra-prediction unit 103 performs intra-prediction of the encoding block to be processed. It generates a predicted image signal from the decoded image signal stored in the decoded image memory 104 by intra-prediction, selects a suitable intra-prediction mode from among a plurality of intra-prediction modes, and supplies the selected intra-prediction mode and the predicted image signal corresponding to the selected intra-prediction mode to the prediction method determination unit 105. An example of intra-prediction is shown in Figure 10. Figure 10(a) shows the correspondence between the prediction direction of intra-prediction and the intra-prediction mode number. For example, intra-prediction mode 50 generates an intra-prediction image by copying pixels in the vertical direction. Intra-prediction mode 1 is DC mode, a mode in which all pixel values of the block to be processed are set to the average value of the reference pixels. Intra-prediction mode 0 is Planar mode, a mode in which a two-dimensional intra-prediction image is created from reference pixels in the vertical and horizontal directions. Figure 10(b) is an example of generating an intra-prediction image in the case of intra-prediction mode 40. For each pixel of the block to be processed, the value of the reference pixel in the direction indicated by the intra-prediction mode is copied. If the reference pixel in intra-prediction mode is not at an integer position, the reference pixel value is determined by interpolation from the reference pixel values at surrounding integer positions.

The decoded image memory 104 stores the decoded image generated by the decoded image signal superimposition unit 110.
The decoded image stored in the decoded image memory is processed by the inter-prediction unit 102 and the intra-prediction unit 1
It will be supplied to 03.

The prediction method determination unit 105 determines the optimal prediction mode (inter-prediction or intra-prediction) for each prediction by evaluating it using encoded information, the code amount of the residual signal, the amount of distortion between the predicted image signal and the image signal to be processed, etc. In the case of the merge mode of inter-prediction, encoded information of the merge index and information indicating whether or not it is a subblock merge mode (subblock merge flag) is supplied to the bit string encoding unit 108, and in the case of the prediction motion vector mode of inter-prediction, the inter-prediction mode, prediction motion vector index, L0, L
Encoded information such as the reference index, differential motion vector, and information indicating whether or not it is in subblock mode (subblock predicted motion vector flag) is supplied to the bit string encoding unit 108. The determined encoded information is supplied to the encoded information storage memory 111.

The residual signal generation unit 106 generates a residual signal by subtracting the predicted image signal from the image signal to be processed, and supplies it to the orthogonal transformation and quantization unit 107.

The orthogonal transformation and quantization unit 107 performs orthogonal transformation and quantization on the residual signal according to the quantization parameters to generate an orthogonally transformed and quantized residual signal, which is then supplied to the bit string encoding unit 108 and the inverse quantization and inverse orthogonal transformation unit 109.

The bit string encoding unit 108 encodes information for each sequence, picture, slice, and encoding block, as well as encoding information according to the prediction method determined by the prediction method determination unit 105 for each encoding block. Specifically, the prediction mode PredMode for each encoding block,
In the PartMode and PRED_INTER modes, the bit string encoding unit 108 encodes information such as a flag to determine whether or not it is in merge mode, a subblock merge flag, a merge index if it is in merge mode, the interpretation mode if it is not in merge mode, a predicted motion vector index, information about the differential motion vector, and a subblock predicted motion vector flag according to the syntax rules described later, and generates a first encoded bit string. The bit string encoding unit 108 also entropy encodes the orthogonal transform and quantized residual signals according to the syntax rules, and generates a second encoded bit string. The first encoded bit string and the second encoded bit string are multiplexed according to the syntax rules, and a bit stream is output.

The inverse quantization/inverse orthogonal transformation unit 109 inversely quantizes and inversely orthogonal transforms the orthogonal transformed residual signal supplied from the orthogonal transformation/quantization unit 107 to calculate the residual signal, and supplies it to the decoded image signal superposition unit 110.

The decoded image signal superposition unit 110 superimposes the predicted image signal determined by the prediction method determination unit 105 with the residual signal that has been inversely quantized and inversely orthogonal transformed by the inverse quantization/inverse orthogonal transformation unit 109 to generate a decoded image, which is then stored in the decoded image memory 104. Furthermore, the decoded image is subjected to a filtering process to reduce distortions such as block distortion caused by encoding before being stored in the decoded image memory 104.
It may be stored in [a different location].

The coding information storage memory 111 stores coding information such as the prediction mode (inter-prediction or intra-prediction) determined by the prediction method determination unit 105.
The encoded information stored is, in the case of interpretation, the determined motion vector, reference list,
In addition to the reference index, in the case of merge mode for interpretation, the merge index,
Encoded information indicating whether or not it is in subblock merge mode (subblock merge flag), interprediction mode if it is in interprediction prediction motion vector mode, prediction motion vector index, L0, L1 reference index, difference motion vector, information indicating whether or not it is in subblock mode (subblock prediction motion vector flag), if it is in intraprediction,
This is the determined intra-prediction mode, etc. The construction of the history candidate list managed in the encoding information storage memory 111 will be described later.

Figure 2 is a block diagram showing the configuration of a video decoding device according to an embodiment of the present invention, corresponding to the video encoding device of Figure 1. The video decoding device of the embodiment comprises a bit sequence decoding unit 201, a block division unit 202, an inter-prediction unit 203, an intra-prediction unit 204, an encoded information storage memory 205, an inverse quantization/inverse orthogonal transformation unit 206, a decoded image signal superimposition unit 207, and a decoded image memory 208.

The decoding process of the video decoding device in Figure 2 corresponds to the decoding process provided inside the video encoding device in Figure 1. Therefore, the configurations of the encoding information storage memory 205, inverse quantization/inverse orthogonal transformation unit 206, decoded image signal superposition unit 207, and decoded image memory 208 in Figure 2 are the same as the inverse quantization/inverse orthogonal transformation unit 109 and decoded image signal superposition unit 110 of the video encoding device in Figure 1.
The encoded information storage memory 111 and the decoded image memory 104 each have corresponding functions.

The bitstream supplied to the bit string decoding unit 201 is separated according to the rules of a defined syntax. The separated first encoded bit string is decoded to obtain sequence, picture, slice, encoded block information, and encoded block information. Specifically, either interpretation (PRED_INTER) or intrapretation (PRED_INTRA) is performed on an encoded block basis.
Prediction modes to determine whether it is PredMode, PartMode, or Interpretation (PRED_INTER)
In this case, the encoded information relating to a flag that determines whether or not it is merge mode, the merge index and subblock merge flag if it is merge mode, and the inter-prediction mode, the predicted motion vector index, the difference motion vector and subblock predicted motion vector flag if it is predicted motion vector mode is decoded according to the syntax rules specified later, and the encoded information is supplied to the inter-prediction unit 203 or intra-prediction unit 204 and the encoded information storage memory 205. The separated second encoded bit sequence is decoded to calculate the orthogonal-transformed and quantized residual signal, and the orthogonal-transformed and quantized residual signal is supplied to the inverse quantization and inverse orthogonal-transformation unit 206.

The inter prediction unit 203, when the prediction mode PredMode of the coding block to be processed is inter prediction (PRED_INTER) and is in prediction motion vector mode, stores coding information in the coding information storage memory 205
Using the encoded information of the already decoded image signal stored in the system, multiple candidate predicted motion vectors are derived and registered in the predicted motion vector candidate list described later. From among the multiple candidate predicted motion vectors registered in the predicted motion vector candidate list, a predicted motion vector corresponding to the predicted motion vector index decoded and supplied by the bit sequence decoding unit 201 is selected.
The bit sequence decoding unit 201 calculates a motion vector from the decoded difference vector and the selected predicted motion vector, and stores it in the coded information storage memory 205 along with other coded information. The coded information of the coded block supplied and stored here is in prediction mode (PredMode) and division mode (Pa).
rtMode, L0 prediction, and flags indicating whether or not to use L1 prediction, predFlagL0[xP][yP], pr
edFlagL1[xP][yP], L0, L1 reference index refIdxL0[xP][yP], refIdxL1[xP][yP]
These are the motion vectors mvL0[xP][yP], mvL1[xP][yP], etc., for L0 and L1. Here, xP and yP are indices indicating the position of the top-left pixel of the encoded block within the picture. Prediction mode
PredMode is for inter-prediction (MODE_INTER), and the inter-prediction mode is for L0 prediction (Pred_L0).
In this case, the flag predFlagL0 indicating whether to use L0 prediction is 1, and the flag predFlagL1 indicating whether to use L1 prediction is 0. When the interpretation mode is L1 prediction (Pred_L1), the flag predFlagL0 indicating whether to use L0 prediction is 0, and the flag predFlagL1 indicating whether to use L1 prediction is 1. When the interpretation mode is bipretation (Pred_BI), both the flag predFlagL0 indicating whether to use L0 prediction and the flag predFlagL1 indicating whether to use L1 prediction are 1. Furthermore, the prediction mode of the coded block to be processed is Pred
When Mode is set to inter prediction (PRED_INTER) and merge mode is active, merge candidates are derived. Using the encoded information of already decoded encoded blocks stored in the encoded information storage memory 205, multiple merge candidates are derived and registered in the merge candidate list described later. From the multiple merge candidates registered in the merge candidate list, a merge candidate corresponding to the decoded and supplied merge index is selected by the bit sequence decoding unit 201, and flags predFlagL0[xP][yP] and predFlagL1[xP][yP] are used to indicate whether or not to use the L0 prediction and L1 prediction of the selected merge candidate.
Interpretation information such as the reference indices of L0 and L1 (refIdxL0[xP][yP], refIdxL1[xP][yP]), motion vectors of L0 and L1 (mvL0[xP][yP], mvL1[xP][yP]), and encoded information storage memory 2
Stored in 05. Here, xP and yP are indices indicating the position of the top-left pixel of the encoded block within the picture. The detailed configuration and operation of the interpretation unit will be described later.

The intra prediction unit 204 performs intra prediction when the prediction mode PredMode of the encoding block to be processed is intra prediction (PRED_INTRA). The encoding information decoded by the bit sequence decoding unit 201 includes the intra prediction mode, and according to the intra prediction mode, it generates a predicted image signal from the decoded image signal stored in the decoded image memory 208 using intra prediction, and supplies the predicted image signal to the decoded image signal superposition unit 207. Intra prediction unit 20
Since 4 corresponds to the intra-prediction unit 103 of the image encoding device 100, it performs the same processing as the intra-prediction unit 103.

The inverse quantization/inverse orthogonal transformation unit 206 performs an inverse orthogonal transformation and inverse quantization on the orthogonal transformation/quantized residual signal decoded by the bit sequence decoding unit 201 to obtain an inverse orthogonal transformation/inverse quantized residual signal.

The decoded image signal superposition unit 207 decodes the decoded image signal by superimposing the predicted image signal inter-predicted by the inter-prediction unit 203 or the predicted image signal intra-predicted by the intra-prediction unit 204 with the residual signal that has been inversely orthogonal-transformed and inversely quantized by the inverse quantization and inverse orthogonal-transformation unit 206, and stores it in the decoded image memory 208. When storing in the decoded image memory 208, the decoded image may be subjected to filtering to reduce block distortion caused by encoding, etc., before being stored in the decoded image memory 208.

Next, the operation of the block division unit 101 in the image encoding device 100 will be described.
Figure 3 is a flowchart showing the process of dividing an image into tree blocks and then further dividing each tree block. First, the input image is divided into tree blocks of a predetermined size (step S1001). Each tree block is scanned in a predetermined order, i.e., in the raster scan order (step S1002), and the inside of the tree block to be processed is divided (step S1003).

Figure 7 is a flowchart showing the detailed operation of the division process in step S1003. First, it is determined whether or not to divide the block to be processed into four parts (step S1101).

If it is determined that the block to be processed should be divided into four parts, the block to be processed is divided into four parts (step S1102). For each of the divided blocks of the block to be processed, in Z-scan order,
That is, the scan is performed in the order of top left, top right, bottom left, and bottom right (step S1103). Figure 5 is an example of the Z-scan order, and 601 in Figure 6 is an example of dividing the processing target block into four parts. The numbers 0 to 3 in 601 in Figure 6 indicate the order of processing. Then, for each block divided in step S1101, the flowchart in Figure 7 is recursively called.

If it is determined that the block to be processed should not be divided into four parts, then a 2-3 division will be performed (Step S1)
105).

Figure 8 is a flowchart showing the detailed operation of the 2-3 division process in step S1105. First, it is determined whether or not to divide the block to be processed into 2-3 parts, that is, whether or not to perform a 2-part division or a 3-part division (step S1201).

If it is not determined that the block to be processed should be divided into two or three parts, i.e., if it is determined not to divide it, the division is terminated (step S1211), and the process returns to the block at the higher level.

If it is determined that the block to be processed should be divided into 2-3 parts, then the block to be processed will be further divided into 2 parts.
A decision is made as to whether or not to divide it (step S1202).

If it is determined that the block to be processed should be divided into two, it is determined whether or not to divide the block to be processed vertically (step S1203), and based on the result, it is determined whether to divide the block to be processed vertically (step S1204) or to divide the block to be processed horizontally.
Step S1205). As a result of step S1204, the block to be processed is divided into two vertical sections as shown in Figure 602, and as a result of step S1205, the block to be processed is
As shown in Figure 604, it is divided into two horizontal sections.

In step S1202, if it is not determined to divide the block to be processed into two parts, that is, if it is determined to divide it into three parts, then it is determined whether or not to divide the block to be processed vertically (step S1206), and based on the result, the block to be processed is divided vertically (step S1207) or horizontally (step S
1208). As a result of step S1207, the block to be processed is divided into three vertical sections as shown in Figure 603, and as a result of step S1208, the block to be processed is divided into three horizontal sections as shown in Figure 605.

After executing any of steps S1204 to S1205, each of the divided blocks of the processing target is scanned from left to right and from top to bottom (step S1209
). Numbers 0 to 3 in Figure 6, from 602 to 605, indicate the order of processing. For each divided block, the flowchart in Figure 8 is called recursively.

The recursive block partitioning described here may restrict the necessity of partitioning based on the number of partitions or the size of the block to be processed. The information restricting the necessity of partitioning may be implemented in a configuration that does not involve the transmission of information, by pre-arranging between the encoding device and the decoding device, or it may be implemented in a configuration where the encoding device determines the information restricting the necessity of partitioning, records it in the encoded bit sequence, and transmits it to the decoding device.

Here, when a block is divided, the original block is called the parent block, and each of the resulting blocks is called a child block.

Next, the operation of the block division unit 202 in the image decoding device 200 will be described. The block division unit 202 divides the tree block using the same processing procedure as the block division unit 101 of the image encoding device 100. However, the block division unit 101 of the image encoding device 100 applies optimization methods such as estimation of the optimal shape by image recognition and distortion rate optimization.
While determining the optimal block division shape is a common approach, the block division unit 202 in the image decoding device 200 differs in that it determines the block division shape by decoding the block division information recorded in the encoded bit sequence.

Syntax (syntax rules for encoded bit sequences) relating to block partitioning in the first embodiment
This is shown in Figure 9. coding_quadtree() represents the syntax for dividing a block into four parts, and multi_type_tree() represents the syntax for dividing a block into two or three parts. qt_split is a flag that indicates whether or not to divide the block into four parts; if the block is to be divided into four parts, qt_split=1; if it is not to be divided into four parts, qt_split=0. If divided into four parts (qt_split=
1) For each of the four divided blocks, recursively perform the four division process (coding_quadtree(0), c
coding_quadtree(1), coding_quadtree(2), coding_quadtree(3)). If not divided into four (qt_
If split=0, the subsequent split is determined according to multi_type_tree(). mtt_split is a flag indicating whether or not to split further. If further splitting is performed (mtt_split=1), refer to mtt_split_vertical, which is a flag indicating whether to split vertically or horizontally, and mtt_split_binary, which is a flag determining whether to split into two or three parts.
`al=1` indicates a vertical split, and `mtt_split_vertical=0` indicates a horizontal split. `mtt_split_binary=1` indicates a 2-way split, and `mtt_split_binary=0` indicates a 3-way split. Hierarchical block splitting is performed by recursively calling `multi_type_tree` until `mtt_split=0`.

<Interface Forecast>
The inter prediction method according to the embodiment is the inter prediction unit 10 of the video encoding device shown in Figure 1.
This is carried out in the inter-prediction unit 203 of the video decoding device shown in Figure 2.

An interpretation method according to an embodiment will be explained with reference to the drawings. The interpretation method is performed by either encoding or decoding on a coding block basis.

<Explanation of the Interpretation Unit 102 on the Encoding Side>
Figure 16 shows a detailed configuration of the inter-prediction unit 102 of the video encoding device shown in Figure 1. The normal prediction motion vector mode derivation unit 301 derives a plurality of normal prediction motion vector candidates, selects a predicted motion vector, and calculates a difference vector from the detected motion vector. The detected inter-prediction mode, reference index, motion vector, and calculated difference vector become the inter-prediction information for the normal prediction motion vector mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305. The detailed configuration and processing of the normal prediction motion vector mode derivation unit 301 will be described later.

The normal merge mode derivation unit 302 derives multiple normal merge candidates, selects a candidate, and obtains inter-prediction information for the normal merge mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305. The detailed configuration and processing of the normal merge mode derivation unit 302 will be described later.

The subblock predicted motion vector mode derivation unit 303 derives multiple subblock predicted motion vector candidates, selects a subblock predicted motion vector, and calculates a difference vector from the detected motion vector. The detected inter-prediction mode, reference index, motion vector, and calculated difference vector become the inter-prediction information for the normal predicted motion vector mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305. The detailed configuration and processing of the subblock predicted motion vector mode derivation unit 303 will be described later.

The subblock merge mode derivation unit 304 derives multiple subblock merge candidates, selects a candidate, and obtains inter-prediction information for the subblock merge mode. This inter-prediction information is supplied to the inter-prediction mode determination unit 305. The detailed configuration and processing of the subblock merge mode derivation unit 304 will be described later.

The inter-prediction mode determination unit 305 determines the inter-prediction mode based on the inter-prediction information supplied from the normal prediction motion vector mode derivation unit 301, the normal merge mode derivation unit 302, the sub-block prediction motion vector mode derivation unit 303, and the sub-block merge mode derivation unit 304. The inter-prediction mode determination unit 305 supplies inter-prediction information corresponding to the determination result to the motion compensation prediction unit 306.

The motion compensation prediction unit 306 uses the determined inter prediction information to process the decoded image memory 1
Interpretation is performed on the reference image signal stored in 04. The detailed configuration and processing will be described later.

<Explanation of the decoding side interpretation unit 203>
Figure 22 shows a detailed configuration of the inter-prediction unit 203 of the video decoding device shown in Figure 2.

The normal prediction motion vector mode derivation unit 401 derives a plurality of normal prediction motion vector candidates, selects a prediction motion vector, and calculates a difference vector from the detected motion vector. The detected inter-prediction mode, reference index, motion vector, and difference vector become the inter-prediction information for the normal prediction motion vector mode. This inter-prediction information is used by switch 408.
It is supplied to the motion compensation prediction unit 406 via the normal prediction motion vector mode derivation unit 40.
The detailed configuration and processing of item 1 will be described later.

The normal merge mode derivation unit 402 derives multiple normal merge candidates, selects a candidate, and obtains inter-prediction information for the normal merge mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the normal merge mode derivation unit 402 will be described later.

The subblock predicted motion vector mode derivation unit 403 derives multiple subblock predicted motion vector candidates, selects a subblock predicted motion vector, and calculates a difference vector from the detected motion vector. The detected inter-prediction mode, reference index, motion vector, and calculated difference vector become the inter-prediction information for the normal predicted motion vector mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the subblock predicted motion vector mode derivation unit 403 will be described later.

The subblock merge mode derivation unit 404 derives multiple subblock merge candidates, selects a candidate, and obtains inter-prediction information for the subblock merge mode. This inter-prediction information is supplied to the motion compensation prediction unit 406 via the switch 408. The detailed configuration and processing of the subblock merge mode derivation unit 404 will be described later.

The motion compensation prediction unit 406 uses the determined inter prediction information to decode the image memory 2
Interpretation is performed on the reference image signal stored in 08. The detailed configuration and processing are the same as on the encoding side.

<Normal Predictive Motion Vector Mode Derivation Unit (Normal AMVP)>
The normal predicted motion vector mode derivation unit 301 in Figure 17 consists of a spatial predicted motion vector candidate derivation unit 321, a time predicted motion vector candidate derivation unit 322, and a history predicted motion vector candidate derivation unit 3
23. Includes a predicted motion vector candidate supplementation unit 325, a normal motion vector detection unit 326, a predicted motion vector candidate selection unit 327, and a motion vector subtraction unit 328.

The normal predicted motion vector mode derivation unit 401 in Figure 23 consists of a spatial predicted motion vector candidate derivation unit 421, a time predicted motion vector candidate derivation unit 422, and a history predicted motion vector candidate derivation unit 4
23. Includes a predicted motion vector candidate supplementation unit 425, a predicted motion vector candidate selection unit 426, and a motion vector addition unit 427.

The processing procedures of the encoding-side normal predicted motion vector mode derivation unit 301 and the decoding-side normal predicted motion vector mode derivation unit 401 will be explained using the flowcharts in Figures 19 and 25, respectively. Figure 19 is a flowchart showing the normal predicted motion vector mode derivation processing procedure by the encoding-side normal predicted motion vector mode derivation unit 301, and Figure 25 is a flowchart showing the normal predicted motion vector mode derivation processing procedure by the decoding-side normal predicted motion vector mode derivation unit 401.

<Explanation of the coding side: Typical predicted motion vector mode derivation unit (typical AMVP):>
The normal prediction motion vector mode derivation procedure on the encoding side will be explained with reference to Figure 19. In the explanation of the processing procedure in Figure 19, the term "motion vector" in the specification and the term "normal motion vector" in Figure 19 will correspond. First, the normal motion vector detection unit 326 detects the normal motion vector for each interprediction mode and reference index (step S100 in Figure 19).

Next, the spatial prediction motion vector candidate derivation unit 321, and the time prediction motion vector candidate derivation unit 3
22. The history prediction motion vector candidate derivation unit 323, the prediction motion vector candidate supplementation unit 325, the prediction motion vector candidate selection unit 327, and the motion vector subtraction unit 328 calculate the difference motion vectors of the motion vectors used in inter prediction of the normal prediction motion vector mode for L0 and L1 respectively (steps S101 to S106 in Figure 19). Specifically, the prediction mode of the block to be processed is inter prediction (MODE_INTER), and the inter prediction mode is L0 prediction (Pr
If the prediction mode of the block to be processed is L1 prediction (Pred_L1), a list of candidate prediction motion vectors for L0, mvpListL0, is calculated, a prediction motion vector mvpL0 is selected, and the difference motion vector mvdL0 of the L0 motion vector mvL0 is calculated. When the interpretation mode of the block to be processed is dual prediction (Pred_BI), both L0 and L1 predictions are performed. A list of candidate motion vectors for L0, mvpListL0, is calculated, the predicted motion vector mvpL0 for L0 is selected, and the difference motion vector mvdL0 of the motion vector mvL0 for L0 is calculated. Simultaneously, a list of candidate motion vectors for L1, mvpListL1, is calculated, the predicted motion vector mvpL1 for L1 is calculated, and the difference motion vector mvdL1 of the motion vector mvL1 for L1 is calculated.

The differential motion vector calculation process is performed for both L0 and L1, but the process is common to both L0 and L1. Therefore, in the following explanation, L0 and L1 will be represented as a common LX. In the process of calculating the differential motion vector for L0, X is 0, and in the process of calculating the differential motion vector for L1, X is 1. Also, during the process of calculating the differential motion vector for LX,
When referring to information from the other list, rather than LX, the other list is represented as LY.

When using the motion vector mvLX for LX (step S102 in Figure 19: YES), LX
Candidates for predicted motion vectors are calculated and a list of LX's predicted motion vector candidates, mvpListLX, is constructed (step S103 in Figure 19). Multiple candidate predicted motion vectors are derived in the spatial predicted motion vector candidate derivation unit 321, time predicted motion vector candidate derivation unit 322, history predicted motion vector candidate derivation unit 323, and predicted motion vector candidate supplementation unit 325 within the normal predicted motion vector mode derivation unit 301 to construct the predicted motion vector candidate list mvpListLX. Figure 19
The detailed processing procedure for step S103 will be described later using the flowchart in Figure 20.

Next, the predicted motion vector candidate selection unit 327 selects the predicted motion vector mvpLX for LX from the predicted motion vector candidate list mvpListLX (step S104 in Figure 19).
The difference motion vector is calculated by comparing the motion vector mvLX with each candidate motion vector mvpListLX[i] stored in the predicted motion vector candidate list mvpListLX.
The code value when those difference motion vectors are encoded is predicted in the motion vector candidate list mvpLis
The calculation is performed for each element of tLX. Then, among the elements registered in the predicted motion vector candidate list mvpListLX, the candidate for the predicted motion vector with the minimum sign amount is selected.
Select mvpListLX[i] as the predicted motion vector mvpLX and obtain its index i. If there are multiple candidates for the predicted motion vector with the smallest generated sign amount in the predicted motion vector candidate list mvpListLX, select the candidate mvpListLX[i] represented by the smallest index i in the predicted motion vector candidate list mvpListLX as the optimal predicted motion vector mvpLX and obtain its index i.

Next, the motion vector subtraction unit 328 subtracts the selected predicted motion vector mvpLX of LX from the motion vector mvLX of LX.
mvdLX = mvLX - mvpLX
The difference motion vector mvdLX is calculated as LX (step S105 in Figure 19).

<Explanation of the decoder side: Normal predicted motion vector mode derivation unit (normal AMVP)>
Next, the normal predicted motion vector mode processing procedure on the decoding side will be explained with reference to Figure 25. On the decoding side, there is a spatial predicted motion vector candidate derivation unit 421 and a time predicted motion vector candidate derivation unit 4
22. History prediction motion vector candidate derivation unit 423, Prediction motion vector candidate supplementation unit 425,
Normally, motion vectors used in inter-prediction in predicted motion vector mode are calculated for L0 and L1 respectively (steps S201 to S206 in Figure 25). Specifically, if the prediction mode PredMode of the block to be processed is inter-prediction (MODE_INTER) and the inter-prediction mode of the block to be processed is L0 prediction (Pred_L0), a list of candidate predicted motion vectors mvpListL0 for L0 is calculated, a predicted motion vector mvpL0 is selected, and the motion vector mvL0 for L0 is calculated. If the inter-prediction mode of the block to be processed is L1 prediction (Pred_L1), a list of candidate predicted motion vectors mvpListL1 for L1 is calculated, a predicted motion vector mvpL1 is selected, and the motion vector mvL1 for L1 is calculated. When the interpretation mode of the block to be processed is dual prediction (Pred_BI), both L0 and L1 predictions are performed. A list of candidate L0 predicted motion vectors, mvpListL0, is calculated, a predicted L0 motion vector mvpL0 is selected, and the L0 motion vector mvL0 is calculated. Simultaneously, a list of candidate L1 predicted motion vectors, mvpListL1, is calculated, the predicted L1 motion vector mvpL1 is calculated, and the L1 motion vector mvL1 is calculated.

Similar to the encoding side, the decoding side also performs motion vector calculation processing for L0 and L1 respectively, but the processing is common to both L0 and L1. Therefore, in the following explanation, L0,
L1 is represented as a common LX. LX represents the inter prediction mode used for inter prediction of the coding block to be processed. In the process of calculating the motion vector of L0, X is 0, and L1
In the process of calculating the motion vector, X is 1. Also, when calculating the motion vector of LX, if information from a different reference list is referenced instead of the same reference list as the LX being calculated, that other reference list is represented as LY.

When using the motion vector mvLX for LX (step S202 in Figure 25: YES), LX
Candidates for predicted motion vectors are calculated and a list of LX's predicted motion vector candidates, mvpListLX, is constructed (step S203 in Figure 25). Multiple candidate predicted motion vectors are calculated in the spatial predicted motion vector candidate derivation unit 421, time predicted motion vector candidate derivation unit 422, history predicted motion vector candidate derivation unit 423, and predicted motion vector candidate supplementation unit 425 within the normal predicted motion vector mode derivation unit 401, and the list of predicted motion vector candidates, mvpListLX, is constructed. Figure 25
The detailed processing procedure for step S203 will be described later using the flowchart in Figure 20.

Next, the predicted motion vector candidate selection unit 426 selects the predicted motion vector candidate list mvpListLX
The index mv of the predicted motion vector, which is decoded and supplied by the bit sequence decoding unit 201.
The candidate predicted motion vectors mvpListLX[mvpIdxLX] corresponding to pIdxLX are extracted as the selected predicted motion vector mvpLX (step S204 in Figure 25).

Next, the motion vector addition unit 427 adds the difference motion vector mvdLX of LX, which is decoded and supplied by the bit sequence decoding unit 201, and the predicted motion vector mvpLX of LX.
mvLX = mvpLX + mvdLX
The motion vector mvLX of LX is calculated (step S205 in Figure 25).

<Normal Prediction Motion Vector Mode Derivation Unit (Normal AMVP): Method for Predicting Motion Vectors>
Figure 20 is a flowchart showing the processing procedure for the normal predicted motion vector mode derivation process, which has functions common to the normal predicted motion vector mode derivation unit 301 of the video encoding device and the normal predicted motion vector mode derivation unit 401 of the video decoding device according to an embodiment of the present invention.

Normal prediction motion vector mode derivation unit 301 and Normal prediction motion vector mode derivation unit 40
In this embodiment, a predicted motion vector candidate list, mvpListLX, is provided. The predicted motion vector candidate list, mvpListLX, has a list structure and is provided with a predicted motion vector index that indicates the location within the predicted motion vector candidate list, and a storage area that stores the predicted motion vector candidate corresponding to the index as an element. The predicted motion vector index starts from 0, and the predicted motion vector candidates are stored in the storage area of the predicted motion vector candidate list, mvpListLX. In this embodiment, the predicted motion vector candidate list, mvpListLX, can register at least two predicted motion vector candidates (inter-prediction information). Furthermore, the variable numCurrMvpCand, which indicates the number of predicted motion vector candidates registered in the predicted motion vector candidate list, mvpListLX, is set to 0.

The spatial prediction motion vector candidate derivation units 321 and 421 derive candidate prediction motion vectors from the block adjacent to the left. In this process, a flag availableFlagLXA indicating whether a prediction motion vector candidate from the block adjacent to the left (A0 or A1) is available, along with the motion vector mvLXA and the reference index refIdxA are derived, and mvLXA is added to the prediction motion vector candidate list mvpListLX (step S301 in Figure 20). Note that when L0, X is 0
When L1, X is set to 1 (the same applies hereafter). Next, the spatial prediction motion vector candidate derivation unit 32
Steps 1 and 421 derive candidate predicted motion vectors from the upper adjacent block (B0, B1, or B2). This process derives a flag availableFlagLXB indicating whether a candidate predicted motion vector from the upper adjacent block is available, as well as the motion vector mvLXB and the reference index refIdxB. If mvLXA and mvLXB are not equal, mvLXB is added to the candidate predicted motion vector list mvpListLX (step S302 in Figure 20). Step S30 in Figure 20
The processes in steps 1 and 302 are similar except for the difference in the position and number of adjacent blocks referenced, and the flag availableFlagLXN indicates whether or not a candidate predicted motion vector for the encoded block is available.
Derive the motion vector mvLXN and the reference index refIdxN (where N is A or B, and so on).

Next, the time-predicted motion vector candidate derivation units 322 and 422 derive candidate predicted motion vectors from encoded blocks in pictures at different time points than the currently processed picture. In this process, a flag availableFlagLXCol and a motion vector mvLXCol are used to indicate whether or not candidate predicted motion vectors from encoded blocks in pictures at different time points are available.
The reference index refIdxCol and reference list listCol are derived, and mvLXCol is added to the predicted motion vector candidate list mvpListLX (step S303 in Figure 20).
The derivation procedure for step 3 will be explained in detail later.

Furthermore, the processing of time-predicted motion vector candidate derivation units 322 and 422 can be omitted in units of sequence (SPS), picture (PPS), or slice.

Next, the history prediction motion vector candidate derivation units 323 and 423 add the history prediction motion vector candidates registered in the history prediction motion vector candidate list HmvpCandList to the prediction motion vector candidate list mvpListLX. (Step S304 in Figure 20). This step S304
The registration process will be explained in detail later using the flowchart in Figure 41.

Next, the predicted motion vector candidate supplementation units 325 and 425 replenish the predicted motion vector candidate list mv
Add a motion vector of a predetermined value, such as (0,0), until pListLX is satisfied (S3 in Figure 20).
05).

<Normal Merge Mode Derivation Section (Normal Merge)>
The normal merge mode derivation unit 302 in Figure 18 includes a spatial merge candidate derivation unit 341, a time merge candidate derivation unit 342, an average merge candidate derivation unit 344, a history merge candidate derivation unit 345, a merge candidate replenishment unit 346, and a merge candidate selection unit 347.

The normal merge mode derivation unit 402 in Figure 24 includes a spatial merge candidate derivation unit 441, a time merge candidate derivation unit 442, an average merge candidate derivation unit 444, a history merge candidate derivation unit 445, a merge candidate replenishment unit 446, and a merge candidate selection unit 447.

Figure 21 shows the normal merge mode derivation unit 302 of the video encoding device according to an embodiment of the present invention.
This is a flowchart illustrating the procedure for the normal merge mode derivation process, which has functions common to both the normal merge mode derivation unit 402 of the video decoding device and the video decoding device.

The following explains the process step by step. Unless otherwise specified, the following explanation assumes a B-slice, but it can also be applied to a P-slice. However, when the slice type is a P-slice, only L0 prediction (Pred_L0) is available as an inter-prediction mode; L1 prediction (Pred_L1) and bi-prediction (Pred_BI) are not available.
Since it is not present, the processing related to L1 can be omitted.

The normal merge mode derivation unit 302 and the normal merge mode derivation unit 402 include a merge candidate list mergeCandList. The merge candidate list mergeCandList has a list structure,
A merge index is provided to indicate the location within the merge candidate list, and a memory area is provided to store the merge candidate corresponding to the index as an element. The merge index number is 0.
The process begins with the merge candidates being stored in the memory area of the merge candidate list, mergeCandList. Subsequent processing involves the merge index i registered in the merge candidate list, mergeCandList.
The merge candidates are represented by mergeCandList[i]. In this embodiment, the merge candidate list mergeCandList can register at least six merge candidates (inter-prediction information). Furthermore, the variable numCurrMergeCand, which indicates the number of merge candidates registered in the merge candidate list mergeCandList, is set to 0.

The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 derive spatial merge candidates A and B from blocks adjacent to the left and above the block to be processed, from the encoded information stored in the encoded information storage memory 111 of the video encoding device or the encoded information storage memory 205 of the video decoding device, and the derived spatial merge candidates are placed in the merge candidate list mergeC.
Registered in andList (step S401 in Figure 21). Here, N is defined to represent either spatial merge candidate A, B or time merge candidate Col. availableFlagN is a flag indicating whether the interpretation information of block N can be used as spatial merge candidate N, the reference index of L0 of spatial merge candidate N is refIdxL0N and the reference index of L1 is refIdxL1N, the L0 prediction flag is predFlagL0N indicating whether L0 prediction is performed and the L1 prediction flag is predFlagL1N indicating whether L1 prediction is performed, the motion vector of L0 is mvL0N and the motion vector of L1 is mvL1
N is derived. However, in this embodiment, merge candidates are derived without referring to other encoding blocks included in the block containing the encoding block to be processed, so spatial merge candidates included in the block containing the encoding block to be processed are not derived.

Next, the time merge candidate derivation unit 342 and the time merge candidate derivation unit 442 derive time merge candidates from pictures of different time periods and register the derived time merge candidates in the merge candidate list mergeCandList (step S402 in Figure 21). A flag availableFlagCol indicating whether a time merge candidate is available, an L0 prediction flag predFlagL0Col indicating whether an L0 prediction is performed for the time merge candidate, an L1 prediction flag predFlagL1Col indicating whether an L1 prediction is performed, and the L0 motion vector mvL0Col and L1 motion vector mvL1Col are derived. The detailed processing procedure for step S402 will be explained in detail later with reference to Figure 56.

Furthermore, the processing of the time merge candidate derivation unit 342 and the time merge candidate derivation unit 442 can be omitted in units of sequence (SPS), picture (PPS), or slice.

Next, the history merge candidate derivation unit 345 and the history merge candidate derivation unit 445 add the history predicted motion vector candidates registered in the history predicted motion vector candidate list HmvpCandList to the merge candidate list mergeCandList (step S403 in Figure 21). Step S40
The detailed processing steps for step 3 will be explained later using the flowchart in Figure 62.

Next, the average merge candidate derivation unit 344 and the average merge candidate derivation unit 444 derive the average merge candidate from the merge candidate list mergeCandList and register the derived average merge candidate in the merge candidate list mergeCandList (step S404 in Figure 21). Step S4
The detailed processing steps for step 04 will be explained later using the flowchart in Figure 41.

Next, the merge candidate replenishment unit 346 and the merge candidate replenishment unit 446 process the merge candidate list.
The number of merge candidates registered in mergeCandList, numCurrMergeCand, is equal to the maximum number of merge candidates, M.
If axNumMergeCand is smaller, the number of merge candidates registered in the merge candidate list mergeCandList, numCurrMergeCand, is used to derive additional merge candidates up to the maximum number of merge candidates MaxNumMergeCand, and these are registered in the merge candidate list mergeCandList (step S4 in Figure 21).
05). With the maximum number of merge candidates MaxNumMergeCand as the upper limit, in a P slice, the prediction mode where the motion vector has a value of (0,0) at different reference indices is L0 prediction (Pred_L0).
Add merge candidates. In B slices, motion vectors with different reference indices (
Prediction modes with a value of 0,0) add merge candidates for dual prediction (Pred_BI).

Next, the merge candidate selection unit 347 and the merge candidate selection unit 447 select the merge candidate list.
A merge candidate is selected from the merge candidates registered in mergeCandList. The encoding-side merge candidate selection unit 347 selects a merge candidate by calculating the code amount and distortion amount, and supplies the merge index indicating the selected merge candidate and the interpretation prediction information of the merge candidate to the motion compensation prediction unit 306. On the other hand, the decoding-side merge candidate selection unit 447 selects a merge candidate based on the decoded merge index and supplies the selected merge candidate to the motion compensation prediction unit 406.

The normal merge mode derivation unit 302 and the normal merge mode derivation unit 402 derive merge candidates in the parent block of a given encoded block if the size (product of width and height) of that encoded block is less than 32. Then, all child blocks use the merge candidates derived in the parent block. However, this is limited to cases where the size of the parent block is 32 or more and fits within the screen.

<Derivation of subblock predicted motion vector modes>
This section explains the derivation of the subblock predicted motion vector mode.

Figure 26 is a block diagram of the subblock predictive motion vector mode derivation unit 303 in the encoding device of this embodiment.

First, the affine inheritance prediction motion vector candidate derivation unit 361 derives affine inheritance prediction motion vector candidates. Details of the affine inheritance prediction motion vector candidate derivation will be described later.

Next, the affine construction prediction motion vector candidate derivation unit 362 derives affine construction prediction motion vector candidates. Details of the affine construction prediction motion vector candidate derivation will be described later.

Next, the affine identical predicted motion vector candidate derivation unit 363 derives affine identical predicted motion vector candidates. Details of the affine identical predicted motion vector candidate derivation will be described later.

The subblock motion vector detection unit 366 detects a subblock motion vector suitable for the subblock predicted motion vector mode and supplies the detected vector to the subblock predicted motion vector candidate selection unit 367 and the difference calculation unit 368.

The subblock predicted motion vector candidate selection unit 367 selects from the subblock predicted motion vector candidates derived in the affine inheritance predicted motion vector candidate derivation unit 361, the affine construction predicted motion vector candidate derivation unit 362, and the affine identical predicted motion vector candidate derivation unit 363, based on the motion vector supplied from the subblock motion vector detection unit 366.
A candidate subblock predicted motion vector is selected, and information regarding the selected subblock predicted motion vector candidate is supplied to the inter-prediction mode determination unit 305 and the difference calculation unit 368.

The difference calculation unit 368 subtracts the subblock predicted motion vector selected by the subblock predicted motion vector candidate selection unit 367 from the motion vector supplied by the subblock motion vector detection unit 366, and supplies the difference predicted motion vector to the inter-prediction mode determination unit 305.

Figure 27 is a block diagram of the subblock predicted motion vector mode derivation unit 403 in the decoding device of this embodiment.

First, the affine inheritance prediction motion vector candidate derivation unit 461 derives affine inheritance prediction motion vector candidates. The processing of the affine inheritance prediction motion vector candidate derivation unit 461 is the same as the processing of the affine inheritance prediction motion vector candidate derivation unit 361 in the encoding device of this embodiment.

Next, the affine construction predicted motion vector candidate derivation unit 462 derives affine construction predicted motion vector candidates. The processing of the affine construction predicted motion vector candidate derivation unit 462 is the same as the processing of the affine construction predicted motion vector candidate derivation unit 362 in the encoding device of this embodiment.

Next, the affine identical predicted motion vector candidate derivation unit 463 derives affine identical predicted motion vector candidates. The processing of the affine identical predicted motion vector candidate derivation unit 463 is the same as the processing of the affine identical predicted motion vector candidate derivation unit 363 in the encoding device of this embodiment.

The subblock predicted motion vector candidate selection unit 466 selects a subblock predicted motion vector candidate from among the subblock predicted motion vector candidates derived in the affine inheritance predicted motion vector candidate derivation unit 461, the affine construction predicted motion vector candidate derivation unit 462, and the affine identical predicted motion vector candidate derivation unit 463, based on the predicted motion vector index transmitted and decoded from the encoding device, and supplies information about the selected subblock predicted motion vector candidate to the motion compensation prediction unit 406 and the addition calculation unit 467.

The addition unit 467 adds the difference motion vector transmitted and decoded from the encoding device to the subblock predicted motion vector selected by the subblock predicted motion vector candidate selection unit 466 and supplies the resulting motion vector to the motion compensation prediction unit 406.

<Derivation of candidate motion vectors for affine inheritance prediction>
The affine inheritance prediction motion vector candidate derivation unit 361 will be explained. The affine inheritance prediction motion vector candidate derivation unit 461 will also be explained.
It is the same as in 1.

The affine inheritance prediction motion vector candidate inherits the motion vector information of the control points.

Figure 30 illustrates the derivation of candidate motion vectors for affine inheritance prediction.

Candidate affine inheritance prediction motion vectors are obtained by searching for motion vectors of control points in spatially adjacent encoded and decoded blocks.

Specifically, a maximum of one affine mode is searched for in the blocks adjacent to the left of the block to be processed (A0, A1) and in the blocks adjacent to the top of the block to be processed (B0, B1, B2), and these are used as the affine inheritance prediction motion vector.

Figure 34 is a flowchart for deriving candidate motion vectors for affine inheritance prediction.

First, the blocks adjacent to the left of the block to be processed (A0, A1) are designated as the left group (step S3101), and it is determined whether the block containing A0 is a block that uses affine transformation motion compensation (affine mode) (step S3102). If A0 is in affine mode (step S3102: YES), the affine mode used by A0 is obtained (step S3103), and the processing of the block adjacent to the upper side is started. If A0 is not in affine mode (step S3102: NO), the target for deriving the affine inheritance predicted motion vector candidate is set to A0 -> A1, and an attempt is made to obtain the affine mode from the block containing A1.

Next, the blocks adjacent to the upper side of the block to be processed (B0, B1, B2) are designated as the upper group (step S3104), and it is determined whether the block containing B0 is in affine mode (step S3105). If B0 is in affine mode (step S310
5: YES), obtain the affine mode used by B0 (step S3106), and terminate the process. If B0 is not in an affine mode (step S3105: NO), the target for deriving the affine inheritance prediction motion vector candidate is set to B0 -> B1, and an attempt is made to obtain the affine mode from the block containing B1. Furthermore, if B1 is not in an affine mode (step S310)
5: NO), the target for deriving candidate motion vectors for affine inheritance prediction is set to B1 -> B2, and an attempt is made to obtain the affine mode from the block containing B2.

In this way, the blocks are divided into left and upper blocks. For the left block, the affine mode is searched in the order from the bottom left to the top left block.
By exploring affine modes in order from the top right to the top left block, it is possible to obtain two affine modes that are as different as possible, and a candidate affine predicted motion vector can be derived in which one of the affine predicted motion vectors is smaller than the difference motion vector.

<Affine Construction Prediction: Derivation of Candidate Motion Vectors>
The affine construction prediction motion vector candidate derivation unit 362 will be explained. The affine construction prediction motion vector candidate derivation unit 462 will also be explained in relation to the affine construction prediction motion vector candidate derivation unit 362.
It is the same as in 2.

The affine construction predicts motion vector candidates by constructing motion vector information for control points from motion information of spatially adjacent blocks.

Figure 31 illustrates the derivation of candidate motion vectors for affine construction prediction.

Affine construction prediction motion vector candidates are obtained by combining motion vectors from spatially adjacent encoded and decoded blocks to construct new affine modes.

Specifically, the motion vector of the upper-left control point CP0 is derived from the blocks adjacent to the upper-left side of the block to be processed (B2, B3, A2), the motion vector of the upper-right control point CP1 is derived from the blocks adjacent to the upper-right side of the block to be processed (B1, B0), and the motion vector of the lower-left control point CP2 is derived from the blocks adjacent to the lower-left side of the block to be processed (A1, A0).

Figure 35 is a flowchart for deriving candidate motion vectors for affine construction prediction.

First, the upper-left control point CP0, the upper-right control point CP1, and the lower-left control point CP2 are derived (step S3201). The upper-left control point CP0 is calculated by searching for reference blocks that have the same reference image as the block to be processed, in the order of priority of reference blocks B2, B3, and A2. The upper-right control point CP1 is calculated by searching for reference blocks that have the same reference image as the block to be processed, in the order of priority of B1, B0
It is calculated by searching for reference blocks in the order of priority of the reference blocks. The lower left control point CP2 is calculated by searching for reference blocks that have the same reference image as the block to be processed, in the order of priority of the A1 and A0 reference blocks.

When selecting the 3-control-point mode as the affine construction prediction motion vector (Step S
3202: YES), determine whether all three control points (CP0, CP1, CP2) have been derived (step S3203). If all three control points (CP0, CP1, CP2) have been derived (step S3203: YES), determine whether all three control points (CP0, CP1, C
The affine model using P2) is used as the affine construction prediction motion vector (step S32).
04). If you do not select the 3-control-point mode and instead select the 2-control-point mode (Step S3)
202: NO), determine whether both control points (CP0, CP1) have been derived (step S3205). If both control points (CP0, CP1) have been derived (step S3205: YES), the affine model using the two control points (CP0, CP1) is used as the affine construction predicted motion vector (step S3206).

<Derivation of candidate affine identical predictive motion vectors>
The affine identical predicted motion vector candidate derivation unit 363 will be explained. The affine identical predicted motion vector candidate derivation unit 463 will also be explained.
It is the same as in 3.

Candidate affine identical predicted motion vectors are obtained by deriving the same motion vector at each control point.

Specifically, it can be obtained by deriving information for each control point and setting all control points to the same value in one of the CP0 to CP2, similar to the affine construction prediction motion vector candidate derivation units 362 and 462. Alternatively, it can be obtained by setting the derived time motion vector to all control points, similar to the normal prediction motion vector mode.

<Derivation of subblock merge mode>
The derivation of the subblock merge mode will be explained.

Figure 28 shows the subblock merge mode derivation unit 304 in the encoding device of this embodiment.
This is a block diagram. The subblock merge mode derivation unit 304 includes a subblock merge candidate list subblockMergeCandList. This is the same as the normal merge mode derivation unit 302
Similar to the merge candidate list mergeCandList in [previous version], it has a list structure and is provided with a merge index that indicates the location within the subblock merge candidate list, and a storage area that stores the subblock merge candidate corresponding to the index as an element. In this embodiment, the subblock merge candidate list subblockMergeCandList can register at least five merge candidates (interpretation information). However, each merge candidate is
Furthermore, it has motion vector information for each subblock, or motion vector information for the control points.

First, the subblock time merge candidate derivation unit 381 derives subblock time merge candidates. Details of the subblock time merge candidate derivation will be described later.

Next, the affine inheritance merge candidate derivation unit 382 derives affine inheritance merge candidates. Details of the affine inheritance merge candidate derivation will be described later.

Next, the affine construction merge candidate derivation unit 383 derives affine construction merge candidates. Details of the affine construction merge candidate derivation will be described later.

Next, the affine fixed merge candidate derivation unit 385 derives affine fixed merge candidates. Details of the affine fixed merge candidate derivation will be described later.

The subblock merge candidate selection unit 386 is a subblock time merge candidate derivation unit 381,
From the subblock merge candidates derived in the affine inheritance merge candidate derivation unit 382, the affine construction merge candidate derivation unit 383, and the affine fixed merge candidate derivation unit 385, a subblock merge candidate is selected, and information regarding the selected subblock merge candidate is supplied to the inter-prediction mode determination unit 305.

Figure 29 is a block diagram of the subblock merge mode derivation unit 404 in the decoding device of this embodiment. The subblock merge mode derivation unit 404 includes a subblock merge candidate list subblockMergeCandList. This is the same as the subblock merge mode derivation unit 304.

First, the subblock time merge candidate derivation unit 481 derives subblock time merge candidates. The processing of the subblock time merge candidate derivation unit 481 is the same as the processing of the subblock time merge candidate derivation unit 381.

Next, the affine inheritance merge candidate derivation unit 482 derives affine inheritance merge candidates. The processing in the affine inheritance merge candidate derivation unit 482 is performed by the affine inheritance merge candidate derivation unit 3
This is the same process as in 82.

Next, the affine construction merge candidate derivation unit 483 derives affine construction merge candidates. The processing in the affine construction merge candidate derivation unit 483 is performed by the affine construction merge candidate derivation unit 3
This is the same process as in 83.

Next, the affine fixed merge candidate derivation unit 485 derives affine fixed merge candidates. The processing of the affine fixed merge candidate derivation unit 485 is performed by the affine fixed merge candidate derivation unit 4
This is the same process as in step 85.

The subblock merge candidate selection unit 486 is a subblock time merge candidate derivation unit 481,
From the subblock merge candidates derived in the affine inheritance merge candidate derivation unit 482, the affine construction merge candidate derivation unit 483, and the affine fixed merge candidate derivation unit 485, a subblock merge candidate is selected based on an index transmitted and decoded from the encoding device, and information regarding the selected subblock merge candidate is supplied to the motion compensation prediction unit 406.

The subblock merge mode derivation unit 304 and the subblock merge mode derivation unit 404 derive a subblock merge candidate in the parent block of a given encoded block if the size (product of width and height) of that encoded block is less than 32. Then, all child blocks use the subblock merge candidate derived in the parent block. However, this is limited to cases where the size of the parent block is 32 or more and fits within the screen.

<Derivation of subblock time merge candidates>
The operation of the subblock time merge candidate derivation unit 381 will be described later.

<Derivation of Affine Inheritance Merge Candidates>
The affine inheritance merge candidate derivation unit 382 will now be described. The affine inheritance merge candidate derivation unit 482 is the same as the affine inheritance merge candidate derivation unit 382.

Affine inheritance merge candidates inherit the affine model of their control points from the affine model of a spatially adjacent block.

Figure 32 illustrates the derivation of affine inheritance merge candidates. The derivation of affine merge inheritance merge mode candidates is obtained by searching for the motion vectors of control points in spatially adjacent encoded and decoded blocks, similar to the derivation of affine inheritance predicted motion vectors.

Specifically, up to one affine mode is searched for from the blocks adjacent to the left of the block to be processed (A0, A1) and from the blocks adjacent to the top of the block to be processed (B0, B1, B2), and used for the affine merge mode.

Figure 36 is a flowchart for deriving candidates for affine inheritance merge.

First, the blocks adjacent to the left of the block to be processed (A0, A1) are designated as the left group (step S3301), and it is determined whether the block containing A0 is in affine mode (step S3302). If A0 is in affine mode (step S3102: YE
S) Obtain the affine model used by A0 (step S3303), and proceed to processing the block adjacent to the upper side. If A0 is not in affine mode (step S3302: NO)
The target for deriving affine inheritance merge candidates is set to A0 -> A1, and an attempt is made to obtain the affine mode from the block containing A1.

Next, the blocks adjacent to the upper side of the block to be processed (B0, B1, B2) are designated as the upper group (step S3304), and it is determined whether the block containing B0 is in affine mode (step S3305). If B0 is in affine mode (step S330
5: YES), obtain the affine model used by B0 (step S3306), and terminate the process. If B0 is not in affine mode (step S3305: NO), set the target of the affine inheritance merge candidate derivation to B0 -> B1, and attempt to obtain the affine mode from the block containing B1. Furthermore, if B1 is not in affine mode (step S3305: NO)
The target for deriving affine inheritance merge candidates is set to B1 -> B2, and an attempt is made to obtain the affine mode from the block containing B2.

<Derivation of Affine Construction Merge Candidates>
The affine construction merge candidate derivation unit 383 will now be described. The affine construction merge candidate derivation unit 483 is the same as the affine construction merge candidate derivation unit 383.

Figure 33 illustrates the derivation of affine construction merge candidates. The affine construction merge candidates construct an affine model of control points from the motion information of spatially adjacent blocks and time-coded blocks.

Specifically, the motion vector of the upper-left control point CP0 is derived from the block adjacent to the upper-left side of the block to be processed (B2, B3, A2), the motion vector of the upper-right control point CP1 is derived from the block adjacent to the upper-right side of the block to be processed (B1, B0), the motion vector of the lower-left control point CP2 is derived from the block adjacent to the lower-left side of the block to be processed (A1, A0), and the motion vector of the lower-right control point CP3 is derived from the encoded block (T0) adjacent to the lower-right side of the block to be processed.

Figure 37 is a flowchart for deriving candidates for affine construction merge.

First, the upper left control point CP0, upper right control point CP1, lower left control point CP2, and lower right control point CP3 are derived (step S3401). The upper left control point CP0 is a block that has motion information,
It is calculated by searching in the priority order of blocks B2, B3, and A2. The upper right control point CP1 is,
The first control point is calculated by searching for blocks containing motion information, prioritizing blocks B1 and B0. The second control point is calculated by searching for blocks containing motion information, prioritizing blocks A1 and A0. The third control point is calculated by searching for motion information in the time block.

Next, it is determined whether an affine model with three control points can be constructed using the derived CP0, CP1, and CP2 (step S3402). If it can be constructed (step S3402: YES), the three-control-point affine model using CP0, CP1, and CP2 is designated as an affine merge candidate (step S3403).

Next, it is determined whether an affine model with three control points can be constructed using the derived CP0, CP1, and CP3 (step S3404). If it can be constructed (step S3404: YES), the three-control-point affine model using CP0, CP1, and CP3 is designated as an affine merge candidate (step S3405).

Next, it is determined whether an affine model with three control points can be constructed using the derived CP0, CP2, and CP3 (step S3406). If it can be constructed (step S3406: YES), the three-control-point affine model using CP0, CP2, and CP3 is designated as an affine merge candidate (step S3407).

Next, it is determined whether an affine model with three control points can be constructed using the derived CP1, CP2, and CP3 (step S3408). If it is possible to construct it (step S3408: YES), the three-control-point affine model using CP1, CP2, and CP3 is designated as an affine merge candidate (step S3409).

Next, it is determined whether an affine model with two control points can be constructed using the derived CP0 and CP1 (step S3410), and if it can be constructed (step S341
0: YES), a two-control-point affine model using CP0 and CP1 is selected as a candidate for affine merge (step S3411).

Next, it is determined whether an affine model with two control points can be constructed using the derived CP0 and CP2 (step S3412), and if it can be constructed (step S341
2: YES), the two-control-point affine model using CP0 and CP2 is selected as a candidate for affine merge (step S3413).

Here, whether or not an affine model can be constructed is conditional on the fact that the reference images of all control points are identical (affine transformation possible). Also, CP0, CP1,
Affine models other than the 3-control-point affine model with CP2 and the 2-control-point affine model with CP0 and CP1 are as follows: For the 3-control-point affine model, CP0, CP1, CP
For the 3-control-point affine model using 2, the 2-control-point affine model is as follows: CP0, C
Convert to a two-control-point affine model using P1.

<Derivation of Affine Fixed Merge Candidates>
The affine fixed merge candidate derivation unit 385 will now be described. The affine fixed merge candidate derivation unit 485 is the same as the affine fixed merge candidate derivation unit 385.

Affine fixed merge candidates fix the motion information of control points using fixed motion information.

Specifically, the motion vector of each control point is fixed to (0,0).

<Time-predicted motion vector derivation>
Prior to explaining the time-predicted motion vector, Figure 49 shows the temporal sequence of events in the pictures.
This will be explained by referring to Figure 49(a). Figure 49(a) shows the relationship between the encoded block to be processed and an encoded picture that is temporally different from the picture to be processed. A specific encoded picture that is referenced in the encoding of the picture to be processed is defined as ColPic. ColPic is identified by syntax.

Furthermore, Figure 49(b) shows the encoded blocks in ColPic that are located at the same position as the encoded block to be processed, and in its vicinity. However, the encoded blocks T0 and T1 shown in Figure 49(b) are schematic, and their actual positions and sizes are not limited to these. Now, for the encoded block to be processed, let the position be (xCb, yCb) and the width be cbWidth
Let h be the height, and cbHeight be the height.
xColBr = xCb + cbWidth
yColBr = yCb + cbHeight
The following is calculated: The coded block on ColPic containing the position ((xColBr >> 3) << 3, (yColBr >> 3) << 3) becomes T0. Also,
xColCtr = xCb + (cbWidth >> 1)
yColCtr = yCb + (cbHeight >> 1)
The following is calculated: The coded block on ColPic containing position ((xColCtr >> 3) << 3, (yColCtr >> 3) << 3) becomes T1.

The above explanation of the temporal sequence of images pertains to the encoding process, but the same applies to the decoding process. In other words, during decoding, the above explanation is similar, simply replacing "encoding" with "decoding."

The operation of the time-predicted motion vector candidate derivation unit 322 in the normal predicted motion vector mode derivation unit 301 shown in Figure 17 will be explained with reference to Figure 50.

First, ColPic is derived (step S4201). The derivation of ColPic will be explained with reference to Figure 51.

If slice type slice_type is B slice and flag collocated_from_l0_flag is 0 (step S4211: YES, step S4212: YES), then the picture ColPic at different times will be the picture RefPicList1[0] whose reference index in reference list L1 is 0.
Step S4213). Otherwise, i.e., if slice type slice_type is B slice and the aforementioned flag collocated_from_l0_flag is 1 (Step S4211: YES,
Step S4212: NO), or if slice type slice_type is P slice (Step S4211: NO, Step S4214: YES), different time ColPic
This results in the picture RefPicList0[0], whose reference index in the reference list L0 is 0 (step S4215). If slice_type is not a P slice (step S4214: NO), the process is terminated.

Refer to Figure 50 again. After deriving ColPic, derive the encoded block colCb and obtain the encoded information (step S4202). This process will be explained with reference to Figure 52.

First, within picture ColPic at different time points, the encoding block containing the same position as the encoding block to be encoded, but in the lower right corner, is defined as the encoding block colCb at a different time point (step S).
4221). An example of this coding block is shown in coding block T0 in Figure 49.

Next, obtain the encoding information of the encoding block colCb at different time points (step S422)
2) If the PredMode of the coding block colCb at a different time is unavailable, or if the PredMode of the coding block colCb at a different time is intraprediction (MODE_INTRA) (step S4223: NO, step S4224: YES), then the picture ColPic at a different time
Within the encoding block to be processed, an encoding block containing the same central lower-right position as the encoding block to be processed is defined as an encoding block colCb at a different time (step S4225). An example of this encoding block is shown in encoding block T1 in Figure 49.

Refer to Figure 50 again. Next, for each reference list, the interpretation information is derived (steps S4203, S4204). Here, for the coded block colCb, the motion vector mvLXCol and the flag availableFlagLXCo indicating whether the coded information is valid are obtained for each reference list.
We derive l. LX represents a reference list; in the derivation of reference list 0, LX becomes L0, and in the derivation of reference list 1, LX becomes L1. The derivation of interpretation information will be explained with reference to Figure 53.

If a coding block colCb of a different time is unavailable (Step S4231: NO)
, or if the prediction mode PredMode is intraprediction (MODE_INTRA) (step S4232
: NO), set both flag availableFlagLXCol and flag predFlagLXCol to 0 (step S
4233) Set the motion vector mvLXCol to (0,0) (step S4234) and terminate the process.

The coding block colCb is available (step S4231: YES), and the prediction mode PredMod
If e is not an intra prediction (MODE_INTRA) (step S4232: YES), calculate mvCol, refIdxCol and availableFlagCol using the following procedure.

PredFlagL0[xPCo
If l[yPCol] is 0 (step S4235: YES), the prediction mode of the coded block colCb is Pred_L1, so the motion vector mvCol is set to the same value as MvL1[xPCol][yPCol], which is the L1 motion vector of the coded block colCb (step S4236), and the reference index refIdxCol is set to the same value as the L1 reference index RefIdxL1[xPCol][yPCol] (
Step S4237), the reference list listCol is set to L1 (Step S4238)
Here, xPCol and yPCol are indices indicating the position of the top-left pixel of the encoded block colCb within the picture ColPic at different times.

On the other hand, if the L0 prediction flag PredFlagL0[xPCol][yPCol] of the coded block colCb is not 0 (step S4235: NO), the L1 prediction flag PredFlagL1[xP
Determine whether [Col][yPCol] is 0 or not. PredFlagL1[x
If [PCol][yPCol] is 0 (step S4239: YES), the motion vector mvCol is set to the same value as MvL0[xPCol][yPCol], which is the motion vector of L0 of the coded block colCb (step S4240), and the reference index refIdxCol is set to the reference index RefIdxL0[xP
It is set to the same value as [Col][yPCol] (step S4241), and the reference list listCol is set to L0 (step S4242).

The L0 prediction flag PredFlagL0[xPCol][yPCol] of the coded block colCb and the coded block col
If the L1 prediction flags PredFlagL1[xPCol][yPCol] for Cb are both not 0 (step S4235)
(S4239: NO), since the interprediction mode of the coded block colCb is Pred_BI, one of the two motion vectors L0 and L1 is selected (step S4243).

Figure 54 is a flowchart showing the procedure for deriving inter-prediction information for an encoded block when the inter-prediction mode of the encoded block colCb is bi-prediction (Pred_BI).

First, it is determined whether the POC of all pictures registered in all reference lists is smaller than the POC of the picture currently being processed (step S4251), and the encoding block
P of all pictures registered in L0 and L1, which are all reference lists of colCb
If OC is smaller than the POC of the currently processed picture (Step S4251: YES)
If LX derives a candidate prediction vector for L0, i.e., the motion vector of the L0 of the coding block to be processed (step S4252: YES), select the inter prediction information for L0 of coding block colCb. If LX derives a candidate prediction vector for L1, i.e., the motion vector of L1 of the coding block to be processed (step S4252: NO), select the inter prediction information for L1 of coding block colCb.
If at least one of the POCs of all the pictures registered in reference lists L0 and L1 is greater than the POC of the currently processed picture (step S4251: NO), and the flag collocated_from_l0_flag is 0 (step S4253: YES), the inter prediction information of the coded block colCb on L0 is selected, and if the flag collocated_from_l0_flag is 1 (step S4253: NO), the inter prediction information of the coded block colCb on L1 is selected.

When selecting the interpretation prediction information for L0 of the coding block colCb (step S42)
52: YES, or step S4253: YES), motion vector mvCol is MvL0[xPCol]
[yPCol] is set to the same value (step S4254), and the reference index refIdxCol is set to RefI
dxL0[xPCol][yPCol] is set to the same value (step S4255), and list listCol is L0
It is set to (step S4256).

When selecting the interpretation prediction information for L1 of the coding block colCb (step S42)
52: NO, or step S4253: NO), motion vector mvCol is MvL1[xPCol][yPC
Set to the same value as ol] (step S4257), and the reference index refIdxCol is set to RefIdxL1
[xPCol][yPCol] are set to the same value (step S4258), and the list listCol is set to L1 (step S4259).

Returning to Figure 53, once the interpretation information is obtained from the coded block colCb, the flag ava
Set both ilableFlagLXCol and flag predFlagLXCol to 1 (step S4244).

Next, the motion vector mvCol is scaled to obtain the motion vector mvLXCol (step S4245). The procedure for scaling the motion vector mvLXCol will be explained using Figure 55.

The distance td between pictures is obtained by subtracting the POC of the reference picture corresponding to the reference index refIdxCol referenced by the list listCol of the coded block colCb from the POC of the picture ColPic at different time points.
td = [POC of picture ColPic at different times] - [list of coded blocks colCb]
[POC of the referenced picture referenced by listCol]
The distance between pictures is calculated as follows (step S4261). Note that if the POC of the referenced picture referenced by the list listCol of the coded block colCb is displayed earlier than the picture ColPic of a different time, the distance between pictures td will be a positive value, and if the POC of the referenced picture referenced by the list listCol of the coded block colCb is displayed later than the picture ColPic of a different time, the distance between pictures td will be a negative value.

Next, subtract the POC of the reference picture referenced by the list of currently processed pictures LX from the POC of the currently processed picture to obtain the picture distance tb.
tb = [POC of the currently processed picture] - [POC of the reference picture corresponding to the reference index of the LX of the time merge candidate]
The distance between pictures tb is calculated as follows (step S4262). Note that if the reference picture referenced in the list LX of the current picture to be processed is displayed earlier than the current picture to be processed, the distance between pictures tb will be a positive value, and if the reference picture referenced in the list LX of the current picture to be processed is displayed later, the distance between pictures tb will be a negative value.

Next, the distances between pictures td and tb are compared (step S4263), and if the distances between pictures td and tb are equal (step S4263: YES), the motion vector mvLXCol is calculated.
mvLXCol = mvCol
The scaling calculation process is then completed (step S4264).

On the other hand, if the distances between pictures td and tb are not equal (step S4263: NO), the variable tx is set as follows:
tx = ( 16384 + Abs( td ) >> 1 ) / td
This is calculated (step S4265). Next, the scaling factor distScaleFactor is calculated as follows:
distScaleFactor = Clip3( -4096, 4095, ( tb * tx + 32 ) >> 6 )
This is calculated (step S4266). Here, Clip3(x,y,z) is calculated so that the minimum value of z is x,
This is a function that restricts the maximum value to y. Next, the motion vector mvLXCol is,
mvLXCol = Clip3( -32768, 32767, Sign( distScaleFactor * mvLXCol )
* ((Abs( distScaleFactor * mvLXCol ) + 127 ) >> 8 ) )
The scaling calculation is then performed (step S4267), and the scaling calculation process is terminated. Here, Sign
(x) is a function that returns the sign of the value x, and Abs(x) is a function that returns the absolute value of the value x.

Refer to Figure 50 again. Then, the motion vector mvL0Col of L0 is added as a candidate to the predicted motion vector candidate list mvpListLX in the normal predicted motion vector mode derivation unit 301 described above (step S4205). However, this addition is only made if the flag availableFlagL0Col=1, which indicates whether the encoded block colCb in reference list 0 is valid or not. Also, L1
The motion vector mvL1Col is added as a candidate to the predicted motion vector candidate list mvpListLX in the normal predicted motion vector mode derivation unit 301 (step S4205). However, this addition is subject to the flag available, which indicates whether the encoded block colCb in reference list 1 is valid or not.
This applies only when eFlagL1Col=1. With this, the processing of the time-predicted motion vector candidate derivation unit 322 is terminated.

The above description of the normal predicted motion vector mode derivation unit 301 pertains to the encoding process, but the same applies to the decoding process. In other words, the operation of the time predicted motion vector candidate derivation unit 422 in the normal predicted motion vector mode derivation unit 401 in Figure 23 can be described similarly, by replacing encoding with decoding in the above description.

<Deriving candidate time merges>
The operation of the time merge candidate derivation unit 342 in the normal merge mode derivation unit 302 of Figure 18 will be explained with reference to Figure 56.

First, ColPic is derived (step S4301). Next, the coded block colCb is derived and coded information is obtained (step S4302). Furthermore, inter prediction information is derived for each reference list (steps S4303, S4304). The above processing is the same as S4201 to S4204 in the time prediction motion vector candidate derivation unit 322, so the explanation is omitted.

Next, calculate the flag availableFlagCol which indicates whether the coded block colCb is valid or not.
Step S4305). If the flag availableFlagL0Col or the flag availableFlagL1Col is
If the value is 1, availableFlagCol will be 1. Otherwise, availableFlagCol will be 0.

Then, the motion vectors mvL0Col for L0 and mvL1Col for L1 are added as candidates to the merge candidate list mergeCandList in the normal merge mode derivation unit 302 (step S4306). However, this addition is only performed if the flag availableFlagCol=1, which indicates whether the encoded block colCb is valid or not. Thus, the time merge candidate derivation unit 34
The process in step 2 is terminated.

The above description of the time merge candidate derivation unit 342 pertains to the encoding process, but the same applies to the decoding process. In other words, the operation of the time merge candidate derivation unit 442 in the normal merge mode derivation unit 402 of Figure 24 can be described similarly, by replacing encoding with decoding in the above description.

<Update to the list of candidate motion vectors for historical prediction>
Next, the encoding information storage memory 111 on the encoding side and the encoding information storage memory 20 on the decoding side
This section provides a detailed explanation of how to initialize and update the HmvpCandList, a list of candidate motion vectors for historical prediction, which is prepared for step 5. Figure 38 is a flowchart illustrating the initialization and update process for the historical prediction motion vector candidate list.

In this embodiment, the updating of the history prediction motion vector candidate list HmvpCandList is performed in the encoded information storage memory 111 and the encoded information storage memory 205. Alternatively, a history candidate list update unit may be installed in the interpretation unit 102 and the interpretation unit 203 to perform the updating of the history prediction motion vector candidate list HmvpCandList.

At the beginning of the slice, the history prediction motion vector candidate list HmvpCandList is initialized. On the encoding side, the prediction method determination unit 105 updates the history prediction motion vector candidate list HmvpCandList if the normal prediction vector mode or normal merge mode is selected. On the decoding side, the bit sequence decoding unit 201 updates the history prediction motion vector candidate list HmvpCandList if the decoded inter prediction mode is the normal prediction vector mode or normal merge mode.

When performing interprediction in normal prediction vector mode or normal merge mode, the interprediction information used is defined as the interprediction information candidate hMvpCand and the historical prediction motion vector candidate list.
Register in HmvpCandList. The interpretation information candidate hMvpCand contains the L0 reference index.
refIdxL0 and L1 reference index refIdxL1, L0 prediction flag predFlagL0 indicating whether or not L0 prediction is performed, and L1 prediction flag predFlagL1 indicating whether or not L1 prediction is performed.
This includes the motion vector mvL0 of L0 and the motion vector mvL1 of L1. Among the elements registered in the history prediction motion vector candidate list HmvpCandList provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side (i.e., inter-prediction information),
If interpretation information with the same value as the interpretation information candidate hMvpCand exists, that element is removed from the historical prediction motion vector candidate list HmvpCandList. On the other hand, if interpretation information with the same value as the interpretation information candidate hMvpCand does not exist, the first element of the historical prediction motion vector candidate list HmvpCandList is removed, and the historical prediction motion vector candidate list HmvpCand
Add the interpretation information candidate hMvpCand to the end of the List.

The encoding information storage memory 111 on the encoding side and the decoding information storage memory 2 on the decoding side of the present invention
The number of elements in the history prediction motion vector candidate list HmvpCandList, which is prepared for 05, is set to 6.

First, the history prediction motion vector candidate list HmvpCandList is initialized for each slice (step S2101 in Figure 38). At the beginning of the slice, the history prediction motion vector candidate list Hm
Clear all elements in vpCandList, and set the value of NumHmvpCand, which is the number of historical prediction motion vector candidates registered in the historical prediction motion vector candidate list HmvpCandList, to 0.

Although the initialization of the history prediction motion vector candidate list HmvpCandList is performed on a slice-by-slice basis (the first encoded block of a slice), it may also be performed on a picture-by-picture, tile-by-tile, or tree block-by-row basis.

Next, for each encoded block within the slice, the following list of candidate historical motion vectors (Hmvp) is presented.
The CandList update process is repeated (steps S2102 to S2107 in Figure 38).

First, initial settings are performed on a per-encoded block basis. The flag `identicalCandExist`, which indicates whether or not identical candidates exist, is set to FALSE, and the index to be deleted, `removeIdx`, is set to 0.
Set this (step S2103 in Figure 38).

Candidate Interpretation Information hMvp to be registered in the Historical Prediction Motion Vector Candidate List HmvpCandList
It is determined whether or not a Cand exists (step S2104 in Figure 38). If the coding side prediction method determination unit 105 determines that it is the normal prediction motion vector mode or the normal merge mode,
Alternatively, if the bit string decoding unit 201 on the decoding side decodes as normal prediction motion vector mode or normal merge mode, the inter prediction mode is set to hMvpCand. If the prediction method determination unit 105 on the encoding side determines that it is intra prediction mode, subblock prediction motion vector mode, or subblock merge mode, or if the bit string decoding unit 201 on the decoding side decodes as intra prediction mode, subblock prediction motion vector mode, or subblock merge mode, the history prediction motion vector candidate list HmvpCandList is not updated, and there are no inter prediction information candidate hMvpCand to register. If there are no inter prediction information candidate hMvpCand to register, steps S2105 to S2106 are skipped (step S2104 in Figure 38: NO). If there are inter prediction information candidate hMvpCand to register, the following steps are performed (step S2104 in Figure 38: YES).
.

Next, it is determined whether there is an element in the list of candidate historical motion vectors HmvpCandList that is identical to the candidate interpretation information hMvpCand to be registered (step S2105 in Figure 38). Figure 39 is a flowchart of this identical element confirmation procedure. If the value of the number of candidate historical motion vectors NumHmvpCand is 0 (step S2121 in Figure 39: NO),
The historical prediction motion vector candidate list HmvpCandList is empty, and there are no identical candidates, as shown in Figure 39.
Steps S2122 to S2125 are skipped, and this identical element verification process is terminated. If the value of the number of historical prediction motion vector candidates NumHmvpCand is greater than 0 (step S2 in Figure 39)
121: YES), the process in step S2123 is repeated as the historical prediction motion vector index hMvpIdx goes from 0 to NumHmvpCand-1 (steps S2122 to S2125 in Figure 39).
First, counting from 0, the hMvpIdx-th element HmvpCandLis
The system compares t[hMvpIdx] with the interpretation information candidate hMvpCand to determine if they are identical (step S2123 in Figure 39). If they are identical (step S2123 in Figure 39: YES), the flag identicalCandExist, which indicates whether an identical candidate exists, is set to TRUE, the value of hMVpIndex is set to the index to be deleted removeIdx, and this identical element verification process is terminated. If they are not identical (step S2123 in Figure 39: NO), hMvpIdx is incremented by 1, and if the historical prediction motion vector index hMvpIdx is less than or equal to NumHmvpCand-1, the process from step S2123 onward is performed (steps S2122 to S2125 in Figure 39).

Returning to the flowchart in Figure 38, the shifting and addition of elements in the history prediction motion vector candidate list HmvpCandList is performed (step S2106 in Figure 38). Figure 40 is a flowchart of the element shifting/addition procedure for the history prediction motion vector candidate list HmvpCandList in step S2106 of Figure 38. First, it is determined whether to remove elements stored in the history prediction motion vector candidate list HmvpCandList and then add new elements, or to add new elements without removing any elements. Specifically, the flag identicalC is used to indicate whether or not identical candidates exist.
andExist is compared to TRUE or NumHmvpCand is 6 (step S21 in Figure 40).
41) The flag identicalCandExist, which indicates whether or not identical candidates exist, is TRUE or
If NumHmvpCand satisfies any of the six conditions (Step S2141 in Figure 40: YES)
Then, remove the elements stored in the history prediction motion vector candidate list HmvpCandList and add a new element. Set the initial value of index i to removeIdx + 1. Repeat the element shifting process in step S2143 from this initial value to NumHmvpCand (steps S2142 to S2144 in Figure 40). Shift the elements forward by copying the elements of HMVPCandList[i] to HMVPCandList[i - 1] (step S2143 in Figure 40), and increment i by 1 (steps S2142 to S2144 in Figure 40).
When Cand becomes +1 and the element shifting process in step S2143 is completed, the inter-prediction information candidate hMvpCand is added to the end of the history prediction motion vector candidate list (step S2 in Figure 40).
145). Here, the last in the list of candidate motion vectors for historical prediction is (NumHmvp) counting from 0.
This is the Cand-1)th HMVPCandList[NumHmvpCand-1]. This completes the element shifting and addition process for the history prediction motion vector candidate list HMVPCandList. On the other hand, if neither the flag identicalCandExist (indicating whether an identical candidate exists) is TRUE and NumHmvpCand is 6 is met (step S2141 in Figure 40: NO), the inter-prediction information candidate hMvpCand is added to the end of the history prediction motion vector candidate list HmvpCandList without removing any elements stored in the list (step S2146 in Figure 40). Here, the end of the history prediction motion vector candidate list is the NumHmvpCandth HMVPC counting from 0.
This is andList[NumHmvpCand]. Then, NumHmvpCand is incremented by 1, and the element shift/addition process of this history prediction motion vector candidate list HMVPCandList is terminated.

Figure 43 illustrates an example of the update process for the historical prediction motion vector list. When six elements (interpretation information) are registered in the historical prediction motion vector candidate list HMVPCandList, and new interpretation information is added, the historical prediction motion vector candidate list HMVP
By comparing each element of CandList with the new interpretation information from the front (Figure 43(a)), if the new interpretation information is the same value as the third element from the top of the historical prediction motion vector candidate list HMVPCandList, then the element HMV
Delete P2 and shift (copy) the subsequent elements HMVP3 to HMVP5 forward one by one, and add new interpretation information to the end of the history prediction motion vector candidate list HMVPCandList (Figure 43)
(b)) Complete the update of the historical prediction motion vector candidate list HMVPCandList (Figure 43 (c)
)).

<Historical prediction motion vector candidate derivation process>
Next, we will explain in detail the method for deriving history prediction motion vector candidates from the history prediction motion vector candidate list HMVPCandList, which is a common process in step S304 of Figure 20 for the history prediction motion vector candidate derivation unit 323 of the encoding side's normal prediction motion vector mode derivation unit 301 and the decoding side's normal prediction motion vector mode derivation unit 401. Figure 41 is a flowchart illustrating the history prediction motion vector candidate derivation process.

The current number of predicted motion vector candidates is numCurrMvpCand, and the list of predicted motion vector candidates is mvpLis.
The number of elements in tLX (which we'll assume is 2) or the number of historically predicted motion vector candidates NumHmvp
If the value of Cand is 0 (step S2201 in Figure 41: NO), then step S220 in Figure 41
The process from step 2 to S2209 is omitted, and the procedure for deriving candidate motion vectors based on historical predictions is terminated.
The current number of predicted motion vector candidates is numCurrMvpCand, and the list of predicted motion vector candidates is mvpListL.
If the number of elements in X is less than 2, and the number of historical prediction motion vector candidates NumHmvpCand
If the value is greater than 0 (step S2201 in Figure 41: YES), step S in Figure 41
Process steps 2202 through S2209.

Next, the process from steps S2203 to S2208 in Figure 41 is repeated until the index i is less than 3 or the number of historical prediction motion vector candidates NumHmvpCand.
Steps S2202-S2209 in Figure 41). Number of current predicted motion vector candidates: numCurrM
If vpCand is 2 or more, which is the maximum number of elements in the predicted motion vector candidate list mvpListLX (Figure 4)
Step S2203: NO), the processing steps S2204 to S2209 in Figure 41 are omitted, and this history prediction motion vector candidate derivation procedure is terminated. The current number of prediction motion vector candidates, numCurrMvpCand, is 2, which is the maximum number of elements in the prediction motion vector candidate list, mvpListLX.
If it is smaller (step S2203 in Figure 41: YES), step S2204 in Figure 41
The following steps will be performed.

Next, steps S2205 to S2207 are performed for Y = 0 and 1 (L0 and L1), respectively (steps S2204 to S2208 in Figure 41). If the current number of predicted motion vector candidates, numCurrMvpCand, is 2 or greater than or equal to the maximum number of elements in the predicted motion vector candidate list, mvpListLX (step S2205 in Figure 41: NO), steps S2206 to S2209 in Figure 41 are omitted, and this history predicted motion vector candidate derivation procedure is terminated. The current number of predicted motion vector candidates, numCurrMvpCand, is 2 or greater than or equal to the maximum number of elements in the predicted motion vector candidate list, mvpListLX
If the number of elements is less than the maximum number of elements, which is 2 (step S2205 in Figure 41: YES), Figure 41
The process from step S2206 onwards is performed.

Next, if the reference index of LY in the historical prediction motion vector candidate list HmvpCandList[i] is the same as the reference index refIdxLX of the motion vector to be encoded/decoded (step S2206 in Figure 41: YES), then the last element of the prediction motion vector candidate list is the numCurrMvpCand-th element counting from 0 of the prediction motion vector candidate list mvpListLX[numCurrMvpCand
The motion vector LY from the historical prediction motion vector candidate list HmvpCandList[i] is added to ] (step S2207 in Figure 41), and the current number of prediction motion vector candidates numCurrMvpCand is incremented by 1. The reference index of LY in the historical prediction motion vector candidate list HmvpCandList[i] is,
If the reference index refIdxLX of the motion vector to be encoded/decoded is not the same (step S2206 in Figure 41: NO), the additional processing in step S2207 is skipped.

The processes from steps S2205 to S2207 in Figure 41 are performed on both L0 and L1 (steps S2204 to S2208 in Figure 41).

The index i is incremented by 1. If the index i is less than or equal to the smaller of 3 and the number of historical prediction motion vector candidates, NumHmvpCand, the process from step S2203 onwards is repeated (steps S2202 to S2209 in Figure 41).

<History merge candidate derivation process>
Next, we will explain in detail the method for deriving history merge candidates from the history merge candidate list HmvpCandList, which is a common process in the history merge candidate derivation unit 345 of the encoding side's normal merge mode derivation unit 302 and the history merge candidate derivation unit 445 of the decoding side's normal merge mode derivation unit 402, as shown in step S404 of Figure 21. Figure 42 is a flowchart illustrating the history merge candidate derivation process.

First, the initialization process is performed (step S2301 in Figure 42). isPruned[i] is set from 0 to (numCu
Set the value of FALSE to each element at the -1th rrMergeCand, and set the variable numOrigMergeCand to the number of elements currently registered in the merge candidate list, numCurrMergeCand.

Next, the initial value of index hMvpIdx is set to 1, and the additional processing from step S2303 to step S2310 in Figure 42 is repeated from this initial value up to NumHmvpCand (Figure 4
Steps S2302 to S2311 of step 2). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is not less than or equal to (maximum number of merge candidates, MaxNumMergeCand-1), then merge candidates have been added to all elements in the merge candidate list, and this history merge candidate derivation process is terminated (step S2303 in Figure 42: NO). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is less than or equal to (maximum number of merge candidates, MaxNumMergeCand-1) (step S2303 in Figure 42: YES), the process from step S2304 onwards is performed.

First, set the value of sameMotion to FALSE (step S2304 in Figure 42). Next, set the initial value of index i to 0, and then from this initial value down to 1 in step S2 in Figure 42.
306. The process in S2307 is performed (S2305 to S2308 in Figure 42).

Next, we compare whether the (NumHmvpCand-hMvpIdx)th element HmvpCandList[NumHmvpCand-hMvpIdx] from the history motion vector prediction candidate list and the i-th element mergeCandList[i] from the merge candidate list have the same value (step S2306 in Figure 42). Here, the merge candidates having the same value means that all the components of the merge candidates (inter-prediction modes,
This indicates that the values of the reference index and motion vector are the same. However, this step S2306 is performed only if hMvpIdx is greater than NumHmvpCand-2, mergeCandList[i] is a spatial merge candidate, and isPruned[i] is FALSE. If the values are the same (step S2306 in Figure 39: YES), both sameMotion and isPruned[i] are set to TRUE (step S2307 in Figure 42). If the values are not the same (step S2306 in Figure 39: N
O) Skip the processing in step S2307. Once the iterative processing from step S2305 to step S2308 in Figure 42 is complete, compare whether sameMotion is FALSE (step S2309 in Figure 42), and if sameMotion is FALSE (step S2309 in Figure 42: YES), the mergeCandList[nu] of the numCurrMergeCand-th merge candidate list
[mCurrMergeCand] counts from 0 in the list of candidate motion vectors for historical prediction (NumHmvpCand - hMvp
The element at the Idxth position, HmvpCandList[NumHmvpCand - hMvpIdx], is added, and numCurrMergeCand is incremented by 1 (step S2310 in Figure 42). The index hMvpIdx is incremented by 1 (step S2302 in Figure 42), and the process of steps S2302 to S2311 in Figure 42 is repeated.

The derivation process for this history merge candidate is completed when all elements in the history prediction motion vector candidate list have been checked, or when merge candidates have been added to all elements in the merge candidate list.

<Process for deriving average merge candidates>
Next, we will explain in detail the method for deriving the average merge candidate, which is a common process in the average merge candidate derivation unit 344 of the encoding side's normal merge mode derivation unit 302 and the average merge candidate derivation unit 444 of the decoding side's normal merge mode derivation unit 402, specifically the processing procedure in step S403 of Figure 21. Figure 62 is a flowchart illustrating the average merge candidate derivation processing procedure.

First, an initialization process is performed (step S1301 in Figure 62). The variable numOrigMergeCand is set to numCurrMergeCand, which is the number of elements currently registered in the merge candidate list.

Next, we scan the merge candidate list from the beginning and determine the two motion information items. Let index i=0 represent the first motion information item, and index j=1 represent the second motion information item.
Steps S1302 to S1303 in Figure 62). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is not less than or equal to (MaxNumMergeCand-1), then merge candidates have been added to all elements in the merge candidate list, and this history merge candidate derivation process is terminated (Step S1304 in Figure 62). If the number of elements registered in the current merge candidate list, numCurrMergeCand, is less than or equal to (MaxNumMergeCand-1), then the process from Step S1305 onwards is performed.

Determine whether both the i-th movement information mergeCandList[i] and the j-th movement information mergeCandList[j] in the merge candidate list are invalid (step S130 in Figure 62).
5) If both are invalid, do not derive the average merge candidate for mergeCandList[i] and mergeCandList[j], and move on to the next element. If both mergeCandList[i] and mergeCandList[j] are valid, set X to 0 and 1 and repeat the following process (from step S1306 to S in Figure 62).
1314).

Determine if the LX prediction for mergeCandList[i] is valid (step S1307 in Figure 62).
If the LX prediction for mergeCandList[i] is valid, it is determined whether the LX prediction for mergeCandList[j] is valid (step S1308 in Figure 62). If the LX prediction for mergeCandList[j] is valid, that is, if both the LX prediction for mergeCandList[i] and the LX prediction for mergeCandList[j] are valid, the average merge candidate for the LX prediction, which has the motion vector of the LX prediction obtained by averaging the motion vector of the LX prediction for mergeCandList[i] and the motion vector of the LX prediction for mergeCandList[j] and the reference index of the LX prediction for mergeCandList[i], is derived and set as the LX prediction for averageCand, and avera
Enable LX prediction for geCand (step S1309 in Figure 62). Step S13 in Figure 62
In 08, if the LX prediction for mergeCandList[j] is not valid, that is, the LX of mergeCandList[i]
If prediction is enabled and LX prediction for mergeCandList[j] is disabled, then LX for mergeCandList[i]
Derive the average merge candidate for LX predictions with prediction motion vectors and reference indices.
Set ageCand to LX prediction and enable averageCand LX prediction (step S13 in Figure 62)
10). In step S1307 of Figure 62, if the LX prediction of mergeCandList[i] is not valid,
Determine whether the LX prediction for mergeCandList[j] is valid (step S1311 in Figure 62)
). If the LX prediction for mergeCandList[j] is valid, i.e., the LX prediction for mergeCandList[i] is invalid and the LX prediction for mergeCandList[j] is valid, then the average merge candidate for the LX prediction with the motion vector and reference index of the LX prediction for mergeCandList[j] is derived and averageC
Set to LX prediction for and and enable LX prediction for averageCand (step S1312 in Figure 62)
). In step S1311 of Figure 62, if the LX prediction for mergeCandList[j] is not valid, that is, if both the LX prediction for mergeCandList[i] and the LX prediction for mergeCandList[j] are invalid,
Disable the LX prediction for averageCand (step S1312 in Figure 62).

Here, LX prediction is considered valid if the reference index refIdxLX is 0 or greater. If LX prediction is invalid, i.e., does not exist, the reference index refIdxLX is set to -1.

The average merge candidate averageCand of the L0, L1, or BI predictions generated as described above is added to the mergeCandList[numCurrMergeCand] at the numCurrMergeCand position in the merge candidate list, and numCurrMergeCand is incremented by 1 (step S1315 in Figure 62). This completes the derivation process of the average merge candidate.

The average merge candidate is calculated by averaging the horizontal and vertical components of the motion vector.

<Derivation of subblock time merge candidates>
The operation of the subblock time merge candidate derivation unit 381 in the subblock merge mode derivation unit 304 of Figure 16 will be explained with reference to Figure 44.

First, it is determined whether the encoded block is less than 8x8 pixels (step S4002).

If the encoded block is less than 8x8 pixels (step S4002: YES), set the flag availableFlagSbCol=0 to indicate the existence of a subblock time merge candidate (step S400
3) The processing of the subblock time merge candidate derivation unit is terminated. Here, if temporal motion vector prediction is prohibited by syntax, or if subblock time merging is prohibited, if the encoded block is less than 8x8 pixels (step S4002:
Perform the same process as YES.

On the other hand, if the encoded block is 8x8 pixels or larger (step S4002: NO), adjacent motion information of the encoded block in the encoded picture is derived (step S4004).

The process for deriving adjacent movement information of encoded blocks will be explained with reference to Figure 45.
The process for deriving adjacent motion information is similar to the process of the spatial prediction motion vector candidate derivation unit 321 described above. However, the order in which adjacent blocks are searched is A0, B0, B1, A1, and B2 is not searched. First, the encoded information is obtained for adjacent block n=A0 (step S4052).
The encoded information includes a flag availableFlagN indicating whether adjacent blocks are available, a reference index refIdxLXN for each reference list, and a motion vector mvLXN.

Next, it is determined whether the adjacent block n is valid or invalid (step S4054). If the flag availableFlagN=1, which indicates whether the adjacent block can be used, it is valid; otherwise, it is invalid.

If the adjacent block n is valid (step S4054: YES), the reference index ref
Let IdxLXN be the reference index refIdxLXn of the adjacent block n (step S4056).
Furthermore, the motion vector mvLXN is used as the motion vector mvLXn of the adjacent block n (step S4)
056) The process of deriving adjacent movement information of blocks is terminated.

On the other hand, if adjacent block n is invalid (step S4054: NO), then adjacent block n = B
The encoding information is obtained by setting it to 0 (step S4052), and it is determined whether the adjacent block n is valid or invalid (step S4054). The same process is then repeated, looping through B1 and A1 in that order. The process of deriving adjacent movement information loops until an adjacent block is valid, and if all adjacent blocks A0, B0, B1, and A1 are invalid, the process of deriving adjacent movement information for the block is terminated.

Refer to Figure 44 again. After deriving the adjacent motion information (step S4004), derive the temporal motion vector (step S4006).

The process for deriving the temporal motion vector will be explained with reference to Figure 46. First,
The temporal motion vector is initialized as tempMv=(0,0) (step S4062).

Next, it is determined whether the adjacent motion information is valid or invalid (step S4064). If the flag availableFlagN=1, which indicates whether the adjacent block can be used, it is valid; otherwise, it is invalid. If the adjacent motion information is invalid (step S4064: NO), the process of deriving the temporal motion vector is terminated.

On the other hand, if adjacent movement information is valid (step S4064: YES), it is determined whether the flag predFlagL1N, which indicates whether or not L1 prediction is being used in adjacent block N, is 1 or not (step S4066). If predFlagL1N=0 (step S4066: NO), the next process (
Proceed to step S4078). If predFlagL1N=1 (step S4066: YES),
It is determined whether the POC of all pictures registered in all reference lists is less than or equal to the POC of the currently processed picture (step S4068). If this determination is true (step S4068: YES), the process proceeds to the next step (step S4070).

If slice type slice_type is B slice and flag collocated_from_l0_flag is 0 (step S4070: YES, step and S4072: YES), determine whether ColPic and reference picture RefPicList1[refIdxL1N] (the picture with reference index refIdxL1N in reference list L1) are the same (step S4074). If this determination is true (step S
4074: YES), set the temporal motion vector tempMv = mvL1N (step S4076)
). If this judgment is false (step S4074: NO), the next process (step S4078
) Proceed. If slice type slice_type is not B slice, flag collocated_from_l0_f
If lag is not 0 (step S4070: NO, or step S4072: NO),
Proceed to the next process (step S4078).

And a flag, predFlagL0N, indicates whether or not L0 prediction is being used in the adjacent block N.
Determine whether it is 1 or not (step S4078). If predFlagL0N=1 (step S40
78: YES), determine whether ColPic and the reference picture RefPicList0[refIdxL0N] (the picture with reference index refIdxL0N in reference list L0) are the same (step S4080).
If this judgment is true (step S4080: YES), the temporal motion vector tempMv = m
Set to vL0N (step S4082). If this determination is false (step S4080: NO)
Then, the process of deriving the temporal motion vector is terminated.

Refer to Figure 44 again. Next, ColPic is derived (step S4016). This process is the same as S4201 in the time prediction motion vector candidate derivation unit 322, so the explanation is omitted.

Next, a coding block colCb at a different time is set (step S4017). This involves setting a coding block located in the lower right center of the same position as the coding block to be processed within the picture ColPic at a different time as colCb. This coding block corresponds to coding block T1 in Figure 49.

Next, the position obtained by adding the temporal motion vector tempMv to the encoded block colCb is set as the new colCb (step S4018). Now, the upper left position of the encoded block colCb is (xCo
lCb, yColCb), temporal motion vector tempMv with 1/16 pixel precision (tempMv[0], tempMv[1])
And,
xColCb = Clip3( xCtb, xCtb + CtbSizeY + 3, xColCb + ( tempMv[0] >> 4 ) )
yColCb = Clip3( yCtb, yCtb + CtbSizeY - 1, yColCb + ( tempMv[1] >> 4 ) )
The following is calculated. Here, the top-left position of the tree block is (xCtb, yCtb), and the size of the tree block is CtbSizeY. The encoded block on ColPic that contains the position ((xColCb >> 3) << 3, (yColCb >> 3) << 3) becomes the new colCb. As shown in the above formula, the position after adding tempMv is t
To prevent significant deviations compared to before the empMv addition, the value is corrected to a range approximately the size of the tree block. If this position falls outside the screen, it is corrected to be within the screen.

And the prediction mode of this coding block colCb is PredMode (MODE_INTER
It is determined whether or not the prediction mode of colCb is interprediction (step S4020: NO). If the prediction mode of colCb is not interprediction, a flag av is displayed indicating the presence of a subblock time merge candidate.
Set ailableFlagSbCol=0 (step S4003) and terminate the processing of the subblock time merge candidate derivation unit.

On the other hand, if the prediction mode for colCb is inter-prediction (step S4020: YES), inter-prediction information is derived for each reference list (steps S4022, S4023). Here, for colCb, the central motion vector ctrMvLX and the flag ctrPredFlagLX indicating whether or not LX prediction is being used are derived for each reference list. LX represents the reference list, and in the derivation of reference list 0, LX becomes L0, and in the derivation of reference list 1, LX becomes L1. The derivation of inter-prediction information will be explained with reference to Figure 47.

If a coding block colCb of a different time is unavailable (Step S4112: NO)
, or if the prediction mode PredMode is intra prediction (MODE_INTRA) (step S4114)
: NO), set both flag availableFlagLXCol and flag predFlagLXCol to 0 (step S
4116) Set the motion vector mvCol to (0,0) (step S4118), and terminate the derivation process of the interpretation information.

The coding block colCb is available (step S4112: YES), and the prediction mode PredMod
If e is not an intra prediction (MODE_INTRA) (step S4114: YES), calculate mvCol, refIdxCol and availableFlagCol using the following procedure.

A flag, PredFlagLX[xPCo], indicates whether or not the LX prediction for the coded block colCb is being used.
If l[yPCol] is 1 (step S4120: YES), the motion vector mvCol is set to the same value as MvLX[xPCol][yPCol] which is the motion vector of LX of the coded block colCb (step S4122), and the reference index refIdxCol is the reference index RefIdxLX[xPCol] of LX.
[yPCol] is set to the same value (step S4124), and the list listCol is set to LX (step S4126). Here, xPCol and yPCol are indices indicating the position of the top-left pixel of the encoded block colCb within the picture ColPic at different times.

On the other hand, the flag PredFlagL indicates whether or not LX prediction for the coded block colCb is being used.
If X[xPCol][yPCol] is 0 (step S4120: NO), the following process is performed. First, it is determined whether the POC of all pictures registered in all reference lists is less than or equal to the POC of the currently processed picture (step S4128). Also, it is determined whether the flag PredFlagLY[xPCol][yPCol], which indicates whether the LY prediction of colCb is being used, is 1 (step S4128). Here, the LY prediction is defined as a different reference list from the LX prediction. That is, if LX=L0, LY=L1, and if LX=L1, LY=L0.

If this determination is true (step S4128: YES), the motion vector mvCol is set to the same value as MvLY[xPCol][yPCol], which is the motion vector of LY in the coded block colCb (step S4130), and the reference index refIdxCol is set to the reference index RefIdxLY[xPCol] of LY.
It is set to the same value as [l][yPCol] (step S4132), and the list listCol is set to LX (step S4134).

On the other hand, if this judgment is false (step S4128: NO), the flag availableFlagLXCol
Set both flag predFlagLXCol to 0 (step S4116), and set motion vector mvCol to (0
The process of deriving interpretation information is terminated (step S4118) as 0.

If interpretation information can be obtained from the coded block colCb, the flag availableFlagLXCo
Set both l and the flag predFlagLXCol to 1 (step S4136).

Next, the motion vector mvCol is scaled to obtain the motion vector mvLXCol (step S4138). This process is performed in S424 of the time-predicted motion vector candidate derivation unit 322.
Since it is the same as number 5, the explanation will be omitted.

Refer to Figure 44 again. After deriving the interpretation information for each reference list, the calculated motion vector mvLXCol is set as the center motion vector ctrMvLX, and the calculated flag predFlagLXCol is set as the flag ctrPredFlagLX (steps S4022, S4023).

Then, it is determined whether the central motion vector is valid or invalid (step S4024).
If dFlagL0=0 and ctrPredFlagL1=0, it is considered invalid; otherwise, it is considered invalid. If the central motion vector is invalid (step S4024: NO), the flag availableFlagSbCol=0, indicating the existence of a subblock time merge candidate, is set (step S4003), and the processing of the subblock time merge candidate derivation unit is terminated.

On the other hand, if the central motion vector is valid (step S4024: YES), set the flag availableFlagSbCol=1 to indicate the presence of a subblock time merge candidate (step S402
5) Derive subblock movement information (step S4026). This process will be explained with reference to Figure 48.

First, from the width cbWidth and height cBheight of the encoded block colCb, we get the number of subblocks in the width direction.
Calculate numSbX and the number of subblocks in the height direction numSbY (step S4152). Also, set refIdxLXSbCol=0 (step S4152). From this point onward, predict subblock c
The process is repeated in units of olSb. This iteration sets the height index ySbIdx to 0.
The process is carried out by changing the width index xSbIdx from 0 to numSbX, from numSbY to numSbY.

If the top-left position of the encoded block colCb is (xCb, yCb), then the top-left position of the prediction subblock colSb is (xSb, ySb).
xSb = xCb + xSbIdx * sbWidth
ySb = yCb + ySbIdx * sbHeight
This is calculated. Next, the position obtained by adding the temporal motion vector tempMv to the predicted subblock colSb is set as the new colSb (step S4154). The upper left position of the predicted subblock colSb is (xColSb, yColSb), and the temporal motion vector tempMv is set to (tempMv[0], with a 1/16 pixel precision).
If we let tempMv[1]), the top-left position of the new colSb is:
xColSb = Clip3( xCtb, xCtb + CtbSizeY + 3, xSb + ( tempMv[0] >> 4 ) )
yColSb = Clip3( yCtb, yCtb + CtbSizeY - 1, ySb + ( tempMv[1] >> 4 ) )
This is the result. Here, the top-left position of the tree block is (xCtb, yCtb), and the size of the tree block is CtbSizeY. As shown in the above equation, the position after tempMv is corrected to a range approximately equal to the size of the tree block so that it does not deviate significantly from the position before tempMv was added. If this position falls outside the screen, it is corrected to be within the screen.

Then, interpretation information is derived for each reference list (steps S4156, S41
58). Here, for the prediction subblock colSb, we have the motion vector mvLXSbCol for each reference list on a subblock basis, and the flag availableFl indicating whether the prediction subblock is valid or not.
We derive agLXSbCol. LX represents a reference list, and in the derivation of reference list 0, LX becomes L0.
In the derivation of reference list 1, LX becomes L1. The derivation of the interpretation information is shown in Figure 47, S4022.
Since it is the same as S4023, the explanation will be omitted.

After deriving the interpretation information (steps S4156, S4158), the prediction subblock col
Determine whether Sb is valid (step S4160). availableFlagL0SbCol=0 and available
If bleFlagL1SbCol=0, colSb is considered invalid; otherwise, it is considered valid. If colSb is invalid (step S4160: NO), the motion vector mvLXSbCol is set to the center motion vector ctrMvLX (step S4162). Furthermore, a flag predFl indicates whether or not LX prediction is being used.
Let agLXSbCol be the flag ctrPredFlagLX in the central motion vector (step S416)
2) This concludes the derivation of subblock movement information.

Refer to Figure 44 again. Then, the motion vectors of L0, mvL0SbCol and L1, mvL1SbCol are added as candidates to the subblock merge candidate list subblockMergeCandList in the subblock merge mode derivation unit 304 (step S4028).
However, this addition is a flag that indicates the presence of a subblock time merge candidate, availableSbCol
This applies only when = 1. With this, the processing of the time merge candidate derivation unit 342 is terminated.

The above description of the subblock time merge candidate derivation unit 381 pertains to the encoding process, but the same applies to the decoding process. In other words, the operation of the subblock time merge candidate derivation unit 481 in the subblock merge mode derivation unit 404 of Figure 22 can be described similarly, by replacing encoding with decoding in the above description.

<Motion compensation prediction processing>
The motion compensation prediction unit 306 obtains the position and size of the block currently being predicted during encoding. The motion compensation prediction unit 306 also obtains inter-prediction information from the inter-prediction mode determination unit 305. From the obtained inter-prediction information, it derives a reference index and a motion vector, and generates a prediction signal after obtaining an image signal of the reference picture identified by the reference index in the decoded image memory, moved by the amount of the motion vector from the same position as the image signal of the prediction block.

In interpretation, if the interpretation mode is L0 prediction or L1 prediction, which is prediction from a single reference picture, the prediction signal obtained from one reference picture is used as the motion-compensated prediction signal. If the interpretation mode is BI prediction, which is prediction from two reference pictures, the prediction signals obtained from the two reference pictures are weighted and averaged to obtain the motion-compensated prediction signal, and this motion-compensated prediction signal is supplied to the prediction method determination unit. Here, the weighted averaging ratio of the two predictions is set to 1:1, but other ratios may be used for weighted averaging.
For example, the closer the interval between the picture being predicted and the reference picture, the greater the weighting ratio. Alternatively, the weighting ratio can be calculated using a correspondence table between combinations of picture intervals and weighting ratios.

The motion compensation prediction unit 406 has the same function as the motion compensation prediction unit 306 on the encoding side. The motion compensation prediction unit 406 uses interprediction information, the normal prediction motion vector mode derivation unit 401,
The normal merge mode derivation unit 402, the subblock predicted motion vector mode derivation unit 403, and the subblock merge mode derivation unit 404 are acquired via switch 408.

The motion compensation prediction unit 406 supplies the obtained motion compensation prediction signal to the decoded image signal superimposition unit 207.

<About Interpretation Mode>
The process of making a prediction from a single reference picture is defined as single prediction. In the case of single prediction, the prediction is made using either L0 prediction or L1 prediction, which are two reference pictures registered in the reference lists L0 and L1. L0 prediction and L1 prediction may be either forward prediction (prediction referring to a forward reference image) or backward prediction (prediction referring to a backward reference image). Figures 57-58 are diagrams illustrating motion compensation prediction in L0 prediction (single prediction).

Figure 57 shows that the interpretation mode is L0 prediction and the reference picture of L0 (RefL0Pi
c) shows the case where the processing time is earlier than that of the picture to be processed (CurPic). Figure 58
This indicates an L0 prediction where the L0 reference picture is at a later time than the picture to be processed. Similarly, single prediction can be performed by replacing the L0 prediction reference picture (RefL1Pic) in Figures 57 and 58 with the L1 prediction reference picture.

The process of making predictions from two reference pictures is defined as dual prediction, and in the case of dual prediction, both L0 prediction and L1 prediction are used and expressed as dual prediction. Figures 59 to 61 are diagrams to explain motion compensation prediction in dual prediction. Figure 59 shows a case of dual prediction where the reference picture for L0 prediction is at a time earlier than the picture to be processed, and the reference picture for L1 prediction is at a time later than the picture to be processed. Figure 60 shows a case of dual prediction where the reference pictures for L0 prediction and L1 prediction are at a time earlier than the picture to be processed. Figure 61 shows a case of dual prediction where the reference pictures for L0 prediction and L1 prediction are at a time later than the picture to be processed.

Thus, the relationship between the prediction type and time of L0/L1 can be used without being limited to L0 being a forward prediction (prediction referring to a forward reference image) and L1 being a backward prediction (prediction referring to a backward reference image). Furthermore, in the case of dual prediction, both L0 and L1 predictions may be performed using the same reference picture. The decision of whether to perform motion compensation prediction as a single prediction or a dual prediction is made based on information (e.g., a flag) indicating whether or not to use L0 prediction and whether or not to use L1 prediction.

<About the reference index>
In an embodiment of the present invention, in order to improve the accuracy of motion compensation prediction, it is possible to select the optimal reference picture from among multiple reference pictures in the motion compensation prediction. Therefore,
The reference picture used in motion compensation prediction is used as the reference index, and the reference index is encoded in the encoding stream along with the encoding vector.

<Motion compensation processing based on the normal predictive motion vector mode>
As shown in Figure 16, also in the Inter Prediction Unit 102 on the encoding side, the motion compensation prediction unit 306, when the inter prediction information from the normal prediction motion vector mode derivation unit 301 is selected in the inter prediction mode determination unit 305, obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, the motion compensation prediction unit 406, as also shown in the interpretation unit 203 on the decoding side in Figure 22, has a switch 408 that normally predicts the motion vector mode during the decoding process.
When connected to unit 1, the unit normally obtains interpretation information from the predicted motion vector mode derivation unit 401, derives the interpretation mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation processing based on normal merge mode>
As also shown in the interpretation prediction unit 102 on the encoding side in Figure 16, the motion compensation prediction unit 306, when the interpretation prediction information from the normal merge mode derivation unit 302 is selected in the interpretation prediction mode determination unit 305, obtains this interpretation prediction information from the interpretation prediction mode determination unit 305 and determines the interpretation prediction mode of the block currently being processed.
A reference index and motion vector are derived, and a motion compensation prediction signal is generated. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, as shown in the inter-prediction unit 203 on the decoding side in Figure 22, when the switch 408 is connected to the normal merge mode derivation unit 402 during the decoding process, the motion compensation prediction unit 406 acquires inter-prediction information from the normal merge mode derivation unit 402, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Subblock Predicted Motion Vector Mode-Based Motion Compensation Processing>
As shown in Figure 16, also in the Inter Prediction Unit 102 on the encoding side, the motion compensation prediction unit 306, when the inter prediction information from the subblock prediction motion vector mode derivation unit 303 is selected in the inter prediction mode determination unit 305, obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, as shown in the inter-prediction unit 203 on the decoding side in Figure 22, when the switch 408 is connected to the sub-block predicted motion vector mode derivation unit 403 during the decoding process, the motion compensation prediction unit 406 acquires inter-prediction information from the sub-block predicted motion vector mode derivation unit 403, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation processing based on subblock merge mode>
As shown in Figure 16, also in the Inter Prediction Unit 102 on the encoding side, the motion compensation prediction unit 306, when the inter prediction information from the subblock merge mode derivation unit 304 is selected in the inter prediction mode determination unit 305, obtains this inter prediction information from the inter prediction mode determination unit 305, derives the inter prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determination unit 105.

Similarly, the motion compensation prediction unit 406, as also shown in the inter prediction unit 203 on the decoding side in Figure 22, has a switch 408 in the decoding process that is a subblock merge mode derivation unit 404
When connected, the subblock merge mode derivation unit 404 acquires inter-prediction information, derives the inter-prediction mode, reference index, and motion vector of the block currently being processed, and generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal superimposition unit 207.

<Motion compensation processing based on affine mode>
In this embodiment, motion compensation using an affine model can be used. Motion compensation using an affine model uses 2 to 4 corners of an encoded block as control points, derives the motion vector of a subblock from the motion vector of the control points, and performs motion compensation on a subblock basis.

The following flags are reflected in the following flags based on the inter-prediction conditions determined by the inter-prediction mode determination unit 305 during the encoding process, and are encoded in the encoded stream. During the decoding process, it is determined whether or not to perform motion compensation using an affine model based on the following flags in the encoded stream.

The sps_affine_enabled_flag indicates whether motion compensation using an affine model is available in interpretation. If sps_affine_enabled_flag is 0, motion compensation using an affine model is suppressed on a sequence-by-sequence basis. Also, inter_affine_flag and
The cu_affine_type_flag is not transmitted in the CU (encoding unit) syntax of the encoded video sequence. If sps_affine_enabled_flag is 1, motion compensation using the affine model can be used in the encoded video sequence.

The sps_affine_type_flag indicates whether motion compensation using the 6-parameter affine mode is available in interpretation. The 6-parameter affine model derives the motion vector of a subblock from the six parameters of the horizontal and vertical components of the motion vectors of each of the three control points, and performs motion compensation on a subblock basis. Although motion vectors are derived on a subblock basis, a common reference index is derived at the coding block level.

If sps_affine_type_flag is 0, motion compensation is suppressed so as not to be performed by a 6-parameter affine model. Also, cu_affine_type_flag is the CU of the encoded video sequence.
It is not transmitted in the syntax. If sps_affine_type_flag is 1, motion compensation using a 6-parameter affine model can be used in the encoded video sequence.

If sps_affine_type_flag does not exist, it will be set to 0.

When decoding a P or B slice, in the CU currently being processed,
If r_affine_flag is 1, motion compensation using an affine model is used to generate a motion compensation prediction signal for the CU currently being processed.

If inter_affine_flag is 0, the affine model is not used for the CU currently being processed.

If `inter_affine_flag` does not exist, it is set to 0.

When decoding a P or B slice, in the CU currently being processed, cu_a
If ffine_type_flag is 1, motion compensation using a 6-parameter affine model is used to generate the motion compensation prediction signal for the CU currently being processed.

If cu_affine_type_flag is 0, motion compensation using a 4-parameter affine model is used to generate the motion compensation prediction signal for the CU currently being processed. The 4-parameter affine model derives the motion vector of a subblock from four parameters: the horizontal and vertical components of the motion vectors of two control points, and performs motion compensation on a subblock basis.

<Merge Difference Motion Vector (MMVD)>
The difference motion vector can be added to the motion vectors of the top two merge candidates (the merge candidates with merge indices of 0 and 1 in the merge candidate list). This difference motion vector is called the merge difference motion vector.

When adding the merge difference motion vector in the merge candidate selection unit 347 on the encoding side,
The motion vector obtained by adding the merged difference motion vectors is determined by the Interpretation Mode Determination Unit 305
It is supplied to the motion compensation prediction unit 306 via this. The bit string encoding unit 108 encodes information about the merged difference motion vector. The information about the merged difference motion vector is an index mmvd_distance_idx that indicates the distance to be added to the motion vector, and an index mmvd_direction_idx that indicates the direction in which the motion vector is added. These indices are defined as shown in the table in Figures 63(a) and 63(b). Then, if the x and y components of the merged difference motion vector offset MmvdOffset are represented by MmvdOffset[0] and MmvdOffset[1] respectively,
MmvdOffset[0] = ( MmvdDistance << 2 ) * MmvdSign[0]
MmvdOffset[1] = ( MmvdDistance << 2 ) * MmvdSign[1]
The merge difference motion vector is the merge difference motion vector offset MmvdOffs in the above formula.
It is derived from et. Details on deriving the merge difference motion vector will be explained in the following section on the decoding side.

On the decoding side, if a merged difference motion vector exists, information regarding the merged difference motion vector is separated from the bitstream supplied to the bit stream decoding unit 201, and the merged difference motion vector offset MmvdOffset is derived. The merge candidate selection unit 447 then derives the merged difference motion vector from the decoded merged difference motion vector offset.
This merged difference motion vector is added to the motion vector, and then that motion vector is supplied to the motion compensation prediction unit 406.

Figure 6 shows the derivation of the merge difference motion vector mMvdLX in the merge candidate selection unit 447.
Refer to the flowchart in 4(a) for explanation. First, determine whether the interprediction mode of the coding block is biprediction (PRED_BI) or not (S4402). If it is not biprediction (
S4402: No), determine whether it is an L0 prediction (PRED_L0) or not (S4404). If it is an L0 prediction (S4404: Yes),
mMvdL0 = MmvdOffset
mMvdL1 = 0
As a result (S4406), the process of deriving the merge difference motion vector is completed. In the case of L1 prediction (S4404: No),
mMvdL0 = 0
mMvdL1 = MmvdOffset
Thus (S4408), the process of deriving the merge difference motion vector is completed.

On the other hand, in the case of biprediction (S4402: Yes), the picture to be processed currPic and the reference picture
The difference in POC is calculated for each reference list and assigned to currPocDiffL0 and currPocDiffL1 respectively.
S4410). Here, the difference in POC between picA and picB, DiffPicOrderCnt(picA, picB), is:
DiffPicOrderCnt( picA, picB ) = [POC of picA] - [POC of picB]
This indicates that the referenced picture RefPicList0[refIdxL0] is the picture pointed to by the reference index refIdxL0 in reference list L0. Similarly, the referenced picture RefPicList1[refIdxL1] is the picture pointed to by the reference index refIdxL1 in reference list L1.

Next, determine whether -currPocDiffL0 * currPocDiffL1 >= 0 (step S4412)
). If this determination is true (step S4412: Yes),
mMvdL0 = MmvdOffset
mMvdL1 = -MmvdOffset
As a result (step S4414), the process of deriving the merge difference motion vector is completed. On the other hand, if this determination is false (step S4412: No),
mMvdL0 = MmvdOffset
mMvdL1 = MmvdOffset
(Step S4416). Next, it is determined whether the absolute value of the difference between the POC with reference list L0 is greater than or equal to the absolute value of the difference between the POC with reference list L1 (Step S4418). If this determination is true (Step S4418: Yes), X=0, Y=1 (Step S4420), and the merge difference motion vector mMvdL1 of L1 is scaled (Step S4424). Here, mMvdLY
This indicates that if Y=0, it is mMvdL0, and if Y=1, it is mMvdL1. On the other hand, if this determination is false (step S4418: No), X=1, Y=0 (step S4422), and the merge difference motion vector mMvdL0 of L0 is scaled (step S4424). The scaling of the merge difference motion vector mMvdLY is as shown in Figure 64(b),
td = Clip3( -128, 127, currPocDiffLX )
tb = Clip3( -128, 127, currPocDiffLY )
tx = ( 16384 + Abs( td ) >> 1 ) / td
distScaleFactor = Clip3( -4096, 4095, ( tb * tx + 32 ) >> 6 )
mMvdLY = Clip3( -32768, 32767, Sign( distScaleFactor * mMvdLY )
* ((Abs( distScaleFactor * mMvdLY ) + 127 ) >> 8 ) )
This is derived as follows: Here, currPocDiffLX is currPocDiffL0 when X=0, and cu when X=1.
This shows that rrPocDiffL1. Similarly, currPocDiffLY is currPocDiffL0 when Y=0.
This shows that when Y=1, it is currPocDiffL1. Also, Clip3(x,y,z) is a function that restricts the value z to a minimum value of x and a maximum value of y. Sign(x) is a function that returns the sign of the value x, and Abs(x)
This is a function that returns the absolute value of the value x. With this, the process of deriving the merge difference motion vector is complete.

The merge difference motion vector may also be added to the top two motion vectors of the subblock merge candidates. In this case, the index mmvd_dis indicates the distance to be added to the motion vector.
tance_idx is defined as shown in the table in Figure 63(c). The operation of the subblock merge candidate selection unit 386 is the same as that of the merge candidate selection unit 347, so its explanation is omitted. Also,
The operation of the subblock merge candidate selection unit 486 is the same as that of the merge candidate selection unit 447, so its explanation is omitted.

As mentioned above, MmvdDistance is defined as shown in the tables in Figures 63(a) and 63(c). Since these tables are defined with 1/4 pixel precision, the generated merged difference motion vector may include fractional pixel precision. However, by encoding/decoding a flag indicating that the pixel precision of these tables is 1 on a slice-by-slice basis, the generated merged difference motion vector can be modified so that it does not include fractional pixel precision.

<Adaptive Motion Vector Resolution (AMVR)>
The resolution of the differential motion vector can be adaptively changed at the coding block level.
This resolution is called adaptive motion vector resolution.

This section describes the case where adaptive motion vector resolution is used for the normal predictive motion vector mode. In this case, the derived candidate motion vectors are rounded according to the resolution in the spatial predictive motion vector candidate derivation units 321 and 421, the time predictive motion vector candidate derivation units 322 and 422, and the history predictive motion vector candidate derivation units 323 and 423. The resolution can be selected from 1/4, 1, or 4 pixel precision, and if the resolution is not changed, it defaults to 1/4 pixel precision. The rounding process is performed in accordance with the resolution of the motion vector in the encoding block being processed.
In other words, the derived candidate motion vector mvX is,
rightShift = leftShift = MvShift + 2
offset = 1 << ( rightShift - 1 )
mvX = ( mvX >= 0 ? ( mvX + offset ) >> rightShift :
- ( ( - mvX + offset ) >> rightShift ) ) << leftShift
And it is rounded. Here, the resolution of the motion vector in the encoding block being processed is
For 1/4 pixel precision, MvShift=0. Similarly, if the motion vector resolution is 1 pixel precision, MvShift=2, and if the motion vector resolution is 4 pixel precision, MvShift=4. According to the above formula, the x and y components of mvX are processed respectively.

Adaptive motion vector resolution can also be used for subblock predictive motion vector modes. In this case, only the resolution differs from the normal predictive motion vector mode described above. That is, in the affine inheritance predictive motion vector candidate derivation units 361 and 461, the affine construction predictive motion vector candidate derivation units 362 and 462, and the affine identical predictive motion vector candidate derivation units 363 and 463, the derived candidate motion vectors are rounded according to the resolution. The resolution can be selected from 1/16, 1/4, and 1 pixel precision, and if the resolution is not changed, it is 1/
This results in 16-pixel precision. Rounding is performed according to the resolution of the motion vector in the coded block being processed. In other words, the derived candidate motion vector mvX is rounded according to the above formula. Here, the resolution of the motion vector in the coded block being processed is 1/
For 4-pixel precision, MvShift=0. Similarly, if the motion vector resolution is 1-pixel precision, MvShift=2. According to the above formula, the x and y components of mvX are processed respectively.

<Triangular Merge Mode>
Triangular merge mode is a type of merge mode that divides the encoded/decoded block into diagonal partitions and performs motion compensation prediction.

The triangular merge mode will be explained using Figure 65. Figure 65 shows the prediction of an encoded/decoded block in triangular merge mode with a 16x16 grid. The encoded/decoded block in triangular merge mode is divided into 4x4 subblocks, with each subblock consisting of partition 0 (UNI0) for single prediction, partition 1 (UNI1) for single prediction, and partition 2 for biprediction.
It is assigned to three partitions (BI). Here, the subblock on the upper side of the diagonal is assigned to partition 0, the subblock on the lower side of the diagonal is assigned to partition 1, and the subblock on the diagonal is assigned to partition 2. merge_triangle_s
If plit_dir is 0, a partition will be allocated as shown in Figure 65(a), and merge_tr
If iangle_split_dir is 1, partitions will be allocated as shown in Figure 65(b).

For motion compensation prediction of partition 0, the single prediction motion information specified by merged triangle index 0 is used. For motion compensation prediction of partition 1, the single prediction motion information specified by merged triangle index 1 is used. For motion compensation prediction of partition 2, the single prediction motion information specified by merged triangle index 0 and merged triangle index 1 are used.
The system uses dual-prediction motion information, which is a combination of single-prediction motion information specified by the system.

Here, single-prediction motion information consists of one set of motion vector and reference index, while double-prediction motion information consists of two sets of motion vector and reference index. Furthermore, motion information refers to either single-prediction motion information or double-prediction motion information.

The merge candidate selection units 347 and 447 select the derived merge candidate list, mergeCandList.
Use this as the triangleMergeCandList for the list of candidate triangle merges.

The flowchart in Figure 66, which relates to the derivation of triangular merge candidates, will now be explained.

First, the merge candidate list is called the triangleMergeCandList.
Use a List (step S3501). Triangle Merge Candidate List Number of Candidates numTriangleMerge
Set Cand to the same value as the number of merge candidates, numCurrMergeCand.

Next, we derive the single predicted motion information for the merged triangular partition (step S3502).
.

Figure 67 is a flowchart illustrating the derivation of single predictive motion information for merged triangular partitions in this embodiment.

In this embodiment, single predictive motion information with the same priority order is derived for merged triangular partition 0 and merged triangular partition 1, thereby reducing the processing load.

First, for the Mth candidate in the derived merge candidate list mergeCandList, candidate M
It is determined whether candidate M has motion information in motion information list L0 (step S3601). If candidate M has motion information in motion information list L0, the motion information in motion information list L0 of candidate M is made a candidate for triangular merge (step S3602).

Next, for the Mth candidate in the derived merge candidate list mergeCandList, it is determined whether candidate M has motion information from the motion information list L1 (step S3603). If candidate M has motion information from the motion information list L1, the motion information from the motion information list L1 of candidate M is set as the triangular merge candidate (step S3604). Candidate M (M = numMergeCand-1, ...)
For , 1, 0), step S3601, step S3602, step S36
03. Step S3604 is performed to derive additional triangular merge candidates.

Figure 68 illustrates an example of motion information for a triangular merge candidate in this embodiment.

Figure 68(a) shows an example of a merge candidate list, where the merge candidate for merge index 0 has an inter-prediction mode of dual prediction (Pred-BI), and the motion information for motion information list L0 is MV0.
_L0, motion information list L1 is MV0_L1. The merge candidate for merge index 1 has an inter-prediction mode of single prediction (Pred-L0), motion information list L0 is MV1_L0, and motion information list L1 does not have any motion information. The merge candidate for merge index 2 has an inter-prediction mode of single prediction (Pred-L1), does not have motion information list L0, and motion information list L1 is MV2_L1. The merge candidate for merge index 3 has an inter-prediction mode of dual prediction (Pred-BI), motion information list L0 is MV3_L0, and motion information list L1 is MV3_L1. The merge candidate for merge index 4 has an inter-prediction mode of single prediction (Pred-L0), motion information list L0 is MV4_L0, and motion information list L1 does not have any motion information.

Figure 68(b) illustrates the relationship between the merge triangular index and the merge triangular partition in the example of the merge candidate list shown in Figure 68(a).

Candidate single predictive motion information for merged triangular partition 0 is MV0_L0, MV0_L1, MV1_L0, MV2_
The motion information is structured in the order of L1, MV3_L0.

Similarly, the single predictive motion information candidates for merged triangular partition 1 are MV0_L0, MV0_L1, and MV1_L0.
The motion information is constructed in the order of MV2_L1, MV3_L0, (MV3_L1), but to ensure that the motion information of merged triangle partition 0 and the motion information of merged triangle partition 1 are not identical, the single predicted motion information selected by merged triangle index 0 (merge_triangle_idx0) is excluded to derive merged triangle index 1 (merge_triangle_idx1).

In this way, by assigning the same priority order to the single predicted motion information candidates for merged triangular partition 0 and merged triangular partition 1 as to the merge list candidate list, an efficient triangular merge mode can be transmitted with a small amount of code. Furthermore, by transmitting merged triangular index 0 and merged triangular index 1 so that the motion information of merged triangular partition 0 and the motion information of merged triangular partition 1 are not identical, redundancy where the motion information of merged triangular partition 0 and the motion information of merged triangular partition 1 are identical, which does not require a triangular merge mode, is eliminated, and the triangular merge mode can be transmitted with a small amount of code.

In all the embodiments described above, the encoded bitstream output by the image encoding device has a specific data format so that it can be decoded according to the encoding method used in the embodiment. The encoded bitstream may be recorded and provided on a recording medium readable by a computer, such as an HDD, SSD, flash memory, or optical disc, or it may be provided from a server via a wired or wireless network. Therefore, the image decoding device corresponding to this image encoding device can decode the encoded bitstream of this specific data format regardless of the means of provision.

In order to exchange encoded bitstreams between the image encoding device and the image decoding device,
When a wired or wireless network is used, the encoded bitstream may be converted to a data format suitable for the transmission mode of the communication channel before transmission. In this case, a transmitting device is provided that converts the encoded bitstream output by the image encoding device into encoded data in a data format suitable for the transmission mode of the communication channel and transmits it to the network, and a receiving device is provided that receives the encoded data from the network, restores it to an encoded bitstream, and supplies it to the image decoding device. The transmitting device includes a memory that buffers the encoded bitstream output by the image encoding device,
The device includes a packet processing unit that packets the encoded bitstream and a transmission unit that transmits the packetized encoded data over a network. The receiving device includes a receiving unit that receives the packetized encoded data over a network, a memory that buffers the received encoded data, and a packet processing unit that packets the encoded data to generate an encoded bitstream and provides it to the image decoding device.

In order to exchange encoded bitstreams between the image encoding device and the image decoding device,
When a wired or wireless network is used, in addition to the transmitting device and receiving device,
A relay device may be provided that receives encoded data transmitted by a transmitting device and supplies it to a receiving device. The relay device includes a receiving unit that receives packetized encoded data transmitted by a transmitting device, a memory that buffers the received encoded data, and a transmitting unit that transmits the packetized encoded data to the network. Furthermore, the relay device may include a receiving packet processing unit that processes the packetized encoded data to generate an encoded bitstream, a recording medium that stores the encoded bitstream, and a transmitting packet processing unit that packets the encoded bitstream.

Furthermore, by adding a display unit to the configuration that displays the image decoded by the image decoding device, it can also be used as a display device. In that case, the display unit reads the decoded image signal generated by the decoded image signal superimposition unit 207 and stored in the decoded image memory 208 and displays it on the screen.

Alternatively, by adding an imaging unit to the configuration and inputting the captured image into an image encoding device, it can also function as an imaging device. In that case, the imaging unit inputs the captured image signal into the block division unit 101.

The above encoding and decoding processes may be implemented not only as hardware-based transmission, storage, and receiving devices, but also by firmware stored in ROM (read-only memory) or flash memory, or by software on a computer. The firmware program or software program may be recorded on a recording medium readable by a computer and provided, provided from a server via a wired or wireless network, or provided as data broadcasting on terrestrial or satellite digital broadcasting.

The present invention has been described above based on embodiments. The embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their components and processing processes, and that such modifications also fall within the scope of the present invention.

100 Image encoding device, 101 Block division unit, 102 Interpretation unit,
103 Intra prediction unit, 104 Decoded image memory, 105 Prediction method determination unit, 10
6 Residual signal generation unit, 107 Orthogonal transformation/quantization unit, 108 Bit sequence encoding unit,
109 Inverse quantization/inverse orthogonal transformation unit, 110 Decoded image signal superposition unit, 111 Encoded information storage memory, 200 Image decoding device, 201 Bit sequence decoding unit, 202 Block division unit, 203 Interpretation unit, 204 Intraprediction unit, 205 Encoded information storage memory, 206 Inverse quantization/inverse orthogonal transformation unit, 207 Decoded image signal superposition unit, 2
08 Decoded image memory.

Claims (8)

A video encoding device using merge mode,
A spatial merge candidate derivation unit derives spatial merge candidates from the movement information of blocks spatially adjacent to the block to be encoded and adds them to a merge candidate list,
A time merge candidate derivation unit derives time merge candidates from the motion information of blocks that are temporally adjacent to the block to be encoded and adds them to the merge candidate list.
A history merge candidate derivation unit derives history merge candidates from memory that holds the movement information of encoded blocks and adds them to the merge candidate list,
A normal merge candidate selection unit that selects a normal merge candidate that will be a single or double predictor from the merge candidate list,
A triangular merge candidate selection unit selects a first triangular merge candidate that is a simple prediction from the merge candidate list and a second triangular merge candidate that is a simple prediction.
Equipped with,
The triangular merge candidate selection unit derives the first triangular merge candidate from the merge candidate list which includes merge candidates used for dual prediction, derives the second triangular merge candidate from the merge candidate list which includes merge candidates used for dual prediction , and derives the second triangular merge candidate by excluding the first triangular merge candidate so that the first triangular merge candidate and the second triangular merge candidate are not the same.
A video encoding device characterized by the following features. A video encoding method using merge mode,
A spatial merge candidate derivation step involves deriving spatial merge candidates from the movement information of blocks spatially adjacent to the block to be encoded and adding them to a merge candidate list, and
A time merge candidate derivation step involves deriving time merge candidates from the motion information of blocks that are temporally adjacent to the block to be encoded and adding them to the merge candidate list.
A history merge candidate derivation step involves deriving history merge candidates from memory that holds the movement information of encoded blocks and adding them to the merge candidate list,
A normal merge candidate selection step of selecting a normal merge candidate that will be a single or biprediction from the merge candidate list,
A triangular merge candidate selection step involves selecting a first triangular merge candidate that is a single prediction and a second triangular merge candidate that is a single prediction from the merge candidate list.
Equipped with,
The triangular merge candidate selection step involves deriving the first triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction, deriving the second triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction , and deriving the second triangular merge candidate by excluding the first triangular merge candidate so that the first triangular merge candidate and the second triangular merge candidate are not the same.
A video encoding method characterized by the present invention. A video encoding program using merge mode,
A spatial merge candidate derivation step involves deriving spatial merge candidates from the movement information of blocks spatially adjacent to the block to be encoded and adding them to a merge candidate list, and
A time merge candidate derivation step involves deriving time merge candidates from the motion information of blocks that are temporally adjacent to the block to be encoded and adding them to the merge candidate list.
A history merge candidate derivation step involves deriving history merge candidates from memory that holds the movement information of encoded blocks and adding them to the merge candidate list,
A normal merge candidate selection step of selecting a normal merge candidate that will be a single or biprediction from the merge candidate list,
A triangular merge candidate selection step involves selecting a first triangular merge candidate that is a single prediction and a second triangular merge candidate that is a single prediction from the merge candidate list.
Equipped with,
The triangular merge candidate selection step involves deriving the first triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction, deriving the second triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction , and deriving the second triangular merge candidate by excluding the first triangular merge candidate so that the first triangular merge candidate and the second triangular merge candidate are not the same.
A video encoding program characterized by the following features. A video decoding device using merge mode,
A spatial merge candidate derivation unit derives spatial merge candidates from the movement information of blocks spatially adjacent to the block to be encoded and adds them to a merge candidate list,
A time merge candidate derivation unit derives time merge candidates from the motion information of blocks that are temporally adjacent to the block to be encoded and adds them to the merge candidate list.
A history merge candidate derivation unit derives history merge candidates from memory that holds the movement information of encoded blocks and adds them to the merge candidate list,
A normal merge candidate selection unit that selects a normal merge candidate that will be a single or double predictor from the merge candidate list,
A triangular merge candidate selection unit selects a first triangular merge candidate that is a simple prediction from the merge candidate list and a second triangular merge candidate that is a simple prediction.
Equipped with,
The triangular merge candidate selection unit derives the first triangular merge candidate from the merge candidate list which includes merge candidates used for dual prediction, derives the second triangular merge candidate from the merge candidate list which includes merge candidates used for dual prediction , and derives the second triangular merge candidate by excluding the first triangular merge candidate so that the first triangular merge candidate and the second triangular merge candidate are not the same.
A video decoding device characterized by the following features. A video decoding method using merge mode,
A spatial merge candidate derivation step involves deriving spatial merge candidates from the movement information of blocks spatially adjacent to the block to be encoded and adding them to a merge candidate list, and
A time merge candidate derivation step involves deriving time merge candidates from the motion information of blocks that are temporally adjacent to the block to be encoded and adding them to the merge candidate list.
A history merge candidate derivation step involves deriving history merge candidates from memory that holds the movement information of encoded blocks and adding them to the merge candidate list,
A normal merge candidate selection step of selecting a normal merge candidate that will be a single or biprediction from the merge candidate list,
A triangular merge candidate selection step involves selecting a first triangular merge candidate that is a single prediction and a second triangular merge candidate that is a single prediction from the merge candidate list.
Equipped with,
The triangular merge candidate selection step involves deriving the first triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction, deriving the second triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction , and deriving the second triangular merge candidate by excluding the first triangular merge candidate so that the first triangular merge candidate and the second triangular merge candidate are not the same.
A method for decoding moving images, characterized by the features described above. A video decoding program using merge mode,
A spatial merge candidate derivation step involves deriving spatial merge candidates from the movement information of blocks spatially adjacent to the block to be encoded and adding them to a merge candidate list, and
A time merge candidate derivation step involves deriving time merge candidates from the motion information of blocks that are temporally adjacent to the block to be encoded and adding them to the merge candidate list.
A history merge candidate derivation step involves deriving history merge candidates from memory that holds the movement information of encoded blocks and adding them to the merge candidate list,
A normal merge candidate selection step of selecting a normal merge candidate that will be a single or biprediction from the merge candidate list,
A triangular merge candidate selection step involves selecting a first triangular merge candidate that is a single prediction and a second triangular merge candidate that is a single prediction from the merge candidate list.
Equipped with,
The triangular merge candidate selection step involves deriving the first triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction, deriving the second triangular merge candidate from the merge candidate list which includes merge candidates to be used for biprediction , and deriving the second triangular merge candidate by excluding the first triangular merge candidate so that the first triangular merge candidate and the second triangular merge candidate are not the same.
A video decoding program characterized by the following features. A storage method for storing a bitstream generated by the video encoding method described in claim 2 into a recording medium. A transmission method for transmitting a bitstream generated by the video encoding method described in claim 2.