JP6841709B2 - Image coding device, image decoding device, image coding program and image decoding program - Google Patents

Description

The present invention relates to an image coding device, an image decoding device, an image coding program, and an image decoding program.

As a standard for moving image coding, there are H.264 / AVC (Advanced Video Coding) and H.265 / HEVC (High Efficiency Video Coding) (hereinafter referred to as "HEVC"). In HEVC, the image coding device divides an image to be coded into a plurality of LCUs (Largest Coding Units) of 64 pixels × 64 pixels, which is the maximum coding unit. The image coding device encodes the image to be coded for each LCU. The image coding apparatus may divide the LCU into quadtrees into CUs (Coding Units), which are coding units. The size of the CU is n pixels x n pixels (n is any of 8, 16, 32 or 64). The image coding device encodes the image to be coded for each CU.

The image coding device encodes the image to be encoded in the intra prediction mode or the inter prediction mode. In the intra prediction mode, the image coding apparatus predicts the brightness value of the pixel of the CU based on the brightness value of the encoded pixel adjacent to the LCU or the CU in the image to be coded. In the inter-prediction mode, the image encoding device is based on the brightness values of the pixels of the reference block in the encoded reference image constituting the moving image, and the brightness values of the pixels of the CU in the encoded target image constituting the moving image. Predict.

In the inter-prediction mode, the image coding device generates a prediction image based on the encoded reference image and the motion information. The motion information includes information indicating a selection pattern of a reference image referred to for generating a predicted image, an index of the reference image, and a motion vector indicating the position of a reference block in the reference image as parameters.

The selection pattern of the reference image includes L0 prediction, L1 prediction, and bidirectional prediction. In L0 prediction, the image encoding device generates a prediction image based on one reference image selected from the L0 reference image list. In L1 prediction, the image encoding device generates a prediction image based on one reference image selected from the L1 reference image list. In bidirectional prediction, the image encoding device generates a prediction image based on one reference image selected from the L0 reference image list and one reference image selected from the L1 reference image list.

The motion vector indicating the position of the reference block in the reference image is expressed by using a two-dimensional vector of a horizontal vector and a vertical vector. For L0 prediction or L1 prediction, one motion vector indicating one reference image is required. Bidirectional prediction requires two motion vectors that represent two reference images.

FIG. 8 is a diagram showing an example of generating a conventional predicted image. In the L1 prediction, the image coding device generates the prediction image based on only one reference image, so that the noise reduction effect cannot be obtained. Therefore, the image encoding device is _{based on the bidirectional prediction using the reference image 400-0 (= P 0} ) and the reference image 400-1 (= P ₁ ), and the prediction image 500 is as shown in the equation (1). (= P) may be generated.

In equation (1), x represents the horizontal index of the pixel in the reference image 400. y represents the vertical index of the pixels in the reference image 400.

In FIG. 8, the image encoding device is based on the average value of the brightness value of the pixel of the reference block 401-0 of the reference image 400-0 and the brightness value of the pixel of the reference block 401-1 of the reference image 400-1. , Generates a predicted image 500, which is a predicted image of the coded block 601 of the coded image 600. The image coding apparatus encodes the coded block 601 using the generated predicted image 500.

As described above, the image coding apparatus reduces the noise in the predicted image 500 by performing the averaging process on each pixel of each reference block 401 of the two reference images 400. Since the noise of the prediction image 500 is significantly reduced, the prediction efficiency of the coded block 601 is improved.

Against this background, a method for further improving the prediction efficiency has been proposed (see Non-Patent Document 1). In the method of Non-Patent Document 1, the image coding apparatus improves the prediction efficiency of the coded block 601 by multiplying the reference image by the weighting coefficient w as in the equation (2).

In the method of Non-Patent Document 1, a set of weighting factors w is predetermined. The image coding device selects the weighting factor w from the set of weighting factors w. The image coding device encodes the index of the selected weighting factor w. Equation (2) is the same as the prediction equation defined in HEVC when the weighting coefficient w is 0.5. Therefore, in the method of Non-Patent Document 1, the image coding apparatus can add a plurality of new prediction images to the prediction image candidates while including the HEVC standard, so that the prediction of the coded block 601 can be predicted. Efficiency can be improved.

Chun-Chi Chen, Xiaoyu Xiu, Yuwen He, and Yan Ye (InterDigital Communications, Inc.), “Generalized Bi-prediction Method for Future VideoCoding”, PCS2016.

In the method of Non-Patent Document 1, only one weighting coefficient is defined for one reference image. Therefore, when the luminance values of the plurality of pixels in the entire region of the reference block 401 change uniformly between the reference image 400-0 and the reference image 400-1, the image coding apparatus determines the image to be encoded. It is possible to improve the prediction efficiency of the coded target block 601 which is the coded unit (CU) of 600.

However, when there are deviation values in the brightness values of a plurality of pixels in the partial region of the reference block 401 while the reference image 400-0 and the reference image 400-1 are present, or when the reference image 400-0 and the reference image 400-1 have deviation values. When the brightness values of the plurality of pixels in the partial region of the reference block 401 change beyond the threshold value, the image coding apparatus may improve the prediction efficiency of the coding target block 601 of the coding target image 600. Can not.

In FIG. 8, there are outliers in the luminance values of the plurality of pixels in the region 402, which is a partial region of the reference block 401-0. In the method of Non-Patent Document 1, the image encoding device multiplies the brightness value of the pixel in the entire region of the reference block 401-0 by one weighting coefficient, and the brightness value of the pixel in the entire region of the reference block 401-1. Is multiplied by one weighting factor. Therefore, in the method of Non-Patent Document 1, the image coding apparatus cannot improve the prediction efficiency of the coded block 601 regardless of the weighting coefficient used.

Further, when the image coding apparatus generates the predicted image 500 only from the reference block 401-1 of the reference image 400-1 by L1 prediction, the noise in the predicted image 500 cannot be reduced, so that the prediction efficiency can be improved. It cannot be improved. Further, in the method of Non-Patent Document 1, since the image coding device encodes the index of the weighting coefficient, when the number of sets of the weighting coefficient is large, the size of the coding parameter including the index of the weighting coefficient is large. Therefore, the coding efficiency of the image 600 to be coded is lowered.

As described above, in the conventional image encoding device, when one or both of the plurality of reference images constituting the moving image have deviation values in the luminance values of a plurality of pixels in the partial region of the reference block, or when the moving image has an deviation value. When the luminance values of a plurality of pixels in the partial region of the reference block change above the threshold value among the plurality of reference images constituting the above, the prediction efficiency of the coded target block of the coded target image cannot be improved. There was a problem.

In view of the above circumstances, the present invention has a case where there is an deviation value in the brightness value of a plurality of pixels in a partial region of the reference block in one or both of the plurality of reference images constituting the moving image, or a plurality of reference images constituting the moving image. Image coding that can improve the prediction efficiency of the coded target block of the coded target image even when the brightness values of a plurality of pixels in the partial region of the reference block change between the reference images of It is an object of the present invention to provide an apparatus, an image decoding apparatus, an image coding program, and an image decoding program.

One aspect of the present invention is whether or not the partial regions of the reference blocks of the plurality of reference images are regions of corresponding positions in the reference image, and the brightness values of the plurality of pixels in the partial regions are outliers. The weight calculation unit that determines the weighting coefficient for the pixel of the brightness value determined to be an outlier value smaller than the weighting coefficient for the pixel of the brightness value determined not to be an outlier, and the pixel of the reference block. A predictive image generation unit that generates a predicted image of the coded target block based on the result of multiplying the brightness value of the above by the weighting coefficient, and a coding unit that encodes the coded target block based on the predicted image. It is an image coding device provided.

One aspect of the present invention is the above-mentioned image coding apparatus, in which the weight calculation unit sets the deviation value of the brightness value in the pixels of the reference block of two or more of the reference images to two or more of the references. The detection is performed based on the difference in the brightness values of the pixels between the reference blocks of the image.

One aspect of the present invention is the above-mentioned image coding apparatus, in which the weight calculation unit sets the deviation value of the brightness value in the pixels of the reference block of three or more of the reference images to three or more of the references. The detection is performed based on the bias of the brightness value of the pixels between the reference blocks of the image.

One aspect of the present invention is whether or not the partial regions of the reference blocks of the plurality of reference images are regions of corresponding positions in the reference image, and the brightness values of the plurality of pixels in the partial regions are outliers. The weight calculation unit that determines the weighting coefficient for the pixel of the brightness value determined to be an outlier value smaller than the weighting coefficient for the pixel of the brightness value determined not to be an outlier, and the pixel of the reference block. An image decoding device including a predictive image generation unit that generates a predicted image of a block to be decoded based on the result of multiplying the brightness value of the above by the weighting coefficient, and a decoding unit that decodes the block to be decoded based on the predicted image. Is.

One aspect of the present invention is the image decoding device, in which the weight calculation unit sets the deviation value of the brightness value in the pixels of the reference block of the two or more reference images as the deviation value of the brightness value of the two or more reference images. The detection is performed based on the difference in the brightness values of the pixels between the reference blocks of the above.

One aspect of the present invention is the above-mentioned image decoding device, in which the weight calculation unit sets the deviation value of the brightness value in the pixels of the reference block of the three or more reference images to three or more of the reference images. The detection is performed based on the bias of the brightness value of the pixels between the reference blocks of the above.

In one aspect of the present invention, in a computer, subregions of reference blocks of a plurality of reference images are regions of corresponding positions in the reference image, and brightness values of a plurality of pixels in the subregions are outliers. A procedure for determining whether or not the image has a weighting coefficient for a pixel having a brightness value determined to be an outlier value smaller than the weighting coefficient for a pixel having a brightness value determined not to be an outlier, and the reference block. To execute a procedure for generating a predicted image of a block to be encoded based on the result of multiplying the brightness value of a pixel by the weighting coefficient, and a procedure for encoding the block to be encoded based on the predicted image. It is an image coding program.

In one aspect of the present invention, in a computer, subregions of reference blocks of a plurality of reference images are regions of corresponding positions in the reference image, and brightness values of a plurality of pixels in the subregions are outliers. A procedure for determining whether or not the image has a weighting coefficient for a pixel having a brightness value determined to be an outlier value smaller than the weighting coefficient for a pixel having a brightness value determined not to be an outlier, and the reference block. An image decoding program for executing a procedure of generating a predicted image of a block to be decoded based on the result of multiplying a pixel brightness value by the weighting coefficient and a procedure of decoding the block to be decoded based on the predicted image. Is.

According to the present invention, when one or both of the plurality of reference images constituting the moving image have deviation values in the brightness values of the plurality of pixels in the partial region of the reference block, or between the plurality of reference images constituting the moving image. It is possible to improve the prediction efficiency of the coded target block of the coded target image even when the luminance values of the plurality of pixels in the partial region of the reference block change to the threshold value or more.

It is a figure which shows the example of the structure of the image coding apparatus in 1st Embodiment. It is a figure which shows the example of the structure of the inter-prediction part of the image coding apparatus in 1st Embodiment. It is a figure which shows the example of the generation of the prediction image in 1st Embodiment. It is a figure which shows the example of the structure of the image decoding apparatus in 1st Embodiment. It is a figure which shows the example of the structure of the inter-prediction part of the image decoding apparatus in 1st Embodiment. It is a flowchart which shows the example of the operation of the inter-prediction part of the image coding apparatus in 1st Embodiment. It is a flowchart which shows the example of the operation of the inter-prediction part of the image decoding apparatus in 1st Embodiment. It is a figure which shows the example of the generation of the conventional prediction image.

Embodiments of the present invention will be described in detail with reference to the drawings.
In the following, the standard to which the image coding device and the image decoding device conform is not limited to a specific standard for video coding, but HEVC is an example. Hereinafter, among the brightness values of a plurality of pixels in the partial region of the reference block of the reference image constituting the moving image, a value that deviates from the reference value related to the brightness value by a threshold value or more is referred to as an “outlier value”. In the following, the outliers do not have to be values that deviate significantly to the extent that the scenes differ, such as scene changes in moving images.

(First Embodiment)
FIG. 1 is a diagram showing an example of the configuration of the image coding device 1. The image coding device 1 is a device that encodes a plurality of images constituting a moving image. The image coding device 1 acquires a coded target image for each coded target block which is a coding unit. The image coding device 1 encodes the coded block by repeatedly executing the raster scan in the coded block.

The image coding device 1 outputs coded data including the result of subjecting the quantization coefficient of the luminance value of the pixel of the coded target block to the variable length coding process to the image decoding device or the like. The image coding device 1 may output coded data including coding parameters to an image decoding device or the like. The coding parameter is, for example, a parameter representing a weighting coefficient, a parameter representing motion information such as a motion vector, and the like.

The image coding device 1 is, for example, an information processing device such as a personal computer device, a smartphone terminal, a tablet terminal, or a server device. The image coding device 1 includes a subtractor 10, an orthogonal conversion / quantization unit 11, a variable length coding unit 12, an inverse quantization / inverse orthogonal conversion unit 13, an adder 14, and a loop filter unit 15. , The decoding picture memory 16, the intra prediction unit 17, the inter prediction unit 18, and the intra / inter changeover switch 19. The image coding device 1 may further include, for example, a non-volatile recording medium (non-temporary recording medium) such as a magnetic hard disk device or a semiconductor storage device as a storage unit.

Subtractor 10, orthogonal conversion / quantization unit 11, variable length coding unit 12, inverse quantization / inverse orthogonal conversion unit 13, adder 14, loop filter unit 15, intra prediction unit 17, inter prediction unit 18, and intra. A part or all of the interchanger switch 19 may be realized by, for example, a processor such as a CPU (Central Processing Unit) executing a program stored in the storage unit, or an LSI (Large Scale Integration). It may be realized by using hardware such as ASIC or ASIC (Application Specific Integrated Circuit).

The subtractor 10 acquires a coded target image for each coded target block which is a coding unit. The subtractor 10 acquires a predicted image of the coded block from the intra prediction unit 17 or the inter prediction unit 18. The subtractor 10 outputs the difference between the coded target block of the coded target image and the predicted image to the orthogonal conversion / quantization unit 11.

The orthogonal conversion / quantization unit 11 performs orthogonal conversion processing and quantization processing on the difference between the coded block and the predicted image. The orthogonal conversion / quantization unit 11 outputs the quantization coefficient resulting from the orthogonal conversion processing and the quantization processing to the variable length coding unit 12 and the inverse quantization / inverse orthogonal conversion unit 13.

The variable-length coding unit 12 is a coding unit that executes the variable-length coding process. The variable-length coding unit 12 outputs coded data including the result of subjecting the quantization coefficient to the variable-length coding process to an image decoding device or the like. The variable-length coding unit 12 may output coded data including coding parameters to an image decoding device or the like. The inverse quantization / inverse orthogonal conversion unit 13 outputs an image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process to the adder 14.

The adder 14 acquires an image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process from the inverse quantization / inverse orthogonal conversion unit 13. The adder 14 acquires a predicted image from the intra prediction unit 17 or the inter prediction unit 18 via the intra / inter changeover switch 19. The adder 14 outputs the result of adding the image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process to the predicted image to the loop filter unit 15 and the intra prediction unit 17.

The loop filter unit 15 applies the loop filter to the result of the adder 14 adding the image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process and the predicted image. The loop filter unit 15 outputs the result of applying the loop filter to the decoding picture memory 16.

The decoded picture memory 16 is, for example, a volatile recording medium such as a RAM (Random Access Memory). The decoded picture memory 16 may be, for example, a non-volatile recording medium (non-temporary recording medium) such as a semiconductor storage device. The decoding picture memory 16 stores the result of applying the loop filter to the result of addition by the adder 14. The decoding picture memory 16 outputs the result of applying the loop filter to the result of addition by the adder 14 to the inter-prediction unit 18.

The intra prediction unit 17 acquires from the adder 14 the result of applying the loop filter to the result added by the adder 14. The intra prediction unit 17 uses the result of applying the loop filter to the result of addition by the adder 14 as a reference image. The intra prediction unit 17 generates a prediction image of the coded block by intra prediction based on the reference image acquired from the adder 14.

The inter-prediction unit 18 acquires a coding target block, which is a coding unit of the coding target image. The inter-prediction unit 18 acquires the result of applying the loop filter to the result of addition by the adder 14 from the decoding picture memory 16. The inter-prediction unit 18 uses the result of applying the loop filter to the result of addition by the adder 14 as a reference image. The inter-prediction unit 18 generates a predicted image of the coded block by inter-prediction based on the reference image acquired from the decoded picture memory 16.

When the prediction mode of the coding target block, which is the coding unit of the coding target image, is intra prediction, the intra-inter selector switch 19 uses the subtractor 10 and the adder 14 to transfer the predicted image generated by the intra prediction unit 17. Output to. When the prediction mode of the coded block, which is the coding unit of the coded image, is inter-prediction, the intra-inter selector switch 19 uses the subtractor 10 and the adder 14 to transfer the predicted image generated by the inter-predicted unit 18. Output to.

Next, an example of the configuration of the inter-prediction unit 18 of the image coding device 1 will be described.
FIG. 2 is a diagram showing an example of the configuration of the inter-prediction unit 18 of the image coding device 1. The inter-prediction unit 18 generates a predicted image of the coded block by inter-prediction based on the reference image. The inter-prediction unit 18 includes a motion search unit 180, a motion compensation unit 181, a weight calculation unit 182, and a prediction image generation unit 183.

FIG. 3 is a diagram showing an example of generation of the predicted image 200. In FIG. 3, the reference block 101 includes a region 102 and a region 103. In the reference block 101-0, the area 102 is an area that does not include pixels whose luminance values are outliers. In the reference block 101-0, the area 103 is an area including pixels whose luminance values are outliers. The pixel whose luminance value is an outlier is, for example, a pixel of an image of a soccer ball temporarily captured in the reference image 100. In the reference block 101-1, the area 102 is an area that does not include pixels whose luminance values are outliers. In the reference block 101-1, the area 103 is an area that does not include pixels whose luminance values are outliers.

The motion search unit 180 shown in FIG. 2 acquires the coding target block 301, which is the coding unit of the coding target image 300 shown in FIG. The motion search unit 180 calculates the motion information of the coded block 301 by the motion search in each reference image 100. The motion search unit 180 outputs the motion information of the coded block 301 to the motion compensation unit 181.

The motion compensation unit 181 acquires the reference images 100-0 and 100-1 from the decoded picture memory 16. The motion compensation unit 181 generates the reference block 101-0 in the reference image 100-0 based on the motion information calculated by the motion search unit 180. The motion compensation unit 181 generates the reference block 101-1 in the reference image 100-1 based on the motion information calculated by the motion search unit 180. That is, the motion compensation unit 181, the motion compensation using the motion information, the two reference images 100, each motion vector to generate a respective reference block P _X pointing respectively. X represents the index of the reference image 100. X is an integer from 0 to (N-1). N represents the number of reference images.

The weight calculation unit 182 acquires the reference images 100-0 and 100-1 from the decoded picture memory 16. The weight calculation unit 182 extracts the index X of the reference image 100 from the motion information. In FIG. 3, the weight calculation unit 182 extracts the index of the reference image 100-0 and the index of the reference image 100-1 from the motion information.

The weight calculation unit 182 calculates the difference δ between the brightness value of the pixel of the reference block 101-0 and the brightness value of the pixel of the reference block 101-1 for each pixel. The weight calculation unit 182 may calculate the difference δ between the brightness value of the pixel of the reference block 101-0 and the brightness value of the pixel of the reference block 101-1 for each partial region of the reference block 101. A pixel whose difference δ is equal to or greater than a threshold value is a pixel whose luminance value is an outlier. When the brightness value of the pixel of the reference block 101-0 is larger than the brightness value of the pixel at the corresponding position of the reference block 101-1, the brightness value of the pixel of the reference block 101-0 is in two or more reference blocks 101. It is the maximum value, and the luminance value of the pixels of the reference block 101-1 is the minimum value in two or more reference blocks 101.

The weight calculation unit 182 can detect outliers of the luminance values in the pixels of the reference block 101 of the two or more reference images 100 for each pixel based on the difference δ. Therefore, the weight calculation unit 182 can encode the image to be encoded even if there are outliers in the brightness values of the plurality of pixels in the partial region of the reference block 101 in one or both of the reference image 100-0 and the reference image 100-1. The prediction efficiency of the coded block 301 of 300 can be improved.

Weight calculator 182, a weight coefficient _{W X} for each area in the reference image 100 defines the pixel. The area in the reference image 100 is a pixel, an entire area or a partial area of the reference block 101. For example, the weight calculation unit 182 determines the _{weight coefficient W X} for each pixel in the reference block 101-X of the reference image 100-X to which the index X is assigned.

The weight calculation unit 182 executes an algorithm that obtains a weighting coefficient such that the outlier becomes small based only on the reference image. Since the image coding device 1 obtains a weighting coefficient such that the outlier becomes small based only on the reference image, it is not necessary to transmit the weighting coefficient to the image decoding device 2. When the difference δ is less than the threshold value th, the weight calculation unit 182 sets the weight coefficient to a value equal to the reciprocal of the number N of the reference images. When the difference δ is equal to or greater than the threshold value th, the weight calculation unit 182 reduces _{the weight coefficient X of one pixel of the reference block 101 in the two reference images 100.} For example, when the difference δ is larger than the maximum noise of the luminance value, the weight calculation unit 182 sets the weighting coefficient to a predetermined maximum value or minimum value. For example, the weight calculation section 182, when the difference δ is the threshold value th or more, determines the weighting factors _{W X} of the pixels of the reference block 101-0 to the maximum value "1", the weights of the pixels of the reference block 101-1 The coefficient W _X may be set to the minimum value “0”.

For example, the weight calculation unit 182, based on pre-selected expression from among the formula (3) or formula (4), determines the weighting factors W _X.

In equations (3) and (4), th represents a predetermined threshold value for the difference δ. x represents the horizontal index of the pixel in the reference image 100. y represents the vertical index of the pixels in the reference image 100. X_min represents the index of the reference image 100 having pixels with a brightness value equal to the minimum value of the brightness value. X_max represents the index of the reference image 100 having pixels with a luminance value equal to the maximum luminance value.

Weight calculator 182, see formula (3) which is selected for each reference block 101 in accordance with the index X of the image 100 or based on the equation (4) may determine a weighting factor W _X. The weight calculation unit 182 selects the equation (3) or the equation (4) for each reference block 101 according to the coding parameters other than the index X of the reference image 100, and the weighting coefficient W _{X is based on the selected equation.} May be determined.

The prediction image generation unit 183 generates a prediction image 200 (= P (x, y)) based on the multiplication result of each pixel in the reference block 101 and the weighting coefficient W _{X for each pixel in the reference block 101.} For example, the prediction image generation unit 183 generates a prediction image P (x, y) as in the equation (5) based on N reference images 100. In the first embodiment, N = 2 or more.

The prediction image generation unit 183 is based on the prediction formula extended as in the formula (6), and the brightness value of the pixels in the vicinity (x + k, y + l) of the corresponding positions (x, y) in the N reference blocks 101. The predicted image P (x, y) may be generated based on the above.

In equation (6), K represents the maximum value of the offset k of the horizontal index x. -K represents the minimum value of the offset k of the horizontal index x. L represents the maximum value of the offset l of the index y in the vertical direction. -L represents the minimum value of the offset l of the index y in the vertical direction.

_{In FIG. 3, the weighting coefficient W X} of each pixel in the region 102-0 of the reference image 100-0 is 0.5. _{The weighting coefficient W X} of each pixel in the region 102-1 of the reference image 100-1 is 0.5. _{The weighting coefficient W X} of each pixel in the region 103-0 of the reference image 100-0 is 0. Weight coefficient _{W X} for each pixel in the area 103-1 of the reference image 100-1 is 1.

Accordingly, the prediction image generation unit 183, even in the presence of outliers in the luminance value of the pixel area 103-0, the weight coefficient _{W X} of pixels in the region 103-0 is 0, the encoding target image 300 It is possible to improve the prediction efficiency of the coded block 301. Further, in the prediction image generation unit 183, even when the brightness value of the area 103-0 changes to the threshold value or more between the reference images 100-0 and 100-1, the weight coefficient W _X of the pixels in the area 103-0 is 0. Therefore, it is possible to improve the prediction efficiency of the coding target block 301 of the coding target image 300.

FIG. 4 is a diagram showing an example of the configuration of the image decoding device 2. The image decoding device 2 is a device that encodes a plurality of images constituting a moving image. The image decoding device 2 acquires coded data for each coded block of the coded image. The image decoding device 2 treats the coded target image in the image coding device 1 as a decoding target image. The image decoding device 2 treats the coding target block in the image coding device 1 as a decoding target block. The image decoding device 2 repeatedly performs decoding processing on a plurality of coded data. The image decoding device 2 outputs a plurality of images constituting a moving image by performing a decoding process on a block to be decoded in the coded data.

The image decoding device 2 includes a variable length decoding unit 20, an inverse quantization / inverse orthogonal conversion unit 21, an adder 22, a loop filter unit 23, a decoding picture memory 24, an intra prediction unit 25, and an inter prediction unit. 26 and an intra-interchangeover switch 27 are provided. The image decoding device 2 may further have a non-volatile recording medium (non-temporary recording medium) such as a magnetic hard disk device or a semiconductor storage device.

A part or all of the variable length decoding unit 20, the inverse quantization / inverse orthogonal conversion unit 21, the adder 22, the loop filter unit 23, the intra prediction unit 25, the inter prediction unit 26, and the intra / inter changeover switch 27 is, for example. , A processor such as a CPU may be realized by executing a program stored in a storage unit, or may be realized by using hardware such as an LSI or an ASIC.

The variable length decoding unit 20 is a decoding unit that executes a variable length decoding process. The variable-length decoding unit 20 extracts the quantization coefficient from the coded data by performing a variable-length decoding process on the coded data. The variable length decoding unit 20 outputs the quantization coefficient to the inverse quantization / inverse orthogonal conversion unit 21. The variable-length decoding unit 20 extracts the coding parameter from the coded data by performing the variable-length decoding process on the coded data. The variable length decoding unit 20 may acquire coding parameters from an external device other than the image coding device 1. The variable length decoding unit 20 outputs the coding parameter to the inter prediction unit 26. The inverse quantization / inverse orthogonal conversion unit 21 is a decoding unit that executes the inverse quantization process and the inverse orthogonal conversion process. The inverse quantization / inverse orthogonal conversion unit 21 outputs an image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process to the adder 22.

The adder 22 is a decoding unit that executes an addition process. The adder 22 acquires an image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process from the inverse quantization / inverse orthogonal conversion unit 21. The adder 22 acquires a prediction image from the intra prediction unit 25 or the inter prediction unit 26 via the intra / inter changeover switch 27. The adder 22 outputs the result of adding the image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process to the predicted image to the loop filter unit 23 and the intra prediction unit 25.

The loop filter unit 23 is a decoding unit that executes a loop filter process. The loop filter unit 23 applies the loop filter to the result of the adder 22 adding the image obtained by subjecting the quantization coefficient to the inverse quantization process and the inverse orthogonal conversion process and the predicted image. The loop filter unit 23 outputs the result of applying the loop filter to the outside of the image decoding device 2 and the decoding picture memory 24.

The decoded picture memory 24 is, for example, a volatile recording medium such as RAM. The decoded picture memory 24 may be, for example, a non-volatile recording medium (non-temporary recording medium) such as a semiconductor storage device. The decoding picture memory 24 stores the result of applying the loop filter to the result of addition by the adder 22. The decoding picture memory 24 outputs the result of applying the loop filter to the result of addition by the adder 22 to the inter-prediction unit 26.

The intra prediction unit 25 acquires the result of addition by the adder 22 from the adder 22. The intra prediction unit 25 uses the result of addition by the adder 22 as a reference image. The intra prediction unit 25 generates a prediction image of the coded block by intra prediction based on the reference image acquired from the adder 22.

The inter-prediction unit 26 acquires coding parameters from the variable-length decoding unit 20. The inter-prediction unit 26 acquires the result of applying the loop filter to the result of addition by the adder 22 from the decoding picture memory 24. The inter-prediction unit 26 uses the result of applying the loop filter to the result of addition by the adder 22 as a reference image. The inter-prediction unit 26 generates a predicted image of the coded block by inter-prediction based on the reference image acquired from the decoded picture memory 24.

When the prediction mode of the decoding target block, which is the decoding unit of the decoding target image, is intra prediction, the intra-inter switching switch 27 outputs the predicted image generated by the intra prediction unit 17 to the subtractor 10 and the adder 14. .. The intra-inter selector switch 19 outputs the predicted image generated by the inter-prediction unit 26 to the adder 22 when the prediction mode of the decoding target block, which is the decoding unit of the decoding target image, is inter-prediction.

Next, an example of the configuration of the inter-prediction unit 26 of the image decoding device 2 will be described.
FIG. 5 is a diagram showing an example of the configuration of the inter-prediction unit 26 of the image decoding device 2. The inter-prediction unit 26 generates a predicted image of the decoding target block by inter-prediction based on the reference image. The inter-prediction unit 26 includes a motion compensation unit 260, a weight calculation unit 261 and a prediction image generation unit 262.

The motion compensation unit 260 acquires the reference images 100-0 and 100-1 from the decoded picture memory 24. The motion compensation unit 260 acquires motion information of the coding target block 301, which is a decoding target block, from the variable length decoding unit 20. The motion compensation unit 260 generates the reference block 101-0 in the reference image 100-0 based on the motion information calculated by the motion search unit 180. The motion compensation unit 260 generates the reference block 101-1 in the reference image 100-1 based on the motion information calculated by the motion search unit 180. That is, the motion compensation unit 260, the motion compensation using the motion information, the two reference images 100, each motion vector to generate a respective reference block P _X pointing respectively.

The weight calculation unit 261 acquires the reference images 100-0 and 100-1 from the decoded picture memory 24. The weight calculation unit 261 extracts the index X of the reference image 100 from the motion information. The weight calculation unit 261 calculates the difference δ between the brightness value of the pixel of the reference block 101-0 and the brightness value of the pixel of the reference block 101-1 for each pixel. The weight calculation unit 261 may calculate the difference δ between the brightness value of the pixel of the reference block 101-0 and the brightness value of the pixel of the reference block 101-1 for each partial region of the reference block 101.

The weight calculation unit 261 determines the _{weight coefficient W X} for each pixel in the reference block 101-X of the reference image 100-X to which the index X is assigned. For example, the weight calculation unit 261, based on pre-selected expression from among the above equation (3) or formula (4), determines the weighting factors W _X. Weight calculator 261, see the above equation (3) which is selected for each reference block 101 in accordance with the index X of the image 100 or based on the equation (4) may determine a weighting factor W _X. That is, the weight calculation unit 261 detects outliers of the luminance values in the pixels of the reference blocks of the two or more reference images based on the difference in the luminance values of the pixels in the reference blocks of the two or more reference images.

Note that the weight calculation unit 261 does not have to calculate the _{weight coefficient W X when the coding parameter including the weight coefficient W X} calculated by the image coding device 1 is acquired from the variable length decoding unit 20. ..

The prediction image generation unit 262 generates a prediction image 200 (= P (x, y)) based on the multiplication result of each pixel in the reference block 101 and the weighting coefficient W _{X for each pixel in the reference block 101.} For example, the prediction image generation unit 183 generates the prediction image P (x, y) based on the N reference images 100 as in the above equation (5).

Next, an example of the operation of the image coding device 1 will be described.
FIG. 6 is a flowchart showing an example of the operation of the inter-prediction unit 18 of the image coding device 1. The motion search unit 180 calculates motion information by motion search (step S101). The motion compensation unit 181 generates each reference block 101 in the two reference images 100 based on the motion information (step S102). The weight calculation unit 182 determines the weighting coefficient for each pixel of the reference block 101 for the two reference images 100 (step S103). The prediction image generation unit 183 generates the prediction image 200 by inter-prediction based on each reference block 101 in the two reference images 100 and the weighting coefficient for each pixel of the reference block 101 (step S104).

Next, an example of the operation of the image decoding device 2 will be described.
The operation of the inter-prediction unit 26 of the image decoding device 2 is the same as the operation of the inter-prediction unit 18 of the image coding device 1 except that the motion compensation process is not executed.

FIG. 7 is a flowchart showing an example of the operation of the inter-prediction unit 26 of the image decoding device 2. The motion compensation unit 260 generates each reference block 101 in the two reference images 100 based on the motion information acquired from the variable length decoding unit 20 (step S201). The weight calculation unit 261 determines the weighting coefficient for each pixel of the reference block 101 for the two reference images 100 (step S202). The prediction image generation unit 262 generates the prediction image 200 by inter-prediction based on each reference block 101 in the two reference images 100 and the weighting coefficient for each pixel of the reference block 101 (step S203).

As described above, the image coding apparatus 1 of the first embodiment includes a weight calculation unit 182, a prediction image generation unit 183, an orthogonal conversion / quantization unit 11, and a variable length coding unit 12. The subregions of the reference blocks 101 of the plurality of reference images 100 are regions at corresponding positions in the reference image 100. The weight calculation unit 182 determines whether or not the luminance values of the plurality of pixels in the partial regions are outliers. Weight calculator 182, a weight coefficient W _X for the pixel of the determined luminance value as an outlier, defined smaller than the weighting factor W _X for the pixel of the determined luminance values not to be outliers. Prediction image generating unit 183, based on the result of multiplying the weight coefficient W _X on the luminance values of the pixels of the reference block 101 generates a predicted image 200 of the encoding target block 301. The orthogonal conversion / quantization unit 11 and the variable-length coding unit 12 encode the coding target block 301 based on the predicted image 200.

The image decoding device 2 of the first embodiment includes an inverse quantization / inverse orthogonal conversion unit 21, a weight calculation unit 261, a prediction image generation unit 262, an adder 22, and a loop filter unit 23. The subregions of the reference blocks 101 of the plurality of reference images 100 are regions at corresponding positions in the reference image 100. The weight calculation unit 261 determines whether or not the luminance values of the plurality of pixels in the partial regions are outliers. Weight calculator 261, a weight coefficient W _X for the pixel of the determined luminance value as an outlier, defined smaller than the weighting factor W _X for the pixel of the determined luminance values not to be outliers. The prediction image generation unit 262 generates the prediction image 200 of the coding target block 301 as the decoding target block based on the result of multiplying the brightness value of the pixel of the reference block 101 by the weighting coefficient W _X. The adder 22 and the loop filter unit 23 decode the coded block 301 based on the predicted image 200.

As a result, the image coding device 1 and the image decoding device 2 of the first embodiment have a plurality of pixels in a partial region of the reference block 101 in one or both of the two reference images 100 constituting the moving image. Even if there is an outlier in the luminance value, or if the luminance values of a plurality of pixels in the partial region of the reference block 101 between the two reference images 100 constituting the moving image change to a threshold value or more, the encoding target is used. It is possible to improve the prediction efficiency of the coded block 301 of the image 300.

(Second Embodiment)
The second embodiment is different from the first embodiment in that the number of reference images 100 is three or more. In the second embodiment, only the differences from the first embodiment will be described.

The upper limit of the number of reference image 100 defined in HEVC is two sheets, the motion compensation unit 181 shown in FIG. 2, also generates a respective reference block P _X in the three or more reference images 100 Good.

The weight calculation unit 182 shown in FIG. 2 detects the maximum value of the brightness value of the pixels of the reference block 101 in the three or more reference images 100 for each pixel. The weight calculation unit 182 detects the minimum value of the brightness value of the pixels of the reference block 101 in the three or more reference images 100 for each pixel. For example, the brightness value of the pixel of the reference block 101 in the reference image 100-0 of the three or more reference images 100 is the maximum value, and the brightness value of the pixel at the corresponding position of the reference block 101 in the reference image 100-2. May be the minimum value. The weight calculation unit 182 calculates the difference δ between the maximum value and the minimum value detected in the three or more reference images 100 for each pixel. Pixels in which the difference δ is equal to or greater than the threshold value are pixels in which an outlier of the luminance value exists.

Weight calculator 182, a weight coefficient _{W X} for each area in the reference image 100 defines the pixel. The area in the reference image 100 is a pixel, an entire area or a partial area of the reference block 101. For example, the weight calculation unit 182 determines the _{weight coefficient W X} for each pixel in the reference block 101-X of the reference image 100-X to which the index X is assigned. The weight calculation unit 182 calculates the average value of the brightness values of the pixels in the reference block 101 of the three or more reference images 100 as a reference value. The weight calculation unit 182 calculates the difference between the brightness value and the average value of the pixels in each reference block 101. Weight calculator 182, a weight coefficient W _X in each reference block 101 is determined for each pixel in accordance with the calculated difference.

Weight calculator 182, based on the deviation of the luminance values of the pixels in the reference block 101 with each other three or more reference images 100, determines the weighting factors W _X for each pixel. The weight calculation unit 182 determines the _{weight coefficient W X} in the pixels of each reference block 101 so that the larger the difference between the luminance value and the average value is, the larger the weight coefficient W _{X is multiplied.} For example, the weight calculation unit 182 determines the _{weight coefficient W X} based on the formulas (7) to (10).

Incidentally, the weight calculator 182 may determine a weighting factor W _X for each partial region in the reference block 101.

As described above, the weight calculation unit 182 of the image coding apparatus 1 of the second embodiment sets the deviation value of the luminance value in the pixels of the reference block 101 of the three or more reference images 100 to the three or more reference images 100. The detection is performed based on the bias of the brightness value of the pixels between the reference blocks 101 of the above.

Further, the weight calculation unit 261 of the image decoding apparatus 2 of the second embodiment sets the deviation value of the brightness value in the pixels of the reference block 101 of the three or more reference images 100 among the reference blocks of the three or more reference images 100. It is detected based on the bias of the brightness value of the pixel in.

As a result, in the image coding device 1 and the image decoding device 2 of the second embodiment, at least one of the three or more reference images 100 constituting the moving image is the brightness value of a plurality of pixels in the partial region of the reference block 101. Even if there is an outlier in the image, or if the brightness values of a plurality of pixels in the partial region of the reference block 101 change above the threshold value among the three or more reference images 100 constituting the moving image, the image to be encoded It is possible to further improve the prediction efficiency of the coded block 301 of 300.

The image coding device 1 and the image decoding device 2 of the second embodiment are robust with respect to the outliers of the brightness values of the pixels in the three or more reference images 100, and the prediction efficiency of the coded target block 301 of the coded target image 300. Can be further improved. The first embodiment and the second embodiment may be combined with respect to the moving image stream.

At least a part of the image coding device and the image decoding device in the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

The present invention is applicable to an image coding device that encodes an image and an image decoding device that decodes a coded image.

1 ... image coding device, 2 ... image decoding device, 10 ... subtractor, 11 ... orthogonal conversion / quantization unit, 12 ... variable length coding unit, 13 ... inverse quantization / inverse orthogonal conversion unit, 14 ... adder , 15 ... Loop filter unit, 16 ... Decoding picture memory, 17 ... Intra prediction unit, 18 ... Inter prediction unit, 19 ... Intra-inter selector switch, 20 ... Variable length decoding unit, 21 ... Inverse quantization / inverse orthogonal conversion unit , 22 ... adder, 23 ... loop filter unit, 24 ... decoded picture memory, 25 ... intra prediction unit, 26 ... inter prediction unit, 27 ... intra / inter changeover switch, 100 ... reference image, 101 ... reference block, 102 ... Area, 103 ... Area, 180 ... Motion search unit, 181 ... Motion compensation unit, 182 ... Weight calculation unit, 183 ... Prediction image generation unit, 200 ... Prediction image, 260 ... Motion compensation unit, 261 ... Weight calculation unit, 262 ... Prediction image generation unit, 300 ... Coding target image, 301 ... Coding target block, 400 ... Reference image, 401 ... Reference block, 402 ... Region, 500 ... Predicted image, 600 ... Coding target image, 601 ... Coding target block

Claims (8)

When the difference in luminance value is equal to or greater than the threshold value for each pixel constituting the reference block of the first reference image and for each pixel at the position corresponding to the constituent pixel of the reference block of the second reference image. so that the weight of the other becomes large is one of the weights is reduced, and the calculation output unit weight Ru determined Me,
The weight obtained for each pixel of the first reference image was multiplied by each pixel of the first reference image to predict the first predicted image, and the weight was obtained for each pixel of the second reference image. the second predicted image to predict by multiplying a weight to each pixel of the second reference image, a third predictive image by adding the first predicted image and the second prediction image and a prediction unit that generates,
An image coding device comprising. The prediction unit obtains the first predicted image by multiplying the brightness value of a pixel in the vicinity of the pixel of the first reference image by a weight to predict the pixel corresponding to the pixel of the first reference image. The second predicted image is predicted by predicting and predicting the pixel corresponding to the pixel of the second reference image by multiplying the brightness value of the pixel in the vicinity of the pixel of the second reference image by a weight. The image coding apparatus according to claim 1, wherein the third predicted image is generated by adding the first predicted image and the second predicted image. The prediction unit multiplies the pixels in the vicinity of each pixel of the first reference image by the weight obtained for each pixel in the vicinity of the pixels of the first reference image to obtain the pixels of the first reference image. The first predicted image is predicted by predicting the corresponding pixel, and the weight obtained for each pixel in the vicinity of the pixel of the second reference image is applied to the pixel in the vicinity of each pixel of the second reference image. The second predicted image is predicted by multiplying by and predicting the pixels corresponding to the pixels of the second reference image, and by adding the first predicted image and the second predicted image. The image coding apparatus according to claim 2, which generates the third predicted image. When the difference in luminance value is equal to or greater than the threshold value for each pixel constituting the reference block of the first reference image and for each pixel at the position corresponding to the constituent pixel of the reference block of the second reference image. so that the weight of the other becomes large is one of the weights is reduced, and the calculation output unit weight Ru determined Me,
The weight obtained for each pixel of the first reference image was multiplied by each pixel of the first reference image to predict the first predicted image, and the weight was obtained for each pixel of the second reference image. the second predicted image to predict by multiplying a weight to each pixel of the second reference image, a third predictive image by adding the first predicted image and the second prediction image and a prediction unit that generates,
An image decoding device comprising. The prediction unit predicts the first predicted image by predicting the pixels corresponding to the pixels of the first reference image using the brightness values of the pixels in the vicinity of the pixels of the first reference image. The second predicted image is predicted by predicting the pixels corresponding to the pixels of the second reference image using the brightness values of the pixels in the vicinity of the pixels of the second reference image, and the first prediction is performed. The image decoding apparatus according to claim 4, wherein the third predicted image is generated by adding the image and the second predicted image. The prediction unit multiplies the pixels in the vicinity of each pixel of the first reference image by the weight obtained for each pixel in the vicinity of the pixels of the first reference image to obtain the pixels of the first reference image. The first predicted image is predicted by predicting the corresponding pixel, and the weight obtained for each pixel in the vicinity of the pixel of the second reference image is applied to the pixel in the vicinity of each pixel of the second reference image. The second predicted image is predicted by multiplying and predicting the pixels corresponding to the pixels of the second reference image, and the first predicted image and the second predicted image are added to the second predicted image. The image decoding apparatus according to claim 5, which generates the predicted image of 3. On the computer
When the difference in luminance value is equal to or greater than the threshold value for each pixel constituting the reference block of the first reference image and for each pixel at the position corresponding to the constituent pixel of the reference block of the second reference image. and instructions to the weight of one increases the weight of the other decreases, Ru weights determined Me,
The weight obtained for each pixel of the first reference image was multiplied by each pixel of the first reference image to predict the first predicted image, and the weight was obtained for each pixel of the second reference image. the second predicted image to predict by multiplying a weight to each pixel of the second reference image, a third predictive image by adding the first predicted image and the second prediction image and procedures you generate,
An image coding program for executing. On the computer
When the difference in luminance value is equal to or greater than the threshold value for each pixel constituting the reference block of the first reference image and for each pixel at the position corresponding to the constituent pixel of the reference block of the second reference image. and instructions to the weight of one increases the weight of the other decreases, Ru weights determined Me,
The weight obtained for each pixel of the first reference image was multiplied by each pixel of the first reference image to predict the first predicted image, and the weight was obtained for each pixel of the second reference image. the second predicted image to predict by multiplying a weight to each pixel of the second reference image, a third predictive image by adding the first predicted image and the second prediction image and procedures you generate,
An image decoding program for executing.

Images