JP3895145B2 - Image encoding apparatus, image decoding apparatus and method, image encoding program, and image decoding program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method for encoding an image by using a correlation between successive images (frames) and reducing the amount of encoded information by motion compensation.
[0002]
[Prior art]
In recent years, with the development of the information-oriented society, there has been a great demand for integration of images and sounds in communication media, that is, audio visualization, and video conferences, videophone systems, etc. have been put into practical use as an example.
In such an image processing system, in order to realize economical image communication, an image encoding technique for compressing a moving image is essential.
[0003]
As an image encoding technique for compressing this moving image, there is an inter-frame predictive encoding method that uses the correlation between consecutive images (frames) and reduces the amount of encoded information by motion compensation.
In moving picture coding using inter-frame predictive coding, there is a reference picture selection coding method that selects a plurality of reference pictures in units of frames or macroblocks.
[0004]
In the Annex N or Annex U of the moving picture coding scheme H.263 (ITU-T international standard for moving picture coding scheme) which is a reference picture selective coding scheme, the encoder includes a reference picture memory that can store a plurality of decoded pictures in advance. The encoded data of each frame is decoded and the decoded image is stored in the reference image memory.
[0005]
When each macroblock is encoded, an image of an appropriate frame is selected from the reference image memory, and encoding and decoding are performed using the selected image as a reference image.
The decoder also includes a reference image memory that can store a plurality of decoded images in advance. When the encoded data of each frame is decoded, the decoded images are stored in this memory.
[0006]
When each macroblock is decoded, an image of an appropriate frame is selected from the reference image memory, and decoding is performed using it as a reference image. In this method, since the number of prediction image candidates used in inter-frame prediction increases, it is possible to generally improve the coding efficiency.
In addition, there is a global motion compensation encoding method that performs interframe prediction encoding using a motion vector (referred to as a global motion vector) of the entire screen in moving image encoding using interframe prediction encoding.
[0007]
In this global motion compensation encoding method, camera parameters such as pan and zoom can be expressed by a global motion vector, compared to a normal motion compensation encoding method using a motion vector for each macroblock.
For this reason, the global motion compensation encoding method can encode the motion vectors for each macroblock as a global motion vector when encoding a moving image with camera parameters. Compared to the method, the coding efficiency in image compression can be improved.
[0008]
For example, in MPEG-4, the motion vectors at the four vertices of the screen are encoded as global motion vectors.
Thus, in an image in which the entire screen is uniformly panned horizontally by camera movement or the like, the motion vectors at the four vertices may be encoded so that the screen pans horizontally.
In this case, the motion vector for each macroblock is a zero vector because the motion of the entire screen is expressed as a motion vector.
[0009]
Further, in the global motion compensation encoding method, it is possible to select whether to perform encoding using the global motion compensation method or normal motion compensation method for each macroblock.
The ability to select this compensation method is effective when backgrounds and foregrounds with different motions are mixed in the same screen, that is, when global motions and local motions are mixed. is there.
For example, a global motion vector is used for macroblocks in the background region, while a macroblock motion vector is used for macroblocks in the foreground region without using a global motion vector, thereby reducing the amount of data to be compensated. Correspondingly, a more effective encoding method can be selected.
[0010]
Here, if it is possible to select whether to encode using the global motion compensation method or the normal motion compensation method, the image encoding device encodes information specifying which motion compensation is used. Yes.
Hereinafter, in the global motion compensation method described in the present application, “information specifying which motion compensation is used” is referred to as a “motion selection flag”.
[0011]
[Problems to be solved by the invention]
When the reference image selection encoding method and the global motion compensation encoding method are combined, a global motion vector is obtained and global motion compensation encoding is performed for each of a plurality of reference images stored in the reference image memory of the encoder. .
At this time, the encoder needs to include a global motion vector for each reference image in the encoded data.
Furthermore, the encoder needs to include in the encoded data the above-described motion selection flag information that specifies the reference image for each macroblock and whether to use the global motion vector.
[0012]
However, the current frame (compression target frame) generally tends to have the smallest difference from the immediately preceding frame.
Therefore, many macroblocks often designate the immediately preceding frame as a reference image, and conversely, the number of macroblocks that designate a frame other than the immediately preceding frame as a reference image is small.
[0013]
Therefore, even if the global motion vector is encoded except for the immediately preceding frame, since there are not many macroblocks that use the global motion vector, unnecessary information that is not used is encoded and attached to the encoded data. .
This leads to a decrease in encoding efficiency.
[0014]
Further, in the encoding method combining the reference image selection encoding method and the global motion compensation encoding method described above, a global motion vector is selected in order to select whether to encode using a global motion vector for each macroblock. In many cases, the number of macroblocks that use is reduced compared to the case of only the global motion compensation encoding method, and the encoding efficiency of the entire encoding process is reduced.
[0015]
The present invention has been made under such a background. When the global motion compensation encoding method is used, encoding of one global motion vector is not performed for each of a plurality of reference images. It is an object of the present invention to provide an image encoding device and an image decoding device that improve the encoding efficiency of the entire image compression.
[0016]
[Means for Solving the Problems]
An image encoding apparatus according to the present invention includes an image input unit that inputs an image of the current frame, an encoding unit that encodes an input image using inter-frame prediction encoding, and encoding using inter-frame prediction encoding. An image encoding apparatus comprising: a decoding unit that decodes data; a global motion vector input unit that inputs a global motion vector; and a global motion compensation unit that creates a predicted image for global motion compensation. A reference image memory for storing decoded images, a reference image selection unit for selecting one image from the reference image memory as a reference image, a global motion vector storage unit for storing a plurality of past global motion vectors, and the global The global motion vector input from the motion vector input unit and the past stored in the global motion vector storage unit From the global motion vector, characterized in that it comprises a global motion vector tracking unit that creates a global motion vector between the images stored in the reference image memory and the current frame.
[0017]
The image encoding device of the present invention includes a motion vector input unit that inputs a motion vector for each region obtained by dividing an image, a motion compensation unit that creates a predicted image for motion compensation for each region, and the encoding unit. And a decoding unit that selects a prediction image used in the decoding unit from a prediction image created in the motion compensation unit and a prediction image created in the global motion compensation unit.
[0018]
In the image encoding device of the present invention, the motion vector input unit inputs a plurality of frames of motion vectors for a plurality of reference images, and the reference image selection unit selects a reference image and a motion vector. Features.
[0019]
An image decoding apparatus according to the present invention includes an encoded data input unit that inputs encoded data of a current frame, a decoding unit that decodes encoded data using inter-frame predictive encoding, and a predicted image for global motion compensation A reference image memory for storing decoded images for a plurality of frames, a reference image selection unit for selecting one image from the reference image memory as a reference image, A global motion vector storage unit that stores a plurality of past global motion vectors, a global motion vector obtained from the encoded data input unit, and a past global motion vector stored in the global motion vector storage unit. A global motion vector that creates a global motion vector between the frame and the image stored in the reference image memory Characterized in that it comprises a Le tracking unit.
[0020]
The image decoding apparatus according to the present invention includes a motion compensation unit that creates a predicted image for motion compensation for each region into which an image is divided, and a predicted image used in the decoding unit, a predicted image created by the motion compensation unit, and a global motion And a motion compensation selection unit that selects from the predicted image created in the compensation unit.
[0021]
The image encoding method according to the present invention includes an image input process in which an image of the current frame is input, an encoding process in which an input image is encoded by inter-frame predictive encoding, and an inter-frame predictive encoding. An image encoding method comprising: a decoding process in which data is decoded; a global motion vector input process in which a global motion vector is input; and a global motion compensation process in which a prediction image for global motion compensation is created. A reference image storage process in which decoded images for frames are accumulated in the reference image memory, a reference image selection process in which one image is selected as a reference image from the reference image memory, and a plurality of past global motion vectors are accumulated. A global motion vector accumulation process, a global motion vector input from the global motion vector input unit, Past the global motion vector of the accumulated fine global motion vector storage unit, and a global motion vector tracking process global motion vector between the images stored in the reference image memory and the current frame is created
It is characterized by having.
[0022]
The image coding method of the present invention includes a motion vector input process in which a motion vector for each region into which an image is divided is input, a motion compensation process in which a prediction image for motion compensation for each region is created, and the code A motion compensation selection process for selecting a prediction image used in the decoding process and a prediction image created in the motion compensation process and a prediction image created in the global motion compensation process. And
[0023]
The image decoding method of the present invention includes an encoded data input process in which encoded data of a current frame is input, a decoding process in which encoded data is decoded using inter-frame predictive encoding, and global motion compensation. A global motion compensation process in which a predicted image is generated, and a reference image storage process in which decoded images for a plurality of frames are stored in a reference image memory; In a reference image selection process selected as a reference image, a global motion vector accumulation process in which a plurality of past global motion vectors are accumulated, a global motion vector obtained in the encoded data input process, and a global motion vector accumulation process The image stored in the current frame and reference image memory from the accumulated past global motion vector. And having a global motion vector tracking process of creating a global motion vector between.
[0024]
The image decoding method according to the present invention includes a motion compensation process in which a predicted image for motion compensation is created for each region into which an image is divided, and a predicted image used in the decoding process. And a motion compensation selection unit process for selecting from the predicted image created in the global motion compensation process.
[0025]
The image encoding program according to the present invention includes an image input process in which an image of the current frame is input, an encoding process in which an input image is encoded by inter-frame prediction encoding, and an encoding by inter-frame prediction encoding. An image for operating the image decoding apparatus, comprising: a decoding process for decoding data; a global motion vector input process for inputting a global motion vector; and a global motion compensation process for generating a prediction image for global motion compensation A reference image storage process in which decoded images for a plurality of frames are accumulated in a reference image memory, a reference image selection process in which one image is selected as a reference image from the reference image memory, and a plurality of frames. Global motion vector accumulation processing in which past global motion vectors are accumulated, and the global motion vector input Global motion vector tracking in which a global motion vector between the current frame and the image stored in the reference image memory is created from the global motion vector input from the past and the past global motion vector stored in the global motion vector storage unit The processing is executed by a computer.
[0026]
The image encoding program of the present invention includes an encoded data input process for inputting encoded data of a current frame, a decoding process for decoding encoded data using inter-frame predictive encoding, and a global motion compensation A reference image storage process in which a decoded image for a plurality of frames is stored in a reference image memory; and a reference image Reference image selection processing in which one image is selected as a reference image from the memory, global motion vector storage processing in which a plurality of past global motion vectors are stored, and global motion obtained in the encoded data input processing From past global motion vectors accumulated in vector and global motion vector accumulation processing Characterized in that to perform the global motion vector tracing process of creating a global motion vector between the stored image in the reference image memory and the current frame to the computer.
[0027]
The computer-readable recording medium of the present invention includes an image input process in which an image of the current frame is input, an encoding process in which an input image is encoded by inter-frame predictive encoding, and an inter-frame predictive encoding. Operates the image decoding apparatus including a decoding process for decoding encoded data, a global motion vector input process for inputting a global motion vector, and a global motion compensation process for generating a prediction image for global motion compensation A reference image storage process in which a plurality of frames of decoded images are stored in a reference image memory, and one image is selected as a reference image from the reference image memory Image selection processing and global motion vector storage processing in which multiple past global motion vectors are stored A global motion vector between the current frame and the image stored in the reference image memory is created from the global motion vector input from the global motion vector input unit and the past global motion vector stored in the global motion vector storage unit. An image encoding program characterized by causing a computer to execute the global motion vector tracking process is recorded.
[0028]
The computer-readable recording medium of the present invention includes an encoded data input process in which encoded data of a current frame is input, a decoding process in which encoded data is decoded using inter-frame predictive encoding, and global motion compensation And a global motion compensation process for generating a predicted image for recording, and a recording medium on which an image decoding program for operating the image decoding device according to claim 4 is recorded, and decoding for a plurality of frames Reference image storage processing in which an image is stored in a reference image memory, reference image selection processing in which one image is selected as a reference image from the reference image memory, and a global in which a plurality of past global motion vectors are stored Motion vector accumulation processing, global motion vector obtained in the encoded data input processing, and global motion vector An image characterized by causing a computer to execute global motion vector tracking processing for creating a global motion vector between a current frame and an image stored in a reference image memory from past global motion vectors stored in the storage processing The decryption program is recorded.
[0029]
The image coding apparatus according to the present invention, when storing global motion vectors obtained from the global motion vector input unit in the past in the global motion vector storage unit, the global motion vector, the current frame and the previous frame. The global motion vector between the current frame and the past frame can be calculated from the global motion vector between.
[0030]
Therefore, according to the image coding apparatus of the present invention, a predicted image is created from a reference image at the same time stored in the reference image memory using the calculated global motion vector, and global motion compensation is performed. Therefore, it is not necessary to encode the global motion vector between the past frames for each reference image, and the encoding efficiency in image compression can be improved.
[0031]
The global motion vector storage unit can also store the global motion vector obtained at the time of encoding the immediately preceding frame in the global motion vector tracking unit.
Furthermore, in the image coding apparatus according to the present invention, it is possible to select a reference image for each frame or to select a reference image for each region such as a macroblock for coding a current image.
[0032]
In the image encoding device of the present invention, when a reference image is selected for each region, a global motion vector tracking unit obtains global motion vectors for a plurality of frames, and the global motion compensation unit generates a predicted image. At the same time, the global motion vector obtained from the global motion vector tracking is selected and used together with the global motion vector input from the global motion vector.
[0033]
In addition, a prediction image used by the encoding unit and the decoding unit is selected for each region from a prediction image created by motion compensation for each region such as a normal macroblock and a prediction image created by global motion compensation. Is possible.
As a result, in an input image with a foreground and a background, it is possible to use a predicted image created by normal motion compensation for the foreground and a predicted image created by global motion compensation for the background, and encoding corresponding to the motion By selecting a method, it is possible to improve the coding efficiency in image compression.
[0034]
In addition, it is possible to prepare a plurality of frames of motion vectors to be used in normal motion compensation, and simultaneously select a reference image and a motion vector.
As a result, when encoding by normal motion compensation is performed in the encoding of the foreground macroblock, an image for a plurality of frames can be used as a reference image. The coding efficiency in image compression can be improved.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. Hereinafter, embodiments of the present invention will be described in the order of an image encoding device and an image decoding device.
<Image encoding device>
FIG. 1 is a block diagram illustrating a configuration example of an image encoding device according to an embodiment of the present invention.
The image encoding apparatus in this figure performs, for example, normal motion compensation or global motion compensation in units of macroblocks, and in the case of encoding by normal motion compensation, the motion vector input unit 104 is provided with a plurality of frames. This is an example in which information on motion vectors is input.
[0036]
The image encoding apparatus includes an image input unit 101, an encoding unit 102, a decoding unit 103, a global motion vector input unit 105, a global motion compensation unit 109, a reference image memory 111, a reference image selection unit 112, and a global motion vector storage unit. 107, the global motion vector tracking unit 106, the motion vector input unit 104, the motion compensation unit 108, and the motion compensation selection unit 110, as well as the motion compensation unit 108 and the global for each reference image selected by the reference image selection unit 112. A difference evaluation unit 113 is provided that calculates a difference between the prediction image created by the motion compensation unit 109 and the current image and selects a prediction image having the smallest difference.
[0037]
Here, the difference evaluation unit 113 calculates the difference between the predicted image and the current image by the absolute value difference sum W in the macroblock, and outputs the prediction error having the smallest absolute value difference as the smallest difference. To do.
The global motion vector is a set of motion vectors (PX, PY), (QX, QY), (RX, RY), and (SX, SY). It shall be expressed as Here, for the sake of simplification, the respective values for the global motion vector GMV are represented by GMV1 to GMV8. For example, the motion vector (PX, PY) = (GMV1, GMV2).
[0038]
As a premise, it is assumed that images of three frames can be stored in the reference image memory 111 in advance.
Assume that these three frames of images are represented by images PA, PB, and PC, and the current image is represented by PD.
For example, the motion vector input unit 104 inputs motion vectors for three frames, and represents the motion vectors between the current image PD and each of the images PA, PB, and PC as motion vectors MAD, MBD, and MCD, respectively.
[0039]
The global motion vector storage unit 107 stores a global motion vector GMAB between the images PA and PB and a global motion vector GMBC between the images PB and PC.
The global motion vector input unit 105 inputs a global motion vector GMCD between the current image PD and the image PC.
Here, GMCD, GMAB, and GMBC are combinations of the motion vectors (PX, PY), (QX, QY), (RX, RY), and (SX, SY) of the four vertices P, Q, R, and S, respectively. It is represented by
[0040]
Based on the premise described above, the encoding process of the macroblock in the current image PD in the image encoding apparatus of the present invention will be described.
First, the reference image selection unit 112 switches the connection of the reference image switching unit 120 so that the reference image PC is selected from the reference image memory 111.
Accordingly, the reference image PC is supplied from the reference image memory 111 to the motion compensation unit 108 and the global motion compensation unit 109 via the reference image switching unit 120.
[0041]
The reference image selection unit 112 switches the connection of the motion vector switching unit 121 so that the motion vector MCD is selected from the motion vector input unit 104.
Thus, the motion vector MCD is supplied from the motion vector input unit 104 to the motion compensation unit 108 via the motion vector switching unit 121.
Then, the motion compensation unit 108 creates a predicted image LC using the motion vector MCD.
[0042]
Also, the global motion compensation unit 109 creates a predicted image GC using the global motion vector GMCD.
Next, the motion compensation selection unit 110 switches the connection of the compensation switching unit 122 so that the predicted image LC is supplied from the motion compensation unit 108 to the difference evaluation unit 113.
Thereby, the difference evaluation unit 113 obtains the absolute value difference sum WLC between the current image PD and the predicted image LC.
[0043]
Then, after the difference evaluation unit 113 obtains the absolute difference sum WLC, the motion compensation selection unit 110 switches the connection of the compensation switching unit 122, and the predicted image GC is supplied from the global motion compensation unit 109 to the difference evaluation unit 113. State.
Thereby, the difference evaluation unit 113 obtains the absolute value difference sum WGC between the current image PD and the predicted image GC.
[0044]
Next, the reference image selection unit 112 switches the connection of the reference image switching unit 120 so that the reference image PB is selected from the reference image memory 111.
Thus, the reference image PB is supplied from the reference image memory 111 to the motion compensation unit 108 and the global motion compensation unit 109 via the reference image switching unit 120.
Further, the reference image selection unit 112 switches the connection of the motion vector switching unit 121 so that the motion vector MBD is selected from the motion vector input unit 104.
[0045]
Thereby, the motion vector MBD is supplied from the motion vector input unit 104 to the motion compensation unit 108 via the motion vector switching unit 121.
Then, the motion compensation unit 108 creates a predicted image LB using the motion vector MBD.
Then, the global motion vector tracking unit 106 obtains a global motion vector GMBD from the global motion vector GMCD and the global motion vector GMBC stored in the global motion vector storage unit 107.
[0046]
Here, the global motion vector GMBD is obtained by the calculation of the following equation (1).
GMDi = GMCBi + GMCDi (i = 1-8) (1)
Then, after obtaining the global motion vector GMBD, the global motion compensation unit 109 creates a predicted image GB using the global motion vector GMBD.
[0047]
Next, the motion compensation selection unit 110 switches the connection of the compensation switching unit 122 so that the predicted image LB is supplied from the motion compensation unit 108 to the difference evaluation unit 113.
Thereby, the difference evaluation unit 113 obtains the absolute value difference sum WLB between the current image PD and the predicted image LB.
Then, after the difference evaluation unit 113 obtains the absolute difference sum WLB, the motion compensation selection unit 110 switches the connection of the compensation switching unit 122, and the predicted image GB is supplied from the global motion compensation unit 109 to the difference evaluation unit 113. State.
Thereby, the difference evaluation unit 113 obtains the absolute value difference sum WGB between the current image PD and the predicted image GB.
[0048]
Next, the reference image selection unit 112 switches the connection of the reference image switching unit 120 so that the reference image PA is selected from the reference image memory 111.
Accordingly, the reference image PA is supplied from the reference image memory 111 to the motion compensation unit 108 and the global motion compensation unit 109 via the reference image switching unit 120.
Further, the reference image selection unit 112 switches the connection of the motion vector switching unit 121 so that the motion vector MAD is selected from the motion vector input unit 104.
[0049]
Accordingly, the motion vector MAD is supplied from the motion vector input unit 104 to the motion compensation unit 108 via the motion vector switching unit 121.
Then, the motion compensation unit 108 creates a predicted image LA using the motion vector MAD.
Then, the global motion vector tracking unit 106 obtains a global motion vector GMAD from the global motion vector GMAD and the global motion vector GMAB stored in the global motion vector storage unit 107.
[0050]
Here, the global motion vector GMAD is obtained by the calculation of the following equation (2).
GMDi = GMABi + GMBCi + GMCDi (i = 1-8) (2)
Then, after obtaining the global motion vector GMAB, the global motion compensation unit 109 creates a predicted image GA using the global motion vector GMAD.
[0051]
Next, the motion compensation selection unit 110 switches the connection of the compensation switching unit 122 so that the predicted image LA is supplied from the motion compensation unit 108 to the difference evaluation unit 113.
Thereby, the difference evaluation unit 113 obtains the absolute value difference sum WLA between the current image PD and the predicted image LA.
Then, after the difference evaluation unit 113 obtains the absolute difference sum WLA, the motion compensation selection unit 110 switches the connection of the compensation switching unit 122, and the predicted image GA is supplied from the global motion compensation unit 109 to the difference evaluation unit 113. State.
Thereby, the difference evaluation unit 113 obtains an absolute value difference sum WGA between the current image PD and the predicted image GA.
[0052]
Next, the difference evaluation unit 113 determines which of the absolute value difference sums WLA, WGA, WLB, WGB, WLC, and WGC is the minimum value.
Then, the encoding unit 102 encodes the predicted image having the minimum absolute value difference sum as the predicted image to be actually used.
Then, the encoding unit 102 determines whether the reference image selection flag Fp for designating the reference image used to create the prediction image and the prediction image from the motion compensation unit 108 or the prediction image from the global motion compensation unit 109 Is included in the encoded data.
[0053]
In addition, when the predicted image from the motion compensation unit 108 is determined to be the minimum value, the encoding unit 102 sets the motion vector used when generating the predicted image as the motion vector MV, and encodes the motion vector MV. Include in data.
On the other hand, when the predicted image from the global motion compensation unit 109 is determined to be the minimum value, the encoding unit 102 does not include the motion vector MV in the encoded data.
The decoding unit 103 stores the decoded image in the reference image memory 111.
The image coding apparatus according to the present invention repeats the above procedure for all macroblocks.
[0054]
In addition, when the encoding of the macroblock is started, the encoding unit 102 attaches the encoded global motion vector to the encoded data at the start time of the encoded data of the image PD.
That is, the encoding unit 102 attaches the encoded global motion vector to the first encoded data of the frame (for example, the image PD) for each frame unit.
The global motion vector tracking unit 106 stores the global motion vector GMCD in the global motion vector storage unit 107.
[0055]
According to the image coding apparatus of the present invention described above, one global motion vector is encoded on a frame basis by providing a configuration for creating a motion vector for each reference image based on the global motion vector with the immediately preceding image. Therefore, since it is not necessary to encode and attach a global motion vector to a reference image for each macroblock, it is possible to improve the encoding efficiency of the entire image compression process.
[0056]
<Image decoding device>
Next, an embodiment of the image decoding apparatus of the present invention will be described with reference to the drawings. FIG. 2 is a block diagram illustrating a configuration example of an image decoding device according to an embodiment of the present invention.
The image decoding apparatus of the present invention includes at least an encoded data input unit 214, a decoding unit 203, a global motion compensation unit 208, a reference image memory 211, a reference image selection unit 212, a global motion vector storage unit 207, and a global motion vector tracking unit. 206, a motion compensation unit 208, and a motion compensation selection unit 210.
[0057]
Hereinafter, a procedure for decoding the encoded data obtained in the image encoding device of the present invention described above by the image decoding device of the present invention will be described.
As a premise, the reference image memory 211 can store images for three frames in advance. Here, the images stored in the reference image memory 211 are represented by images PA, PB, and PC, respectively.
In addition, it is assumed that the global motion vector storage unit 207 stores a global motion vector GMAB between the image PA and the image PB and a global motion vector GMBC between the image PB and the image PC.
[0058]
Based on the premise described above, encoded data of an image PD is input to the encoded data input unit 214 from the image encoding device.
Here, the decoding unit 203 decodes the global motion vector GMCD included in the encoded data of the image PD.
Next, the decoding unit 203 decodes the reference image selection flag Fp and the motion selection flag Fm for each sequentially input macroblock.
[0059]
Based on the decoded reference image selection flag Fp and motion selection flag Fm, decoding is performed in units of macroblocks.
The decoding procedure based on the reference image selection flag Fp and the motion selection flag Fm will be described below.
That is, two examples of decoding will be described: a case where the motion selection flag Fm indicates that normal motion compensation is used and a case where global motion compensation is used.
[0060]
(A) The decoding procedure when the reference image selection flag Fp indicates the reference image PB and the motion selection flag Fm indicates that global motion compensation is used will be described.
The reference image selection unit 212 changes the connection state of the reference image switching unit 220 so that the reference image PB is supplied from the reference image memory 211 when the reference image selection flag Fp indicates the reference image PB.
In addition, the global motion vector tracking unit 206 calculates the global motion vector GMBD based on the global motion vector GMCD and the global motion vector GMBC stored in the global motion vector storage unit 207 by an operation based on Expression (1). Ask.
[0061]
Thereby, the global motion compensation unit 209 uses the global motion vector GMBD obtained by the global motion vector tracking unit 206 to create a predicted image GB from the reference image PB.
Then, based on the motion selection flag Fm indicating that the global motion compensation is used, the motion compensation selection unit 210 is supplied to the decoding unit 203 so that the predicted image GB created by the global motion compensation unit 209 is supplied. The connection state of the compensation switching unit 222 is changed.
[0062]
That is, the motion compensation selection unit 210 selects the prediction image GB created by the global motion compensation unit 209 as a prediction image used for decoding by the decoding unit 203.
Next, the decoding unit 203 decodes the encoded data of the image PD using the predicted image GB set by the motion compensation selection unit 210.
Then, the decoding unit 203 stores the decoded image of the encoded data of the image PD in the reference image memory 211.
[0063]
(B) The decoding procedure when the reference image selection flag Fp indicates the reference image PA and the motion selection flag Fm indicates that normal motion compensation is used will be described.
The reference image selection unit 212 changes the connection state of the reference image switching unit 220 so that the reference image PA is supplied from the reference image memory 211 when the reference image selection flag Fp indicates the reference image PA.
In addition, the decoding unit 203 decodes the motion vector MV included in the input encoded data and outputs the decoded motion vector MV to the motion compensation unit 208.
[0064]
Thereby, the motion compensation unit 208 creates a predicted image LA from the reference image PA using the motion vector MV (that is, the motion vector MAD).
Then, based on the motion selection flag Fm indicating that the normal motion compensation is used, the motion compensation selection unit 210 is supplied to the decoding unit 203 so that the predicted image LA created by the motion compensation unit 208 is supplied. The connection state of the compensation switching unit 222 is changed.
[0065]
That is, the motion compensation selection unit 210 selects the predicted image LA created by the motion compensation unit 208 as a predicted image used for decoding by the decoding unit 203.
Next, the decoding unit 203 decodes the encoded data of the image PD using the predicted image LA set by the motion compensation selection unit 210.
Then, the decoding unit 203 stores the decoded image of the encoded data of the image PD in the reference image memory 211.
[0066]
As described above, each macroblock is repeatedly decoded based on the motion selection flag Fm indicating whether normal motion compensation or global motion compensation is used for encoding each macroblock.
The global motion vector GMCD is stored in the global motion vector storage unit 207.
[0067]
According to the image decoding apparatus of the present invention described above, one global motion vector (global motion vector between an image to be compressed and an image immediately before this image) is set in the frame, and based on this global motion vector, Since other global vectors can be obtained from the calculations based on equations (1) and (2), the global motion vector is not decoded for each reference image of the macroblock, that is, for each frame (for example, image PD). Since a plurality of global motion vectors are not decoded at the time of decoding, the amount of encoded data can be reduced, and the encoding efficiency of the entire image compression process can be improved.
[0068]
In the image coding apparatus according to the present embodiment, the configuration in which the motion vector input unit 104 inputs motion vectors for a plurality of frames has been described, but a motion vector for one frame may be input.
In this case, the motion compensation unit 108 creates a predicted image only for the reference image targeted by the input motion vector.
[0069]
Further, in the present embodiment, normal motion compensation and global motion compensation are selected in the image encoding device and the image decoding device, but it is also possible to configure to perform only global motion compensation.
In this case, the image coding apparatus does not include the motion vector input unit 104, the motion compensation unit 108, and the motion compensation selection unit 110, and the image decoding apparatus includes the motion compensation unit 208 and the motion compensation selection unit 210. It is set as the structure which is not provided.
Since all use global motion compensation, it is not necessary to include the motion selection flag Fm and the motion vector MV in the encoded data.
[0070]
In the above-described embodiment of the present invention, the procedure executed by each of the image encoding device and the image decoding device is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system. By executing, the user attribute registration information automatic updating apparatus of the present invention is realized. The computer system referred to here includes an OS and hardware such as peripheral devices.
[0071]
Further, the “computer system” includes a homepage providing environment (or display environment) if the WWW system is used.
The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a client or a system when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.
[0072]
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.
[0073]
As mentioned above, although one embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and there are design changes and the like without departing from the gist of the present invention. Are also included in the present invention.
[0074]
【The invention's effect】
According to the image encoding device and the image decoding device of the present invention, the global motion between the current frame and the past frame based on the past global motion vector and the global motion vector between the current frame and the previous frame. It is possible to calculate a vector.
[0075]
Then, the image encoding device and the image decoding device according to the present invention use the global motion vector obtained by calculation to create a prediction image from the reference image stored at the same time in the reference image memory and perform global motion compensation. Can do.
For this reason, the image encoding device and the image decoding device of the present invention provide a global motion vector between a frame at the current time (image to be encoded) and a past frame (reference image) for each reference image of the macroblock. Therefore, it is possible to improve the coding efficiency in the entire image compression.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of an image encoding device according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a configuration example of an image decoding device according to an embodiment of the present invention.
[Explanation of symbols]
101 Image input unit
102 Encoding unit
103, 203 Decoding unit
104 Motion vector input unit
105 Global motion vector input unit
106 Global motion vector tracking unit
107 Global motion vector storage unit
108,208 Motion compensation unit
109,209 Global motion compensation unit
110, 210 Motion compensation selection unit
111, 211 Reference image memory
112, 212 Reference image selection unit
113 Difference evaluation unit
120, 220 Reference image switching unit
121 Motion vector switching unit
122, 222 Compensation switching unit
214 Encoded data input section

Claims (10)

An image input unit for inputting an image of the current frame;
An encoding unit that encodes an input image using inter-frame prediction encoding;
A decoding unit that decodes encoded data using inter-frame prediction encoding;
A global motion vector input unit for inputting a global motion vector;
A motion vector input unit for inputting a motion vector for each region obtained by dividing the image;
A motion compensation unit that creates a prediction image for motion compensation for each region;
An image encoding device comprising:
A reference image memory for storing decoded images for a plurality of frames;
A reference image selection unit that sequentially selects and outputs the reference images of the reference image memory one by one ;
A global motion vector storage unit for storing a plurality of past global motion vectors;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector input from the global motion vector input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking unit for creating a global motion vector,
A global motion compensation unit that creates a prediction image for global motion compensation using the global motion vector created by the global motion vector tracking unit;
A difference evaluation unit that calculates a difference between the predicted image and the original image and selects a predicted image having the smallest difference;
A compensation switching unit that uses the prediction image used in the difference evaluation unit as one of a prediction image created in a motion compensation unit and a prediction image created in a global motion compensation unit;
With
The motion vector input unit inputs motion vectors for a plurality of frames for a plurality of reference images, the reference image selection unit selects a reference image and a motion vector, the motion compensation unit, the global motion compensation unit, Generating a predicted image for each reference image selected by the reference image selection unit. An encoded data input unit for inputting encoded data of the current frame;
A decoding unit that decodes encoded data using inter-frame prediction encoding;
A global motion compensation unit that creates a prediction image for global motion compensation, a motion compensation unit that creates a prediction image for motion compensation for each region into which the image is divided, and
An image decoding device comprising:
A reference image memory for storing decoded images for a plurality of frames;
In the image encoding device, the difference between the predicted image and the original image is calculated, and one image is selected from the reference image memory by the reference image selection flag that specifies the reference image from which the predicted image having the smallest difference is obtained. A reference image selection unit to select as,
A global motion vector storage unit for storing a plurality of past global motion vectors;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector obtained from the encoded data input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking unit for creating a global motion vector;
In the image encoding device, a motion selection flag indicating whether a prediction image obtained by calculating a difference between a prediction image and an original image and obtaining a prediction image having the smallest difference is generated by either global motion compensation or motion compensation By means of this, an image decoding apparatus comprising: a motion compensation selection unit that selects one of the prediction images created by the motion compensation unit and the global motion compensation unit. 3. The image decoding according to claim 2, wherein the motion compensation selection unit selects a prediction image to be used by the decoding unit from a prediction image created by the motion compensation unit and a prediction image created by the global motion compensation unit. apparatus. An image input process in which an image of the current frame is input;
An encoding process in which an input image is encoded by inter-frame predictive encoding;
A decoding process in which encoded data is decoded by inter-frame predictive encoding;
A global motion vector input process in which a global motion vector is input;
A motion vector input process in which a motion vector for each region obtained by dividing an image is input;
A motion compensation process in which a predicted image for motion compensation for each region is created, and
A reference image storage process in which decoded images for a plurality of frames are stored in a reference image memory;
A reference image selection process for sequentially selecting and outputting the reference images of the reference image memory one by one ;
A global motion vector accumulation process in which a plurality of past global motion vectors are accumulated;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector input from the global motion vector input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking process in which a global motion vector is created,
A global motion compensation process for creating a prediction image for global motion compensation using the global motion vector created by the global motion vector tracking unit;
A difference evaluation process for calculating a difference between the predicted image and the original image and selecting a predicted image with the smallest difference;
Compensation switching process in which the prediction image used in the difference evaluation process is one of a prediction image created in the motion compensation process and a prediction image created in the global motion compensation process ,
In the motion vector input process, motion vectors for a plurality of frames for a plurality of reference images are input, a reference image and a motion vector are selected in the reference image selection process, and the motion compensation process and the global motion compensation process in, images coding method you characterized by creating a predicted image with respect to the reference image selected each reference image selected by the process. Encoded data input process in which encoded data of the current frame is input;
A decoding process in which encoded data is decoded using inter-frame predictive encoding;
A global motion compensation process in which a predicted image for global motion compensation is created;
A motion compensation process in which a prediction image for motion compensation for each region obtained by dividing the image is created, and
A reference image storage process in which decoded images for a plurality of frames are stored in a reference image memory;
In the image encoding device, one image is calculated from the reference image memory by using a reference image selection flag that designates a reference image that calculates a difference between the predicted image and the original image and obtains a predicted image having the smallest difference. A reference image selection process selected as:
A global motion vector accumulation process in which a plurality of past global motion vectors are accumulated;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector obtained from the encoded data input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking process to create a global motion vector;
In the image encoding device, a motion selection flag indicating whether a prediction image obtained by calculating a difference between a prediction image and an original image and obtaining a prediction image having the smallest difference is generated by either global motion compensation or motion compensation And a motion compensation selection process for selecting one of the predicted images created by the motion compensation process and the global motion compensation process. In the motion compensation selection process claims and selects from the predicted image to be used in the decoding process, a prediction image generated in the predicted image and the global motion compensation process that was created in the motion compensator process 5 The image decoding method described in 1. An image input process in which an image of the current frame is input;
An encoding process in which an input image is encoded by inter-frame predictive encoding;
A decoding process in which encoded data is decoded by inter-frame predictive encoding;
A global motion vector input process in which a global motion vector is input;
A motion vector input process in which a motion vector for each region obtained by dividing an image is input;
A motion compensation process in which a predicted image for motion compensation for each region is created, and an image coding program for operating the image decoding device according to claim 1 ,
A reference image storage process in which decoded images for a plurality of frames are accumulated in a reference image memory;
A reference image selection process for sequentially selecting and outputting the reference images in the reference image memory one by one ;
A global motion vector accumulation process in which a plurality of past global motion vectors are accumulated;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector input from the global motion vector input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking process in which a global motion vector is created,
A global motion compensation process for creating a prediction image for global motion compensation using the global motion vector created by the global motion vector tracking unit;
A difference evaluation process for calculating a difference between the predicted image and the original image and selecting a predicted image having the smallest difference;
Compensation switching processing, wherein the prediction image used in the difference evaluation processing is one of a prediction image created in motion compensation processing and a prediction image created in global motion compensation processing ,
In the motion vector input process, motion vectors for a plurality of frames for a plurality of reference images are input, a reference image and a motion vector are selected in the reference image selection process, and the motion compensation process and the global motion compensation process are performed. A computer-readable recording medium storing an image coding program, comprising: causing a computer to create a predicted image for each reference image selected by the reference image selection process. Encoded data input processing in which encoded data of the current frame is input;
A decoding process in which encoded data is decoded using inter-frame predictive encoding;
A global motion compensation process in which a prediction image for global motion compensation is created;
And a motion compensation processing predicted image for motion compensation of each region obtained by dividing an image is created, an image decoding program for operating the image decoding apparatus according to claim 2 or claim 3,
A reference image storage process in which decoded images for a plurality of frames are accumulated in a reference image memory;
In the image encoding device, one image is calculated from the reference image memory by using a reference image selection flag that designates a reference image that calculates a difference between the predicted image and the original image and obtains a predicted image having the smallest difference. A reference image selection process selected as
A global motion vector accumulation process in which a plurality of past global motion vectors are accumulated;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector obtained from the encoded data input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking process for creating a global motion vector;
In the image encoding device, a motion selection flag indicating whether a prediction image obtained by calculating a difference between a prediction image and an original image and obtaining a prediction image having the smallest difference is generated by either global motion compensation or motion compensation According to the present invention, there is provided an image decoding program that causes a computer to execute a motion compensation selection process that selects any one of the predicted images created by the motion compensation process and the global motion compensation process. An image input process in which an image of the current frame is input;
An encoding process in which an input image is encoded by inter-frame predictive encoding;
A decoding process in which encoded data is decoded by inter-frame predictive encoding;
A global motion vector input process in which a global motion vector is input;
A motion vector input process in which a motion vector for each region obtained by dividing an image is input;
And a motion compensation process in which a prediction image for motion compensation for each region is created.A recording medium on which an image encoding program for operating the image decoding apparatus according to claim 1 is recorded,
A reference image storage process in which decoded images for a plurality of frames are accumulated in a reference image memory;
A reference image selection process for sequentially selecting and outputting the reference images in the reference image memory one by one ;
A global motion vector accumulation process in which a plurality of past global motion vectors are accumulated;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector input from the global motion vector input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking process in which a global motion vector is created,
A global motion compensation process for creating a prediction image for global motion compensation using the global motion vector created by the global motion vector tracking unit;
A difference evaluation process for calculating a difference between the predicted image and the original image and selecting a predicted image having the smallest difference;
Compensation switching processing, wherein the prediction image used in the difference evaluation processing is one of a prediction image created in motion compensation processing and a prediction image created in global motion compensation processing ,
In the motion vector input process, motion vectors for a plurality of frames for a plurality of reference images are input, a reference image and a motion vector are selected in the reference image selection process, and the motion compensation process and the global motion compensation process are performed. A computer-readable recording medium recorded with an image encoding program, wherein the computer is caused to generate a predicted image for each reference image selected by the reference image selection process. Encoded data input processing in which encoded data of the current frame is input;
A decoding process in which encoded data is decoded using inter-frame predictive encoding;
A global motion compensation process in which a prediction image for global motion compensation is created;
4. A recording medium on which an image decoding program for operating the image decoding apparatus according to claim 2 or 3 is recorded, comprising: a motion compensation process for generating a predicted image for motion compensation for each region into which an image is divided And
A reference image storage process in which decoded images for a plurality of frames are stored in a reference image memory;
In the image encoding device, one image is calculated from the reference image memory by using a reference image selection flag that designates a reference image that calculates a difference between the predicted image and the original image and obtains a predicted image having the smallest difference. A reference image selection process selected as
A global motion vector accumulation process in which a plurality of past global motion vectors are accumulated;
The global motion vector between the past images stored in the global motion vector storage unit is added to the global motion vector obtained from the encoded data input unit, and between the current frame and the reference image stored in the reference image memory. A global motion vector tracking process for creating a global motion vector;
In the image encoding device, a motion selection flag indicating whether a prediction image obtained by calculating a difference between a prediction image and an original image and obtaining a prediction image having the smallest difference is generated by either global motion compensation or motion compensation A computer-readable recording medium on which an image decoding program is recorded, which causes a computer to execute a motion compensation selection process for selecting one of the predicted images created by the motion compensation process and the global motion compensation process.

Images