JP2003264833A - Image processing apparatus and image processing method, and program and recording medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To efficiently decode encoded data into a decoded image with good image quality. SOLUTION: An inverse quantization circuit 40 has a function of D for each block.
The prediction tap extraction circuit 41 outputs the CT coefficient, and extracts a DCT coefficient used for predicting the target pixel as a prediction tap. Then, the mode control unit 81 determines the pixel position mode of the target pixel by using the information indicating the position of each pixel in the block as the pixel position mode, and degenerates the pixel position mode of the target pixel, thereby setting the degenerated pixel position mode. Output. On the other hand, the code control unit 82 performs the DCT forming the prediction tap in accordance with the pixel position mode of the target pixel.
The sign of the coefficient is manipulated and output to the product-sum operation circuit 45. The product-sum operation circuit 45 acquires a tap coefficient corresponding to the degenerate pixel position mode of the target pixel from the tap coefficients obtained by performing the learning for each degenerate pixel position mode, and calculates the tap coefficient and the prediction tap. To predict the pixel value of the target pixel.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium, and particularly, for example, at least a two-dimensional DCT (D
The present invention relates to an image processing device, an image processing method, a program, and a recording medium that enable efficient decoding of encoded data that has been encoded by performing iscrete Cosine Transform).

[0002]

2. Description of the Related Art For example, since digital image data has a large amount of data, it is necessary to directly record or transmit it.
A large capacity recording medium or transmission medium is required. Therefore, in general, image data is compressed and encoded to reduce the amount of data, and then recording and transmission are performed.

As a method of compression-encoding image data, for example, there is a method using a two-dimensional DCT conversion for performing DCT conversion on the image data in two directions, horizontal and vertical, and a typical example thereof is , JPEG (Joint Photographic Ex
perts Group) or MP, which is a compression encoding method for moving images
There is an EG (Moving Picture Experts Group) system and the like.

For example, encoding / decoding of image data by the JPEG system is performed as shown in FIG.

That is, FIG. 1A shows the configuration of an example of a conventional JPEG encoding apparatus.

The image data to be coded is input to the block forming circuit 1, and the block forming circuit 1 divides the image data input thereto into blocks each having 64 pixels, which is 8 × 8 pixels in width × length. To do. Each block obtained by the blocking circuit 1 is supplied to the DCT circuit 2. DCT circuit 2
Is the DCT for the block from the blocking circuit 1.
After processing (2D DCT conversion), one DC (Direc
t Current) component (direct current component) and 63 AC (Alternating Current) components (alternating current component) in the horizontal and vertical directions are converted into a total of 64 DCT coefficients.
The 64 DCT coefficients for each block are calculated by the DCT circuit 2
Is supplied to the quantization circuit 3.

The quantization circuit 3 quantizes the DCT coefficient from the DCT circuit 2 according to a predetermined quantization table, and uses the quantization result (hereinafter, appropriately referred to as quantized DCT coefficient) for the quantization. The data is supplied to the entropy coding circuit 4 together with the quantization table.

Here, FIG. 1B shows an example of a quantization table used in the quantization circuit 3. Generally, in the quantization table, considering the human visual characteristics,
The low-frequency DCT coefficient of high importance is finely quantized,
The quantization step is set so that the high-frequency DCT coefficient of low importance is roughly quantized, whereby the deterioration of the image quality of the image is suppressed and efficient compression is performed.

The entropy coding circuit 4 performs entropy coding processing such as Huffman coding on the quantized DCT coefficient from the quantization circuit 3 and adds the quantization table from the quantization circuit 3. Then, the encoded data obtained as a result is output as a JPEG encoding result.

Next, FIG. 1C shows the JPE of FIG.
The structure of an example of the conventional JPEG decoding device which decodes the encoded data which a G encoding device outputs is shown.

The encoded data is the entropy decoding circuit 1
1, the entropy decoding circuit 11 separates the encoded data into entropy-encoded quantized DCT coefficients and a quantization table. Further, the entropy decoding circuit 11 uses the entropy-coded quantized D
The CT coefficient is entropy-decoded, and the resulting quantized DCT coefficient is supplied to the inverse quantization circuit 12 together with the quantization table. The inverse quantization circuit 12 inversely quantizes the quantized DCT coefficient from the entropy decoding circuit 11 in accordance with the quantization table from the entropy decoding circuit 11, and the resulting DCT coefficient is inverse DCT.
Supply to the circuit 13. The inverse DCT circuit 13 performs inverse DCT processing (two-dimensional inverse DCT conversion) on the DCT coefficient from the inverse quantization circuit 12 and supplies the resulting decoded block of 8 × 8 pixels to the block decomposition circuit 14. To do. The block decomposing circuit 14 obtains and outputs a decoded image by unblocking the decoded block from the inverse DCT circuit 13.

[0012]

[Problems to be Solved by the Invention] JPEG of FIG.
In the encoding device, the amount of encoded data can be reduced by increasing the quantization step of the quantization table used for the block quantization in the quantization circuit 3. That is, high compression can be realized.

However, if the quantization step is increased, the so-called quantization error also increases.
The image quality of the decoded image obtained by the JPEG decoding device of (C) deteriorates. That is, blurring, block distortion, mosquito noise, and the like are prominent in the decoded image.

Therefore, in order to prevent the image quality of the decoded image from deteriorating while reducing the data amount of the encoded data, or to maintain the data amount of the encoded data and improve the image quality of the decoded image. In general, it is necessary to perform some kind of processing for improving image quality after JPEG decoding.

However, performing the process for improving the image quality after the JPEG decoding makes the process complicated,
It also takes a long time to finally obtain a decoded image.

In the MPEG system as well, as in the JPEG system, image data (moving image data) is two-dimensionally DCT-converted and quantized, so that the quality of the decoded image deteriorates as in the case of the JPEG system.

The present invention has been made in view of such a situation, and image data is JPEG encoded or MPEG.
The encoded data, which has been encoded, can be efficiently decoded into a decoded image with high image quality.

[0018]

According to a first image processing apparatus of the present invention, image data is processed into a two-dimensional DC for each predetermined block.
A two-dimensional DCT coefficient acquisition unit that acquires a two-dimensional DCT coefficient obtained by T conversion from encoded data, and predicts a pixel of interest, which is a pixel of interest among pixels forming image data. A prediction tap extraction unit that extracts the used two-dimensional DCT coefficient as a prediction tap, a pixel position mode determination unit that determines the pixel position mode of the pixel of interest using the information indicating the position of each pixel in the block as the pixel position mode, Corresponding to the pixel position mode degenerating means for degenerating the pixel position mode of the pixel and outputting the degenerate pixel position mode which is the pixel position mode in which the total number is smaller than the number of pixels in the block, and the pixel position mode of the pixel of interest,
A tap coefficient corresponding to the degenerate pixel position mode of the pixel of interest is calculated from the code operating means for operating the sign of the two-dimensional DCT coefficient forming the prediction tap and the tap coefficient obtained by learning for each degenerate pixel position mode. It is characterized by including a prediction unit that acquires the pixel value of the pixel of interest using the tap coefficient and the prediction tap.

A first image processing method of the present invention comprises a two-dimensional DCT coefficient acquisition step of acquiring from a coded data a two-dimensional DCT coefficient obtained by two-dimensional DCT transforming image data for each predetermined block, A prediction tap extraction step of extracting, as a prediction tap, a two-dimensional DCT coefficient used for predicting a pixel of interest, which is a pixel of interest among the pixels forming the image data, and a position of each pixel in the block. A pixel position mode determination step of determining the pixel position mode of the pixel of interest using the information as the pixel position mode, and a degenerate pixel that is a pixel position mode in which the pixel position mode of the pixel of interest is degenerated and the total number is smaller than the number of pixels in the block. Configure prediction taps corresponding to the pixel position mode degeneration step that outputs the position mode and the pixel position mode of the pixel of interest. A tap coefficient corresponding to the degenerate pixel position mode of the pixel of interest is acquired from the code operation step of operating the code of the two-dimensional DCT coefficient and the tap coefficient obtained by learning for each degenerate pixel position mode, and the tap coefficient is obtained. It is characterized by including a prediction step of predicting a pixel value of a target pixel using a coefficient and a prediction tap.

A first program of the present invention is a two-dimensional DCT coefficient acquisition step of acquiring from a coded data a two-dimensional DCT coefficient obtained by two-dimensional DCT transforming image data for each predetermined block, and image data. The prediction tap extraction step of extracting, as a prediction tap, the two-dimensional DCT coefficient used to predict the pixel of interest, which is the pixel of interest, among the pixels that form the block, and information indicating the position of each pixel in the block. As the pixel position mode, a pixel position mode determination step of determining the pixel position mode of the pixel of interest, and a degenerate pixel position mode that is a pixel position mode in which the pixel position mode of the pixel of interest is degenerated and the total number is smaller than the number of pixels in the block. A prediction tap corresponding to the pixel position mode degeneration step for outputting A tap coefficient corresponding to the degenerate pixel position mode of the pixel of interest is acquired from the code operation step of operating the sign of the three-dimensional DCT coefficient and the tap coefficient obtained by learning for each degenerate pixel position mode, and the tap coefficient And a prediction step of predicting the pixel value of the target pixel using the prediction tap.

The first recording medium of the present invention is obtained by subjecting image data to two-dimensional DCT conversion for each predetermined block.
Two-dimensional D that obtains three-dimensional DCT coefficient from encoded data
A CT coefficient acquisition step, a prediction tap extraction step of extracting, as a prediction tap, a two-dimensional DCT coefficient used for predicting a pixel of interest, which is a pixel of interest among pixels forming image data, The pixel position mode determining step of determining the pixel position mode of the target pixel using the information indicating the position of each pixel as the pixel position mode, and the pixel position mode of the target pixel are degenerated, and the total number of pixels is less than the number of pixels in the block. A pixel position mode degeneration step of outputting a degenerate pixel position mode which is a position mode; a code operation step of operating a sign of a two-dimensional DCT coefficient forming a prediction tap corresponding to a pixel position mode of a pixel of interest; From the tap coefficient obtained by learning for each position mode, the degenerate pixel position mode of the pixel of interest is compared. It gets the tap coefficients, and the tap coefficients, by using the prediction taps, a program and a prediction step of predicting the pixel value of the pixel of interest is characterized in that it is recorded.

The second image processing apparatus of the present invention encodes the image data for learning and obtains the encoded data obtained as a result of at least two-dimensional DCT conversion of the image data for each predetermined block. Of the teacher data, student data acquisition means for acquiring the two-dimensional DCT coefficient as student data to be a student for learning, and each pixel constituting the image data for learning as teacher data to be a teacher for learning. The student data used to predict the teacher data of interest, which is the teacher data being processed, as prediction tap extraction means, and information indicating the position of each pixel in the block as the pixel position mode. Pixel position mode determination means for determining the pixel position mode and the number of pixels in the block by degenerating the pixel position mode of the teacher data of interest Pixel position mode degeneracy means for outputting a degenerate pixel position mode, which is a pixel position mode with a smaller total number, and code operation for manipulating the code of the student data forming the prediction tap corresponding to the pixel position mode of the teacher data of interest. Means, learning means for obtaining the tap coefficient for each degenerate pixel position mode by performing learning for each degenerate pixel position mode using the teacher data of interest, and the prediction tap.

The second image processing method of the present invention is obtained by encoding learning image data and performing at least two-dimensional DCT conversion of the image data for each predetermined block from the encoded data obtained as a result. A student data acquisition step of acquiring a two-dimensional DCT coefficient as student data to be a student of learning, and each pixel constituting image data for learning as teacher data to be a teacher of learning, The prediction tap extraction step of extracting student data used to predict attention teacher data, which is the teacher data being used, as prediction taps, and information indicating the position of each pixel in the block as the pixel position mode. The pixel position mode determination step of determining the pixel position mode and the pixel position mode of the teacher data of interest are degenerated and stored in blocks. Corresponding to the pixel position mode degeneration step that outputs a degenerate pixel position mode that is a pixel position mode in which the total number of pixels is smaller than the number of pixels A learning step of obtaining a tap coefficient for each degenerate pixel position mode by performing learning for each degenerate pixel position mode using a code operation step of operating, attention teacher data, and a prediction tap. To do.

A second program of the present invention encodes image data for learning, and based on the encoded data obtained as a result, at least the image data is divided into two blocks for each predetermined block.
A two-dimensional DCT coefficient obtained by the three-dimensional DCT transformation is acquired as student data to be a student for learning, a student data acquisition step, and pixels constituting image data for learning are
As teacher data to be a teacher for learning, a prediction tap extraction step of extracting, as a prediction tap, student data used to predict attention teacher data that is attention teacher data among teacher data, The pixel position mode determining step of determining the pixel position mode of the teacher data of interest and the pixel position mode of the teacher data of interest being degenerated so that the total number of pixels is less than the number of pixels in the block. A pixel position mode degeneration step of outputting a degenerate pixel position mode which is a pixel position mode, a code operation step of operating a code of student data forming a prediction tap corresponding to the pixel position mode of the teacher data of interest, and a teacher of interest Data and
A learning step of obtaining a tap coefficient for each degenerate pixel position mode by performing learning for each degenerate pixel position mode using a prediction tap.

The second recording medium of the present invention encodes image data for learning, and at least the image data is encoded at least in predetermined blocks from the obtained encoded data.
A two-dimensional DCT coefficient obtained by the three-dimensional DCT transformation is acquired as student data to be a student for learning, a student data acquisition step, and pixels constituting image data for learning are
As teacher data to be a teacher for learning, a prediction tap extraction step of extracting, as a prediction tap, student data used to predict attention teacher data that is attention teacher data among teacher data, The pixel position mode determining step of determining the pixel position mode of the teacher data of interest and the pixel position mode of the teacher data of interest being degenerated so that the total number of pixels is less than the number of pixels in the block. A pixel position mode degeneration step of outputting a degenerate pixel position mode which is a pixel position mode, a code operation step of operating a code of student data forming a prediction tap corresponding to the pixel position mode of the teacher data of interest, and a teacher of interest Data and
A program including a learning step of obtaining a tap coefficient for each degenerate pixel position mode by performing learning for each degenerate pixel position mode using a prediction tap is characterized by being recorded.

In the first image processing apparatus, the image processing method, and the program of the present invention, the two-dimensional DCT coefficient obtained by the two-dimensional DCT conversion of the image data for each predetermined block is obtained from the encoded data. Of the pixels forming the image data, the two-dimensional DCT coefficient used to predict the pixel of interest, which is the pixel of interest, is extracted as a prediction tap. Further, the information indicating the position of each pixel in the block is set as the pixel position mode, the pixel position mode of the pixel of interest is determined, the pixel position mode of the pixel of interest is degenerated, and the pixel position of which the total number is smaller than the number of pixels in the block is determined. The degenerate pixel position mode, which is the mode, is output. Then, the sign of the two-dimensional DCT coefficient forming the prediction tap is manipulated in accordance with the pixel position mode of the pixel of interest, and the degeneration of the pixel of interest is deduced from the tap coefficient obtained by learning for each degenerate pixel position mode. The tap coefficient corresponding to the pixel position mode is acquired, and the pixel value of the target pixel is predicted using the tap coefficient and the prediction tap.

In the second image processing apparatus, the image processing method, and the program of the present invention, the image data for learning is encoded, and at least the image data for every predetermined block is encoded from the resulting encoded data. To 2
The two-dimensional DCT coefficient obtained by the three-dimensional DCT transformation is acquired as student data to be a learning student. Further, each pixel constituting the image data for learning is used as teacher data to be a teacher for learning, and student data used to predict attention teacher data that is attention teacher data among teacher data is The pixel position mode of the teacher data of interest is determined by using the information extracted as a prediction tap and representing the position of each pixel in the block as the pixel position mode. After that, the pixel position mode of the teacher data of interest is degenerated, and the degenerated pixel position mode, which is a pixel position mode in which the total number is smaller than the number of pixels in the block, is output. And
The sign of the student data forming the prediction tap is manipulated corresponding to the pixel position mode of the attention teacher data, and the degeneration is performed by performing learning for each degenerate pixel position mode using the attention teacher data and the prediction tap. A tap coefficient is obtained for each pixel position mode.

[0028]

FIG. 2 shows an example of the configuration of an embodiment of an image transmission system to which the present invention is applied.

The image data to be transmitted is the encoder 21.
The encoder 21 encodes the image data supplied thereto by at least two-dimensional DCT conversion for each predetermined block. That is, in the encoder 21, when the image data supplied thereto is still image or moving image data, the still image or moving image data is, for example, JPEG encoded or MPEG encoded, respectively. .

The encoded data obtained by encoding the image data by the encoder 21 is recorded on a recording medium 23 such as a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape, or a phase change disk. , Or, for example, terrestrial, satellite lines,
It is transmitted via a transmission medium 24 such as a CATV (Cable Television) network, the Internet, or a public line.

The decoder 22 receives the encoded data provided via the recording medium 23 or the transmission medium 24,
Decode to image data. The decoded image data (decoded image data) is supplied to, for example, a monitor (not shown) and displayed.

Next, when the image data supplied to the encoder 21 of FIG. 2 is a still image, the encoder 21
Encodes the image data by JPEG, for example. In this case, the encoder 21 uses, for example, the above-mentioned FIG.
The image data is JPEG-encoded and output as encoded data by the same configuration as the JPEG-encoding apparatus shown in FIG.

FIG. 3 shows a case where image data of a still image is JPEG encoded in the encoder 21 of FIG.
2 shows an example of the configuration of the decoder 22.

The encoded data is the entropy decoding circuit 3
1, the entropy decoding circuit 31 entropy-decodes the encoded data, and obtains the quantized DCT coefficient Q for each block obtained as a result.
It is supplied to the coefficient conversion circuit 32.

The coefficient conversion circuit 32 dequantizes the quantized DCT coefficient Q from the entropy decoding circuit 31 to obtain 8 ×.
8 DCT coefficients for each block, and by using the DCT coefficient and a tap coefficient obtained by performing learning to be described later, a predetermined prediction calculation is performed to obtain the DCT coefficient for each block of 8 × 8 pixels. Decrypt to original block.

The block decomposition circuit 33 includes a coefficient conversion circuit 3
A decoded image is obtained and output by solving the block formation of the decoded block (decoding block) obtained in 2.

Next, referring to the flow chart of FIG.
The processing of the decoder 22 of FIG. 3 will be described.

The encoded data is the entropy decoding circuit 3
1, and the entropy decoding circuit 31 entropy-decodes the encoded data and supplies the quantized DCT coefficient Q for each block to the coefficient conversion circuit 32 in step S1. The coefficient conversion circuit 32 executes step S2.
, Inverse quantization of the quantized DCT coefficient Q for each block from the entropy decoding circuit 31 and a prediction operation using the DCT coefficient obtained as a result and the tap coefficient obtained in advance by learning are performed. Thus, the pixel value for each block is decoded and supplied to the block decomposing circuit 33. In step S3, the block decomposition circuit 33 performs block decomposition for unblocking the block of pixel values (decoding block) from the coefficient conversion circuit 32, outputs the decoded image obtained as a result, and ends the process.

Next, in the coefficient conversion circuit 32 of FIG. 3, the quantized DCT coefficient can be decoded into a pixel value by utilizing, for example, the class classification adaptive processing previously proposed by the applicant of the present application. .

The class classification adaptive process is composed of a class classification process and an adaptation process. The class classification process classifies data based on its property and applies the adaptation process to each class. Is based on the following method.

That is, in the adaptive processing, for example, the DCT coefficient is decoded into the original pixel value by obtaining the predicted value of the original pixel by linear combination of the DCT coefficient and a predetermined tap coefficient.

Specifically, for example, a certain image is now used as teacher data, and the image is subjected to DCT processing in block units and further quantized to obtain a DCT coefficient (or the DCT coefficient is further inversely quantized). The predicted value E [y] of the pixel value y of the pixel, which is the teacher data, is used as the student data.
It is considered that a linear linear combination model defined by a linear combination of a set of coefficients x ₁ , x ₂ , ... And predetermined tap coefficients w ₁ , w ₂ ,. In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ + ... (1)

In order to generalize the equation (1), a matrix W consisting of a set of tap coefficients w _j , a matrix X consisting of a set of student data x _ij , and a matrix Y consisting of a set of prediction values E [y _j ]. '
To

[Equation 1] , The following observation equation holds.

XW = Y ′ (2) Here, the component x _ij of the matrix X is a set of i-th student data (a set of student data used for prediction of _i-th teacher data y _i ). Means the j-th student data, and the component w _{j of the} matrix W represents the tap coefficient with which the product of the j-th student data in the set of student data is calculated. Also, y
_i represents the i-th teacher data, and therefore E [y _i ]
Represents the predicted value of the i-th teacher data. It should be noted that y on the left side of the equation (1) is obtained by omitting the suffix i of the component y _i of the matrix Y, and x ₁ , x ₂ , ... On the right side of the equation (1) is also the matrix X. The suffix i of the component x _ij is omitted.

Then, it is considered that the least squares method is applied to this observation equation to obtain the predicted value E [y] close to the original pixel value y. In this case, the true pixel value y that becomes the teacher data
Matrix Y consisting of a set of P and prediction value E for pixel value y
Matrix E, which is a set of residuals e of [y],

[Equation 2] , The following residual equation is established from the equation (2).

[0047]       XW = Y + E                                                       ... (3)

In this case, the predicted value E close to the original pixel value y
The tap coefficient w _j for obtaining [y] is the squared error

[Equation 3] Can be obtained by minimizing.

Therefore, the above squared error is converted into the tap coefficient w _j.
When the differentiated value becomes 0, that is, the tap coefficient w _j satisfying the following equation is the optimum value for obtaining the predicted value E [y] close to the original pixel value y.

[0050]

[Equation 4] ... (4)

Therefore, first, the equation (3) is changed to the tap coefficient w
By differentiating with _j , the following equation holds.

[0052]

[Equation 5] ... (5)

From equations (4) and (5), equation (6) is obtained.

[0054]

[Equation 6] ... (6)

Further, considering the relationship among the student data x _ij , the tap coefficient w _j , the teacher data y _i , and the residual e _{i in} the residual equation of the equation (3), the following equation is obtained from the equation (6). You can get the normal equation.

[0056]

[Equation 7] ... (7)

In the normal equation shown in the equation (7), the matrix (covariance matrix) A and the vector v are

[Equation 8] When the vector W is defined as shown in Formula 1, it can be expressed by the formula AW = v ... (8).

Each normal equation in the equation (7) is prepared by preparing a certain number of sets of the student data x _ij and the teacher data y _i , so that the same number as the number J of tap coefficients w _{j to} be obtained is obtained. Can be, therefore, equation (8)
By solving for the vector W (however, in order to solve the equation (8), the matrix A in the equation (8) needs to be regular), and the optimum tap coefficient (here, the square error is minimized. The tap coefficient) w _j can be obtained. In addition,
In solving equation (8), for example, the sweep method (Ga
Uss-Jordan elimination method) can be used.

As described above, the optimum tap coefficient w _j
Is calculated, and the tap coefficient w _j is further used to calculate a prediction value E [y] close to the original pixel value y by the equation (1) is an adaptive process.

Note that, for example, as teacher data, JPE
While using an image of the same quality as the image to be G-encoded,
When the DCT coefficient obtained by DCT and quantization of the teacher data is used as the student data, as the tap coefficient, there is a prediction error in decoding the JPEG-coded image data into the original image data. The statistically smallest one will be obtained.

Therefore, even if the compression rate at the time of performing JPEG coding is increased, that is, even if the quantization step used for quantization is rough, the prediction error is statistically minimized according to the adaptive processing. Will be subjected to the decoding process
The decoding process of the JPEG-encoded image and the process for improving the image quality thereof are performed at the same time.
As a result, the image quality of the decoded image can be maintained even if the compression rate is increased.

Further, for example, as teacher data, JPE
An image having a higher image quality than the image to be G-encoded is used, and the image quality of the teacher data as student data is
The image quality is degraded to the same as that of the image to be EG encoded, and D
When a DCT coefficient obtained by CT conversion and quantization is used, the tap coefficient has a statistically minimum prediction error when decoding JPEG-coded image data into high-quality image data. Things will be obtained.

Therefore, in this case, according to the adaptive processing, J
The decoding process of the PEG-coded image and the process for further improving the image quality will be performed at the same time. From the above, by changing the image quality of the image that is the teacher data or the student data, it is possible to obtain the tap coefficient that sets the image quality of the decoded image to an arbitrary level.

Note that the above is the same for the JPEG-encoded image as well as the MPEG-encoded image, for example.

FIG. 5 shows a configuration example of the coefficient conversion circuit 32 of FIG. 3 which decodes the quantized DCT coefficient into a pixel value by the class classification adaptive processing as described above.

The quantized DCT coefficient for each block output from the entropy decoding circuit 31 (FIG. 3) is supplied to the inverse quantization circuit 40.

The inverse quantization circuit 40 inversely quantizes the quantized DCT coefficient for each block, acquires a block of the DCT coefficient by this, and supplies it to the prediction tap extraction circuit 41 and the class tap extraction circuit 42.

The prediction tap extracting circuit 41 sequentially sets the pixels of the block of the DCT coefficient supplied thereto as the target pixel in the so-called raster scan order, for example. further,
The prediction tap extraction circuit 41 extracts the DCT coefficient used for predicting the pixel value of the target pixel from the output of the inverse quantization circuit 40, and sets it as a prediction tap.

That is, the prediction tap extraction circuit 41 extracts all DCT coefficients of the block including the pixel of interest, that is, 64 DCT coefficients of 8 × 8 as prediction taps, as shown in FIG. 6, for example. . Therefore, here, the same prediction tap is configured for all pixels of a block. However, the prediction tap is
It is possible to configure with different DCT coefficients.

The prediction tap for the pixel of interest obtained in the prediction tap extraction circuit 41 is supplied to the product-sum calculation circuit 45.

The class tap extraction circuit 42 extracts from the output of the dequantization circuit 40 DCT coefficients used for classifying the pixel of interest into one of several classes, and outputs the class. Tap.

That is, the class tap extraction circuit 42 extracts all DCT coefficients of the block including the pixel of interest, that is, 64 DCT coefficients of 8 × 8, as class taps, as shown in FIG. 6, for example. To do. Therefore, here, the same class tap is configured for all the pixels of a certain block. However, the class tap can be configured with different DCT coefficients for each pixel of interest.

The prediction taps and class taps are not limited to the tap structures described above.

The class tap of the pixel of interest obtained in the class tap extraction circuit 42 is supplied to the class classification circuit 43.
Based on the class tap from the class tap extraction circuit 42, the pixel of interest is classified into classes, and a class code corresponding to the class obtained as a result is output.

Here, as a method for classifying,
For example, ADRC (Adaptive Dynamic Range Coding) or the like can be adopted.

In the method using ADRC, the DCT coefficients forming the class tap are ADRC-processed and the result is obtained.
The class of the pixel of interest is determined according to the ADRC code.

In the K-bit ADRC, for example,
The maximum value MAX and the minimum value MIN of the DCT coefficient composing the class tap are detected, DR = MAX-MIN is set as the local dynamic range of the set, and the DCT coefficient composing the class tap is based on this dynamic range DR. Is requantized to K bits. That is, the DCT that constitutes the class tap
The minimum value MIN is subtracted from the coefficients, and the subtracted value is DR /
It is divided (quantized) by 2 ^K. Then, each of the K-bit DCTs constituting the class tap obtained as described above
A bit string in which the coefficients are arranged in a predetermined order is output as an ADRC code. Therefore, the class tap is, for example, 1
When the bit ADRC processing is performed, each DCT coefficient forming the class tap is, after the minimum value MIN is subtracted,
It is divided by the average value of the maximum value MAX and the minimum value MIN, so that each DCT coefficient becomes 1 bit (binarized). A bit string in which the 1-bit DCT coefficients are arranged in a predetermined order is used as an ADRC code.
The code is output as a class code.

In the class classification circuit 43, for example,
It is possible to output the pattern of the level distribution of the DCT coefficients forming the class taps as it is as a class code, but in this case, the number of class taps is N DCs.
Assuming that each DCT coefficient is composed of T coefficients and K bits are assigned to each DCT coefficient, the number of class codes output from the class classification circuit 43 is (2 ^N ) ^K , and DC
It is a huge number exponentially proportional to the number K of bits of the T coefficient.

Therefore, in the class classification circuit 43,
It is preferable to perform class classification by compressing the amount of information of class taps by the above-mentioned ADRC process, vector quantization, or another method.

The class classification circuit 43 is obtained, for example, by ADRC processing the class taps as described above.
The ADRC code is output as a class code representing the class of the pixel of interest, and this class code is given to the coefficient table storage unit 44 as an address.

The coefficient table storage unit 44 stores a coefficient table in which tap coefficients obtained by performing a learning process described later are registered, and an address corresponding to the class code output from the class classification circuit 43 is stored. And outputs the tap coefficient stored in the product-sum calculation circuit 45.

Here, the block is subjected to DCT transform / inverse DCT
In the case of conversion, position information representing the spatial position of each pixel in the block is considered in the form of the phase of the cosine function used for DCT conversion / inverse DCT conversion.

On the other hand, in the adaptive processing, the linear first-order prediction calculation of the equation (1) using the DCT coefficient forming the prediction tap and the tap coefficient is performed. The position information of the existing pixel (pixel of interest) is not considered (the formula (1) does not have a function that considers the position information). Therefore, in order to perform the linear primary prediction calculation considering the position information of the pixel of interest, the product-sum calculation circuit 45 uses different tap coefficients depending on the position of the pixel of interest. That is, even if the pixels are classified into the same class, if the positions in the blocks are different, the linear primary prediction calculation is performed using different tap coefficients. In this case, linear 1
The tap coefficient used in the next prediction calculation is switched depending on the position of the pixel of interest in the block, and the information indicating the position of this pixel of interest is hereinafter referred to as a pixel position mode, as appropriate.

In this case, since the block is composed of 8 × 8 pixels, there are 64 positions. Here, for example, the nth position in the raster scan order is the pixel position mode # n−1. Shall be expressed as Therefore, here, the numbers attached to the squares of the 8 × 8 block shown in FIG.
Indicates the pixel position mode.

From the above, here, the pixel position mode is
There are 64 modes from 0 to 63, 6 for each class
A set of tap coefficients corresponding to each of the four pixel position modes, that is, 64 sets of tap coefficients are prepared, and 64 sets of tap coefficients are prepared for the address corresponding to one class code in the coefficient table storage unit 44. The tap coefficient is stored.

The sum-of-products calculation circuit 45 includes the prediction tap output from the prediction tap extraction circuit 41 and the coefficient table storage unit 44.
And the tap coefficient output from the target pixel are acquired, and the prediction pixel and the tap coefficient are used to perform the linear prediction operation (sum of products operation) shown in Expression (1) to decode the pixel of interest.
Then, when the sum of products arithmetic circuit 45 obtains the decoding result of the pixel value of 8 × 8 pixels of the block including the target pixel (hereinafter, appropriately referred to as the target block), the pixel value of the target block is determined by the block decomposition circuit 33. (Fig. 3).

Here, in the prediction tap extraction circuit 41, as described above, each pixel of the target block is sequentially
Although it is regarded as the pixel of interest, the product-sum calculation circuit 45 performs processing in the operation mode corresponding to the position of the pixel of interest in the block of interest, that is, the pixel position mode.

That is, for example, among the pixels of the block of interest, the nth pixel in the raster scan order is represented as p _n−1, and when the pixel p _n−1 is the pixel of interest, the sum of products operation is performed. The circuit 45 performs the pixel position mode # n-1.

Specifically, as described above, the coefficient table storage unit 44 outputs 64 sets of tap coefficients for the class of the pixel of interest, but the tap coefficient for decoding the pixel p _n−1 of them. _Let W _n-1 be the set of
The operation mode of the product-sum calculation circuit 45 is pixel position mode #
When n−1, a prediction tap and a set W _{n−1 of} 64 sets of tap coefficients corresponding to the class of the pixel of interest
Using and, the product-sum operation of the equation (1) is performed, and the product-sum operation result is used as the decoding result of the pixel p _n-1 .

The product-sum calculation circuit 45 recognizes the pixel position mode of the pixel of interest based on the output of the mode control unit 46. That is, the prediction tap extracting circuit 41 is adapted to supply the position information (for example, the coordinates of the pixel of interest in the block of interest) of the pixel which is currently the pixel of interest to the mode control unit 46. 46 determines the pixel position mode of the pixel of interest based on the position information of the pixel of interest supplied from the prediction tap extraction circuit 41.
Then, the mode control unit 46 supplies the pixel position mode of the target pixel to the product-sum calculation circuit 45.

Next, referring to the flowchart of FIG.
The processing of the coefficient conversion circuit 32 of FIG. 5 will be described.

The quantized DCT coefficient for each block output from the entropy decoding circuit 31 (FIG. 3) is supplied to the dequantization circuit 40, and the dequantization circuit 40 proceeds to step S11.
, The quantized DC for each block supplied thereto
Dequantize the T coefficient. The DCT coefficients for each block obtained by the inverse quantization circuit 40 performing inverse quantization are sequentially supplied to the prediction tap extraction circuit 41 and the class tap extraction circuit 42.

The prediction tap extraction circuit 41 sequentially sets the blocks of the DCT coefficients supplied from the inverse quantization circuit 40 as the target block. Then, the prediction tap extraction circuit 41
Selects a pixel of interest in the raster scan order that has not been selected as a pixel of interest in step S12, outputs the position information of the pixel of interest to the mode control unit 46, and then executes step S13. Proceed to.

In step S13, the mode control unit 46
Determines the pixel position mode #n of the target pixel based on the position information from the prediction tap extraction circuit 41, supplies the pixel position mode #n to the product-sum calculation circuit 45, and proceeds to step S14.

In step S14, the class tap extraction circuit 42 extracts, from the DCT coefficients supplied from the dequantization circuit 40, the one used for classifying the pixel of interest to form a class tap. That is, here
The class tap extraction circuit 42 extracts the 64 DCT coefficients of the block including the target pixel, for example, in the raster scan order, and sets the sequence of the 64 DCT coefficients as the class tap of the target pixel. The class tap of this pixel of interest is from the class tap extraction circuit 42 to the class classification circuit 4
3 is supplied.

In step S15, the class classification circuit 43 classifies the pixel of interest using the class tap from the class tap extraction circuit 42, and outputs the resulting class code to the coefficient table storage unit 44.

Upon receiving the class code from the class classification circuit 43, the coefficient table storage unit 44 receives the step S1.
In 6, the 64 sets of tap coefficients stored in the address corresponding to the class code are read out and output to the product-sum operation circuit 45.

Then, proceeding to step S17, the prediction tap extraction circuit 41 extracts the DCT coefficient used to predict the pixel value of the pixel of interest, and forms a prediction tap. That is, the prediction tap extraction circuit 41 extracts the 64 DCT coefficients of the block including the pixel of interest, for example, in the raster scan order, and sets the sequence of the 64 DCT coefficients as the prediction tap of the pixel of interest. This prediction tap is supplied from the prediction tap extraction circuit 41 to the product-sum calculation circuit 45.

Here, in the coefficient conversion circuit 32 of FIG. 5, the same prediction taps are formed for all the pixels of each block in each block, so that the prediction taps of step S17 are actually used. The processing for constructing the above is performed only for the first pixel of interest in the target block, and does not need to be performed for the remaining 63 pixels. The same applies to the process of configuring the class tap in step S14 and the class classification process in step S15.

In step S18, the product-sum calculation circuit 45 obtains the set of tap coefficients corresponding to the pixel position mode of the pixel of interest from the 64 sets of tap coefficients output from the coefficient table storage section 44 in step S16. Then, the process proceeds to step S19 and the product-sum operation shown in equation (1) is performed using the set of tap coefficients and the prediction tap supplied from the prediction tap extraction circuit 41 in step S17 to obtain the pixel value of the pixel of interest. Get the decoded value of.

Then, in step S20, the prediction tap extracting circuit 41 determines whether or not all the pixels of the target block have been processed as the target pixel. When it is determined in step S16 that all the pixels of the target block have not been processed as the target pixel, the process returns to step S12, and the prediction tap extracting circuit 41
Is the pixel of the block of interest, in raster scan order,
The same process is repeated from now on, assuming that the pixel not yet set as the target pixel is newly set as the target pixel.

If it is determined in step S20 that all the pixels of the target block have been processed as the target pixel, that is, if the decoded values of all the pixels of the target block are obtained, the product-sum operation circuit The 45 outputs a block (decoded block) composed of the decoded value to the block decomposition circuit 33 (FIG. 3), and proceeds to step S21.

In step S21, the prediction tap extracting circuit 41 determines whether or not all the blocks of DCT coefficients output by the inverse quantization circuit 40 have been processed as the block of interest. In step S21, the inverse quantization circuit 40
If it is determined that all the blocks of the DCT coefficients output by the block are not considered as the block of interest, the prediction tap extraction circuit 41 determines that the block of the DCT coefficients output by the inverse quantization circuit 40 is the block of interest. The block that is not marked is newly set as the block of interest, and step S1
Returning to 2, the same processing is repeated thereafter.

If it is determined in step S21 that all the blocks of DCT coefficients output by the inverse quantization circuit 40 have been processed as the block of interest, the process ends.

Next, FIG. 8 shows an example of the configuration of a learning device for performing the learning process of the tap coefficient stored in the coefficient table storage section 44 of FIG.

The blocking circuit 61 is supplied with one or more pieces of image data for learning (here, image data of a still image). The blocking circuit 61,
Further, the DCT circuit 62 and the quantization circuit 6 in the subsequent stage
3, and the inverse quantization circuit 64 encodes the learning image data in the same manner as in the case of JPEG encoding, and from the resulting encoded data, the DCT coefficient becomes a student of tap coefficient learning. It is acquired as student data.

That is, the blocking circuit 61 converts the learning image data into 8 pieces, as in the case of JPEG encoding.
Block into blocks of × 8 pixels. The image data for each block is read by the DCT circuit 62, and is also read by the normal equation adding circuit 68 as teacher data for teaching tap coefficients.

The DCT circuit 62 sequentially reads the blocks of the image data divided into blocks by the blocking circuit 61 and performs DCT conversion on the blocks to obtain blocks of DCT coefficients. This block of DCT coefficients is supplied to the quantization circuit 63.

The quantization circuit 63 quantizes the block of DCT coefficients from the DCT circuit 62 according to the same quantization table used for JPEG encoding, and obtains the quantized DCT coefficient for each block obtained as a result. , To the inverse quantization circuit 64. The inverse quantization circuit 64 inversely quantizes the quantized DCT coefficient for each block supplied from the quantization circuit 64, thereby obtaining the DCT coefficient for each block as student data.

The prediction tap extraction circuit 65 sequentially sets the blocks of the learning image data divided into blocks by the block formation circuit 61 as a target block, and further, among the pixels of the target block, in the raster scan order. , A pixel that is not a target pixel is set as a target pixel, and the same prediction tap as the prediction tap extraction circuit 41 of FIG. It is configured by extracting (reading out) student data). This prediction tap is supplied from the prediction tap extraction circuit 65 to the normal equation addition circuit 68.

Further, the prediction tap extraction circuit 65 supplies the position information of the pixel of interest to the mode control unit 71, similarly to the prediction tap extraction circuit 41 of FIG.

From the dequantization circuit 64, the class tap extraction circuit 66 outputs the same class tap as the class tap extraction circuit 42 of FIG.
It is configured by extracting (reading out) necessary DCT coefficients. This class tap is supplied from the class tap extraction circuit 66 to the class classification circuit 67.

The class classification circuit 67 uses the class tap from the class tap extraction circuit 66 to perform the same processing as that of the class classification circuit 43 of FIG. The code is supplied to the normal equation adding circuit 68.

The normal equation adding circuit 68 receives the pixel of interest (pixel value) as the teacher data from the blocking circuit 61.
The teacher data and the prediction tap extraction circuit 6
Prediction tap (student data from 5
CT coefficient) and addition for each class,
And it is performed for each pixel position mode.

That is, the normal equation adding circuit 68 outputs prediction taps (student data) for each class corresponding to the class code supplied from the class classification circuit 67 and for each pixel position mode output from the mode control unit 71. By using, multiplications (x _in x _im ) of student data, which are the respective components in the matrix A of Expression (8), and calculations corresponding to the summation (Σ) are performed.

Further, the normal equation adding circuit 68 also predicts taps (student data) for each class corresponding to the class code supplied from the class classification circuit 67 and for each pixel position mode output from the mode control unit 71. ) And the target pixel (teacher data), which is each component in the vector v of the equation (8), the multiplication (x _in y _i ) of the student data and the teacher data, and the calculation corresponding to the summation (Σ). I do.

The normal equation adding circuit 68 performs the above addition with all the pixels of the image data for learning as the teacher data as the target pixel, whereby the expression ( Create the normal equation shown in 8).

The tap coefficient determination circuit 69 solves the normal equation obtained in the normal equation addition circuit 68 for each class and for each pixel position mode, and thus 64 for each class corresponds to 64 pixel position modes. The tap coefficient of the set is calculated and supplied to the address corresponding to each class in the coefficient table storage unit 70.

Depending on the number of images prepared as images for learning, the contents of the images, etc., the normal equation adding circuit 68 may not obtain the number of normal equations required to obtain the tap coefficient. Although a pixel position mode may occur, the tap coefficient determination circuit 69 outputs, for example, a default tap coefficient for such a class or pixel position mode.

The coefficient table storage unit 70 stores 64 sets of tap coefficients for each class supplied from the tap coefficient determination circuit 69.

The mode control section 71 determines the pixel position mode of the target pixel based on the position information of the target pixel supplied from the prediction tap extraction circuit 65. Then, the mode control unit 71 supplies the pixel position mode of the target pixel to the normal equation adding circuit 68.

Next, referring to the flow chart of FIG.
The processing (learning processing) of the learning device in FIG. 8 will be described.

In the learning process, first, in step S3
1, the block circuit 61, the DCT circuit 62, the quantization circuit 63, and the dequantization circuit 64 JPEG-encode the learning image data, and further, from the encoded data, the DCT coefficient that becomes the student data. get.

That is, the blocking circuit 61 converts the learning image data into 8 blocks, as in the case of JPEG encoding.
Block into blocks of × 8 pixels. DCT circuit 62
By sequentially reading blocks of the image data blocked by the blocking circuit 61 and performing DCT conversion,
It is a block of CT coefficients. The quantization circuit 63 uses the DCT
The blocks of DCT coefficients obtained in the circuit 62 are sequentially read and quantized in accordance with the same quantization table used for JPEG encoding to form blocks including quantized DCT coefficients. Then, the inverse quantization circuit 6
4 sequentially reads blocks of quantized DCT coefficients from the quantization circuit 63 and inversely quantizes them to obtain DCT coefficients for each block as student data.

After that, the prediction tap extracting circuit 65 sequentially sets the blocks of the image data divided by the blocking circuit 61 as the target block. Then, in step S32, the prediction tap extraction circuit 65 selects, as the pixel of interest, a pixel of the block of interest that has not yet been set as the pixel of interest in the raster scan order, and the position information of the pixel of interest is set to the mode control unit 71. Output to step S33.

In step S33, the mode control unit 71
Determines the pixel position mode #n of the target pixel based on the position information from the prediction tap extraction circuit 65, supplies the pixel position mode #n to the normal equation addition circuit 68, and proceeds to step S34.

In step S34, the class tap extraction circuit 66 reads the DCT coefficient used for classifying the pixel of interest from the inverse quantization circuit 64, and forms a class tap. That is, here, the class tap extraction circuit 64 extracts the 64 DCT coefficients of the block including the pixel of interest in the raster scan order, similarly to the class tap extraction circuit 42 of FIG. Let the sequence be the class tap of the pixel of interest. The class tap of the pixel of interest is supplied from the class tap extraction circuit 66 to the class classification circuit 67.

In step S35, the class classification circuit 67 classifies the pixel of interest using the class tap from the class tap extraction circuit 66, and outputs the resulting class code to the normal equation addition circuit 68.

On the other hand, the prediction tap extraction circuit 65 reads the DCT coefficient used for predicting the pixel value of the target pixel from the inverse quantization circuit 64 in step S36, and configures it as a prediction tap. That is, the prediction tap extraction circuit 6
5 extracts the 64 DCT coefficients of the block including the pixel of interest, for example, in raster scan order, and uses the sequence of the 64 DCT coefficients as the prediction tap of the pixel of interest. This prediction tap is supplied from the prediction tap extraction circuit 65 to the normal equation addition circuit 68.

Here, in the learning device of FIG. 8, as in the case of the coefficient conversion circuit 32 of FIG. 5, the same prediction taps are formed for all the pixels of each block for each block. In the process of configuring the prediction tap in step S36,
It is not necessary to perform it for the remaining 63 pixels if it is performed only for the pixel of interest first. This is
The same applies to the process of configuring the class tap in step S34 and the class classification process of step S35.

Then, in step S37, the normal equation adding circuit 68 reads out the target pixel as the teacher data (target teacher data) from the blocking circuit 61,
For the prediction tap (the DCT coefficient that constitutes the prediction tap) as the student data from the prediction tap extraction circuit 65 and the target pixel as the teacher data, the matrix A and the vector v of the equation (8) are added as described above. Is performed for each class code of the pixel of interest from the class classification circuit 67 and for each pixel position mode of the pixel of interest from the mode control unit 71.

Then, in step S38, the prediction tap extracting circuit 65 determines whether or not addition has been performed with all the pixels of the target block as the target pixels.
If it is determined in step S38 that all pixels of the target block have not been added as target pixels, the process returns to step S32, and the prediction tap extraction circuit 65 performs raster scan among the pixels of the target block. In this order, a pixel which has not yet been set as a target pixel is newly set as a target pixel, and the same processing is repeated thereafter.

If it is determined in step S38 that all pixels of the target block have been added as target pixels, the process proceeds to step S39, where the prediction tap extraction circuit 65 is blocked by the blocking circuit 61. It is determined whether or not all the blocks of the learned image data for learning have been processed as the block of interest. When it is determined in step S39 that all blocks of the learning image data have not been processed as the blocks of interest, the prediction tap extraction circuit 65 determines that all the blocks of the image data for learning have not been processed. A block that is not a block is newly set as a block of interest, the process returns to step S32, and the same process is repeated thereafter.

On the other hand, if it is determined in step S39 that all the blocks of the image data for learning have been processed as the block of interest, the normal equation adding circuit 68
Supplies the normal equation for each pixel position mode for each class to the tap coefficient determination circuit 69, and in step S4.
Go to 0.

In step S40, the tap coefficient determination circuit 69 solves the normal equation for each pixel position mode of each class, thereby giving 64 sets of taps corresponding to each of the 64 pixel position modes of the class. The coefficient is obtained, supplied to the address corresponding to each class in the coefficient table storage unit 70 to be stored therein, and the processing is ended.

As described above, the coefficient table storage unit 7
The tap coefficient for each class stored in 0 is stored in the coefficient table storage unit 44 of FIG.

Therefore, the tap coefficient stored in the coefficient table storage unit 44 has a prediction error (square error here) of the prediction value of the original pixel value obtained by performing the linear prediction calculation using the DCT coefficient. , Which is obtained by learning so as to be statistically minimized. As a result, according to the coefficient conversion circuit 32 of FIG. 5, the JPEG encoded image is as close as possible to the original image. Can be decoded into images.

Further, as described above, the decoding process for the JPEG-encoded image and the process for improving the image quality are performed at the same time, so that the JPEG-encoded image can be efficiently converted. Therefore, a decoded image with high image quality can be obtained.

By the way, in the coefficient conversion circuit 32 of FIG.
As shown in FIG. 10A, in each block, there are 64 pixel position modes of 0 to 63, and in the product-sum calculation circuit 45, using the tap coefficient for each pixel position mode, Equation (1) Since the linear prediction calculation is performed, even if the target pixel is not classified, that is, even if the total number of classes is one, 64 sets of tap coefficients are required.

Further, in the coefficient conversion circuit 32 of FIG.
As shown in 0 (B), in the prediction tap extraction circuit 41, all 64 DCT coefficients of the block including the pixel of interest (block of interest) are extracted in raster scan order,
Since the sequence of the 64 DCT coefficients is used as a prediction tap, it is possible to perform the linear prediction calculation of Expression (1) using the DCT coefficient (hereinafter, appropriately referred to as DCT coefficient tap) forming the prediction tap. Requires 64 tap coefficients.

That is, assuming that the pixel position mode of a certain pixel in the block is #n, the product-sum calculation circuit 4 of FIG.
5, the predicted value L _n of the pixel value of that pixel is obtained according to the following equation corresponding to equation (1).

L _n = F _{n, 0} × f ₀ + F _{n, 1} × f ₁ + ... + F _{n, 63} × f ₆₃ (9)

However, f _i-1 is 64 as a prediction tap.
It represents the i-th DCT coefficient from the beginning of the sequence of DCT coefficients. Further, F _{n, i−1} is multiplied by the _i- th DCT coefficient f _i−1 from the beginning of the sequence of 64 DCT coefficients as prediction taps in the set of tap coefficients in the pixel position mode #n. Represents the tap coefficient.

In equation (9), pixel position mode #n
Can take 64 values from 0 to 63. Therefore, even if the total number of classes is 1, the number of tap coefficients is 64 × 64 = 4096. Further, when the target pixel is classified into classes, 4096 × C tap coefficients are required, where C is the total number of classes. Therefore, at least the coefficient table storage unit 44 of FIG. 5 needs to have a storage capacity capable of storing such a number of tap coefficients.

In the learning device, for each class,
A normal equation is established for each of the 64 pixel position modes. Therefore, as compared with the case where the position of each pixel in the block is not considered, that is, the pixel position mode is only one mode, the set of teacher data and student data is
It is distributed to the normal equation for each of the 64 pixel position modes, which reduces the learning efficiency to 1/64.

Therefore, FIG. 11 shows a configuration example of the first embodiment of the coefficient conversion circuit 32 to which the present invention is applied.
In the figure, parts corresponding to those in FIG. 5 are designated by the same reference numerals, and the description thereof will be given below.
Omitted as appropriate. That is, the coefficient conversion circuit 32 of FIG. 11 is basically the same as the case of FIG. 5 except that the mode control unit 81 is provided instead of the mode control unit 46 and the code control unit 82 is newly provided. Is similarly configured.

The mode control unit 81 determines the pixel position mode based on the position information of the pixel of interest supplied from the prediction tap extraction circuit 41, as in the case of the mode control unit 46 of FIG. Furthermore, the mode control unit 81
The pixel position mode of the pixel of interest is degenerated to, for example, 1/4, and a degenerate pixel position mode that is a pixel position mode in which the total number is smaller than the number of pixels in the block (total number is 1/4) is obtained, and the sum of products is calculated. It is supplied to the arithmetic circuit 45. Further, the mode control unit 81 obtains area information of a pixel of interest, which will be described later, and controls the code control unit 82 based on the area information.

The code control unit 82 is supplied with the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41. The code control unit 82 follows the control of the mode control unit 81 and performs prediction. The sign of the DCT coefficient forming the prediction tap from the tap extraction circuit 41 is operated and output to the product-sum calculation circuit 45.

Here, in the embodiment of FIG. 11, the pixel position mode is degenerated (reduced) to 1/4 in the mode control unit 81 to be the degenerated pixel position mode. Then, the degenerate pixel position mode is supplied from the mode control unit 81 to the product-sum calculation circuit 45, and the product-sum calculation circuit 45 is supplied.
Then, the linear prediction calculation of Expression (1) is performed using the set of tap coefficients corresponding to the degenerate pixel position mode of the pixel of interest.

Therefore, in the embodiment of FIG. 11, it suffices that the tap coefficient exists for each degenerate pixel position mode for each class, and the degenerate pixel position mode here is 1/4 of the total number of pixel position modes. , That is, 16 modes (= 6
Since 4 modes / 4), the total number of tap coefficients can be reduced.

That is, the coefficient table storage unit 44 of FIG. 11 stores a set of 16 tap coefficients for each degenerate pixel position mode, that is, 16 sets of tap coefficients for each class. So for each class,
A set of tap coefficients for each of 64 pixel position modes (64
As compared with the case of FIG. 5 in which the tap coefficient of the set) is stored, it is possible to use the coefficient table storage unit 44 having a small storage capacity.

Next, referring to FIG. 12, the mode control unit 8
The degeneracy of the pixel position mode by 1 (conversion of the pixel position mode to the degenerate pixel position mode) and the operation of the sign of the prediction tap by the sign control unit 82 will be described.

The mode control unit 81 converts the pixel position mode of the target pixel into the degenerate pixel position mode depending on which of the plurality of regions in the target block the target pixel is located in.

That is, the mode control section 81 determines which of the four 4 × 4 regions of upper left, upper right, lower left and lower right obtained by dividing the target block into two equal parts in the vertical direction and the horizontal direction. It is determined whether the pixel of interest is located in the area.

When the area in which the pixel of interest is located (hereinafter referred to as the area of interest as appropriate) is the upper left area (upper left area) of the block of interest, the mode control unit 81 provides information indicating the position of the pixel of interest in the block of interest. Is output as the degenerate pixel position mode. That is, in this case, the mode control unit 81
For example, as shown in FIG. 12 (A-1), depending on which of the 4 × 4 upper left area of the target block the target pixel is located, among the degenerate pixel position modes of 16 modes of 0 to 15 Is output as the degenerate pixel position mode of the pixel of interest. Here, in the embodiment of FIG. 12A-1, in the raster scan order of the 4 × 4 upper left region,
Degenerate pixel position modes 0 to 15 are assigned. That is, for example, if the pixel at the x-th position from the left and the y-th pixel from the top of the block of interest is represented by p (x, y), the pixels p (1, 1), p located in the upper left region are represented.
(2,1), p (3,1), p (4,1), p (1,
2), p (2, 2), p (3, 2), p (4, 2), p
(1, 3), p (2, 3), p (3, 3), p (4
3), p (1,4), p (2,4), p (3,4), p
For (4, 4), degenerate pixel position modes # 0, #
1, # 2, # 3, # 4, # 5, # 6, # 7, # 8, #
9, # 10, # 11, # 12, # 13, # 14, # 15
Are output respectively.

Further, when the attention area is the upper left area, the mode control unit 81 determines the DCT which constitutes the prediction tap.
The code control unit 82 keeps the code of the coefficient as it is.
To control. Therefore, in this case, the code control unit 82 outputs the DCT coefficient as the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41 to the product-sum operation circuit 45 as it is, as shown in FIG. Supply.

[0157] product-sum operation circuit 45, the operation mode is degenerate pixel position mode #n 'when are the prediction taps, set W _n of the tap coefficients of the 16 sets corresponding to the class of the pixel of _interest' with the , (1) is performed, and the product-sum operation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the pixel of interest located in the upper left area is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = F _{n ′, 0} × f ₀ + F _{n ′, 1} × f ₁ + ... + F _{n ′, 63} × f ₆₃・ ・ ・ (10)

However, F _{n ', i-1} is the degenerate pixel position mode #
Of the set of n'tap coefficients, it represents the tap coefficient to be multiplied by the _i- th DCT coefficient f _i-1 from the beginning of the sequence of 64 DCT coefficients as prediction taps.

Next, when the attention area is the upper right area (upper right area) of the attention block, the mode control unit 81 divides the attention block into left and right lines (hereinafter appropriately referred to as vertical dividing lines). With respect to, the degenerate pixel position mode, which is information indicating the position of the pixel in the upper left region of the block of interest that is line-symmetric with the pixel of interest, is output as the degenerate pixel position mode of the pixel of interest. That is, the mode control unit 81
For example, as shown in FIG. 12 (B-1), depending on which of the 4 × 4 upper right area of the target block the target pixel is located, which of the 16 degenerate pixel position modes from 0 to 15 is selected. Is output as the degenerate pixel position mode of the pixel of interest. Therefore, the pixel p located in the upper right area
(8,1), p (7,1), p (6,1), p (5
1), p (8, 2), p (7, 2), p (6, 2), p
(5, 2), p (8, 3), p (7, 3), p (6
3), p (5,3), p (8,4), p (7,4), p
For (6,4) and p (5,4), the degenerate pixel position modes # 0 to # 15 are the same as in the case of the pixels p (1,1) to p (4,4) located in the upper left region. , Respectively.

Further, when the attention area is the upper right area, the mode control unit 81 determines the DCT which constitutes the prediction tap.
The sign control unit 82 is controlled to invert the sign of the coefficient located in the even column (the leftmost column is the first column) of the block of interest. Therefore, in this case, the code control unit 8
2 is a DCT coefficient included in the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41, and FIG.
The sign of the even-numbered column of the block of interest shown shaded in (B-2) is inverted and supplied to the product-sum operation circuit 45.

[0163] product-sum operation circuit 45, the operation mode is degenerate pixel position mode #n 'when are the prediction taps, set W _n of the tap coefficients of the 16 sets corresponding to the class of the pixel of _interest' with the , (1) is performed, and the product-sum operation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the pixel of interest located in the upper right area is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = F _{n ′, 0} × f ₀ + F _{n ′, 1} × (−f ₁ ) + ... + F _{n ′, 63} × (−f ₆₃ ) ... (11)

In the equation (11), among the 64 DCT coefficients of the block of interest forming the prediction tap,
By multiplying (-1) by the one located in the even column (hereinafter, appropriately referred to as the even column DCT coefficient),
The sign of the DCT coefficient of the even column is inverted (changed from + to − or − to +).

Here, when the attention area is the upper right area, the mode control unit 81, as described above, the pixels in the upper left area which are line-symmetrical to the attention pixel with respect to the vertical dividing line (hereinafter referred to as appropriate). The pixel position mode is degenerated by adopting the degenerate pixel position mode of the vertical line symmetric pixel) as the degenerate pixel position mode of the target pixel.

As described above, when the degenerate pixel position mode of the pixel in the upper right area is also used as the degenerate pixel position mode of the vertical line symmetric pixel, if the classification is not performed, the pixel in the upper right area, The vertically axisymmetric pixel will be predicted using the same set of tap coefficients.

That is, the pixel in the upper right region and its vertical line symmetric pixel have a left-right symmetrical positional relationship with respect to the vertical dividing line, and the pixel value of the pixel in the upper right region has a left-right symmetrical relationship. Prediction is performed using a set of tap coefficients corresponding to the same degenerate pixel position mode as the vertical line-symmetrical pixel. Therefore, the cosine waveform that is the basis of the DCT coefficient at the pixel position in the upper right region is assigned the cosine waveform that is the left and right of the cosine waveform that is the basis of the DCT coefficient at the position of the vertically axisymmetric pixel. Therefore, when predicting the pixel value of the pixel in the upper right area, the sign of the even-numbered column DCT coefficient is inverted as shown in Expression (11).

That is, FIG. 13 shows the cosine waveform (FIG. 13A) which is the basis of f ₁ which is _one of the DCT coefficients which is the even column DCT coefficient and the DCT coefficient which is not the even column DCT coefficient. FIG. 13B shows a cosine waveform (FIG. 13B) which is the basis of f ₂ which is one of the coefficients.

The base cosine waveform of the even column DCT coefficient f ₁ shown by the solid line in FIG. 13A becomes the waveform shown by the dotted line in FIG. 13A when the left and right thereof are inverted. The waveform) is the original cosine waveform (solid line waveform) with the sign inverted (multiplied by -1).

On the other hand, the cosine waveform of the base of the DCT coefficient f ₂ which is not the even-numbered DCT coefficient shown by the solid line in FIG. 13B becomes the waveform shown by the solid line in FIG. Match.

Therefore, in the spatial domain, associating the pixel in the upper right region with the vertical line-symmetrical pixel is equivalent to inverting the sign of the cosine waveform of the base of the even column DCT coefficient in the frequency domain.

Therefore, when the degenerate pixel position mode of pixels in the upper right region is also used as the degenerate pixel position mode of vertical line symmetric pixels to predict the pixel value of the pixel in the upper right region, the formula (11) is used. As shown, the sign of the even column DCT coefficients is inverted.

As described above, the base cosine waveform of a DCT coefficient that is not an even column DCT coefficient matches the original cosine waveform even if its left and right sides are inverted, so that even column D
For DCT coefficients that are not CT coefficients, it is not necessary to invert the sign.

Next, when the attention area is the lower left area of the attention block (lower left area), the mode control unit 81 divides the attention block into upper and lower halves (hereinafter appropriately referred to as horizontal dividing line). With respect to, the degenerate pixel position mode, which is information indicating the position of the pixel in the upper left region of the block of interest that is line-symmetric with the pixel of interest, is output as the degenerate pixel position mode of the pixel of interest. That is, the mode control unit 81
For example, as shown in FIG. 12C-1, depending on which of the 4 × 4 lower left area of the target block the target pixel is located, any one of the 16-mode degenerate pixel position modes 0 to 15 is selected. Is output as the degenerate pixel position mode of the pixel of interest. Therefore, the pixel p located in the lower left area
(1,8), p (2,8), p (3,8), p (4
8), p (1,7), p (2,7), p (3,7), p
(4, 7), p (1, 6), p (2, 6), p (3
6), p (4, 6), p (1, 5), p (2, 5), p
For (3,5) and p (4,5), the degenerate pixel position modes # 0 to # 15 are set in the same manner as the case of the pixels p (1,1) to p (4,4) located in the upper left area. , Respectively.

Further, when the attention area is the lower left area, the mode control unit 81 determines the DCT which constitutes the prediction tap.
The sign control unit 82 is controlled to invert the sign of the coefficient located in the even-numbered row (the uppermost row is the first row) of the block of interest. Therefore, in this case, the code control unit 8
2 is a DCT coefficient included in the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41, and FIG.
The sign of the even block of the target block indicated by (C-2) shaded is inverted and supplied to the product-sum calculation circuit 45.

[0178] product-sum operation circuit 45, the operation mode is degenerate pixel position mode #n 'when are the prediction taps, set W _n of the tap coefficients of the 16 sets corresponding to the class of the pixel of _interest' with the , (1) is performed, and the product-sum operation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the pixel of interest located in the lower left area is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = F _{n ′, 0} × f ₀ + F _{n ′, 1} × f ₁ + ... + F _{n ′, 63} × (−f ₆₃ ) ... (12)

In the equation (12), among the 64 DCT coefficients of the block of interest constituting the prediction tap,
By multiplying (-1) by the one located in an even-numbered row (hereinafter, appropriately referred to as even-numbered row DCT coefficient),
The sign of the even row DCT coefficient is inverted.

Here, when the attention area is the lower left area, the mode control unit 81, as described above, the pixels in the upper left area (hereinafter, appropriately, with respect to the horizontal dividing line, which are line symmetrical to the attention pixel). The degenerate pixel position mode (referred to as a horizontal line symmetric pixel) is adopted as the degenerate pixel position mode of the target pixel, thereby degenerating the pixel position mode.

As described above, when the degenerate pixel position mode of the pixel in the lower left area is also used as the degenerate pixel position mode of the horizontal line symmetrical pixel, if the classification is not performed, the pixel in the lower left area and its Horizontally symmetric pixels will be predicted using the same set of tap coefficients.

That is, the pixel in the lower left area and the horizontal line symmetrical pixel have a positional relationship that is vertically symmetrical with respect to the horizontal dividing line, and the pixel values of the pixel in the lower left area have a vertically symmetrical positional relationship. Prediction is performed using a set of tap coefficients corresponding to the same degenerate pixel position mode as the horizontal symmetric pixel. Therefore, the cosine waveform that is the basis of the DCT coefficient at the pixel position in the lower left region is assigned the cosine waveform that is the inverted cosine waveform that is the basis of the DCT coefficient at the position of the horizontal axisymmetric pixel. Therefore, when predicting the pixel value of the pixel in the lower left area, the sign of the even-numbered row DCT coefficient is inverted as shown in Expression (12).

That is, when the upper and lower sides of the base cosine waveform of the even row DCT coefficient are inverted, the sign of the original cosine waveform is inverted as in the case of the even column DCT coefficient shown in FIG. 13A. (Multiplied by -1).

On the other hand, when the cosine waveform of the base of the DCT coefficient which is not the even row DCT coefficient is inverted vertically,
Similar to the case of the DCT coefficient that is not the even-numbered DCT coefficient shown in FIG. 3B, the waveform matches the original waveform.

Therefore, in the spatial domain, associating the pixel in the lower left area with the horizontal line-symmetrical pixel is equivalent to inverting the sign of the base cosine waveform of the even row DCT coefficient in the frequency domain.

For this reason, when the degenerate pixel position mode of the pixel in the lower left area is also used as the degenerate pixel position mode of the horizontal line symmetric pixel to predict the pixel value of the pixel in the lower left area, it is expressed by equation (12). As described above, the sign of the even row DCT coefficient is inverted.

As described above, the base cosine waveform of the DCT coefficient that is not the even row DCT coefficient matches the original cosine waveform even if it is inverted upside down.
For DCT coefficients that are not CT coefficients, it is not necessary to invert the sign.

Next, when the attention area is the lower right area (lower right area) of the attention block, the mode control unit 81 causes the intersection point of the vertical division line and the horizontal division line of the attention block (hereinafter,
The degenerate pixel position mode, which is information indicating the position of the pixel in the upper left region of the target block that is point-symmetric with respect to the target pixel, is output as the degenerate pixel position mode of the target pixel. That is, for example, as shown in FIG. 12 (D-1), the mode control unit 81 selects 16 modes from 0 to 15 depending on which of the 4 × 4 lower right region of the target block the target pixel is located. Any one of the degenerate pixel position modes of is output as the degenerate pixel position mode of the pixel of interest. Therefore, the pixels p (8,8), p (7,8), p (6,6) located in the lower right area
8), p (5, 8), p (8, 7), p (7, 7), p
(6,7), p (5,7), p (8,6), p (7,
6), p (6, 6), p (5, 6), p (8, 5), p
For (7,5), p (6,5), p (5,5),
Pixels p (1,1) to p (4,4) located in the upper left area
In the same manner as in the above, the degenerate pixel position modes # 0 to #
15 are output respectively.

Further, when the attention area is the lower right area, the mode control unit 81 makes the DCT constituting the prediction tap.
Of the coefficients, the sign control unit 82 is controlled so that the sign of the even-numbered column DCT coefficient is inverted and the sign of the even-numbered row DCT coefficient is inverted. Therefore, in this case, the code control unit 8
2 is a DCT coefficient included in the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41, and FIG.
Even-numbered column DC of the block of interest shown by shadowing (D-2)
The signs of only the T coefficient and those of the even row DCT coefficient are inverted and supplied to the product-sum operation circuit 45. In this case, the DC that constitutes the prediction tap
Of the T coefficients, those that are also even column DCT coefficients and even row DCT coefficients have their signs inverted twice, so that they are eventually returned to their original signs.

[0192] product-sum operation circuit 45, the operation mode is degenerate pixel position mode #n 'when are the prediction taps, set W _n of the tap coefficients of the 16 sets corresponding to the class of the pixel of _interest' with the , (1) is performed, and the product-sum operation result is used as the decoding result of the pixel of interest.

[0193] Therefore, when the degenerate pixel position mode of the target pixel located in the lower right area and #n ', in the product-sum operation circuit 45 of FIG. 11, the predicted value L _n of the pixel value of the pixel is,
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = F _{n ′, 0} × f ₀ + F _{n ′, 1} × (−f ₁ ) + ... + F _{n ′, 63} × f ₆₃・ ・ ・ (13)

In equation (13), among the 64 DCT coefficients of the block of interest forming the prediction tap,
For both even column DCT coefficients and even row DCT coefficients,
By being multiplied by (−1), the sign of the DCT coefficient which is only one of the even column DCT coefficient or the even row DCT coefficient is inverted.

Here, when the attention area is the lower right area, the mode control unit 81, as described above, the pixels in the upper left area (hereinafter, appropriately referred to as the center point of the block) are symmetrical with respect to the attention pixel. , A point-symmetrical pixel) is adopted as the degenerate pixel position mode of the target pixel, thereby degenerating the pixel position mode.

As described above, when the degenerate pixel position mode of the pixel in the lower right area is also used as the degenerate pixel position mode of the point-symmetrical pixel, if the class classification is not performed, the pixel in the lower right area is regarded as the pixel. , The point-symmetrical pixel will be predicted using the same set of tap coefficients.

That is, the pixel in the lower right area and the point-symmetrical pixel have a positional relationship that is point-symmetrical with respect to the block center point, that is, moves in line symmetry with respect to the vertical dividing line, and , The pixel value of the pixel in the lower right area has a positional relationship that is moved in line symmetry with respect to the horizontal dividing line, and corresponds to the same degenerate pixel position mode as the point-symmetrical pixel whose positional relationship is point symmetric. Is predicted using a set of tap coefficients that Therefore, the cosine waveform that is the basis of the DCT coefficient at the pixel position in the lower right region is the cosine waveform that is the inverted cosine waveform that is the basis of the DCT coefficient at the point-symmetrical pixel position, and that is the inverted cosine waveform. Will be assigned. Therefore, when predicting the pixel value of the pixel in the lower right area, the sign of the even-numbered column DCT coefficient is inverted and the sign of the even-numbered row DCT coefficient is inverted as shown in Expression (13). There is.

That is, as described above, when the right and left sides of the base cosine waveform of the even-numbered column DCT coefficient are inverted, the sign of the original cosine waveform is inverted, and even-numbered row DCT coefficients are inverted.
When the upper and lower sides of the cosine waveform of the base of the T coefficient are inverted, the sign of the original cosine waveform is inverted.

Therefore, in the spatial domain, associating the pixel in the lower right region with the point-symmetrical pixel inverts the sign of the base cosine waveform of the even-row DCT coefficient in the frequency domain, and also makes the even-row DCT coefficient It is equivalent to inverting the sign of the cosine waveform of the basis of.

Therefore, when predicting the pixel value of the pixel in the lower left area, as shown in equation (13), the even column DC
The signs of the DCT coefficient which is only the T coefficient and the DCT coefficient which is only the even row DCT coefficient are inverted.

For the DCT coefficient which is both the even-column DCT coefficient and the even-row DCT coefficient, the sign is inverted twice to return to the original sign. Therefore, it is not necessary to invert the sign. Absent.

In equations (10) to (13), the degenerate pixel position mode #n 'can take 16 values from 0 to 15, so if the total number of classes is one, the required tap coefficient Is 64 × 16 = 1024. Therefore, as compared with the case where the pixel position mode is not degenerated (the total number of required tap coefficients is 4096 as described above), the coefficient table storage unit 44 has a small storage capacity (simply, FIG. Of the storage capacity of 1/4).

In the above case, the code control unit 82 operates the code of the DCT coefficient forming the prediction tap output from the prediction tap extraction circuit 41.
In the sign control unit 82, the sign of the tap coefficient used by the product-sum calculation circuit 45 is manipulated, and in the product-sum calculation circuit 45, the prediction formulas of Expressions (10) to (13) are substantially calculated. It is also possible to do so.

Next, FIG. 14 shows the mode control unit 8 of FIG.
1 shows a configuration example of No. 1.

The area determination section 91 is supplied with the position information of the pixel of interest output from the prediction tap extraction circuit 41 (FIG. 11), and the area determination section 91 draws attention based on the position information. Determine the area. That is, the area determination unit 9
1 determines whether the pixel of interest is located in the upper left region, upper right region, lower left region, or lower right region of the block of interest. Then, the region determination unit 91 outputs the determination result of the attention region as region information (information indicating whether the attention region is the upper left region, the upper right region, the lower left region, or the lower right region of the attention block). To do. This area information is supplied to the mode degeneration unit 93 and also to the code control unit 82 (FIG. 11). Then, in the code control unit 82,
Based on this area information, the operation (control) of the code of the prediction tap as described in FIG. 12 is performed.

The position information of the pixel of interest output from the prediction tap extraction circuit 41 (FIG. 11) is supplied to the pixel position mode determination unit 92, and the pixel position mode determination unit 92 receives the position information. On the basis of the above, it is determined which of the 64 pixel position modes shown in FIG. 6 the pixel position mode of the pixel of interest is, and the result is supplied to the mode degeneracy unit 93.

The mode degeneracy unit 93 degenerates the pixel position mode of the pixel of interest supplied from the pixel position mode determination unit 92 based on the region information on the pixel of interest supplied from the region determination unit 91, and degenerates the pixel position mode. Is output.

That is, when the attention area is the upper left area, the mode degeneracy section 93 corresponds to the position of the attention pixel represented by the pixel position mode output by the pixel position mode determination section 92, as shown in FIG. Any one of the values 0 to 15 shown in is output as the degenerate pixel position mode.

Further, when the attention area is the upper right area, the mode degenerating section 93 corresponds to the position of the attention pixel indicated by the pixel position mode output from the pixel position mode determination section 92, as shown in FIG. 12 (B-1). Any one of the values 0 to 15 shown in is output as the degenerate pixel position mode. Further, when the attention area is the lower left area, the mode degeneration unit 93 corresponds to the position of the attention pixel indicated by the pixel position mode output by the pixel position mode determination unit 92, and is illustrated in FIG. 12 (C-1). When any one of 0 to 15 is output as the degenerate pixel position mode and the attention area is the lower right area, it corresponds to the position of the attention pixel represented by the pixel position mode output by the pixel position mode determination unit 92. Then, one of the values 0 to 15 shown in FIG. 12D-1 is output as the degenerate pixel position mode.

The degenerate pixel position mode of the pixel of interest output from the mode degenerating unit 93 is supplied to the product-sum operation circuit 45 (FIG. 11), and the product-sum operation circuit 45 obtains the tap coefficient corresponding to the degenerate pixel position mode. Then, the linear prediction calculation of Expression (1) is performed.

Next, the processing of the coefficient conversion circuit 32 of FIG. 11 will be described with reference to the flowchart of FIG.

The quantized DCT coefficient for each block output from the entropy decoding circuit 31 (FIG. 3) is supplied to the dequantization circuit 40, and the dequantization circuit 40 proceeds to step S51.
, The quantized DC for each block supplied thereto
Dequantize the T coefficient. The DCT coefficients for each block obtained by the inverse quantization circuit 40 performing inverse quantization are sequentially supplied to the prediction tap extraction circuit 41 and the class tap extraction circuit 42.

The prediction tap extracting circuit 41 sequentially sets the blocks of DCT coefficients supplied from the inverse quantizing circuit 40 as the target block. Then, the prediction tap extraction circuit 41
In step S52, among the pixels of the block of interest, those in the raster scan order that have not yet been selected as the pixel of interest are selected as the pixel of interest, and the position information of the pixel of interest is output to the mode control unit 81, and in step S53. Proceed to.

At step S53, the mode controller 81
Area determination unit 91 and pixel position mode determination unit 9 of FIG.
2 determines the area information of the pixel of interest and the pixel position mode #n based on the position information from the prediction tap extraction circuit 41, and proceeds to step S54. In step S54, the mode degeneracy unit 93 of the mode control unit 81 degenerates the pixel position mode #n of the pixel of interest based on the area information, and obtains a degenerate pixel position mode #n '. Then, the area information of the target pixel is obtained from the area determination unit 91 to the code control unit 82.
And the degenerate pixel position mode #n ′ of the pixel of interest is
It is supplied from the mode degeneracy unit 93 to the product-sum calculation circuit 45.

After that, in step S55, the class tap extraction circuit 42 extracts the DCT coefficients supplied from the inverse quantization circuit 40, which are used for classifying the pixel of interest, to form a class tap. To do. That is, here, the class tap extraction circuit 42, for example,
Similar to the case in step S14 of FIG. 7, a class tap of the pixel of interest is configured and supplied to the class classification circuit 43.

In step S56, the class classification circuit 43 classifies the pixel of interest using the class tap from the class tap extraction circuit 42, and outputs the resulting class code to the coefficient table storage unit 44.

Upon receiving the class code from the class classification circuit 43, the coefficient table storage section 44 receives the step S5.
In 7, the 16 sets of tap coefficients stored at the address corresponding to the class code are read out and output to the product-sum operation circuit 45.

That is, the coefficient table storage unit 44 of FIG.
As described above, 16 sets of tap coefficients, which are the total number of degenerate pixel position modes, are stored for each class, and 16 sets of tap coefficients are stored at the address corresponding to the class code of the pixel of interest. The tap coefficient of is read and output to the product-sum calculation circuit 45.

Then, proceeding to step S58, the prediction tap extraction circuit 41 extracts the DCT coefficient used for predicting the pixel value of the target pixel, and configures it as a prediction tap.
That is, the prediction tap extraction circuit 41 configures a prediction tap of the pixel of interest and supplies the prediction tap to the code control unit 82, for example, as in the case of step S17 of FIG. 7.

Further, in step S58, the code control unit 82 operates the code of the prediction tap from the prediction tap extraction circuit 41 based on the area information of the pixel of interest from the mode control unit 81.

That is, when the region information indicates that the region of interest is the upper left region, the code control unit 82 outputs from the prediction tap extraction circuit 41 as described in FIG. 12 (A-2). The prediction tap is directly supplied to the product-sum calculation circuit 45.

When the region information indicates that the region of interest is the upper left region, the code control unit 82 outputs the prediction tap extraction circuit 41 from the prediction tap extraction circuit 41 as described with reference to FIG. 12B-2. Of the DCT coefficients forming the prediction tap, the sign of the DCT coefficient in the even column is inverted, and the product-sum operation circuit 4
Supply to 5.

Further, when the region information indicates that the region of interest is the upper left region, the code control unit 82 outputs from the prediction tap extraction circuit 41 as described in FIG. 12 (C-2). Of the DCT coefficients forming the prediction tap, the sign of the DCT coefficient in the even-numbered row is inverted, and the product-sum operation circuit 4
Supply to 5.

When the region information indicates that the region of interest is the upper left region, the code control unit 82 outputs from the prediction tap extraction circuit 41 as described with reference to FIG. 12D-2. Of the DCT coefficients forming the prediction tap, the sign of the DCT coefficient that is only the even-column DCT coefficient or the even-row DCT coefficient is inverted, and the product-sum operation circuit 45
Supply to.

In step S59, the product-sum calculation circuit 45 sets the tap coefficient corresponding to the degenerate pixel position mode #n 'of the pixel of interest among the 16 sets of tap coefficients output from the coefficient table storage section 44 in step S57. Is obtained, the process proceeds to step S60, and the product-sum operation shown in equation (1) is performed using the set of tap coefficients and the prediction tap supplied from the code control unit 82 in step S58 to obtain the pixel of interest. Get the decoded value of the pixel value of.

Then, in step S61, the prediction tap extracting circuit 41 determines whether or not all the pixels of the target block have been processed as the target pixel. When it is determined in step S61 that all the pixels of the target block have not been processed as the target pixel, the process returns to step S52, and the prediction tap extracting circuit 41
Is the pixel of the block of interest, in raster scan order,
The same process is repeated from now on, assuming that the pixel not yet set as the target pixel is newly set as the target pixel.

If it is determined in step S61 that all the pixels of the target block have been processed as the target pixel, that is, if the decoded values of all the pixels of the target block have been obtained, the product-sum operation circuit The 45 outputs a block (decoded block) composed of the decoded value to the block decomposition circuit 33 (FIG. 3), and proceeds to step S62.

In step S62, the prediction tap extraction circuit 41 determines whether or not all the blocks of DCT coefficients output by the inverse quantization circuit 40 have been processed as the block of interest. In step S62, the inverse quantization circuit 40
If it is determined that all the blocks of the DCT coefficients output by the block are not considered as the block of interest, the prediction tap extraction circuit 41 determines that the block of the DCT coefficients output by the inverse quantization circuit 40 is the block of interest. The block that has not been set is newly set as a block of interest, and step S5 is performed.
Returning to 2, the same processing is repeated thereafter.

If it is determined in step S62 that all the blocks of DCT coefficients output by the inverse quantization circuit 40 have been processed as the block of interest, the process ends.

As described above, the coefficient conversion circuit 32 of FIG.
Then, since the pixel position mode is degenerated, the total number of tap coefficients may be small, and as the coefficient table storage unit 44,
It becomes possible to efficiently decode a high-quality image by adopting one having a small storage capacity.

Next, FIG. 16 shows an example of the structure of a learning device for carrying out the learning process of the tap coefficient stored in the coefficient table storage section 44 of FIG. In the figure, parts corresponding to those in FIG. 8 are designated by the same reference numerals, and description thereof will be omitted below as appropriate. That is, FIG.
In the learning device of No. 6, a mode control unit 101 is provided instead of the mode control unit 71, and the code control unit 10
The configuration is basically the same as the case in FIG. 8 except that 2 is newly provided.

The mode control unit 101 is configured similarly to the mode control unit 81 (FIG. 14) of FIG. 11, and determines the pixel position mode based on the position information of the pixel of interest supplied from the prediction tap extraction circuit 65. Then, by further degenerating the pixel position mode, the degenerate pixel position mode is obtained and supplied to the normal equation adding circuit 68. In addition, the mode control unit 101 is also the mode control unit 81 of FIG.
Similarly, the area information of the target pixel is determined, and the code control unit 102 is controlled based on the area information.

The code control unit 102 is supplied with the prediction taps for the pixel of interest output from the prediction tap extraction circuit 65.
Similarly to the sign control unit 82, the sign of the DCT coefficient forming the prediction tap from the prediction tap extraction circuit 65 is operated under the control of the mode control unit 101, and is output to the normal equation addition circuit 68.

Next, the processing (learning processing) of the learning device in FIG. 16 will be described with reference to the flowchart in FIG.

In the learning process, first, in step S7.
1, the blocking circuit 61, the DCT circuit 62, the quantization circuit 63, and the dequantization circuit 64 JPEG-encode the learning image data, and further, in the same manner as in step S31 of FIG. From the digitized data,
A DCT coefficient that becomes student data is acquired.

After that, the prediction tap extracting circuit 65 sequentially sets the blocks of the image data divided by the blocking circuit 61 as the target block. Then, in step S72, the prediction tap extraction circuit 65 selects, as the pixel of interest, a pixel of the block of interest that has not yet been set as the pixel of interest in the raster scan order, and the position information of the pixel of interest is set to the mode control unit 101. Output to step S73.

At step S73, the mode control unit 101
Determines the pixel position mode #n of the pixel of interest and the region information based on the position information from the prediction tap extraction circuit 65, and proceeds to step S74. In step S74, the mode control unit 101 sets the pixel position mode #n of the target pixel to
Degenerate and degenerate pixel position mode based on the region information #
Find n '. Then, the mode control unit 101 supplies the degenerate pixel position mode #n ′ of the pixel of interest to the normal equation adding circuit 68, and the area information is also supplied to the code control unit 10.
2 and the process proceeds to step S75.

In step S75, the class tap extraction circuit 66 reads out the DCT coefficient used for classifying the pixel of interest from the inverse quantization circuit 64, for example, as shown in FIG.
In the same manner as in step S34, the class tap is constructed and supplied to the class classification circuit 67.

In step S76, the class classification circuit 67 classifies the pixel of interest using the class tap from the class tap extraction circuit 66 in the same manner as in step S35 of FIG. 9, and obtains the class code obtained as a result. Is output to the normal equation adding circuit 68.

On the other hand, the prediction tap extraction circuit 65 reads the DCT coefficient used for predicting the pixel value of the target pixel from the inverse quantization circuit 64 in step S77, and performs the same process as in step S36 of FIG. , Configure prediction taps. This prediction tap is supplied from the prediction tap extraction circuit 65 to the code control unit 102.

Further, in step S77, the code control unit 102 sets the code of the prediction tap from the prediction tap extraction circuit 65 to the mode control unit 101 in the same manner as the code control unit 82 does in step S58 of FIG. The pixel is operated based on the area information of the pixel of interest from and is supplied to the normal equation adding circuit 68, and the process proceeds to step S78.

In step S78, the normal equation adding circuit 68 reads out the target pixel as the teacher data from the blocking circuit 61, and the prediction taps (the DCT coefficients forming the prediction taps) as the student data from the prediction tap extracting circuit 65, And for the target pixel as the teacher data, the above-described addition of the matrix A and the vector v in Expression (8) is performed for each class code of the target pixel from the class classification circuit 67, and the mode control unit 101. Is performed for each degenerate pixel position mode of the target pixel from.

Then, in step S79, the prediction tap extracting circuit 65 determines whether or not addition has been performed by setting all the pixels of the target block as the target pixels.
If it is determined in step S79 that all the pixels in the target block have not been added as the target pixels, the process returns to step S72, and the prediction tap extraction circuit 65 selects the raster scan among the pixels in the target block. In this order, a pixel which has not yet been set as a target pixel is newly set as a target pixel, and the same processing is repeated thereafter.

If it is determined in step S79 that all pixels in the target block have been added as target pixels, the process proceeds to step S80, and the prediction tap extraction circuit 65 is blocked by the blocking circuit 61. It is determined whether or not all the blocks of the learned image data for learning have been processed as the block of interest. If it is determined in step S80 that all blocks of the learning image data have not been processed as the blocks of interest, the prediction tap extraction circuit 65 determines that all the blocks of the image data for learning have not been processed. A block that is not a block is newly set as a block of interest, the process returns to step S72, and the same process is repeated thereafter.

On the other hand, if it is determined in step S80 that all blocks of the image data for learning have been processed as the block of interest, the normal equation adding circuit 68
Supplies the normal equation for each pixel position mode for each class to the tap coefficient determination circuit 69, and step S8
Go to 1.

In step S81, the tap coefficient determination circuit 69 solves the normal equation for each degenerate pixel position mode of each class, thereby
Sixteen sets of tap coefficients corresponding to each of the six degenerate pixel position modes are obtained, supplied to the address corresponding to each class of the coefficient table storage unit 70, and stored therein, and the processing is ended.

As described above, in the learning device of FIG. 16, for each class, for each of the 16 degenerate pixel position modes,
A normal equation is created. Therefore, as compared with the case where the normal equation is set for each of the 64 pixel position modes as in the learning device of FIG. 8, the learning efficiency is four times (= (1/1
6) / (1/64)).

Next, in the coefficient conversion circuit 32 of FIG.
As shown in FIG. 18 (A), which is the same as 0 (A), there are 64 pixel position modes of 0 to 63 for each block. Also (even if the total number of classes is one class), 64 sets of tap coefficients are required.

Further, in the coefficient conversion circuit 32 of FIG.
As shown in FIG. 18 (B), which is the same as 0 (B), all the 64 DCT coefficients of the block including the pixel of interest (block of interest) are used as prediction taps. 64 tap coefficients are required to perform the linear prediction operation of).

By the way, the prediction tap is not limited to all the 64 DCT coefficients of the target block, but also other information,
That is, for example, as shown in FIG. 18C, four pixels a, b, and
It may be configured by using c and d as well.

Here, the pixel a is a pixel located in the same column as the target pixel among the eight pixels adjacent to the target block, and the pixel b is adjacent to the left of the target block.
Among the pixels, the pixels are located in the same row as the pixel of interest. The pixel c is a pixel located in the same row as the pixel of interest among the eight pixels adjacent to the right of the block of interest, and the pixel d is the pixel of interest among the eight pixels adjacent to the block of interest. It is a pixel located in the same column as the pixel.

The pixels a, b, c and d will be referred to as appropriate below.
Are referred to as upper adjacent block pixels, left adjacent block pixels, right adjacent block pixels, and lower adjacent block pixels, respectively.
Further, hereinafter, the pixels a to d are collectively referred to as an adjacent block pixel as appropriate.

As described above, when the prediction tap is composed of all 64 DCT coefficients of the block of interest and 4 pixels adjacent to the block of interest, the linear prediction calculation of the equation (1) requires 68. (= 64 + 4) tap coefficients are required.

That is, in this case, assuming that the pixel position mode of a certain pixel in the block is #n, the predicted value L _n of the pixel value of the pixel in the product-sum operation circuit 45 of FIG. 5 corresponds to the equation (1). It is calculated according to the following formula.

[0256] _{L n = A n × a +} B n × b + C n × c + D n × d + F n, 0 × f 0 + F n, 1 × f 1 + ··· + F n, 63 × f 63 ··· (14)

However, f _i-1 is 64 as a prediction tap.
It represents the i-th DCT coefficient from the beginning of the sequence of DCT coefficients. Further, F _{n, i−1} is multiplied by the _i- th DCT coefficient f _i−1 from the beginning of the sequence of 64 DCT coefficients as prediction taps in the set of tap coefficients in the pixel position mode #n. Represents the tap coefficient. Also, A _n , B _n , C _n , and D _n are multiplied by (a pixel value of) a, b, c, and d as a prediction tap in the set of tap coefficients in the pixel position mode #n. Each tap coefficient is shown.

In equation (14), pixel position mode #n
Can take 64 values from 0 to 63. Therefore, even if the total number of classes is 1, the number of tap coefficients is 64 × 68 = 4352. Further, when the target pixel is classified into classes, assuming that the total number of classes is C, 4352 × C tap coefficients are required. Therefore, at least the coefficient table storage unit 44 of FIG. 5 needs to have a storage capacity capable of storing such a number of tap coefficients.

That is, the prediction tap is set to 64 of the target block.
If not only all the DCT coefficients but also other information is used, the coefficient table storage unit 44 of FIG. 5 compares the prediction tap with only the 64 DCT coefficients of the block of interest. , Larger capacity is required.

Further, also in this case, the pixel position mode #n
Can take 64 values from 0 to 63, so
Compared with the case where the position of each pixel in the block is not considered (when the pixel position mode is only one mode), the learning efficiency in the learning device is reduced to 1/64.

Therefore, FIG. 19 shows a configuration example of the second embodiment of the coefficient conversion circuit 32 to which the present invention is applied.
Note that, in the figure, parts corresponding to those in FIG. 11 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 32 of FIG.
Is a tap structure storage unit 111 and an inverse DCT circuit 112.
The configuration is basically the same as the case in FIG. 11 except that is newly provided.

Area information is supplied to the tap structure storage section 111 from the mode control section 81.
The tap structure storage unit 111 controls the tap structure of the prediction tap formed by the prediction tap extraction circuit 41. That is, the tap structure storage unit 111 stores the attention area and the tap structure information indicating the tap structure of the prediction tap in association with each other, and associates the attention area with the attention area represented by the area information supplied from the mode control unit 81. The tap structure information stored is read and supplied to the prediction tap extraction circuit 41.

The inverse DCT circuit 112 includes an inverse quantization circuit 4
The DCT coefficient for each block output by 0 is supplied, and the inverse DCT circuit 112 performs inverse DCT conversion on the DCT coefficient for each block to obtain the pixel value for each block, and the prediction tap extraction circuit. 41.

In the embodiment of FIG. 19, the prediction tap extraction circuit 41 constitutes a prediction tap having a tap structure represented by the tap structure information supplied from the tap structure storage unit 111. Further, the prediction tap extraction circuit 41 extracts the DCT coefficient of the target block from the DCT coefficient supplied from the inverse quantization circuit 40, and the pixel supplied from the inverse DCT circuit 112 to the adjacent block pixel of the target pixel. a to d are extracted, and adjacent block pixels a to d are also used to configure a prediction tap having a tap structure represented by the tap structure information supplied from the tap structure storage unit 111.

Next, the control of the tap structure of the prediction tap by the tap structure information storage unit 111 will be described with reference to FIG.

When the region of interest is the upper left region, the mode control unit 81 is the same as that shown in FIG.
As shown in 1), depending on which of the 4 × 4 upper left area of the target block the target pixel is located, 0 to 15
One of the 16 degenerate pixel position modes is output as the degenerate pixel position mode of the pixel of interest.

On the other hand, the tap structure storage section 111 stores the 64 blocks of the target block in the sequence (of pixel values) of the upper adjacent block pixel a, the left adjacent block pixel b, the right adjacent block pixel c, and the lower adjacent block pixel d. The tap structure information indicating the sequence in which the sequences of DCT coefficients are connected is stored in association with the area information indicating that the attention area is the upper left area. Note that, here, the tap structure represented by the tap structure information associated with the area information indicating that the attention area is the upper left area is, for example, the default tap structure.

When the attention area is the upper left area, the prediction tap extraction circuit 41 extracts the DCT coefficient from the output of the inverse quantization circuit 40 according to the tap structure information representing the default tap structure as described above. With reverse DC
By extracting pixels from the output of the T circuit 112,
Configure a predictive tap with a default tap structure.

That is, the prediction tap extracting circuit 41 uses the inverse DC.
As shown in FIG. 20 (A-3), an upper adjacent block pixel a, a left adjacent block pixel b, a right adjacent block pixel c, and a lower adjacent block pixel d are extracted from the output of the T circuit 40,
The taps (a, b, c, d) arranged in that order are configured. Further, the prediction tap extraction circuit 41 extracts all 64 DCT coefficients of the target block from the output of the dequantization circuit 40 to form taps arranged in the line scan order. Then, the prediction tap extraction circuit 41 configures a prediction tap by connecting (arranging) DCT coefficient taps after the taps of adjacent block pixels, and the code control unit 82.
Supply to.

When the region of interest is the upper left region, the mode control unit 81, as described above, forms the prediction tap.
The sign control unit 82 is controlled so that the sign of the CT coefficient remains unchanged. Therefore, in this case, the code control unit 82
20 shows the prediction taps of the pixel of interest output by the prediction tap extraction circuit 41, which are the same as those in FIG.
As shown in (A-2), it is supplied to the product-sum calculation circuit 45 as it is.

When the operation mode is the degenerate pixel position mode #n ', the product-sum operation circuit 45 uses the prediction tap and the set W _n of the 16 sets of tap coefficients corresponding to the class of the pixel of interest. , Perform the sum of products operation of equation (1),
The product-sum calculation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the pixel of interest located in the upper left area is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = A _{n ′} × a + B _{n ′} × b + C _{n ′} × c + D _{n ′} × d + F _{n ′, 0} × f ₀ + F _{n ′, 1} × f ₁ ++ ... + F _{n ′, 63} × f ₆₃ (15)

However, A _{n ′} , B _{n ′} , C _{n ′} , and D _{n ′} are adjacent blocks forming a prediction tap of the default tap structure in the set of tap coefficients in the degenerate pixel position mode #n ′. Represents a tap coefficient that is multiplied by the pixel (pixel value thereof) a, b, c, d.

The expression (15) is the above expression (10).
On the right-hand side of, the terms A _{n ′} × a + B _{n ′} × b + C _{n ′} × c + D _{n ′} × regarding the adjacent block pixels a to d forming the prediction taps
d is added.

Next, when the attention area is the upper right area, the mode control unit 81 operates as shown in FIG.
As shown in (B-1), depending on which of the 4 × 4 upper right region of the target block the target pixel is located, one of the 16 modes of degenerate pixel position modes 0 to 15 is selected. Output as a pixel degenerate pixel position mode.

On the other hand, the tap structure storage section 111 stores the sequence of 64 DCT coefficients of the target block in the sequence of the upper adjacent block pixel a, the right adjacent block pixel c, the left adjacent block pixel b, and the lower adjacent block pixel d. The tap structure information indicating the connected sequence is stored in association with the area information indicating that the attention area is the upper right area. That is, the tap structure storage unit 111 uses the tap structure information representing the tap structure in which the left adjacent block pixel b and the right adjacent block pixel c are replaced in the default tap structure as the area information indicating that the attention area is the upper right area. It is associated and stored.

When the attention area is the upper right area, the prediction tap extraction circuit 41 extracts the DCT coefficient from the output of the inverse quantization circuit 40 according to the tap structure information representing the tap structure as described above, and Inverse DCT circuit 112
A prediction tap is constructed by extracting pixels from the output of

That is, the prediction tap extraction circuit 41 uses the inverse DC.
As shown in FIG. 20 (B-3), the upper adjacent block pixel a, the left adjacent block pixel b, the right adjacent block pixel c, and the lower adjacent block pixel d are extracted from the output of the T circuit 40,
A tap (a, c, b, d) in which the order of the left adjacent block pixel b and the right adjacent block pixel c is exchanged is formed. Further, the prediction tap extraction circuit 41 extracts all 64 DCT coefficients of the target block from the output of the dequantization circuit 40 to form taps arranged in the line scan order. Then, the prediction tap extraction circuit 41 connects the tap of the DCT coefficient after the tap of the adjacent block pixel,
A prediction tap is configured and supplied to the code control unit 82.

When the region of interest is the upper right region, the mode control unit 81, as described above, forms the prediction tap.
The sign control unit 82 is controlled to invert the sign of the even-numbered column DCT coefficient among the CT coefficients. So in this case,
The code control unit 82 shades FIG. 20 (B-2), which is the same as FIG. 12 (B-2), among the DCT coefficients forming the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41. The sign of the even-numbered column DCT coefficient indicated by is inverted and supplied to the product-sum operation circuit 45.

[0281] product-sum operation circuit 45, the operation mode is degenerate pixel position mode #n 'when are the prediction taps, set W _n of the tap coefficients of the 16 sets corresponding to the class of the pixel of _interest' with the Then, the sum of products operation of equation (1) is performed,
The product-sum calculation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the pixel of interest located in the upper right area is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = A _{n ′} × a + B _{n ′} × c + C _{n ′} × b + D _{n ′} × d + F _{n ′, 0} × f ₀ + F _{n ′, 1} × (−f ₁ ) + ... + F _{n ′ , 63} × (-f ₆₃ ) ... (16)

The equation (16) is the above-mentioned equation (11).
On the right side of the term, A _{n ′} × a + B _{n ′} × c + C _{n ′} × b + D _{n ′} × relating to adjacent block pixels a to d that form the prediction tap.
d is added.

Here, when the attention area is the upper right area, the mode control unit 81 adopts the degenerate pixel position mode of the vertical line symmetrical pixel as the degenerate pixel position mode of the attention pixel as described above. , Degenerate the pixel position mode.

As described above, when the degenerate pixel position mode of the pixel in the upper right area is also used as the degenerate pixel position mode of the vertical line symmetric pixel, if the classification is not performed, the pixel in the upper right area, The vertically axisymmetric pixel will be predicted using the same set of tap coefficients.

That is, the pixel in the upper right area and the vertical line symmetric pixel have a positional relationship that is bilaterally symmetric with respect to the vertical dividing line, and the pixel value of the pixel in the upper right area has the following positional relationship.
Prediction is performed using a set of tap coefficients corresponding to the same degenerate pixel position mode as the vertically axisymmetric pixel that is the opposite of the vertical dividing line. Therefore, in predicting the pixel value of the pixel in the upper right region, the positional relationship between adjacent block pixels forming the prediction tap also needs to be the opposite of the lateral positional relationship in the case of the default tap structure.

Therefore, in the expression (16), the positional relationship between the left adjacent block pixel b and the right adjacent block pixel c in the expression (15) is exchanged.

Note that, in the equation (16), the sign of the even-numbered column DCT coefficient is also inverted, as in the case of the equation (11).

Next, when the attention area is the lower left area, the mode controller 81 shown in FIG.
As shown in (C-1), depending on which of the 4 × 4 lower left area of the target block the target pixel is positioned, one of the 16 degenerate pixel position modes of 0 to 15 is selected. Output as a pixel degenerate pixel position mode.

On the other hand, the tap structure storage section 111 stores the sequence of 64 DCT coefficients of the target block in the sequence of the lower adjacent block pixel d, the left adjacent block pixel b, the right adjacent block pixel c, and the upper adjacent block pixel a. The tap structure information indicating the connected sequence is stored in association with the area information indicating that the attention area is the lower left area. That is, the tap structure storage unit 111 converts the tap structure information indicating the tap structure in which the upper adjacent block pixel a and the lower adjacent block pixel d are replaced in the default tap structure into the area information indicating that the attention area is the lower left area. It is associated and stored.

When the attention area is the lower left area, the prediction tap extraction circuit 41 extracts the DCT coefficient from the output of the inverse quantization circuit 40 according to the tap structure information representing the tap structure as described above, and Inverse DCT circuit 112
A prediction tap is constructed by extracting pixels from the output of

That is, the prediction tap extracting circuit 41 uses the inverse DC.
As shown in FIG. 20C-3, the upper adjacent block pixel a, the left adjacent block pixel b, the right adjacent block pixel c, and the lower adjacent block pixel d are extracted from the output of the T circuit 40,
A tap (d, b, c, a) in which the order of the upper adjacent block pixel a and the lower adjacent block pixel d is exchanged is configured. Further, the prediction tap extraction circuit 41 extracts all 64 DCT coefficients of the target block from the output of the dequantization circuit 40 to form taps arranged in the line scan order. Then, the prediction tap extraction circuit 41 connects the tap of the DCT coefficient after the tap of the adjacent block pixel,
A prediction tap is configured and supplied to the code control unit 82.

When the region of interest is the lower left region, the mode control unit 81, as described above, forms the prediction tap.
The sign control unit 82 is controlled to invert the sign of the even-numbered DCT coefficient among the CT coefficients. So in this case,
The code control unit 82 shades FIG. 20 (C-2), which is the same as FIG. 12 (C-2), among the DCT coefficients forming the prediction tap for the pixel of interest output from the prediction tap extraction circuit 41. The sign of the even-numbered row DCT coefficient shown by is inverted and supplied to the product-sum operation circuit 45.

When the operation mode is the degenerate pixel position mode #n ', the product-sum operation circuit 45 uses the prediction tap and the set W _{n' of} 16 tap coefficients corresponding to the class of the pixel of interest. Then, the sum of products operation of equation (1) is performed,
The product-sum calculation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the target pixel located in the lower left area is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = A _{n ′} × d + B _{n ′} × b + C _{n ′} × c + D _{n ′} × a + F _{n ′, 0} × f ₀ + F _{n ′, 1} × f ₁ ++ ... + F _{n ′, 63} × (-F ₆₃ ) ... (17)

The expression (17) is the above-mentioned expression (12).
On the right-hand side of, the terms A _{n ′} × d + B _{n ′} × b + C _{n ′} × c + D _{n ′} × regarding the adjacent block pixels a to d forming the prediction taps
a is added.

When the attention area is the lower left area, the mode control unit 81 adopts the degenerate pixel position mode of the horizontal line symmetrical pixel as the degenerate pixel position mode of the attention pixel as described above. Degenerate the pixel position mode.

As described above, when the degenerate pixel position mode of the pixel in the lower left area is also used as the degenerate pixel position mode of the horizontal line symmetrical pixel, if the class classification is not performed, the pixel in the lower left area and its Horizontally symmetric pixels will be predicted using the same set of tap coefficients.

That is, the pixel in the lower left area and its horizontal line symmetric pixel have a positional relationship of being vertically symmetrical with respect to the horizontal dividing line, and the pixel value of the pixel in the lower left area has a positional relationship of
Prediction is performed using the set of tap coefficients corresponding to the same degenerate pixel position mode as the horizontal line symmetrical pixel which is upside down with respect to the horizontal dividing line. Therefore, when predicting the pixel value of the pixel in the lower left area, it is necessary to make the positional relationship between the adjacent block pixels forming the prediction tap reverse the vertical positional relationship in the case of the default tap structure.

Therefore, in the expression (17), the positional relationship between the upper adjacent block pixel a and the lower adjacent block pixel d in the expression (15) is exchanged.

In the equation (17), the sign of the even-numbered row DCT coefficient is also inverted as in the case of the equation (12).

Next, when the attention area is the lower right area, the mode controller 81 shown in FIG.
As shown in (D-1), depending on which of the 4 × 4 lower right area of the target block the target pixel is located, one of the 16-mode degenerate pixel position modes of 0 to 15 is set, It is output as the degenerate pixel position mode of the pixel of interest.

On the other hand, the tap structure storage section 111 stores the sequence of 64 DCT coefficients of the target block in the sequence of the lower adjacent block pixel d, the right adjacent block pixel c, the left adjacent block pixel b, and the upper adjacent block pixel a. The tap structure information indicating the connected sequence is stored in association with the area information indicating that the attention area is the lower right area. That is, the tap structure storage unit 111 replaces the left adjacent block pixel b and the right adjacent block pixel c in the default tap structure and also replaces the upper adjacent block pixel a and the lower adjacent block pixel d with each other. The structure information is stored in association with the area information indicating that the attention area is the lower right area.

When the attention area is the lower right area, the prediction tap extraction circuit 41 extracts the DCT coefficient from the output of the inverse quantization circuit 40 according to the tap structure information representing the tap structure as described above. , Inverse DCT circuit 112
A prediction tap is constructed by extracting pixels from the output of

That is, the prediction tap extraction circuit 41 uses the inverse DC.
As shown in FIG. 20D-3, the upper adjacent block pixel a, the left adjacent block pixel b, the right adjacent block pixel c, and the lower adjacent block pixel d are extracted from the output of the T circuit 40,
Taps (d, c, b,) in which the order of the left adjacent block pixel b and the right adjacent block pixel c is changed and the order of the upper adjacent block pixel a and the lower adjacent block pixel d is changed.
a). Furthermore, the prediction tap extraction circuit 41
From the output of the inverse quantization circuit 40, all 64 DCT coefficients of the target block are extracted to form taps arranged in the line scan order. Then, the prediction tap extraction circuit 41
Is connected to the tap of the DCT coefficient after the tap of the adjacent block pixel to form a prediction tap, and supplies the prediction tap to the code control unit 82.

When the attention area is the lower right area, the mode control unit 81, as described above, forms the prediction tap.
The sign control unit 82 is controlled so as to invert the sign of the even-numbered column DCT coefficient among the CT coefficients and invert the sign of the even-numbered row DCT coefficient. Therefore, in this case, the code control unit 82, among the DCT coefficients that form the prediction tap for the pixel of interest output by the prediction tap extraction circuit 41, the same as in FIG. The sign of the DCT coefficient which is only the even-column DCT coefficient or the even-row DCT coefficient indicated by shading is inverted, and the product-sum operation circuit 45
Supply to.

[0309] product-sum operation circuit 45, the operation mode is degenerate pixel position mode #n 'when are the prediction taps, set W _n of the tap coefficients of the 16 sets corresponding to the class of the pixel of _interest' with the Then, the sum of products operation of equation (1) is performed,
The product-sum calculation result is used as the decoding result of the pixel of interest.

Therefore, assuming that the degenerate pixel position mode of the pixel of interest located in the lower right region is #n ', the predicted value L _n of the pixel value of that pixel in the product-sum operation circuit 45 of FIG.
It is obtained according to the following equation corresponding to the equation (1).

L _{n ′} = A _{n ′} × d + B _{n ′} × c + C _{n ′} × d + D _{n ′} × a + F _{n ′, 0} × f ₀ + F _{n ′, 1} × (−f ₁ ) + ... + F _{n ′ , 63} × f ₆₃・ ・ ・ (18)

The equation (18) is the above equation (13).
On the right side of the term, A _{n ′} × d + B _{n ′} × c + C _{n ′} × b + D _{n ′} × relating to adjacent block pixels a to d that form the prediction tap.
a is added.

Here, when the attention area is the lower right area, the mode control unit 81 adopts the degenerate pixel position mode of the point symmetric pixel as the degenerate pixel position mode of the attention pixel as described above. , Degenerate the pixel position mode.

As described above, when the degenerate pixel position mode of the pixel in the lower right area is also used as the degenerate pixel position mode of the point-symmetrical pixel, if the class classification is not performed, the pixel is in the lower right area. , The dotted line symmetric pixel will be predicted using the same set of tap coefficients.

That is, the pixel in the lower right region and its point-symmetrical pixel have a positional relationship of being bilaterally symmetric with respect to the vertical dividing line and vertically symmetrical with respect to the horizontal dividing line, and The pixel value of the pixel corresponds to the same degenerate pixel position mode as the point-symmetrical pixel whose positional relationship is left-right opposite to the vertical dividing line and upside down with respect to the horizontal dividing line. Predicted using a set of tap coefficients.
Therefore, when predicting the pixel value of the pixel in the lower right area, the positional relationship between the adjacent block pixels forming the prediction tap is the same as the positional relationship between the left and right in the case of the default tap structure, but the vertical positional relationship is reversed. And need to.

Therefore, in the expression (18), the positional relationship between the left adjacent block pixel b and the right adjacent block pixel c in the expression (15) is exchanged, and the upper adjacent block pixel a and the lower adjacent block pixel d are changed. The positional relationship has been changed.

In the equation (18), as in the case of the equation (13), the even column DCT coefficient or the even row DCT coefficient is used.
The sign of the DCT coefficient, which is only a coefficient, is also inverted.

Also in formulas (15) to (18), as in the case of formulas (10) to (13), the degenerate pixel position mode #n 'can take 16 values of 0 to 15, Assuming that the total number of classes is one, the total number of required tap coefficients is 68 × 16 = 1088. Therefore, as compared with the case where the pixel position mode is not degenerated (the total number of required tap coefficients is 4352 as described above), the coefficient table storage unit 44 having a small storage capacity can be adopted. Become.

Next, the processing of the coefficient conversion circuit 32 of FIG. 19 will be described with reference to the flowchart of FIG.

The quantized DCT coefficients for each block output from the entropy decoding circuit 31 (FIG. 3) are supplied to the dequantization circuit 40, and in step S91, the dequantization circuit 40 is supplied to each block supplied thereto. Quantized DC
Dequantize the T coefficient. The DCT coefficients for each block obtained by the inverse quantization circuit 40 performing inverse quantization are sequentially supplied to the prediction tap extraction circuit 41, the class tap extraction circuit 42, and the inverse DCT circuit 112.
Further, in step S91, the inverse DCT circuit 112
The inverse DCT transform of the DCT coefficient for each block output from the inverse quantization circuit 40 is started, and the pixel (pixel value) for each block obtained by the inverse DCT transform is sequentially supplied to the prediction tap extracting circuit 41.

The prediction tap extracting circuit 41 sequentially sets the blocks of DCT coefficients supplied from the inverse quantizing circuit 40 as the target block. Then, hereinafter, in steps S92 to S97, the same processing as that in steps S52 to S57 of FIG. 15 is performed.

Then, in step S97, 16 sets of tap coefficients corresponding to the class of the pixel of interest are supplied from the coefficient table storage section 44 to the product-sum calculation circuit 45, the flow proceeds to step S98, and tap structure storage is performed. Part 1
11 reads the tap structure information associated with the area information output by the mode control unit 81 in step S93, supplies the tap structure information to the prediction tap extraction circuit 41, and then performs step S9.
Proceed to 9.

In step S99, the prediction tap extraction circuit 41 extracts the DCT coefficient and the adjacent block pixel used for predicting the pixel value of the target pixel from the outputs of the inverse quantization circuit 40 and the inverse DCT circuit 112, respectively. Step S9
At 8, the prediction tap of the tap structure represented by the tap structure information supplied from the tap structure storage unit 111 is configured. This prediction tap is from the prediction tap extraction circuit 41 to the code control unit 8
2 is supplied.

Further, in step S99, the code control unit 82 determines the code of the prediction tap of the pixel of interest from the prediction tap extraction circuit 41 based on the area information of the pixel of interest supplied from the mode control unit 81, as shown in FIG. Step S
The operation is performed in the same manner as in 58, and the product-sum calculation circuit 45 is supplied.

Then, the process proceeds to step S100, and
In steps S100 to S103, the same processing as in steps S59 to S62 of FIG. 15 is performed, respectively, and in step S103, it is determined that all the blocks of the DCT coefficients output by the inverse quantization circuit 40 have been processed as the target block. If so, the process ends.

As described above, when the prediction tap is configured by using not only the DCT coefficient of the target block but also the adjacent block pixel, it is possible to obtain a decoded image of higher quality.

In the above case, D of the target block
In addition to the CT coefficient, adjacent block pixels are used to form the prediction taps. The prediction taps relate to pixels located above, below, on the left side, or on the right side of the block of interest other than the adjacent block pixels. It is possible to configure by using upper side pixel information, lower side pixel information, left side pixel information, or right side pixel information which is information. Here, as the upper side pixel information other than the adjacent block pixels, the lower side pixel information, the left side pixel information, or the right side pixel information, for example, DC of the adjacent block above, below, left, or right of the target block
It is possible to adopt a T coefficient or the like. However, no matter what information is used as the upper side pixel information, the lower side pixel information, the left side pixel information, and the right side pixel information, the horizontal or vertical positional relationship of the prediction taps is the same as in the case of the adjacent block pixel described above. Need to be reversed.

Next, FIG. 22 shows an example of the structure of a learning device for carrying out the learning process of the tap coefficient stored in the coefficient table storage section 44 of FIG. Note that, in the figure, parts corresponding to those in FIG. 16 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device of FIG. 22 has the tap structure storage unit 121 and the inverse DCT.
The configuration is basically the same as that in FIG. 16 except that a circuit 122 is newly provided.

The tap structure storage unit 121 has the same structure as the tap structure storage unit 111 of FIG.
Controls the tap structure of the prediction taps that That is,
Area information is supplied from the mode control unit 101 to the tap structure storage unit 121. And
The tap structure storage unit 121 stores the attention area and the tap structure information indicating the tap structure of the prediction tap in association with each other, and associates the attention area with the attention area represented by the area information supplied from the mode control unit 101. Existing tap structure information is read out and supplied to the prediction tap extraction circuit 65.

The inverse DCT circuit 122 includes an inverse quantization circuit 6
4 outputs DCT coefficients for each block, and the inverse DCT circuit 122 uses the inverse D
Similar to the CT circuit 112, the DCT coefficient for each block output from the inverse quantization circuit 64 is inversely DCT-transformed to obtain the pixel value for each block, and the pixel value is supplied to the prediction tap extraction circuit 65.

In the embodiment of FIG. 22, the prediction tap extraction circuit 65, like the prediction tap extraction circuit 41 of FIG. 19, predicts the tap structure represented by the tap structure information supplied from the tap structure storage unit 121. Configure the tap. Furthermore, the prediction tap extraction circuit 65 also extracts the DCT coefficient of the target block from the DCT coefficient supplied from the inverse quantization circuit 64, as in the prediction tap extraction circuit 41 of FIG. The adjacent block pixels a to d for the pixel of interest are extracted from the pixels supplied from, and the adjacent block pixels a to d are also used to configure a prediction tap having a tap structure represented by the tap structure information.

Next, the processing (learning processing) of the learning device in FIG. 22 will be described with reference to the flowchart in FIG.

In the learning process, first, in step S1
11, in the same manner as in step S71 of FIG. 17, the blocking circuit 61, the DCT circuit 62, the quantization circuit 63, and the dequantization circuit 64 JPEG-encode the learning image data, and further, the code thereof. From the digitized data, a DCT coefficient that becomes student data is acquired. further,
In step S111, the inverse DCT circuit 122 inverts the DCT coefficient for each block obtained by the inverse quantization circuit 64 by inverse D.
The CT conversion is performed, and the pixel (pixel value) for each block is thereby acquired.

Then, the prediction tap extraction circuit 65 sequentially sets the blocks of the image data blocked by the blocking circuit 61 as the block of interest, and then, at step S1.
In 12 to S116, the same processes as in steps S72 to S76 of FIG. 17 are performed.

Then, in step S116, the class classification circuit 67 instructs the normal equation addition circuit 68 to
When the class code of the pixel of interest is output, step S1
In step 17, the tap structure storage unit 121 reads the tap structure information associated with the area information output by the mode control unit 101 in step S113, supplies the tap structure information to the prediction tap extraction circuit 6, and proceeds to step S118.

In step S118, the prediction tap extraction circuit 65 uses the DCT used to predict the pixel value of the target pixel from the outputs of the inverse quantization circuit 64 and the inverse DCT circuit 122.
The coefficient and the adjacent block pixel are respectively extracted, and step S
At 117, a prediction tap having a tap structure represented by the tap structure information supplied from the tap structure storage unit 121 is configured. This prediction tap is supplied from the prediction tap extraction circuit 65 to the code control unit 102.

Further, in step S118, the code control unit 102 determines the code of the prediction tap of the pixel of interest from the prediction tap extraction circuit 65 based on the area information of the pixel of interest supplied from the mode control unit 101, as shown in FIG. The operation is performed in the same manner as in step S77 in step S77, and the normal equation adding circuit 68 is supplied.

Then, the process proceeds to step S119, and
In steps S119 to S122, the same processing as that in steps S78 to S81 of FIG. 17 is performed, respectively, whereby the tap coefficient determination circuit 69 sets each of the 16 degenerate pixel position modes of the class for each class. Corresponding 16 sets of tap coefficients are obtained, supplied to the address corresponding to each class in the coefficient table storage unit 70, and stored therein, and the processing is ended.

As described above, even when the prediction tap is formed by using information other than the DCT coefficient of the target block, by degenerating the pixel position mode, as in the case of the embodiment of FIG. Learning efficiency can be improved.

Next, in the degeneracy of the pixel position mode, as described above, when the pixel of interest is a pixel in the upper right region, the lower left region, or the lower right region, the pixel position mode of the pixel of interest is changed to the vertical division line. With respect to the pixel of interest (vertical line symmetrical pixel) with respect to the pixel of interest, the pixel in the upper left region (horizontal line symmetrical pixel) of line of interest with respect to the horizontal dividing line, or the block center point On the other hand, the prediction tap is performed by also using the pixel position mode of the pixel in the upper left region (point-symmetrical pixel) that is point-symmetrical to the pixel of interest, and thus the prediction tap is as described in FIGS. 12 and 20. To
It is necessary to reverse (reverse the positional relationship) the cosine waveform that is the basis of the DCT coefficient and the positional relationship between the positions of adjacent block pixels on the left and right, the top and bottom, or both the left and right and the top and bottom.

On the other hand, in the above case, the coefficient conversion circuit 32
Class classification circuit 43 and the class classification circuit 6 of the learning device
In FIG. 7, all DCT coefficients of the block of interest are used as class taps, and the class taps are subjected to ADRC processing to perform class classification. That is, the class classification is performed by using the pixel position in the pixel position mode as described above. Since the method is performed by the method that is not affected by the degeneracy of the mode, it is not necessary to perform the positional reversal, which is necessary for the classification tap, for the classification.

However, when the classification is performed based on the relationship between the block of interest and each of the blocks vertically, horizontally, and adjacently to each other, the same operation as the reversal of the positional relationship for the prediction tap is performed. Needs to be done for the class code.

Therefore, FIG. 24 shows a configuration example of the third embodiment of the coefficient conversion circuit 32 to which the present invention is applied.
Note that, in the figure, parts corresponding to those in FIG. 19 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 32 of FIG.
Is basically the same as the case in FIG. 11 except that a class code operation unit 131 is newly provided.

However, in the embodiment of FIG. 24, in the class tap extraction circuit 42, there is some continuity such as an edge being connected between the block of interest and each of the blocks vertically and horizontally adjacent thereto. A class tap including information indicating whether or not it is extracted from the output of the inverse quantization circuit 40, and the class classification circuit 43 performs class classification based on such a class tap.

The class code operation unit 131 is supplied with the class code of the target pixel output from the class classification circuit 43 and the area information of the target pixel output from the mode control unit 81. Then, the class code operation unit 13
1 operates the class code of the pixel of interest based on the area information of the pixel of interest and supplies it to the coefficient table storage unit 44.

Next, FIG. 25 shows a configuration example of the class tap extraction circuit 42 of FIG.

The one-dimensional inverse DCT transform section 141 has a DCT for each block output from the inverse quantization circuit 40 (FIG. 24).
Coefficients are being supplied. Then, the one-dimensional inverse DCT transform unit 141 performs a one-dimensional inverse DCT transform on the DCT coefficient for each block supplied from the inverse quantization circuit 40, so that a horizontal 1 representing a spatial frequency component in the horizontal direction is obtained.
A three-dimensional DCT coefficient and a vertical one-dimensional DCT coefficient representing a spatial frequency component in the vertical direction are obtained.

Here, referring to FIGS. 26 and 27,
One-dimensional inverse D performed by the one-dimensional inverse DCT transform unit 141 in FIG.
The CT conversion will be described.

In the image encoding method using DCT conversion such as MPEG and JPEG, the image data is 8 pixels in the horizontal and vertical directions.
Two-dimensional DCT transforms (two-dimensional DCT transforms) / inverse DCT transforms (two-dimensional inverse DCT transforms) in the horizontal and vertical directions are performed in units of × 8 blocks.

Here, the DCT coefficients included in the encoded data that has been MPEG or JPEG encoded represent spatial frequency components in two directions, the horizontal direction and the vertical direction, and are distinguished from the one-dimensional DCT coefficients. , 2D D
It is called CT coefficient.

Further, hereinafter, two-dimensional D is calculated from the pixel value as appropriate.
Two-dimensional DCT transform and two-dimensional DC transform to CT coefficient
The conversion from the T coefficient to the pixel value is a two-dimensional inverse DCT conversion,
Say each. The DCT conversion performed by the DCT circuit 2 in FIG. 1 described above is a two-dimensional DCT conversion, and the inverse DCT circuit 112 (FIG. 19 and the like) and the inverse DCT circuit 1 described above are also used.
The inverse DCT transform performed at 22 (FIG. 22 and the like) is a two-dimensional inverse DCT transform.

Now, the pixel value in the 8 × 8 block shown in FIG. 26A is represented by a matrix X of 8 rows × 8 columns, and DC in the 8 × 8 block shown in FIG.
If the T coefficient is represented by a matrix F of 8 rows × 8 columns, 2
The dimensional DCT transform / two-dimensional inverse DCT transform can be expressed by the following equation.

CXC ^T = F (19) C ^T FC = X (20)

Here, the superscript T represents transposition. Also,
C is a DCT transformation matrix of 8 rows × 8 columns, and the component c _ij of the i + 1-th row and the j + 1-th column is represented by the following equation.

C _ij = A _i × cos ((2j + 1) × i × π / 16) (21)

However, in the equation (21), when i = 0, A _i = 1 / (2√2), and when i ≠ 0, A _i
= 1/2. Further, i and j are integer values in the range of 0 to 7.

Equation (19) is a two-dimensional DCT for the pixel value X.
A two-dimensional DCT transform for transforming into a coefficient F is expressed by equation (20).
Represents a two-dimensional inverse DCT transform that transforms a two-dimensional DCT coefficient F into a pixel value X.

Therefore, according to the equation (20), the two-dimensional DC
The T coefficient F is multiplied by the matrix C ^T from its left side,
The pixel value X is converted by multiplying the matrix C from the right side, but the one-dimensional inverse DCT conversion unit 141 uses the two-dimensional D
A one-dimensional DCT coefficient is obtained by only multiplying the CT coefficient F by the matrix C ^T from the left side or the matrix C from the right side.

That is, the one-dimensional inverse DCT transform unit 141 is
For the dimension DCT coefficient F, only the matrix C ^T is multiplied from the left side. In this case, as shown in FIG. 26C, the vertical one-dimensional inverse DCT transform in which the vertical direction in the two-dimensional DCT coefficient F is transformed into the spatial domain and the horizontal direction remains the frequency domain is performed, As a result, the horizontal one-dimensional DCT coefficient vXhF representing the spatial frequency component in the horizontal direction
Can be obtained.

Further, the one-dimensional inverse DCT transform section 141 is
Multiply the matrix C from the right side of the dimensional DCT coefficient F. In this case, as shown in FIG. 26D, the horizontal direction of the two-dimensional DCT coefficient F is converted into the spatial domain,
Horizontal one-dimensional inverse DC whose vertical direction remains the frequency domain
The T transform is performed, and as a result, the vertical one-dimensional DCT coefficient hXvF representing the spatial frequency component in the vertical direction can be obtained.

When the vertical 1-dimensional inverse DCT transformation is performed on the 2-dimensional DCT coefficient F of horizontal × vertical 8 × 8, 8 sets (8 rows) of horizontal 1-dimensional DCT coefficient of 8 × 1 are obtained. (FIG. 26 (C)). Further, when the two-dimensional DCT coefficient F is subjected to the horizontal one-dimensional inverse DCT conversion, eight sets (8 columns) of 1 × 8 vertical one-dimensional DCT coefficients are obtained (FIG. 26 (D)). .

For the 8 × 1 horizontal one-dimensional DCT coefficient in a certain row, the DCT coefficient at the left end is the DC component (DC component) of the pixel values of the eight pixels in that row (the average of the pixel values of the eight pixels). Value), and the other seven DCT coefficients are
It represents the horizontal AC component (AC component) of the row. Also, regarding the 1 × 8 vertical one-dimensional DCT coefficient in a certain column, the DCT coefficient in the uppermost row represents the DC component of the pixel value of the 8 pixels in that column, and the other seven DCT coefficients are It represents the AC component in the vertical direction.

According to the equation (19), the horizontal one-dimensional DCT coefficient is the pixel value X corresponding to the two-dimensional DCT coefficient F.
, The horizontal one-dimensional D by multiplying the matrix C ^T from its right side
It can also be obtained by performing CT conversion. The vertical one-dimensional DCT coefficient can also be obtained by performing a vertical one-dimensional DCT transformation by multiplying the pixel value X corresponding to the two-dimensional DCT coefficient F by the matrix C from the left side of the pixel value X.

FIG. 27 shows an actual image, the two-dimensional DCT coefficient, the horizontal one-dimensional DCT coefficient, and the vertical one-dimensional DCT coefficient for the image.

Note that FIG. 27 shows an 8 × 8 block image and the two-dimensional DCT coefficient, horizontal one-dimensional DCT coefficient, and vertical one-dimensional DCT coefficient for the image. In addition, FIG. 27A shows an actual image in FIG.
27B shows a two-dimensional DCT coefficient, FIG. 27C shows a horizontal one-dimensional DCT coefficient, and FIG. 27D shows a vertical one-dimensional DCT coefficient.
Each T coefficient is shown.

The image of FIG. 27A has a pixel value of 8 bits, and the DCT coefficient obtained from such a pixel value can take a negative value. However, FIG.
In the embodiment of FIGS. 7 (B) to 27 (D), 128 (= 2 ⁷ ) is added to the obtained DCT coefficient, and if the added value is less than 0, it is clipped to 0. , If the added value is 256 or more, the DCT coefficient in the range of 0 to 255 is clipped to 255,
It is shown.

Since information of the entire block of 8 × 8 pixels is reflected in the two-dimensional DCT coefficient, local information such as information of a specific pixel in the block is grasped from the two-dimensional DCT coefficient. Is difficult. On the other hand, the horizontal one-dimensional DCT coefficient or the vertical one-dimensional DCT coefficient reflects the information of only one row or one column in which the block is present, so that it is compared with the two-dimensional DCT coefficient in the block. The local information of can be easily grasped.

That is, the feature of a certain row of a block can be understood from the 8 × 1 horizontal one-dimensional DCT coefficient of that row, and the feature of a certain column is a 1 × 8 vertical one-dimensional DC of that column.
It can be understood from the T coefficient. Further, the feature of a pixel in a block is that the horizontal 1-dimensional DCT coefficient of 8 × 1 in the row in which the pixel is located and the 1 × 8 in the column in which the pixel is located.
Can be understood from the vertical one-dimensional DCT coefficient of

The state of the boundary between adjacent blocks on the left and right is a two-dimensional D in which the information of the entire block is reflected.
Using a vertical one-dimensional DCT coefficient that represents a spatial frequency component in the vertical direction at the boundary of blocks is more preferable than using a CT coefficient.
It can be grasped more accurately. Furthermore, the state of the boundary between vertically adjacent blocks is also a horizontal one-dimensional D that represents the spatial frequency component in the horizontal direction at the block boundary, rather than the two-dimensional DCT coefficient that reflects the information of the entire block.
It is possible to grasp more accurately by using the CT coefficient.

The vertical one-dimensional DCT coefficient and the horizontal one-dimensional DCT coefficient will be collectively referred to as a one-dimensional DCT coefficient hereinafter.

Returning to FIG. 25, the one-dimensional inverse DCT transform unit 14
1 is the one-dimensional DCT coefficient of the block of interest and the one-dimensional DC of each of the blocks vertically, horizontally, and adjacently to the block of interest.
When the T-coefficients are obtained, the one-dimensional DCT coefficients are calculated from the boundary one-dimensional DCT coefficient extraction unit 144 and the adjacent one-dimensional DC.
It is supplied to the T coefficient extraction unit 145.

The boundary one-dimensional DCT coefficient extracting section 144 uses
From the output of the dimensional inverse DCT transform unit 141, the horizontal 1-dimensional DCT coefficient of the upper boundary (first row) of the target block (hereinafter, appropriately referred to as the upper boundary 1-dimensional DCT coefficient) and the lower boundary (8 rows) Vertical one-dimensional DCT coefficient (hereinafter, as appropriate,
Lower boundary 1-dimensional DCT coefficient), left boundary (first column) vertical 1-dimensional DCT coefficient (hereinafter appropriately referred to as left boundary 1-dimensional DCT coefficient), and right boundary (8th column) vertical A one-dimensional DCT coefficient (hereinafter, appropriately referred to as a right boundary one-dimensional DCT coefficient) is extracted and output.

Here, the upper boundary one-dimensional DCT
The coefficient, the lower boundary one-dimensional DCT coefficient, the left boundary one-dimensional DCT coefficient, and the right boundary one-dimensional DCT coefficient are collectively referred to as a boundary one-dimensional DCT coefficient.

The adjacent one-dimensional DCT coefficient extraction unit 145 sets 1
From the output of the dimensional inverse DCT transform unit 141, the horizontal one-dimensional DCT coefficient adjacent to the target block in the block adjacent to the target block (hereinafter, appropriately referred to as the upper adjacent one-dimensional D
CT coefficient), a horizontal one-dimensional DCT adjacent to the target block in a block adjacent to the target block
Coefficient (hereinafter, appropriately referred to as lower adjacent one-dimensional DCT coefficient),
In the block adjacent to the left of the block of interest, the vertical one-dimensional DCT coefficient adjacent to the block of interest (hereinafter, appropriately,
The left adjacent one-dimensional DCT coefficient) and the vertical one-dimensional DCT coefficient adjacent to the target block in the block adjacent to the right of the target block (hereinafter, appropriately referred to as right adjacent one-dimensional DCT coefficient) are extracted and output.

Hereafter, the upper adjacent one-dimensional DCT will be referred to as appropriate.
The coefficient, the lower adjacent one-dimensional DCT coefficient, the left adjacent one-dimensional DCT coefficient, and the right adjacent one-dimensional DCT coefficient are collectively referred to as an adjacent one-dimensional DCT coefficient.

The class tap extraction circuit 42 uses the output of the dequantization circuit 40 to determine the boundary one-dimensional D as described above.
The CT coefficient and the adjacent one-dimensional DCT coefficient are extracted as class taps and supplied to the class classification circuit 43 (FIG. 24).

Next, FIG. 28 shows a configuration example of the class classification circuit 43 of FIG.

The AC power calculator 151 receives the boundary one-dimensional DCT coefficient as a class tap and the adjacent one-dimensional DCT coefficient output from the class tap extraction circuit 42, and outputs the power of each AC component (hereinafter, AC Power), and supplies it to the continuity determination unit 154.

That is, the AC power calculating section 151 is shown in FIG.
As shown in, the upper 1-dimensional DCT coefficient and the lower 1-dimensional DCT coefficient, which are horizontal 1-dimensional DCT coefficients, are extracted from the 1-dimensional boundary DCT coefficient, and the horizontal 1-dimensional DCT coefficient is extracted from the adjacent 1-dimensional DCT coefficient. One upper one-dimensional DC
The T coefficient and the lower adjacent one-dimensional DCT coefficient are extracted to obtain the AC power in the horizontal direction. Further, the AC power calculation unit 151
As shown in FIG. 29, from the boundary one-dimensional DCT coefficient,
The left boundary one-dimensional DCT coefficient and the right boundary one-dimensional DCT coefficient, which are vertical one-dimensional DCT coefficients, are extracted, and the vertical adjacent one-dimensional DCT coefficient is extracted from the adjacent one-dimensional DCT coefficient. Extract the DCT coefficients,
Determine the AC power in the vertical direction.

Here, if the seven AC components of the one-dimensional DCT coefficient are represented by AC _n (n = 1, 2, ...,
7), AC power P _AC is calculated, for example, by the following equation.

P _AC = ΣAC _n ² (22)

However, in the equation (22), Σ is a variable n
Represents the summation from 1 to 7.

As described above, the AC power obtained from the one-dimensional DCT coefficient can be regarded as the power of the AC component of 8 pixels corresponding to the one-dimensional DCT coefficient, and therefore represents the image activity. Can be said.

Returning to FIG. 28, the DC component extracting section 152
A boundary one-dimensional DCT coefficient as a class tap output by the class tap extraction circuit 42 (FIG. 24) and an adjacent one-dimensional D
The CT coefficient is received, each DC component (DC component) is extracted, and is supplied to the continuity determination unit 154.

That is, the DC component extracting unit 152 determines the boundary one-dimensional DCT coefficient and the 8 × 1 of the adjacent one-dimensional DCT coefficients.
For the horizontal one-dimensional DCT coefficient (upper boundary one-dimensional DCT coefficient, lower boundary one-dimensional DCT coefficient, upper adjacent one-dimensional DCT coefficient, lower adjacent one-dimensional DCT coefficient), the DC component located on the leftmost side is extracted, It is supplied to the continuity determination unit 154. In addition, the DC component extraction unit 152 uses the boundary one-dimensional DCT.
Coefficient and 1 × 8 vertical 1 in adjacent one-dimensional DCT coefficient
For the three-dimensional DCT coefficient (left boundary one-dimensional DCT coefficient, right boundary one-dimensional DCT coefficient, left-adjacent one-dimensional DCT coefficient, right-adjacent one-dimensional DCT coefficient), the DC component located on the uppermost side is extracted to determine continuity. Supply to the section 154.

The AC inner product calculating unit 153 determines the AC component of the boundary one-dimensional DCT coefficient as the class tap supplied from the class tap extraction circuit 42 (FIG. 24) and the adjacent one-dimensional D.
The AC component of the CT coefficient is regarded as a vector component, and the inner product (AC inner product) of the two vectors is calculated and supplied to the continuity determination unit 154.

That is, as shown in FIG. 30, the AC inner product calculating unit 153 detects the AC component of the upper boundary one-dimensional DCT coefficient and the AC component of the upper adjacent one-dimensional DCT coefficient adjacent to the upper boundary one-dimensional DCT coefficient. Is regarded as a component of each vector, and an AC inner product (hereinafter referred to as an upper inner product) of the two vectors is obtained according to the following equation.

I = Σ (AC _n × AC _n ') (23)

However, in the equation (23), I represents an AC inner product. Further, AC _n represents the n-th AC component of the upper boundary one-dimensional DCT coefficient for the block of interest, and A n
C _n 'is the upper adjacent one-dimensional DCT for the block of interest
It represents the nth AC component of the coefficient. Further, Σ represents the summation in which n is changed from 1 to 7.

The AC inner product calculating unit 153 determines the lower boundary one-dimensional D
CT coefficient, lower one-dimensional DCT coefficient, left boundary one-dimensional DC
For each of the T coefficient and the left adjacent one-dimensional DCT coefficient, or the right boundary one-dimensional DCT coefficient and the right adjacent one-dimensional DCT coefficient, the AC inner product is calculated according to the equation (23).

Here, hereinafter, the AC inner product obtained from the lower boundary one-dimensional DCT coefficient and the lower adjacent one-dimensional DCT coefficient for the target block is appropriately calculated as the lower inner product, the left boundary one-dimensional DCT coefficient, and the left adjacent one-dimensional DCT coefficient. The AC inner product obtained from the DCT coefficient is called the left inner product, and the AC inner product obtained from the right boundary one-dimensional DCT coefficient and the right adjacent one-dimensional DCT coefficient is called the right inner product.

The AC inner product is adjacent to a vector having the AC component of the boundary 1-dimensional DCT coefficient as a component, when the AC component of the boundary 1-dimensional DCT coefficient and the AC component of the adjacent 1-dimensional DCT coefficient are similar to each other. One-dimensional DC
When the angle formed by the vector having the AC component of the T coefficient as a component is less than 90 degrees (or more), it has a positive value (value of 0 or more). Therefore, the positive value of the AC inner product means that the boundary one-dimensional DCT that sandwiches the boundary of the block of interest.
It shows that the waveform pattern of 8 pixels corresponding to the coefficient is similar to the waveform pattern of 8 pixels corresponding to the adjacent one-dimensional DCT coefficient. For example, at the boundary between the block of interest and the block adjacent to it, the waveform crosses the boundary. Indicates that there are continuous edges.

Returning to FIG. 28 again, the continuity determining unit 154
Is the AC power calculated from the boundary one-dimensional DCT coefficient supplied from the AC power calculation unit 151, the AC power calculated from the adjacent one-dimensional DCT coefficient, and the boundary one-dimensional DCT coefficient supplied from the DC component extraction unit 152. DC component of A, and DC component of adjacent one-dimensional DCT coefficient, and A
Based on the upper inner product, the lower inner product, the left inner product, and the right inner product supplied from the C inner product calculating unit 153, the continuity of the images at the boundary portions of the target block in the upper, lower, left, and right directions is adjacent to the target block in the upper, lower, left, and right directions. It is determined as the upper block information, the lower block information, the left block information, and the right block information related to the block. Furthermore, the continuity determination unit 154 determines the upper inter-block information, the lower inter-block information, the left inter-block information, and the right inter-block information, which are the results of determining the continuity of the image at the upper, lower, left, and right boundary portions of the target block. Based on this, the pixel of interest is classified into classes, and a class code representing that class is output.

That is, the continuity judging section 154 judges the continuity of the image showing the connection state of the image between the block of interest and each of the blocks vertically and horizontally adjacent to it.
Then, the continuity determination unit 154 represents the presence / absence of image continuity for each of the upper, lower, left, and right boundary portions of the target block.
Output bits as class code. Therefore, here, the pixel of interest is classified into any of 16 (= 2 ⁴ ) classes.

[0395] Here, 4 output from the continuity determination unit 154
Bit class code, in order from the most significant bit,
If it is expressed as (B ₄ , B ₃ , B ₂ , B ₁ ), the bit B
₄ , B ₃ , B ₂ and B ₁ are respectively above the block of interest,
It represents continuity at the lower, left, and right boundaries. Also, for example, a bit of 1 indicates that the image is continuous (having continuity), and a bit of 0 indicates that the image is discontinuous (no continuity). ) Represents.

The continuity determination unit 154 determines the continuity as follows, for example.

That is, for example, the continuity determining unit 154, for example, the DC component DC of the upper boundary one-dimensional DCT coefficient for the block of interest and the upper adjacent first-order DCT coefficient corresponding (adjacent) to the upper boundary one-dimensional DCT coefficient. The difference absolute value | DC-DC '| of the DC component DC' of the above is larger than (or more than) a predetermined threshold value TH1 as a boundary edge condition, and such a boundary edge condition is satisfied. Is determined.

When the boundary edge condition is satisfied, that is, the DC component DC of the upper boundary first-order DCT coefficient and the DC component DC 'of the upper adjacent first-order DCT coefficient for the target block are satisfied.
When there is an edge at the upper boundary of the block of interest and there is no connection between the image pattern of the block of interest and the block adjacent to the block of interest, the continuity determination unit 154 determines that of the 4-bit class code, (and the least significant bit first bit) fourth bits representing the upper continuity B _4, 0 is set indicating that there is no continuity.

When the boundary edge condition is not satisfied, the continuity determining unit 154 determines the AC power calculated from the upper boundary one-dimensional DCT coefficient and the AC power calculated from the upper adjacent one-dimensional DCT coefficient for the target block. Further, based on the DC component of the upper boundary one-dimensional DCT coefficient and the DC component of the upper adjacent one-dimensional DCT coefficient, it is determined whether the flatness condition that the upper boundary portion of the target block is flat is satisfied.

That is, the continuity determination unit 154 determines, for example, whether the flatness condition is satisfied by using the condition represented by the following equation as the flatness condition.

(P _AC ≦ TH2) ∩ (P _AC '≦ TH2) ∩ (| DC-DC' | ≦ TH3) (24)

P _AC '≦ TH4 (25)

However, in the equations (24) and (25), TH2, TH3, and TH4 are predetermined threshold values, and the threshold value TH4 is sufficiently smaller than the threshold value TH2.
Further, in the equations (24) and (25), P _AC represents the AC power obtained from the upper boundary one-dimensional DCT coefficient for the target block, and P _AC ′ is the upper adjacent one-dimensional DCT coefficient for the target block. Represents the AC power obtained from Further, in Expression (24), DC represents the DC component of the upper boundary one-dimensional DCT coefficient for the target block, and DC ′ represents the DC component of the upper adjacent one-dimensional DCT coefficient for the target block. Further, ∩ represents a logical product.

Equation (24) is such that AC powers P _AC and P _AC ′ obtained from the upper boundary one-dimensional DCT coefficient and the upper adjacent one-dimensional DCT coefficient for the target block are both below (or less than) the threshold value TH2. ), And the absolute value of the difference between the DC components DC and DC '| DC-DC' |
Is true (or less) than the threshold TH3, it is true. Further, the equation (25) is the AC power P _AC ′ obtained from the upper adjacent one-dimensional DCT coefficient for the block of interest.
Is true (or less) than the threshold TH4, it is true.

Here, the threshold TH4 is, as described above,
The value is sufficiently smaller than the threshold value TH2, for example, a value close to 0. Therefore, the equation (25) indicates that the AC power P _AC 'determined from the upper adjacent one-dimensional DCT coefficient for the target block.
Is true when is close to zero.

The flatness condition here is, for example,
If either one of equations (24) and (25) is true, it is satisfied.

When the flatness condition is satisfied, the continuity determining section 154 sets the fourth bit B ₄ representing the continuity on the upper side of the 4-bit class code to 1 which indicates continuity. .

If the flatness condition is not satisfied, the continuity determining unit 154 determines whether or not the continuity condition that the boundary portion of the target block is continuous is satisfied based on the AC inner product obtained for the target block. To judge.

That is, the continuity determining section 154 sets such a continuity condition that the upper inner product of the target block is a positive value (or 0 or more), and the continuity condition is satisfied. Determine whether or not.

When the continuity condition is satisfied, the continuity determining unit 154 sets the fourth bit B ₄ representing the upper continuity of the 4-bit class code to 1 which indicates continuity. .

If the continuity condition is not satisfied, that is, the boundary edge condition is not satisfied, and the flatness condition and the continuity condition are also not satisfied, the continuity determining unit 154 determines that the 4-bit class is satisfied. In the code, the fourth bit B ₄ indicating the continuity on the upper side is set to 0 indicating that there is no continuity.

The continuity determining unit 154 performs the same processing as for the upper boundary portion on the lower, left, and right boundary portions in addition to the upper boundary portion,
Thereby, the fourth bit B ₄ , the third bit B ₃ , the second bit B ₂ , and the first bit B of the 4-bit class code
_1, the upper side of the target block, lower, left, and outputs the set value representing the continuity of the image in the right respective boundary.

Here, in order to simplify the explanation, the target pixel is classified according to the presence / absence of continuity at the upper, lower, left, and right boundaries of the target block. However, the target pixel is classified as follows. In addition to the presence / absence of continuity at the boundary portion of the block of interest, it is possible to perform it based on the flatness and the like.

Further, as described below, as described above,
Continuity determination unit 154 of () class classification circuit 43 of FIG.
(Fig. 28) outputs the 4th bit of the 4-bit class code.
B _Four, 3rd bit B₃, The second bit B₂Or first
Bit B₁The upper block code and the lower block, respectively
Block-to-block code, left block-to-block code, or right
It is called a code between blocks.

Next, the operation of the class code by the class code operation unit 131 of FIG. 24 will be described with reference to FIG.

As described above, the class code operation unit 131 operates the class code output by the class classification circuit 43 based on the area information of the pixel of interest supplied from the mode control unit 81.

That is, when the attention area is the upper left area, the class code operation unit 131, as shown in FIG. 31A, outputs the 4-bit class code (B ₄ , B ₃ , B ₂ , B ₁ ) is output as it is as the class code of the pixel of interest.

That is, when the attention area is the upper left area and the attention pixel is located at any one of 16 positions in the upper left area, among the values 0 to 15 indicating the position of the attention pixel in the upper left area. Since any one of the above is used as the degenerate pixel position mode of the pixel of interest as it is, the class code operation unit 131 causes the class code operation unit 131 to output the class code (B ₄ , B ₃ , B ₂ , B ₁ ) Is output as it is as the class code of the pixel of interest.

Next, when the attention area is the upper right area, the class code operation unit 131, as shown in FIG. 31B, outputs the 4-bit class code (B ₄ , B ₃ ) output from the class classification circuit 43. , B ₂ , B ₁ ), the left inter-block code B ₂ and the right inter-block code B ₁ are exchanged to generate a new class code (B ₄ , B ₃ , B ₁ , B ₂ ). Output as a pixel class code.

That is, when the attention area is the upper right area, the degenerate pixel position mode of the attention pixel which is the pixel in the upper right area is
As described above, although it is also used as the degenerate pixel position mode of the vertical line-symmetrical pixel in the upper left region which is line-symmetrical with respect to the vertical division line (although it is the same), in this case, In the spatial area, the left-right positional relationship is opposite to that of the line-symmetrical pixel. Thus, the class code operation unit 131, class code classification circuit 43 outputs _{(B 4, B 3, B} 2, B 1) for also, inter left blocks representing continuity of the right and left boundary of the block of interest The operation of exchanging the code B ₂ and the code B ₁ between the right blocks is performed, and a new class code (B ₄ , B ₃ ,
B ₁ , B ₂ ) are output as the class code of the pixel of interest.

Therefore, in this case, the class code output from the class classification circuit 43 is, for example, (0,0,0,1).
Then, the class code operation unit 131 exchanges the second bit and the first bit to (0, 0, 1,
0) is output as the class code of the pixel of interest.

Next, when the attention area is the lower left area, the class code operation unit 131, as shown in FIG. 31C, outputs the 4-bit class code (B ₄ , B ₃ ) output from the class classification circuit 43. , B ₂ , B ₁ ), the upper block code B ₄ and the lower block code B ₃ are replaced with each other to generate a new class code (B ₃ , B ₄ , B ₂ , B ₁ ), which is It is output as the class code of the pixel of interest.

That is, when the attention area is the lower left area, the degenerate pixel position mode of the attention pixel which is the pixel in the lower left area is
As described above, it is also used as the degenerate pixel position mode of the horizontal line-symmetrical pixel in the upper left region located in the line-symmetrical position with respect to the horizontal division line (although the same), but in this case, the target pixel and the horizontal line-symmetrical In the spatial area, the vertical positional relationship is opposite to that of the pixel. Therefore, the class code manipulating unit 131, even for the class codes (B ₄ , B ₃ , B ₂ , B ₁ ) output by the class classification circuit 43, between upper blocks that represent continuity of the upper and lower boundary portions of the target block. The code B ₄ and the code B ₃ between the lower blocks are exchanged, and a new class code (B ₄ , B ₃ ,
B ₂ , B ₁ ) is output as the class code of the pixel of interest.

Next, when the attention area is the lower right area, the class code operation unit 131, as shown in FIG. 31D, outputs the 4-bit class code (B ₄ , B ₄ ) output from the class classification circuit 43. ₃ , B ₂ , B ₁ ), the left inter-block code B ₂ and the right inter-block code B ₁ are exchanged, and the upper inter-block code B ₄ and the lower inter-block code B ₃ are exchanged to obtain a new class code ( B ₃ , B ₄ ,
B ₁ , B ₂ ) is generated and this is output as the class code of the pixel of interest.

That is, when the attention area is the lower right area, the degenerate pixel position mode of the attention pixel which is the pixel in the lower right area is
As described above, it is also used as the degenerate pixel position mode of the point-symmetrical pixel in the upper left region located at the point-symmetrical position with respect to the block center point (although it is the same), but in this case, the target pixel and the point-symmetrical In the spatial area, the pixel has a left-right positional relationship and a vertical positional relationship opposite to each other. Therefore, the class code operation unit 131 also applies the class codes (B ₄ , B ₃ , B ₂ , B ₁ ) output by the class classification circuit 43 to
The code B ₂ between the left blocks and the code B ₁ between the right blocks are exchanged, and the code B ₄ between the upper blocks and the code B ₃ between the lower blocks are exchanged. As a result, a new class is obtained. Code (B
₃ , B ₄ , B ₁ , B ₂ ) are output as the class code of the pixel of interest.

Next, the processing of the coefficient conversion circuit 32 of FIG. 24 will be described with reference to the flowchart of FIG.

In the coefficient conversion circuit 32 of FIG. 24, the same processing as that in FIG. 21 is performed except that the class code operation unit 131 operates the class code output by the class classification circuit 43 in step S137. .

That is, in steps S131 to S136, the same processes as in steps S91 to S96 in FIG. 21 are performed.

However, in step S135, the class tap extraction circuit 42 extracts the class taps described in FIGS. 25 to 27, and in step S136, the class classification circuit 43 in the class classification circuit 43 described in FIGS. 28 to 30. Is done.

Then, in step S136, when the class classification circuit 43 outputs the class code of the pixel of interest, the process proceeds to step S137, and the class code operation unit 13
1 operates the class code output from the class classification circuit 43 based on the area information of the pixel of interest output from the mode control unit 81 as described in FIG. 31, and supplies it to the coefficient table storage unit 44.

When the coefficient table storage unit 44 receives the class code of the pixel of interest from the class code operation unit 131, the process proceeds to step S138 and step S97 in FIG.
As in the case of, the 16 sets of tap coefficients stored at the address corresponding to the class code of the pixel of interest are read out and output to the product-sum operation circuit 45.

Thereafter, the process proceeds to steps S139 to S144 in sequence, and the same processes as those in steps S98 to S103 of FIG. 21 are performed, respectively, and step S144.
In, when it is determined that all the blocks of the DCT coefficient output by the inverse quantization circuit 40 have been processed as the block of interest, the process ends.

Next, FIG. 33 shows an example of the structure of a learning device for carrying out the learning process of the tap coefficient stored in the coefficient table storage section 44 of FIG. Note that, in the figure, parts corresponding to those in FIG. 22 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device of FIG. 33 is basically configured similarly to the case of FIG. 24 except that the class code operation unit 161 is newly provided.

However, in the embodiment of FIG. 33, the class tap extraction circuit 66 and the class classification circuit 67 are the same as those of FIG.
The same processing as in the case of the class tap extraction circuit 42 and the class classification circuit 43 of No. 4 is performed.

The class code operation unit 131 performs the same operation as the case of the class code operation unit 131 of FIG. 24 on the class code output by the class classification circuit 67 based on the area information output by the mode control unit 101.
It is supplied to the normal equation adding circuit 68.

Next, the processing (learning processing) of the learning device in FIG. 33 will be described with reference to the flowchart in FIG.

In the learning device of FIG. 33, step S157.
Then, the class code operation unit 161 causes the class classification circuit 67 to
23 except that the class code output by
The same processing as in the case of is performed.

That is, in steps S151 to S156, the same processes as in steps S111 to S116 of FIG. 23 are performed.

However, in step S155, the class tap extraction circuit 66 extracts the class tap described in FIGS. 25 to 27, and in step S156, the class classification circuit 67 in the class classification circuit 67 described in FIGS. 28 to 30. Is done.

When the class classification circuit 67 outputs the class code of the pixel of interest in step S156, the process proceeds to step S157, and the class code operation unit 16
1 operates the class code output from the class classification circuit 67 based on the area information of the pixel of interest output from the mode control unit 101 as described in FIG. 31, and supplies it to the normal equation addition circuit 68.

After that, the process proceeds to step S158, and thereafter, in steps S158 to S163, the same processes as in steps S117 to S122 of FIG. 23 are performed. As a result, the tap coefficient determination circuit 69 obtains 16 sets of tap coefficients corresponding to each of the 16 degenerate pixel position modes for each class, and supplies the tap coefficients to the address corresponding to each class in the coefficient table storage unit 70. Is stored, and the process ends.

In the embodiments of FIGS. 5, 11, 19, and 24 described above, the inverse quantizing circuit 40 is provided to form the coefficient converting circuit 32. 32 can be configured without providing the inverse quantization circuit 40. However, when the coefficient conversion circuit 32 is configured without providing the inverse quantization circuit 40, a learning device for learning the tap coefficient used in the coefficient conversion circuit 32 (FIG. 8, FIG. 16, FIG. 22, FIG. 33). Also, it is necessary to configure without providing the inverse quantization circuit 64.

Next, in the above-mentioned embodiment, the case where the image data supplied to the encoder 21 in FIG. 2 is a still image and the image data is JPEG encoded in the encoder 21 has been described. The image transmission system No. 2 is also applicable to the case of encoding moving image data.

Therefore, FIG. 35 shows a configuration example of the encoder 21 of FIG. 2 for encoding the image data of a moving image.

In the embodiment shown in FIG. 35, the image data is MPEG coded, for example.

That is, a frame (or field) constituting image data of a moving image which is an object of MPEG encoding.
Are sequentially supplied to the motion detection circuit 201 and the arithmetic unit 202.

The motion detection circuit 201 detects a motion vector in macroblock units for the frame supplied thereto, and supplies it to the entropy coding circuit 206 and the motion compensation circuit 210.

If the image supplied thereto is an I (Intra) picture, the arithmetic unit 202 supplies it to the blocking circuit 203 as it is, and P (Predictive) or B (Bidirec).
In the case of a (tionally predictive) picture, the difference from the prediction image supplied from the motion compensation circuit 100 is calculated, and the difference value is supplied to the blocking circuit 203.

The blocking circuit 203 blocks the output of the arithmetic unit 202 into blocks of 8 × 8 pixels, and
Supply to the circuit 204. The DCT circuit 204 performs a two-dimensional DCT transformation on the block from the blocking circuit 203, and outputs a two-dimensional DCT coefficient obtained as a result of the quantization circuit 205.
Supply to. The quantization circuit 205 quantizes the block-based two-dimensional DCT coefficient from the DCT circuit 203 in a predetermined quantization step, and supplies the quantized DCT coefficient obtained as a result to the entropy coding circuit 206. The entropy encoding circuit 206 entropy-encodes the quantized DCT coefficient from the quantization circuit 205 by variable-length encoding or the like, and adds the motion vector from the motion detection circuit 201 and other necessary information. Then, the entropy coding circuit 206 outputs the coded data obtained as a result as the MPEG coding result.

Quantized DCT output from the quantization circuit 205
Among the coefficients, the I picture and P picture need to be locally decoded to be used as reference pictures for P pictures and B pictures to be coded later. Therefore, in addition to the entropy coding circuit 206, the inverse quantization circuit 207 is also available. Will also be supplied.

The inverse quantization circuit 207 is the quantization circuit 205.
By dequantizing the quantized DCT coefficients from
The dimensional DCT coefficient is supplied to the inverse DCT circuit 208.
The inverse DCT circuit 208 performs two-dimensional inverse DCT conversion on the two-dimensional DCT coefficient from the inverse quantization circuit 207 and outputs it to the calculator 209. The calculator 209 is supplied with not only the output of the inverse DCT circuit 208 but also the prediction image output from the motion compensation circuit 210. The calculator 209 outputs the output of the inverse DCT circuit 208 to the P picture. If it is one,
By adding the output and the output of the motion compensation circuit 210, the original image is decoded and supplied to the motion compensation circuit 210. Further, when the output of the inverse DCT circuit 208 is for an I picture, the arithmetic unit 209 outputs the output as I
Since it is a decoded image of a picture, it is supplied to the motion compensation circuit 210 as it is.

The motion compensation circuit 210 uses the locally decoded image supplied from the arithmetic unit 209 as a reference image and performs motion compensation on the reference image according to the motion vector from the motion detection circuit 201. ,
A predicted image is generated and supplied to the arithmetic units 202 and 209.

Next, FIG. 36 shows a configuration example of the decoder 22 of FIG. 2 when the image data is MPEG encoded in the encoder 21.

The coded data (video stream) output by the encoder 21 (FIG. 35) is supplied to the separation unit 221. The separation unit 221 extracts a coded block pattern (coded block pattern) (hereinafter, appropriately referred to as CBP), a DCT type, a VLC (variable length coding) code of a quantized DCT coefficient, a quantization scale, and a motion vector from encoded data. , Motion compensation type (frame motion type, fie
ld motion type) etc. are output separately.

The DCT coefficient extraction / inverse quantization section 222 receives the VLC code, the quantization scale, and the DCT type of the quantized DCT coefficient output from the separation section 221, and receives the 2
Decode the dimensional DCT coefficients. That is, the DCT coefficient extraction / inverse quantization unit 2 performs variable length decoding on the VLC code of the quantized DCT coefficient output by the separation unit 221, and obtains the quantized DCT coefficient for each block of 8 × 8 pixels. Further, the DCT coefficient extraction / inverse quantization unit 2 uses the quantized DCT for each block.
The coefficient is inversely quantized by the quantization scale output by the separating unit 221, and a two-dimensional DCT coefficient for each block is obtained. The two-dimensional DCT coefficient for each block obtained by the DCT coefficient extraction / inverse quantization section 2 is supplied to the frequency domain motion compensation addition section 223.

On the other hand, the motion compensating section 224 has the separating section 221.
Receives a motion vector and a motion compensation type output from, and reads a frame (or field) serving as a reference image from an image memory 229 described later based on the motion compensation type. Furthermore, the motion compensation unit 224 separates the reference image read from the image memory 229 from the separation unit 221.
Motion compensation according to the motion vector output by
The predicted image obtained as a result is supplied to and stored in the image memory 225.

Here, the block which is the object of decoding now is referred to as a block of interest as appropriate.

The DCT conversion unit 226 recognizes the DCT type of the block of interest output by the DCT coefficient extraction / inverse quantization unit 2 based on the DCT type output by the separation unit 221. Furthermore, the DCT conversion unit 226 determines that the D
Based on the CT type, 8 × 8 pixels of the same size as the block are selected from the predicted image stored in the image memory 225, and the two-dimensional DCT conversion is performed by the two-dimensional DCT conversion. Two-dimensional DCT obtained from this predicted image
The coefficient (hereinafter, appropriately referred to as a predicted DCT coefficient) is the DCT
It is supplied from the conversion unit 226 to the frequency domain motion compensation addition unit 223.

The frequency domain motion compensation adder 223 has D
The 8 × 8 predicted DCT coefficient is supplied from the CT conversion unit 226, and the macroblock including the target block is also supplied from the separation unit 221 (hereinafter, appropriately referred to as the target macroblock).
CBP is supplied, and the target block is supplied from the DCT coefficient extraction / inverse quantization unit 222.

The frequency domain motion compensation adder 223 determines each two-dimensional DCT coefficient of the target block and the corresponding prediction DCT, if necessary, based on the CBP of the target macroblock.
By adding the coefficient and the coefficient, the pixel value of the block of interest is subjected to two-dimensional DCT conversion to obtain a DCT block of interest.

That is, when the target block is intra-coded, the two-dimensional DC of the target block is
Since the T coefficient is a two-dimensional DCT transform of a block of pixel values (pixel block), the frequency domain motion compensation addition unit 223 does not change the block of interest as it is.
Let it be the DCT block of interest.

When the block of interest is non-intra coded, the block of interest (block of pixels) and the difference value (residual image) between the predicted image and the block of interest are two-dimensional DCT. Since it has been converted, the frequency domain motion compensation addition unit 223 is obtained by performing two-dimensional DCT conversion of each 2D DCT coefficient of the block of interest and the 8 × 8 pixel prediction image in the DCT conversion unit 226. The DCT block of interest is obtained by adding the corresponding 8x8 predicted DCT coefficient.

The DCT block of interest obtained by the frequency domain motion compensation adder 223 is the buffer memory 22.
7 and is temporarily stored.

Here, the two-dimensional DCT coefficient of the block stored in the buffer memory 227 is the original image regardless of the picture type of the block and whether the block is intra-coded or non-intra-coded. The pixel value of the above (not the residual image but the original image) is DCT-transformed.

The original image here means an image obtained by adding the residual image and the predicted image, and is the MPEG.
It is not the original image itself that was the target of encoding.

The coefficient conversion circuit 228 is used for the buffer memory 2
By performing the above-described class classification adaptation processing using the two-dimensional DCT coefficient stored in 27, the block of the two-dimensional DCT coefficient is decoded into a block of pixel values and supplied to the image memory 229 and the picture selection unit 230. To do.

The image memory 229 has a coefficient conversion circuit 228.
Of the decoded image data supplied from
The decoded image data of a picture is temporarily stored as a reference image of encoded data that is subsequently decoded. In addition, MPE
In G, the B picture is not used as a reference image. Therefore, if the decoded image output by the coefficient conversion circuit 228 is a B picture image, the decoded image is stored in the image memory 22.
Not remembered in 9.

[0468] The picture selection unit 230 includes the coefficient conversion circuit 2
The decoded image output by the 28 or the frame (or field) of the decoded image stored in the image memory 229,
Select the display order and output. That is, in MPEG, the display order of the frames (or fields) of the image and the decoding order (coding order) do not match, so the picture selection unit 23
In the case of 0, the frames (or fields) of the decoded images arranged in the decoding order are rearranged in the display order and output.

Next, FIG. 37 shows a configuration example of the frequency domain motion compensation addition section 223 and the DCT conversion section 226 of FIG.

The DCT transform unit 226 is composed of a (8 × 8 pixel) sampling unit 241 and a DCT unit 242, and generates a predicted DCT coefficient which is a two-dimensional DCT transform of the predicted image stored in the image memory 225. ing.

That is, the sampling unit 241 is supplied with the predicted image stored in the image memory 225 and the DCT type of the target block (including the macro block) output from the separating unit 221 (FIG. 36). ing.

Here, the image memory 225 is supplied from the motion compensating unit 224 (FIG. 36) with a predicted image of 16 × 16 pixels having the same size as the macroblock. 225 stores a predicted image of 16 × 16 pixels corresponding to the block of interest. Therefore, the image memory 225 has a storage capacity capable of storing an image of at least 16 × 16 pixels.

The sampling section 241 controls the image memory 22.
The 16 × 16 pixel prediction image stored in 5 is sampled according to the DCT type of the target block, and an 8 × 8 pixel prediction image having the same size as the block is generated.

That is, for example, assuming that the encoder 21 encodes image data by the MPEG2 system,
In MPEG2, as the DCT type, frame DCT
And the field DCT can be selected. On the other hand, the 16 × 16 pixel prediction image stored in the image memory 225 has a frame structure and may or may not match the structure of the target block.

Specifically, when the block of interest has a frame structure, the block of interest has a frame structure, and the predicted image of 16 × 16 pixels stored in the image memory 225 also has a frame structure. It has a structure (frame structure).

Therefore, as shown in FIG. 38 (A), if the target block is the upper left, lower left, upper right, or lower right block of the macro block (target macro block),
The sampling unit 241 is, as shown in FIG.
Of the 16 × 16 pixel prediction image stored in the image memory 225, the upper left, lower left, upper right, or lower right 8 × 8 pixels are respectively sampled, and thereby spatially correspond to each pixel of the target block. The predicted image of 8 × 8 pixels at the position to be obtained is obtained and supplied to the DCT unit 242.

Here, in FIG. 38 (see FIG.
Similarly, the shaded line represents an odd line (top field), and the non-shadowed line represents an even line (bottom field).

On the other hand, when the target block has a field structure, the target block has a field structure and the 16 × 16 pixel prediction image stored in the image memory 225 has a frame structure. Becomes

That is, in this case, in the macroblock including the target block (target macroblock), as shown in FIG. 39 (A), the upper eight lines are composed of odd lines (top field), and the lower eight lines are composed. The line is composed of even lines (bottom field).

Therefore, when the block of interest is the block at the upper left of the macro block of interest, 8 × 8 pixels of the block of interest are stored in the image memory 225 as shown in FIG. It corresponds to 8 pixels on the left side of 8 odd lines (shaded lines) in the predicted image of pixels. If the block of interest is the block at the lower left of the macro block of interest, the 8 × 8 pixels of the block of interest are predicted to be 16 × 16 pixels stored in the image memory 225, as shown in FIG. It corresponds to the eight pixels on the left side of the eight even lines (lines not shaded) of the image. Further, when the block of interest is the block on the upper right of the macro block of interest, the 8 × 8 pixels of the block of interest are 16 × 16 pixels stored in the image memory 225 as shown in FIG. Corresponding to the eight pixels on the right side of the eight odd lines (shaded lines) of the prediction image of the target image, and if the target block is the lower right block of the target macro block, The 8 × 8 pixels of the block of interest are 16 × 16 pixels stored in the image memory 225, as shown in FIG.
It corresponds to the eight pixels on the right side of eight even lines (lines not shaded) in the pixel prediction image.

Therefore, when the target block has a field structure, the sampling unit 241 selects from among the 16 × 16 pixel prediction images stored in the image memory 225.
The 8 × 8 pixels as described above, which correspond to the position of the block of interest in the macro block of interest, are sampled to obtain a prediction image of 8 × 8 pixels at a position spatially corresponding to each pixel of the block of interest. And supplies it to the DCT unit 242.

The DCT unit 242 is the sampling unit 241.
The two-dimensional DCT transformation is performed on the 8 × 8 pixel prediction image that is spatially corresponding to each pixel of the block of interest, which is supplied from, to obtain the 8 × 8 prediction DCT coefficient, and to perform the frequency domain motion. It is supplied to the compensation addition unit 223.

In the sampling unit 241, the structure of the block of interest is the DCT of the block of interest (including the macro block of interest) output by the separation unit 221 (FIG. 36).
Recognized based on type.

The frequency domain motion compensation adder 223 uses the DC
T coefficient selection unit 251, addition unit 252, and selection unit 25
3 and includes a DCT coefficient extraction / inverse quantization unit 222.
Two-dimensional DCT of the block of interest supplied from (FIG. 36)
The coefficient and the predicted DCT supplied from the DCT conversion unit 226
The coefficient and the coefficient are added as necessary to obtain the two-dimensional DCT coefficient of the original image of the block of interest.

That is, the DCT coefficient selecting section 251
The two-dimensional DCT coefficient of the target block output by the T coefficient extraction / inverse quantization unit 222 (FIG. 36) and the separation unit 221 (FIG. 3).
The CBP of the target block (including the target macro block) output by 6) is supplied.

If the block of interest is intra-coded, the DCT coefficient selecting section 251 outputs the block of interest as it is. The output of the DCT coefficient selecting unit 251 is supplied to the adding unit 252 and the selecting unit 253.

If the block of interest is intra-coded, the adding section 252 does not perform any particular processing, and the selecting section 253 selects the output of the DCT coefficient selecting section 251 and the buffer memory 227 at the subsequent stage. (FIG. 36).

Therefore, when the target block is intra-coded, that is, the two-dimensional D of the target block output from the DCT coefficient extraction / inverse quantization unit 222 (FIG. 36).
If the CT coefficient is a two-dimensional DCT transform of the original image of the block of interest, the two-dimensional D of the block of interest output by the DCT coefficient extraction / inverse quantization unit 222 (FIG. 36).
The CT coefficient remains unchanged as it is in the buffer memory 227 (see FIG.
6).

On the other hand, when the block of interest is non-intra coded, the DCT coefficient selecting unit 251 refers to the CBP of the block of interest and refers to the two-dimensional D of the residual image.
Recognize the presence or absence of CT coefficients. That is, when the target block is non-intra-coded, the two-dimensional DCT coefficient of the residual image is basically arranged in the target block, but the two-dimensional DCT coefficient of the residual image ( When all the quantized two-dimensional DCT coefficients) become 0,
The CBP is set to 0, and the two-dimensional DCT coefficient is not arranged.
Then, in this case, the image of the target block matches the predicted image.

Therefore, if the block of interest is non-intra coded and its CBP is 0, DCT coefficient selecting section 251 outputs 0 as the two-dimensional DCT coefficient of the residual image.

Further, the DCT coefficient selecting section 251 is a block in which the block of interest is non-intra coded, and its C
When BP is 1, the DCT coefficient extraction / inverse quantization unit 22
Since the two-dimensional DCT coefficient of the residual image is arranged in the block of interest output by 2 (FIG. 36), its two-dimensional D
Select and output the CT coefficient.

The output of the DCT coefficient selecting unit 251 is supplied to the adding unit 252 and the selecting unit 253, as described above.

If the target block is non-intra-coded, the adder 252 selects the DCT coefficient selector 2
51 output and the DCT conversion unit 226 (the DCT unit 24 of the
2) and the two-dimensional DCT coefficient of the predicted image output by the 2) are added.
The dimensional DCT coefficient is obtained and supplied to the selection unit 253.

When the target block is non-intra coded, selecting section 253 selects the output of arithmetic section 72 and supplies it to buffer memory 227 (FIG. 36) in the subsequent stage.

Therefore, when the target block is non-intra coded, the CB of the target block is
When P is 0, the image of the block of interest matches the predicted image. Therefore, in the addition unit 252, 0 output from the DCT coefficient selection unit 251 and the DCT coefficient of the predicted image output from the DCT unit 242 (predicted DCT 2D DC of the original image of the block of interest by adding
The T coefficient is determined.

When the CBP of the target block is 1, the DCT coefficient selecting section 25 in the adding section 252 is added.
1 outputs the DCT coefficient of the residual image of the block of interest,
By adding the DCT coefficient (predicted DCT coefficient) of the predicted image output by the DCT unit 242, the two-dimensional DCT coefficient of the original image of the block of interest is also obtained.

Then, in the selecting unit 253, the adding unit 252
In 2), the two-dimensional DCT coefficient of the original image of the block of interest obtained as described above is selected and output.

In the decoder 22 of FIG. 36, the residual image and the predicted image are added in the frequency domain to obtain the two-dimensional DCT coefficient of the original image.
Two-dimensional DCT coefficient of residual image and two-dimensional DC of predicted image
The two-dimensional DCT coefficient of the original image is calculated by adding the T coefficient and the two-dimensional DCT coefficient of the original image. However, the two-dimensional DCT coefficient of the original image is obtained by MPEG decoding the encoded data, for example. It is also possible to obtain the decoded image data by performing two-dimensional DCT conversion in block units.

The two-dimensional DCT coefficient of the original image is D
The output of the CT coefficient extraction / inverse quantization unit 222 is the two-dimensional inverse DC
A residual image obtained by T-transforming and a prediction image obtained by performing motion compensation using the output of the coefficient conversion circuit 228 as a reference image are added in the time domain,
The addition result may be obtained by two-dimensional DCT conversion.

Next, FIG. 40 shows the coefficient conversion circuit 2 of FIG.
28 shows a configuration example of No. 28. In the figure, the parts corresponding to the case of the coefficient conversion circuit 32 of FIG.
The same reference numerals are given, and the description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 228 of FIG. 40 is basically configured similarly to the coefficient conversion circuit 32 of FIG. 11 except that the inverse quantization circuit 40 is not provided.

Therefore, in the coefficient conversion circuit 228 of FIG. 40, the two-dimensional DCT coefficient stored in the buffer memory 227 (FIG. 36) is basically processed in the same manner as in the coefficient conversion circuit 32 of FIG. Is decoded into a pixel value.

However, in MPEG, there are three picture types, I-picture, P-picture, and B-picture, depending on the difference in the prediction method, so the coefficient table storage unit 4 in FIG.
4, tap coefficients are stored for each of the three picture types, and decoding of an image of each picture type is performed by
The tap coefficient for the picture type is used.

Therefore, the learning of the tap coefficient used in the coefficient conversion circuit 228 of FIG. 40 is performed for each picture type.

That is, FIG. 41 shows the coefficient conversion circuit 2 of FIG.
28 illustrates a configuration example of an embodiment of a learning device when learning tap coefficients for an I picture used in No. 28.
Note that, in the figure, parts corresponding to those in FIG. 16 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

Image data for learning (here, image data of a moving image) is supplied to the student data generation unit 261, and the student data generation unit 261 converts the image data for learning from the student image data. Generate data.

[0506] That is, the student data generation unit 261 uses the MPE
The G encoder 262, the separation unit 263, and the DCT coefficient extraction / inverse quantization unit 264 are included.

The MPEG encoder 262 has the same structure as the encoder 21 of FIG. 35, and sequentially encodes moving image data as image data for learning by MPEG encoding (for example, by the MPEG2 system), The resulting encoded data is supplied to the separating unit 263. The separation unit 263 and the DCT coefficient extraction / inverse quantization unit 264 correspond to the separation unit 221 and the DCT coefficient extraction / inverse quantization unit 22 in FIG.
It has the same configuration as that of No. 2 and separates and extracts quantized DCT coefficients from encoded data, dequantizes and outputs.

The separation unit 263 and the DCT coefficient extraction / inverse quantization unit 264 perform processing only on I pictures.

Therefore, the DCT coefficient extraction / inverse quantization unit 26
From 4, the two-dimensional DCT coefficient for the I picture is output, and the data for the P picture and B picture is not output.

The two-dimensional DCT coefficient for the I picture output from the DCT coefficient extraction / inverse quantization unit 264 is supplied to the prediction tap extraction circuit 65 and the class tap extraction circuit 66 as student data.

On the other hand, the image data for learning is supplied as teacher data to the normal equation addition circuit 68, and thereafter, the prediction tap extraction circuit 65, the class tap extraction circuit 66, the class classification circuit 67, the normal equation addition circuit 68, In the tap coefficient determination circuit 69, the coefficient table storage unit 70, the mode control unit 101, and the code control unit 102, the same processing as in FIG. 16 is performed using the above-described student data and teacher data. A tap coefficient for a picture is obtained for each class and for each degenerate pixel position mode.

Next, FIG. 42 shows the coefficient conversion circuit 2 of FIG.
28 shows an example of the configuration of an embodiment of a learning device for learning the tap coefficient for the P picture used in No. 28.
Note that, in the figure, parts corresponding to those in FIG. 16 or FIG. 41 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device of FIG. 42 is basically configured in the same manner as in the case of FIG. 41 except that a student data generation unit 271 is provided instead of the student data generation unit 261.

Here, since the P picture is predictively coded (non-intra coded) with the I or P picture decoded (encoded) preceding in time as the reference image, that is, the P picture is predicted from the original image. The residual image obtained by subtracting the image is a two-dimensional D
Since CT conversion is performed, the decoder 22 of FIG. 36 performs motion compensation on the previously decoded I or P picture, and as a result, the 2D DCT coefficient of the predicted image and the 2D of the residual image of the P picture are obtained. After the addition of the DCT coefficient, the coefficient conversion circuit 228 performs the class classification adaptation processing.

Therefore, in learning the tap coefficient for the P picture, it is necessary to use, as the student data, the one obtained by adding the two-dimensional DCT coefficient of the prediction image to the two-dimensional DCT coefficient of the residual image of the P picture.

By the way, a predicted picture of a P picture can be obtained by using the previously decoded I or P picture as a reference picture and performing motion compensation on the reference picture. There are no tap coefficients for P-pictures because we are trying to learn the tap coefficients.

On the other hand, the tap coefficient for I-picture is shown in FIG.
In the first learning device, it can be obtained in advance.

Therefore, in the learning apparatus shown in FIG. 42, the student data generation unit 271 generates student data from a block of a P picture that is predictively coded using an I picture as a reference image.

That is, in the student data generation unit 271,
The MPEG encoder 272 is the encoder 21 of FIG.
The image data for learning is MPEG-encoded and the encoded data obtained as a result is separated by the separation unit 273.
Supply to. The separation unit 273 and the DCT coefficient extraction / inverse quantization unit 274 are the same as the separation unit 221 and the DCT coefficient extraction / dequantization unit 274 of FIG.
The dequantization unit 222 has the same configuration as that of the dequantization unit 222, and separates and extracts quantized DCT coefficients from encoded data and dequantizes them.

The separation unit 273 and the DCT coefficient extraction / inverse quantization unit 274 perform processing only on the predictively coded blocks of the I picture and P picture. Further, the separating unit 273 uses the encoded data to quantize the DCT.
In addition to the coefficient, side information such as a quantization scale, a DCT type, a motion vector, etc. is separated as necessary.

The DCT coefficient extraction / inverse quantization unit 274 uses the I
When the two-dimensional DCT coefficient of the picture is obtained, the two-dimensional DCT coefficient of the I picture is supplied to the coefficient conversion circuit 276.

Also, the DCT coefficient extraction / inverse quantization unit 274
Is the two-dimensional D of the predictively coded block of the P picture.
CT coefficient, that is, two-dimensional DCT of the residual image of P picture
When the coefficient is obtained, the two-dimensional DCT coefficient of the residual image is
It is supplied to the frequency domain motion compensation addition unit 275.

Further, the DCT coefficient extraction / inverse quantization unit 27
4 obtains the motion vector of the predictive-coded block of the P picture, the motion vector is set to the motion compensation unit 2
Supply to 78.

The coefficient conversion circuit 276 stores the tap coefficient for the I picture obtained by the learning device of FIG.
The tap coefficient and the DCT coefficient extraction / inverse quantization unit 274
By using the two-dimensional DCT coefficient of the I picture supplied from the same as that in the coefficient conversion circuit 228 of FIG. 40, a decoded image of the I picture is obtained.
The decoded image of this I picture is supplied to and stored in the I picture storage 277 as a reference image.

After that, the motion compensation unit 278 reads from the I picture storage 277 an I picture as a reference image to be motion compensated by the motion vector supplied from the DCT coefficient extraction / inverse quantization unit 274, and the I picture
A predicted image is generated by performing motion compensation on the picture. This predicted image is supplied to and stored in the image memory 279. The predicted image stored in the image memory 279 is
The DCT transform unit 280 transforms it into a two-dimensional DCT coefficient and supplies it to the frequency domain motion compensation addition unit 275. The frequency domain motion compensation addition unit 275 includes a two-dimensional DCT coefficient of the residual image of the P picture supplied from the DCT coefficient extraction / inverse quantization unit 274 and a two-dimensional DCT coefficient of the prediction image supplied from the DCT conversion unit 280. And are added.

That is, the motion compensation unit 278 and the image memory 27
9, the DCT transform unit 280, and the frequency domain motion compensation addition unit 275 are configured in the same manner as the motion compensation unit 224, the image memory 225, the DCT transformation unit 226, and the frequency domain motion compensation addition unit 223 of FIG. Therefore, in the frequency domain motion compensation addition unit 275, the two-dimensional DCT coefficient of the residual image of the P picture from the DCT coefficient extraction / inverse quantization unit 274 and the two-dimensional DCT coefficient of the prediction image from the DCT conversion unit 280 are used. Is added, the original image of the P picture (which is not the original image as described above) becomes 2
A two-dimensional DCT coefficient obtained by the three-dimensional DCT conversion is obtained.

The two-dimensional DCT coefficient of the original image of the P picture obtained by the frequency domain motion compensation addition unit 275 is supplied to the prediction tap extraction circuit 65 and the class tap extraction circuit 66 as student data.

On the other hand, the image data for learning is supplied to the normal equation adding circuit 68 as teacher data, and thereafter, in the prediction tap extracting circuit 65 to the coefficient table storage unit 70, the mode control unit 101, and the code control unit 102. ,
By performing the same processing as in the case of FIG. 16 using the above-mentioned student data and teacher data, the tap coefficients for P pictures (more accurately, for predictively coded blocks of P pictures) are It is obtained for each class and for each degenerate pixel position mode.

Next, FIG. 43 shows the coefficient conversion circuit 2 of FIG.
28 shows a configuration example of an embodiment of a learning device for learning the tap coefficient for B picture used in No. 28.
As with the learning device for learning tap coefficients for B pictures, the learning device for learning tap coefficients for P pictures in FIG. Since it is different from the learning device for learning, only the student data generation unit of the learning device for learning the tap coefficient for B picture is shown in FIG. 43.

Similarly to P pictures, B pictures are also predictively coded (non-intra coded) using I or P pictures decoded earlier in time as reference pictures.
That is, since the residual image obtained by subtracting the predicted image from the original image is two-dimensionally DCT-transformed, the decoder 22 shown in FIG. 36 performs motion compensation on the previously decoded I or P picture, and the predicted image obtained as a result. After adding the two-dimensional DCT coefficient of and the two-dimensional DCT coefficient of the residual image of the B picture,
The coefficient conversion circuit 228 is adapted to perform class classification adaptation processing.

Therefore, in learning the tap coefficient for the B picture, it is necessary to use, as the student data, the one obtained by adding the two-dimensional DCT coefficient of the prediction image to the two-dimensional DCT coefficient of the residual image of the B picture.

Therefore, the learning apparatus shown in FIG. 43 is designed to generate such student data for learning.

That is, the MPEG encoder 291 is supplied with the image data for learning. MP
The EG encoder 291 is configured similarly to the encoder 21 of FIG. 35, MPEG-encodes the image data for learning, and the encoded data obtained as a result is separated by the separation unit 29.
Supply to 2. The separation unit 292 and the DCT coefficient extraction / inverse quantization unit 293 are configured similarly to the separation unit 221 and the DCT coefficient extraction / inverse quantization unit 222 of FIG. 36, respectively.
From the encoded data, the quantized DCT coefficient is separated and extracted,
Dequantize and output.

The two-dimensional DCT coefficient of the I picture output by the DCT coefficient extraction / inverse quantization unit 293 is the coefficient conversion circuit 2
95. Also, the two-dimensional DCT coefficient of the P picture output by the DCT coefficient extraction / inverse quantization section 293 is supplied to the frequency domain motion compensation addition section 294. further,
The two-dimensional DCT coefficient of the B picture output by the DCT coefficient extraction / inverse quantization section 293 is the frequency domain motion compensation addition section 3
It is supplied to 05.

Here, the separating unit 292 determines, from the encoded data, not only the quantized DCT coefficient but also the side information such as the quantization scale, the DCT type, and the moving vector.
The DCT coefficient extraction / inverse quantization unit 29 is separated if necessary.
Via 3 to the required blocks. Regarding the side information, in the embodiment of FIG. 43, the supply of the motion vector of the P picture from the DCT coefficient extraction / inverse quantization unit 293 to the motion compensation unit 297, and the DCT
Only the supply of B-picture motion vectors from the coefficient extractor / inverse quantizer 293 to the motion compensator 302 is shown.

Also, the DCT coefficient extraction / inverse quantization unit 293
Is the two-dimensional D of the predictively coded block of the B picture.
Only the CT coefficient, that is, the two-dimensional DCT coefficient of the residual image,
It is supplied to the frequency domain motion compensation addition unit 305.

Further, the DCT coefficient extraction / inverse quantization unit 29
For the P picture, 3 supplies only the two-dimensional DCT coefficient of the predictively coded block to the frequency domain motion compensation adder 294. In the DCT coefficient extraction / inverse quantization unit 293, the two-dimensional DCT coefficients of the intra-coded blocks of P and B pictures are converted into the coefficient conversion circuit 29.
5 and is processed in the same way as the intra-coded I picture.

The coefficient conversion circuit 295 stores the tap coefficient for I picture obtained by the learning device of FIG.
The tap coefficient and the DCT coefficient extraction / inverse quantization unit 274
Using the two-dimensional DCT coefficient of the I picture block or the two-dimensional DCT coefficient of the intra-coded P picture block supplied from the above, the same processing as in the coefficient conversion circuit 228 of FIG. 40 is performed. By I
A decoded image of a picture or a decoded image of an intra-coded P picture is obtained. The decoded image of the I picture is supplied to and stored in the I picture storage 296 as a reference image, and the decoded image of the P picture is supplied to and stored in the P picture storage 301.

After that, the motion compensating section 297 reads from the I picture storage 296 an I picture as a reference picture to be motion compensated by the motion vector supplied from the DCT coefficient extracting / inverse quantizing section 293, and the I picture
By performing motion compensation on the picture, a predicted image of the P picture is generated. The predicted image of the P picture is supplied to and stored in the image memory 298. The predicted image stored in the image memory 298 is stored in the DCT conversion unit 299.
It is converted into a three-dimensional DCT coefficient and supplied to the frequency domain motion compensation addition unit 294. Frequency domain motion compensation adder 294
Adds the two-dimensional DCT coefficient of the P-picture residual image supplied from the DCT coefficient extraction / inverse quantization section 293 and the two-dimensional DCT coefficient of the predicted image supplied from the DCT conversion section 299.

That is, the motion compensation unit 297 and the image memory 29
8, the DCT transform unit 299, and the frequency domain motion compensation addition unit 294 are configured similarly to the motion compensation unit 224, the image memory 225, the DCT conversion unit 226, and the frequency domain motion compensation addition unit 223 of FIG. Therefore, in the frequency domain motion compensation addition unit 294, the two-dimensional DCT coefficient of the residual image of the P picture from the DCT coefficient extraction / inverse quantization unit 293 and the two-dimensional DCT coefficient of the predicted image from the DCT conversion unit 299. Is added, the original image of the P picture (which is not the original image as described above) becomes 2
A two-dimensional DCT coefficient obtained by the three-dimensional DCT conversion is obtained.

The two-dimensional DCT coefficient of the original image of the P picture obtained by the frequency domain motion compensation addition unit 294 is supplied to the coefficient conversion circuit 300.

The coefficient conversion circuit 300 stores the tap coefficient for P picture obtained by the learning device of FIG.
The tap coefficient and the DCT coefficient extraction / inverse quantization unit 294
By using the two-dimensional DCT coefficient of the block of the non-intra coded P picture supplied from the above, the same processing as in the coefficient conversion circuit 228 of FIG. Get the decoded image of. The non-intra-coded P picture decoded image is supplied to the P picture storage 301 and stored therein. As described above, the P picture storage 301 also stores the decoded image of the intra-coded block of the P picture supplied from the coefficient conversion circuit 295.

After that, the motion compensating unit 302 stores the I or P picture as the reference image to be motion compensated by the motion vector of the B picture supplied from the DCT coefficient extracting / inverse quantizing unit 293 into the I picture storage 296.
Alternatively, a predicted image of a B picture is generated by reading from the P picture storage 301 and subjecting the I or P picture to motion compensation. The predicted image of this B picture is supplied to and stored in the image memory 303. The predicted image stored in the image memory 303 is the DCT conversion unit 30.
In step 4, it is converted into a two-dimensional DCT coefficient and supplied to the frequency domain motion compensation addition section 305. The frequency domain motion compensation addition unit 305 has a two-dimensional DCT coefficient of the residual image of the B picture supplied from the DCT coefficient extraction / inverse quantization unit 293 and a two-dimensional DCT coefficient of the prediction image supplied from the DCT conversion unit 304. And are added.

That is, the motion compensation unit 302 and the image memory 30.
3, the DCT transform unit 304, and the frequency domain motion compensation addition unit 305 are configured similarly to the motion compensation unit 224, the image memory 225, the DCT transformation unit 226, and the frequency domain motion compensation addition unit 223 of FIG. Therefore, in the frequency domain motion compensation addition unit 305, the two-dimensional DCT coefficient of the residual image of the B picture from the DCT coefficient extraction / inverse quantization unit 293 and the two-dimensional DCT coefficient of the predicted image from the DCT conversion unit 304 are calculated. Is added to the original image of the B picture (which is not the original image as described above).
A two-dimensional DCT coefficient obtained by the three-dimensional DCT conversion is obtained.

The two-dimensional DCT coefficient of the original image of the B picture obtained by the frequency domain motion compensation addition unit 305 is output as student data. Then, thereafter, the same processing as in the case of the learning device of FIG. 41 is performed, whereby
A tap coefficient for a B picture (more precisely, for a predictively coded block of the B picture) is obtained for each class and for each degenerate pixel position mode.

In the coefficient table storage unit 44 of FIG. 40, 1 obtained for each class is obtained by the above learning.
The tap coefficients for the I, P, and B pictures for each of the six degenerate pixel position modes are stored.

In the coefficient conversion circuit 228 of FIG. 40,
For I picture blocks, class classification adaptation processing is performed using tap coefficients for I pictures. Also,
For blocks of P pictures or B pictures, if the blocks are predictively coded (non-intra coded), the class classification adaptation processing is performed using tap coefficients for P pictures or B pictures, respectively. , If the block is intra-coded, then I
The class classification adaptation process is performed using the tap coefficient for the picture.

In the above case, the tap coefficient for each of I, P and B pictures is learned.
Learning the tap coefficient for a picture can be omitted. In this case, the coefficient conversion circuit 228 of FIG.
For each of the predictively coded blocks of the picture and the B picture, for example, the class classification adaptation process is performed using the tap coefficient for the P picture.

Further, in the above-mentioned case, the tap coefficient is learned for each picture type, but the tap coefficient can be learned regardless of the picture type.

Next, FIG. 44 shows the coefficient conversion circuit 2 of FIG.
28 shows a second configuration example of No. 28. In addition, in FIG.
The parts corresponding to those in the coefficient conversion circuit 32 are given the same reference numerals, and the description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 228 of FIG.
19 except that the inverse quantization circuit 40 is not provided.
The coefficient conversion circuit 32 of FIG.

Therefore, in the coefficient conversion circuit 228 of FIG. 44, the two-dimensional DCT coefficient stored in the buffer memory 227 (FIG. 36) is basically processed in the same manner as in the coefficient conversion circuit 32 of FIG. Is decoded into a pixel value.

However, also in the embodiment of FIG. 44, as in the case of FIG. 40, the tap coefficient is stored in the coefficient table storage unit 44 for each of the three picture types, and the image of each picture type is stored. Is decoded using the tap coefficient for the picture type.

Next, FIG. 45 shows the coefficient conversion circuit 2 of FIG.
28 illustrates a configuration example of an embodiment of a learning device when learning tap coefficients for an I picture used in No. 28.
Note that, in the figure, parts corresponding to those in the case of FIG. 22 or FIG. 41 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

In the embodiment shown in FIG. 45, the student data generation unit 261 operates in the same manner as in the case shown in FIG.
Student data is generated.

Then, the image data for learning is supplied to the normal equation adding circuit 68 as teacher data, and
Prediction tap extraction circuit 65, class tap extraction circuit 66,
Class classification circuit 67, normal equation addition circuit 68, tap coefficient determination circuit 69, coefficient table storage unit 70, mode control unit 101, code control unit 102, tap structure storage unit 12
1 and the inverse DCT circuit 122 perform the same processing as in the case of FIG. 22 using the above-mentioned student data and teacher data, so that the tap coefficient for I-picture is reduced for each class and for each degeneration. It is obtained for each pixel position mode.

Next, FIG. 46 shows the coefficient conversion circuit 2 of FIG.
28 shows an example of the configuration of an embodiment of a learning device for learning the tap coefficient for the P picture used in No. 28.
Note that, in the figure, parts corresponding to those in the case of FIG. 22, FIG. 42, or FIG. 45 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, FIG.
The learning device of No. 6 is basically configured similarly to the case in FIG. 45 except that a student data generation unit 271 is provided instead of the student data generation unit 261.

In the embodiment shown in FIG. 46, in the student data generation unit 271, as in the case of FIG. 42,
Student data is generated.

Then, the image data for learning is supplied to the normal equation adding circuit 68 as teacher data, and
Prediction tap extraction circuit 65, class tap extraction circuit 66,
Class classification circuit 67, normal equation addition circuit 68, tap coefficient determination circuit 69, coefficient table storage unit 70, mode control unit 101, code control unit 102, tap structure storage unit 12
1, and the inverse DCT circuit 122 performs the same processing as in FIG. 22 using the above-described student data and teacher data, so that the tap coefficient for the P picture is reduced for each class and for each degeneration. It is obtained for each pixel position mode.

Next, FIG. 47 shows the coefficient conversion circuit 2 of FIG.
28 shows a configuration example of an embodiment of a learning device for learning the tap coefficient for B picture used in No. 28.
As with the learning device for learning tap coefficients for B pictures, the learning device for learning tap coefficients for P pictures uses only the student data generation unit for tap coefficients for I pictures in FIG. 45. Since it is different from the learning device for learning, only the student data generation unit of the learning device for learning tap coefficients for B pictures is shown in FIG. 47.

In the embodiment of FIG. 47, student data is generated in the same manner as in the case of FIG.

Thereafter, the same processing as in the case of the learning device of FIG. 45 is performed, whereby tap coefficients for B pictures are obtained for each class and for each degenerate pixel position mode.

In the coefficient table storage unit 44 of FIG. 44, 1 obtained for each class is obtained by the above learning.
The tap coefficients for the I, P, and B pictures for each of the six degenerate pixel position modes are stored.

In the coefficient conversion circuit 228 of FIG. 44,
For I picture blocks, class classification adaptation processing is performed using tap coefficients for I pictures. Also,
For blocks of P pictures or B pictures, if the blocks are predictively coded, class classification adaptation processing is performed using tap coefficients for P pictures or B pictures, respectively, but the blocks are intra-coded. If so, the class classification adaptation process is performed using the tap coefficient for the I picture.

In the above case, the tap coefficient for each of I, P and B pictures is learned.
Learning the tap coefficient for a picture can be omitted. In this case, the coefficient conversion circuit 228 of FIG.
For each of the predictively coded blocks of the picture and the B picture, for example, the class classification adaptation process is performed using the tap coefficient for the P picture.

Further, in the above case, the tap coefficient is learned for each picture type, but the tap coefficient can be learned regardless of the picture type.

Next, FIG. 48 shows the coefficient conversion circuit 2 of FIG.
28 shows a third configuration example of No. 28. In addition, in FIG.
The parts corresponding to those in the coefficient conversion circuit 32 are given the same reference numerals, and the description thereof will be omitted below as appropriate. That is, the coefficient conversion circuit 228 of FIG.
24, except that the inverse quantization circuit 40 is not provided.
The coefficient conversion circuit 32 of FIG.

Therefore, in the coefficient conversion circuit 228 of FIG. 48, the two-dimensional DCT coefficient stored in the buffer memory 227 (FIG. 36) is basically processed in the same manner as in the coefficient conversion circuit 32 of FIG. Is decoded into a pixel value.

However, also in the embodiment of FIG. 48, as in the case of FIG. 40, the tap coefficient is stored in the coefficient table storage unit 44 for each of the three picture types, and the image of each picture type is stored. Is decoded using the tap coefficient for the picture type.

Next, FIG. 49 shows the coefficient conversion circuit 2 of FIG.
28 illustrates a configuration example of an embodiment of a learning device when learning tap coefficients for an I picture used in No. 28.
Note that, in the figure, parts corresponding to those in the case of FIG. 33 or FIG. 41 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

In the embodiment shown in FIG. 49, the student data generation unit 261 operates in the same manner as in FIG.
Student data is generated.

Then, the image data for learning is supplied to the normal equation adding circuit 68 as teacher data, and
Prediction tap extraction circuit 65, class tap extraction circuit 66,
Class classification circuit 67, normal equation addition circuit 68, tap coefficient determination circuit 69, coefficient table storage unit 70, mode control unit 101, code control unit 102, tap structure storage unit 12
1, inverse DCT circuit 122, and class code operation unit 1
In 61, the above-described student data and teacher data are used to perform the same processing as in FIG. 33, whereby tap coefficients for I-picture are obtained for each class and for each degenerate pixel position mode. To be

Next, FIG. 50 shows the coefficient conversion circuit 2 of FIG.
28 shows an example of the configuration of an embodiment of a learning device for learning the tap coefficient for the P picture used in No. 28.
Note that, in the figure, parts corresponding to those in the case of FIG. 33, FIG. 42, or FIG. 49 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, FIG.
The learning device of No. 0 has basically the same configuration as the case in FIG. 49 except that a student data generation unit 271 is provided instead of the student data generation unit 261.

In the embodiment shown in FIG. 50, in the student data generation unit 271, as in the case of FIG.
Student data is generated.

Then, the image data for learning is supplied to the normal equation adding circuit 68 as teacher data, and
Prediction tap extraction circuit 65, class tap extraction circuit 66,
Class classification circuit 67, normal equation addition circuit 68, tap coefficient determination circuit 69, coefficient table storage unit 70, mode control unit 101, code control unit 102, tap structure storage unit 12
1, inverse DCT circuit 122, and class code operation unit 1
At 61, the above-described student data and teacher data are used to perform the same processing as in FIG. 33, whereby the tap coefficient for the P picture is obtained for each class and for each degenerate pixel position mode. To be

Next, FIG. 51 shows the coefficient conversion circuit 2 of FIG.
28 shows a configuration example of an embodiment of a learning device for learning the tap coefficient for B picture used in No. 28.
As with the learning device for learning the tap coefficient for the P picture in FIG. 50, the learning device for learning the tap coefficient for the B picture only uses the student data generation unit to change the tap coefficient for the I picture in FIG. 49. Since it is different from the learning device for learning, FIG. 51 shows only the student data generation unit of the learning device for learning tap coefficients for B pictures.

In the embodiment shown in FIG. 51, student data is generated in the same manner as in the case shown in FIG.

Thereafter, the same processing as in the case of the learning device of FIG. 49 is performed, whereby tap coefficients for B pictures are obtained for each class and for each degenerate pixel position mode.

In the coefficient table storage unit 44 of FIG. 48, 1 obtained for each class is obtained by the above learning.
The tap coefficients for the I, P, and B pictures for each of the six degenerate pixel position modes are stored.

Note that in the coefficient conversion circuit 228 of FIG.
For I picture blocks, class classification adaptation processing is performed using tap coefficients for I pictures. Also,
For blocks of P pictures or B pictures, if the blocks are predictively coded, class classification adaptation processing is performed using tap coefficients for P pictures or B pictures, respectively, but the blocks are intra-coded. If so, the class classification adaptation process is performed using the tap coefficient for the I picture.

Also, in the above case, the tap coefficients for each of I, P and B pictures are learned.
Learning the tap coefficient for a picture can be omitted. In this case, the coefficient conversion circuit 228 of FIG.
For each of the predictively coded blocks of the picture and the B picture, for example, the class classification adaptation process is performed using the tap coefficient for the P picture.

Furthermore, in the above-mentioned case, the tap coefficient is learned for each picture type, but the tap coefficient can be learned regardless of the picture type.

Next, the series of processes described above can be performed by hardware or software. When performing a series of processing by software, the program that constitutes the software,
It is installed on a general-purpose computer or the like.

Therefore, FIG. 52 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

The program is stored in a hard disk 405 or ROM 4 as a recording medium built in the computer.
03 can be recorded in advance.

Alternatively, the program is a flexible disk, a CD-ROM (Compact Disc Read Only Memory),
MO (Magneto Optical) disc, DVD (Digital Versatile)
Disc), magnetic disk, semiconductor memory, or other removable recording medium 411 can be stored (recorded) temporarily or permanently. Such removable recording medium 411 can be provided as so-called package software.

The program is installed in the computer from the removable recording medium 411 as described above, and is also wirelessly transferred from the download site to the computer via an artificial satellite for digital satellite broadcasting or LAN (Local Area). It is possible to transfer the program by wire to a computer via a network such as Network) and the Internet. In the computer, the program thus transferred can be received by the communication unit 408 and installed in the built-in hard disk 405.

[0586] The computer is a CPU (Central Processing
Unit) 402 is built in. The CPU 402 has a bus 4
The input / output interface 410 is connected via 01, and the CPU 402 operates the input / output interface 410 via the input unit 407 including a keyboard, a mouse, a microphone, etc., by the user. When a command is input, the ROM (Read O
The program stored in the nly memory) 403 is executed. Alternatively, the CPU 402 also reads from a program stored in the hard disk 405, a program transferred from a satellite or a network, received by the communication unit 408 and installed in the hard disk 405, or a removable recording medium 411 mounted in the drive 409. The programs read and installed in the hard disk 405 are loaded into RAM (Random Access Memor
y) Load in 404 and execute. This allows the CPU 40
2 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 402 displays the processing result as needed, for example, via the input / output interface 410.
The data is output from the output unit 406 including an LCD (Liquid CryStal Display), a speaker, or the like, or transmitted from the communication unit 408, and further recorded on the hard disk 405.

Here, in the present specification, the processing steps for writing a program for causing a computer to perform various kinds of processing do not necessarily have to be processed in a time series in the order described as a flowchart, but in parallel. Alternatively, it also includes processes that are executed individually (for example, parallel processes or processes by objects).

Further, the program may be processed by one computer or may be processed in a distributed manner by a plurality of computers. Further, the program may be transferred to a remote computer and executed.

In this embodiment, the predicted value of the teacher data is obtained by the linear first-order prediction calculation of the equation (1). However, the predicted value of the teacher data may be, for example,
It is possible to obtain it by using a higher-order equation of second or higher order.

Also, for example, as in the embodiment of FIG. 24 or 33, the class code operating unit 131 or 16
When 1 is provided, the tap structure storage unit 111 and the inverse DC
Inverse DC with the T circuit 112 or the tap structure storage unit 121
It is possible to configure the device without providing the T circuit 122.

Furthermore, in the present embodiment, the operation of the code and class code of the prediction tap and the control of the tap structure of the prediction tap are performed based on the attention area. However, the attention area is the pixel position of the attention pixel. Since it can be recognized from the mode, the operation of the code of the prediction tap can be performed based on the pixel position mode of the pixel of interest.

Further, in the present embodiment, the operation of the code of the prediction tap, the operation of the class code, and the control of the tap structure are described with reference to the upper left area of the block of interest. However, the operation of the code of the prediction tap is It is possible to use the upper right region other than the upper left region, the lower left region, and the lower right region as a reference. That is, the operation of the sign of the prediction tap based on the upper right area, the lower left area, or the lower right area is performed by considering the state in which the target block is rotated 90 degrees to the right, 90 degrees to the left, or 180 degrees, respectively. , Can be performed in the same manner as in the case of using the upper left area as a reference.

The present invention can be applied to the case of decoding image data coded by the MPEG1, 2, 4, JPEG system or other coding system using two-dimensional DCT conversion.

[0594]

According to the first image processing apparatus, the image processing method, and the program of the present invention, the two-dimensional DCT coefficient obtained by the two-dimensional DCT conversion of the image data for each predetermined block is the encoded data. The two-dimensional DCT coefficient, which is obtained from the image data and is used to predict the pixel of interest, which is the pixel of interest, of the pixels forming the image data is extracted as a prediction tap. Further, the information indicating the position of each pixel in the block is set as the pixel position mode, the pixel position mode of the pixel of interest is determined, the pixel position mode of the pixel of interest is degenerated, and the pixel position of which the total number is smaller than the number of pixels in the block is determined. The degenerate pixel position mode, which is the mode, is output. Then, the sign of the two-dimensional DCT coefficient forming the prediction tap is manipulated in accordance with the pixel position mode of the pixel of interest, and the degeneration of the pixel of interest is deduced from the tap coefficient obtained by learning for each degenerate pixel position mode. The tap coefficient corresponding to the pixel position mode is acquired, and the pixel value of the target pixel is predicted using the tap coefficient and the prediction tap. Therefore, it is possible to efficiently decode the encoded data into a decoded image with high image quality.

According to the second image processing apparatus, image processing method, and program of the present invention, learning image data is encoded, and from the encoded data obtained as a result,
At least a two-dimensional DCT coefficient obtained by performing a two-dimensional DCT conversion on the image data for each predetermined block is acquired as student data to be a student for learning. Further, each pixel constituting the image data for learning is used as teacher data to be a teacher for learning, and student data used to predict attention teacher data that is attention teacher data among teacher data is Information that is extracted as a prediction tap and represents the position of each pixel in the block is set as the pixel position mode,
The pixel position mode of the teacher data of interest is determined. After that, the pixel position mode of the teacher data of interest is degenerated, and the degenerated pixel position mode, which is a pixel position mode in which the total number is smaller than the number of pixels in the block, is output. Then, the sign of the student data forming the prediction tap is manipulated corresponding to the pixel position mode of the attention teacher data, and learning is performed for each degenerate pixel position mode using the attention teacher data and the prediction tap. , The tap coefficient for each degenerate pixel position mode is obtained. Therefore, it is possible to improve the learning efficiency of learning the tap coefficient.

[Brief description of drawings]

FIG. 1 is a diagram for explaining conventional JPEG encoding and JPEG decoding.

FIG. 2 is a block diagram showing a configuration example of an embodiment of an image transmission system to which the present invention has been applied.

FIG. 3 is a block diagram showing a first configuration example of a decoder 22.

FIG. 4 is a flowchart illustrating a process of a decoder 22.

FIG. 5 is a block diagram showing a first configuration example of a coefficient conversion circuit 32.

FIG. 6 is a diagram for explaining a prediction tap.

FIG. 7 is a flowchart illustrating processing of the coefficient conversion circuit 32.

FIG. 8 is a block diagram showing a first configuration example of a learning device for learning tap coefficients.

FIG. 9 is a flowchart illustrating a process of the learning device.

FIG. 10 is a diagram for explaining the total number of pixel position modes and the total number of tap coefficients.

FIG. 11 is a block diagram showing a second configuration example of the coefficient conversion circuit 32.

FIG. 12 is a diagram for explaining a degenerate pixel position mode and control of a code of a prediction tap.

FIG. 13 is a diagram for explaining left and right inversion of a cosine waveform that is the basis of a DCT coefficient.

FIG. 14 is a block diagram showing a configuration example of a mode control unit 81.

FIG. 15 is a flowchart illustrating a process of a coefficient conversion circuit 32.

FIG. 16 is a block diagram showing a second configuration example of a learning device for learning tap coefficients.

FIG. 17 is a flowchart illustrating a process of the learning device.

[Fig. 18] Fig. 18 is a diagram for describing a prediction tap configured using DCT coefficients of a target block and adjacent block pixels.

FIG. 19 is a block diagram showing a third configuration example of the coefficient conversion circuit 32.

FIG. 20 is a diagram for explaining inversion of the positions of adjacent block pixels.

FIG. 21 is a flowchart illustrating a process of the coefficient conversion circuit 32.

FIG. 22 is a block diagram showing a third configuration example of a learning device for learning tap coefficients.

FIG. 23 is a flowchart illustrating a process of the learning device.

FIG. 24 is a block diagram showing a fourth configuration example of the coefficient conversion circuit 32.

FIG. 25 is a block diagram showing a configuration example of a class tap extraction circuit 42.

FIG. 26 is a diagram for explaining one-dimensional DCT coefficients.

FIG. 27 is a half-tone photograph displayed on the display for explaining the one-dimensional DCT coefficient.

FIG. 28 is a block diagram showing a configuration example of a class classification circuit 43.

FIG. 29 is a diagram for explaining the processing of the AC power calculation unit 151.

FIG. 30 is a diagram for explaining the processing of an AC inner product calculation unit 153.

FIG. 31 is a diagram for explaining the processing of the class code operation unit 131.

FIG. 32 is a flowchart illustrating processing of the coefficient conversion circuit 32.

FIG. 33 is a block diagram illustrating a fourth configuration example of a learning device that learns tap coefficients.

FIG. 34 is a flowchart illustrating a process of the learning device.

FIG. 35 is a block diagram showing a configuration example of an encoder 21.

[Fig. 36] Fig. 36 is a block diagram illustrating a second configuration example of the decoder 22.

[Fig. 37] Fig. 37 is a block diagram illustrating a configuration example of a frequency domain motion compensation addition unit 223 and a DCT conversion unit 226.

FIG. 38 is a diagram for explaining the processing of the sampling unit 241.

FIG. 39 is a diagram for explaining the processing of the sampling unit 241.

FIG. 40 is a block diagram showing a first configuration example of the coefficient conversion circuit 228.

FIG. 41 is a block diagram showing a first configuration example of a learning device for learning tap coefficients for I pictures.

[Fig. 42] Fig. 42 is a block diagram illustrating a first configuration example of a learning device that learns a tap coefficient for a P picture.

FIG. 43 is a block diagram showing a first configuration example of a learning device for learning tap coefficients for B pictures.

FIG. 44 is a block diagram showing a second configuration example of the coefficient conversion circuit 228.

[Fig. 45] Fig. 45 is a block diagram illustrating a second configuration example of a learning device that learns a tap coefficient for an I picture.

[Fig. 46] Fig. 46 is a block diagram illustrating a second configuration example of a learning device that learns a tap coefficient for P pictures.

[Fig. 47] Fig. 47 is a block diagram illustrating a second configuration example of a learning device that learns tap coefficients for B pictures.

48 is a block diagram showing a third configuration example of the coefficient conversion circuit 228. FIG.

[Fig. 49] Fig. 49 is a block diagram illustrating a third configuration example of a learning device that learns a tap coefficient for an I picture.

[Fig. 50] Fig. 50 is a block diagram illustrating a third configuration example of a learning device that learns a tap coefficient for a P picture.

[Fig. 51] Fig. 51 is a block diagram illustrating a third configuration example of a learning device that learns tap coefficients for B pictures.

FIG. 52 is a block diagram showing a configuration example of an embodiment of a computer to which the present invention has been applied.

[Explanation of symbols]

21 encoder, 22 decoder, 23 recording medium, 24 transmission medium, 31 entropy decoding circuit, 32 coefficient conversion circuit, 33 block decomposition circuit, 40 inverse quantization circuit, 41 prediction tap extraction circuit, 42 class tap extraction circuit, 43 class classification Circuit, 44 coefficient table storage unit, 45 product-sum operation circuit, 46 mode control unit, 61 block circuit, 62 DCT circuit, 63 quantization circuit, 64
Inverse quantization circuit, 65 prediction tap extraction circuit, 66
Class tap extraction circuit, 67 class classification circuit, 6
8 normal equation addition circuit, 69 tap coefficient determination circuit,
70 coefficient table storage unit, 71, 81 mode control unit, 82 code control unit, 91 area determination unit, 92
Pixel position mode determination unit, 93 mode degeneration unit, 1
01 mode control unit, 102 code control unit, 111
Tap structure storage unit, 112 inverse DCT circuit, 12
1 tap structure storage unit, 122 inverse DCT circuit, 1
31 class code operation unit, 141 one-dimensional inverse DCT
Transform unit, 144 boundary one-dimensional DCT coefficient extraction unit, 1
45 Adjacent one-dimensional DCT coefficient extraction unit, 151 AC power calculation unit, 152 DC component extraction unit, 153 A
C inner product calculation unit, 154 continuity determination unit, 161 class code operation unit, 201 motion detection circuit, 202
Arithmetic unit, 203 block circuit, 204 DC
T circuit, 205 quantization circuit, 206 entropy coding circuit, 207 inverse quantization circuit, 208 inverse DCT circuit, 209 arithmetic unit, 210 motion compensation circuit, 221 separation unit, 222 DCT coefficient extraction / inverse quantization unit, 223 frequency Regional motion compensation adder, 22
4 motion compensation unit, 225 image memory, 226 D
CT conversion unit, 227 buffer memory, 228 coefficient conversion circuit, 229 image memory, 230 picture selection unit, 241 sampling unit, 242 DC
T section, 251 DCT coefficient selection section, 252 addition section, 253 selection section, 261 student data generation section,
262 MPEG encoder, 263 separation unit, 2
64 DCT coefficient extraction / inverse quantization unit, 271 student data generation unit, 272 MPEG encoder, 273
Separation unit, 274 DCT coefficient extraction / inverse quantization unit,
275 frequency domain motion compensation addition unit, 276 coefficient conversion circuit, 277 I picture storage, 278 motion compensation unit, 279 image memory, 280 DCT conversion unit, 291 MPEG encoder, 292 separation unit, 293 DCT coefficient extraction / inverse quantization unit, 294
Frequency domain motion compensation addition unit, 295 coefficient conversion circuit, 296 I picture storage, 297 motion compensation unit, 298 image memory, 299 DCT conversion unit, 300 coefficient conversion circuit, 301 P picture storage, 302 motion compensation unit, 303 image memory, 304 DCT transform unit, 305 Frequency domain motion compensation addition unit, 401 bus, 402 CPU, 40
3 ROM, 404 RAM, 405 hard disk,
406 output unit, 407 input unit, 408 communication unit, 409 drive, 410 input / output interface, 411 removable recording medium

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Toshihiko Hamamatsu             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Hideki Otsuka             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F term (reference) 5C059 MA00 MA23 MC11 MC32 MC34                       MC38 ME01 NN01 PP01 PP05                       PP06 PP07 SS06 SS11 SS20                       TA41 TB08 TC02 TC04 TC10                       TC27 TC34 TC41 TC42 TC43                       TD13 TD15 TD16 UA05 UA38                 5C078 AA04 BA21 BA57 CA21 DA01

Claims (36)

[Claims] 1. A two-dimensional DCT (Discrete Cosine Transform) of image data at least for each predetermined block.
An image processing apparatus for decoding encoded data obtained by conversion, comprising: a two-dimensional acquisition of two-dimensional DCT coefficients obtained by performing two-dimensional DCT conversion of the image data for each predetermined block from the encoded data. DCT coefficient acquisition means, and the two-dimensional D used for predicting a pixel of interest, which is a pixel of interest among pixels forming the image data.
Prediction tap extraction means for extracting a CT coefficient as a prediction tap; pixel position mode determination means for determining a pixel position mode of the pixel of interest using information indicating the position of each pixel in the block as a pixel position mode; Pixel position mode degeneracy means for degenerating the pixel position mode of pixels and outputting a degenerate pixel position mode that is a pixel position mode in which the total number is smaller than the number of pixels in the block, and corresponding to the pixel position mode of the pixel of interest. , A code manipulating means for manipulating the sign of the two-dimensional DCT coefficient forming the prediction tap, and a tap coefficient obtained by performing learning for each of the degenerate pixel position modes, to a degenerate pixel position mode of the pixel of interest. The corresponding tap coefficient is acquired, and the pixel value of the pixel of interest is predicted using the tap coefficient and the prediction tap. An image processing apparatus, comprising: a prediction unit for measuring. 2. The code operating means operates the code of a tap coefficient corresponding to the degenerate pixel position mode of the pixel of interest, instead of the two-dimensional DCT coefficient forming the prediction tap. Item 1. The image processing device according to item 1. 3. An area determination unit that determines an attention area in which the attention pixel is located among a plurality of areas in a attention block that is a block including the attention pixel, wherein the code operation unit is configured to change the attention area to the attention area. The image processing apparatus according to claim 1, wherein the sign of the two-dimensional DCT coefficient forming the prediction tap is correspondingly operated. 4. The area determination means sets any one of four areas of an upper left, an upper right, a lower left, and a lower right obtained by equally dividing the block of interest into two parts in a vertical direction and a horizontal direction. The image processing apparatus according to claim 3, wherein it is determined whether the pixel of interest is located. 5. The prediction tap extraction means extracts at least all of the two-dimensional DCT coefficients of the block of interest as the prediction taps, and the code operation means has the area of interest at the upper left of the block of interest. 2 of the blocks of interest constituting the prediction tap,
If the sign of the dimensional DCT coefficient is left as it is and the region of interest is the upper right region of the block of interest, 2 of the blocks of interest forming the prediction taps are used.
Among the dimensional DCT coefficients, the sign of the one located in the even column of the block of interest is inverted, and when the region of interest is the lower left region of the block of interest, 2 of the blocks of interest forming the prediction tap are reversed.
Among the dimensional DCT coefficients, the sign of the one located in the even-numbered row of the block of interest is inverted, and when the region of interest is the lower right region of the block of interest, 2 of the blocks of interest forming the prediction tap are
The image processing apparatus according to claim 4, wherein among the dimensional DCT coefficients, the sign of the one located in the even column of the target block is inverted and the sign of the one located in the even row of the target block is also inverted. . 6. The pixel position mode degeneracy means outputs, as the degenerate pixel position mode, information representing the position of the pixel of interest in the block of interest when the pixel of interest is located in the upper left region of the block of interest. If the attention area is the upper right area of the attention block, a pixel in the upper left area of the attention block that is line-symmetrical to the attention pixel with respect to a line that divides the attention block into left and right parts. Is output as the degenerate pixel position mode, and when the region of interest is the lower left region of the block of interest, with respect to the line that bisects the block of interest up and down, the line of interest and the line of interest. Information indicating the position of a pixel in the upper left area of the block of interest at a symmetrical position is output as the degenerate pixel position mode, In the lower right area of the block of interest, the block of interest is located symmetrically with respect to the pixel of interest with respect to the intersection point of a line bisecting the block of interest left and right and a line bisecting the block vertically. The image processing apparatus according to claim 4, wherein the information indicating the position of the pixel in the upper left area of is output as the degenerate pixel position mode. 7. The prediction tap extracting means includes at least all of the two-dimensional DCT coefficients of the block of interest and upper pixel information that is information about pixels located above, below, left, or right of the block of interest. , Lower pixel information, left pixel information, or right pixel information,
The image processing apparatus according to claim 4, further comprising a tap structure control unit that extracts the prediction taps and controls a tap structure of the prediction taps corresponding to a pixel position mode of the pixel of interest. 8. The tap structure control means, when the attention area is an upper left area of the attention block, sets the tap structure of the prediction tap to a predetermined default tap structure, and the attention area is the In the case of the upper right region of the block of interest, the tap structure of the prediction tap, the left side pixel information and the right side pixel information in the default tap structure is replaced, the region of interest, the lower left of the block of interest. In the case of a region, the tap structure of the prediction tap is obtained by replacing the upper pixel information and the lower pixel information in the default tap structure, and the attention region is the lower right region of the attention block. In this case, the tap structure of the prediction tap is set to the left side pixel information and the right side pixel information in the default tap structure. Together replaced, the image processing apparatus according to claim 7, characterized in that it is assumed that interchanged with the upper pixel information and the lower pixel information. 9. The class classification is performed to classify the pixel of interest into any one of a plurality of classes,
Further comprising a class classification means for outputting a class code representing the class of the pixel of interest, the prediction means, of the tap coefficient obtained by learning for each class and for each degenerate pixel position mode From the tap coefficient corresponding to the class represented by the class code of the pixel of interest, obtain a tap coefficient corresponding to the degenerate pixel position mode of the pixel of interest, and using the tap coefficient and the prediction tap, The image processing apparatus according to claim 1, wherein a pixel value is predicted. 10. The class code operating means for operating the class code of the pixel of interest in accordance with the pixel position mode of the pixel of interest is further provided.
The image processing device according to item 1. 11. An area determination unit that determines an attention area in which the attention pixel is located among a plurality of areas in a attention block that is a block including the attention pixel, wherein the class code operation unit is the attention area. The image processing apparatus according to claim 10, wherein the class code of the pixel of interest is operated in response to the above. 12. The area determining means sets any one of four areas of an upper left, an upper right, a lower left, and a lower right obtained by dividing the block of interest into two equal parts in a vertical direction and a horizontal direction, respectively. The image processing apparatus according to claim 11, wherein it is determined whether the pixel of interest is located. 13. The class classification means is at least:
Based on the upper block information, the lower block information, the left block information, or the right block information, which is information about the block of interest and blocks located on the upper side, the lower side, the left side, or the right side, respectively. , By classifying the pixel of interest, the upper block information, the lower block information, the left block information,
Or, output the class code including at least an upper inter-block code, a lower inter-block code, a left inter-block code, or a right inter-block code respectively corresponding to the right inter-block information, wherein the class code operating means is the attention area. Is the upper left area of the block of interest, the class code output by the class classification means,
As it is, it is output as the class code of the pixel of interest, and when the region of interest is the upper right region of the block of interest, the left inter-block code and the right inter-block code in the class code output by the class classification means are set. Swap, output as the class code of the pixel of interest, when the region of interest is the lower left region of the block of interest, the upper inter-block code and the lower inter-block code in the class code output by the class classification means And output as the class code of the pixel of interest, and when the region of interest is the lower right region of the block of interest, the left inter-block code and the right inter-block code in the class code output by the class classification means. And the upper block cord and the lower block Tsu interchanging the click between codes, claim and outputs as the class code of the pixel of interest 12
The image processing device according to item 1. 14. The image processing apparatus according to claim 1, wherein the prediction unit uses the tap coefficient and the prediction tap to perform linear primary prediction of the pixel value of the target pixel. 15. The encoded data is obtained by encoding the image data in accordance with a JPEG (Joint Photographic Experts Group) or MPEG (Moving Picture Experts Group) standard.
The image processing device according to item 1. 16. A two-dimensional DCT (Discrete Cosine Transfor) for at least predetermined blocks of image data.
m) An image processing method for decoding encoded data obtained by transforming, wherein a two-dimensional DCT coefficient obtained by subjecting the image data to two-dimensional DCT transformation for each predetermined block is obtained from the encoded data. A two-dimensional DCT coefficient acquisition step, and the two-dimensional D used to predict a pixel of interest, which is a pixel of interest among the pixels forming the image data.
A prediction tap extraction step of extracting a CT coefficient as a prediction tap; a pixel position mode determination step of determining a pixel position mode of the target pixel using information indicating the position of each pixel in the block as a pixel position mode; A pixel position mode degeneration step of degenerating the pixel position mode of the pixel and outputting a degenerate pixel position mode which is a pixel position mode in which the total number is smaller than the number of pixels in the block; , A code operation step of operating a code of the two-dimensional DCT coefficient forming the prediction tap, and a tap coefficient obtained by performing learning for each of the degenerate pixel position modes, to a degenerate pixel position mode of the pixel of interest. The corresponding tap coefficient is acquired and the tap coefficient and the prediction tap are used to perform the attention. An image processing method comprising: a prediction step of predicting a pixel value of a pixel. 17. A two-dimensional DCT (Discrete Cosine Transfor) for at least predetermined blocks of image data.
m) A program that causes a computer to perform image processing for decoding encoded data obtained by conversion, wherein the two-dimensional DCT coefficient obtained by performing two-dimensional DCT conversion on the image data for each predetermined block is A two-dimensional DCT coefficient acquisition step of acquiring from encoded data, and the two-dimensional D used to predict a pixel of interest, which is a pixel of interest among pixels forming the image data
A prediction tap extraction step of extracting a CT coefficient as a prediction tap; a pixel position mode determination step of determining a pixel position mode of the target pixel using information indicating the position of each pixel in the block as a pixel position mode; A pixel position mode degeneration step of degenerating the pixel position mode of the pixel and outputting a degenerate pixel position mode which is a pixel position mode in which the total number is smaller than the number of pixels in the block; , A code operation step of operating a code of the two-dimensional DCT coefficient forming the prediction tap, and a tap coefficient obtained by performing learning for each of the degenerate pixel position modes, to a degenerate pixel position mode of the pixel of interest. The corresponding tap coefficient is acquired and the tap coefficient and the prediction tap are used to perform the attention. And a prediction step of predicting a pixel value of a pixel. 18. A two-dimensional DCT (Discrete Cosine Transfor) for at least every predetermined block of image data.
m) A recording medium in which a program for causing a computer to perform image processing for decoding encoded data obtained by conversion is recorded, which is obtained by performing two-dimensional DCT conversion on the image data in predetermined blocks. A two-dimensional DCT coefficient acquisition step of acquiring a two-dimensional DCT coefficient from the coded data; and the two-dimensional DCT coefficient used for predicting a pixel of interest, which is a pixel of interest among pixels forming the image data. D
A prediction tap extraction step of extracting a CT coefficient as a prediction tap; a pixel position mode determination step of determining a pixel position mode of the target pixel using information indicating the position of each pixel in the block as a pixel position mode; A pixel position mode degeneration step of degenerating the pixel position mode of the pixel and outputting a degenerate pixel position mode which is a pixel position mode in which the total number is smaller than the number of pixels in the block; , A code operation step of operating a code of the two-dimensional DCT coefficient forming the prediction tap, and a tap coefficient obtained by performing learning for each of the degenerate pixel position modes, to a degenerate pixel position mode of the pixel of interest. The corresponding tap coefficient is acquired and the tap coefficient and the prediction tap are used to perform the attention. A recording medium in which a program including a prediction step of predicting a pixel value of a pixel is recorded. 19. A two-dimensional DCT (Discrete Cosine Transfor) for at least predetermined blocks of image data.
m) An image processing device for learning tap coefficients used for decoding encoded data obtained by converting, encoding image data for learning, and at least the image from the encoded data obtained as a result. Two-dimensional DC obtained by performing two-dimensional DCT conversion on the data for each predetermined block
Student data acquisition means for acquiring a T coefficient as student data to be a student for learning, and each pixel constituting the image data for learning as teacher data to be a teacher for learning, among the teacher data,
Prediction tap extraction means for extracting, as prediction taps, the student data used to predict attention teacher data which is attention data, and information indicating the position of each pixel in the block as a pixel position mode, Pixel position mode determining means for determining a pixel position mode of the teacher data of interest, and a degenerate pixel position mode which is a pixel position mode in which the pixel position mode of the teacher data of interest is degenerated so that the total number is smaller than the number of pixels in the block. Pixel position mode degeneracy means for outputting, code operation means for operating the code of the student data constituting the prediction tap corresponding to the pixel position mode of the teacher data of interest, the teacher data of interest, and the prediction tap By performing learning for each of the degenerate pixel position modes by using And a learning unit that obtains the tap coefficient. 20. The learning means performs learning so that a prediction error of a prediction value of the teacher data obtained by performing a predetermined prediction calculation using the tap coefficient and the prediction tap is statistically minimized. 20. The image processing apparatus according to claim 19, wherein the tap coefficient for each of the degenerate pixel position modes is obtained by performing 21. An area determination unit that determines an attention area in which the attention teacher data is located among a plurality of areas in a attention block that is a block including the attention teacher data, wherein the code operation unit is configured to perform the attention operation. A two-dimensional DCT that is the student data that constitutes the prediction tap corresponding to a region
The image processing apparatus according to claim 19, wherein the sign of the coefficient is operated. 22. The area determining means sets any one of four areas of an upper left, an upper right, a lower left, and a lower right obtained by dividing the block of interest into two equal parts in a vertical direction and a horizontal direction. 22. The image processing apparatus according to claim 21, wherein it is determined whether the attention teacher data is located. 23. The prediction tap extracting means is at least a two-dimensional D that is the student data of the block of interest.
All the CT coefficients are extracted as the prediction taps, and when the attention area is the upper left area of the attention block, the code operation unit is the student data of the attention block that constitutes the prediction tap. If the sign of the three-dimensional DCT coefficient is left as it is and the region of interest is the upper right region of the block of interest, the attention of the two-dimensional DCT coefficient that is the student data of the block of interest forming the prediction tap is selected. If the sign of the one located in the even column of the block is inverted, and if the attention area is the lower left area of the attention block, among the two-dimensional DCT coefficients that are the student data of the attention block that constitutes the prediction tap, , The sign of the even-numbered row of the block of interest is reversed, and the region of interest is a region at the lower right of the block of interest. In the case of the two-dimensional DCT coefficients that are the student data of the block of interest constituting the prediction tap, the signs of those located in the even columns of the block of interest are inverted, and they are located in the even rows of the block of interest. The image processing apparatus according to claim 22, wherein the sign of the object is inverted. 24. When the teacher data of interest is located in the upper left area of the block of interest, the pixel position mode degeneracy means provides information indicating the position of the teacher data of interest in the block of interest with the degenerate pixel position mode. When the attention area is an upper right area of the attention block, a line that bisects the attention block to the left and right is located in the upper left corner of the attention block at a position symmetrical to the attention teacher data. Information indicating the position of the teacher data in the region is output as the degenerate pixel position mode, and when the attention region is the lower left region of the attention block, a line that divides the attention block into upper and lower halves, Information indicating the position of the teacher data in the upper left area of the target block that is line-symmetric with the target teacher data is When the region of interest is the lower right region of the block of interest, the intersection point of the line that bisects the block of interest and the line that bisects the block vertically is 23. The information indicating the position of the teacher data in the upper left area of the block of interest located symmetrically with respect to the teacher data of interest is output as the degenerate pixel position mode.
The image processing device according to item 1. 25. The prediction tap extracting means is at least a two-dimensional D that is the student data of the block of interest.
All the CT coefficients and the upper side pixel information, the lower side pixel information, the left side pixel information, or the right side pixel information, which is information regarding pixels located on the upper side, the lower side, the left side, or the right side of the block of interest, 23. The image processing apparatus according to claim 22, further comprising a tap structure control unit that controls the tap structure of the prediction tap corresponding to the pixel position mode of the teacher data of interest. 26. The tap structure control unit, when the attention area is an upper left area of the attention block, sets the tap structure of the prediction tap to a predetermined default tap structure, and the attention area is the In the case of the upper right region of the block of interest, the tap structure of the prediction tap, the left side pixel information and the right side pixel information in the default tap structure is replaced, the region of interest, the lower left of the block of interest. In the case of a region, the tap structure of the prediction tap is obtained by replacing the upper pixel information and the lower pixel information in the default tap structure, and the attention region is the lower right region of the attention block. In this case, the tap structure of the prediction tap is set to the left pixel information and the right pixel information in the default tap structure. With Order is, the image processing apparatus according to claim 25, characterized in that it is assumed that interchanged with the upper pixel information and the lower pixel information. 27. Further comprising a class classification means for classifying the teacher data of interest into any of a plurality of classes and outputting a class code representing the class of the teacher data of interest. The learning unit performs learning for each of the classes and for each of the degenerate pixel position modes by using the attention teacher data and the prediction taps, and thus for each of the classes and each of the degenerate pixel position modes. 20. The image processing apparatus according to claim 19, wherein the tap coefficient is calculated. 28. The image processing apparatus according to claim 27, further comprising class code operating means for operating a class code of the teacher data of interest corresponding to a pixel position mode of the teacher data of interest. 29. An area determination unit that determines an attention area in which the attention teacher data is located among a plurality of areas in a attention block that is a block including the attention teacher data, the class code operation means is provided. 29. The image processing apparatus according to claim 28, wherein the class code of the teacher data of interest is operated in correspondence with a region of interest. 30. The area determining means sets any one of four areas of an upper left, an upper right, a lower left, and a lower right obtained by dividing the block of interest into two equal parts in a vertical direction and a horizontal direction. 30. The image processing apparatus according to claim 29, wherein it is determined whether the teacher data of interest is located. 31. At least the class classification means,
Based on the upper block information, the lower block information, the left block information, or the right block information, which is information about the block of interest and blocks located on the upper side, the lower side, the left side, or the right side, respectively. , By classifying the noted teacher data into upper block information, lower block information, left block information, or right block information, the upper block code, the lower block code, and the left block code, respectively. The class code including at least an inter-block code or a right-side inter-block code is output, and the class code operation unit outputs a class output by the class classification unit when the attention area is an upper left area of the attention block. the code,
As it is, it is output as the class code of the attention teacher data, and when the attention area is the upper right area of the attention block, the left inter-block code and the right inter-block code in the class code output by the class classification means And output as the class code of the attention teacher data, and when the attention area is the lower left area of the attention block, between the upper inter-block code and the lower block in the class code output by the class classification means. When the attention area is a lower right area of the attention block, the left inter-block code and the right inter-block code in the class code output by the class classification means are replaced with each other. Replace the code between blocks and Interchanging the code between between codes and the lower block, the image processing apparatus according to claim 30, characterized in that the output as a class code of the target teacher data. 32. The learning means statistically minimizes a prediction error of a prediction value of the teacher data obtained by linearly predicting the teacher data using the tap coefficient and the prediction tap. By learning so that
The image processing apparatus according to claim 19, wherein a tap coefficient is obtained for each of the degenerate pixel position modes. 33. The student data acquisition means uses the JPEG (Joint Photographic Expe
rts Group) or MPEG (Moving Picture Experts Gro)
20. The image processing apparatus according to claim 19, wherein the encoded data is obtained by encoding in accordance with the up) standard. 34. At least two-dimensional DCT (Discrete Cosine Transfor
m) An image processing method for learning tap coefficients used for decoding encoded data obtained by conversion, wherein image data for learning is encoded, and at least the image is obtained from the obtained encoded data. Two-dimensional DC obtained by performing two-dimensional DCT conversion on the data for each predetermined block
A student data acquisition step of acquiring a T coefficient as student data to be a student of learning; and each pixel forming the image data for learning as teacher data to be a teacher of learning, of the teacher data,
A prediction tap extraction step of extracting, as a prediction tap, the student data used to predict attention teacher data that is the attention teacher data, and information indicating the position of each pixel in the block as a pixel position mode, A pixel position mode determination step of determining the pixel position mode of the teacher data of interest, and a degenerate pixel position mode which is a pixel position mode in which the pixel position mode of the teacher data of interest is degenerated and the total number is smaller than the number of pixels in the block. A pixel position mode degenerating step of outputting, a code operation step of operating a code of the student data constituting the prediction tap corresponding to the pixel position mode of the teacher data of interest, the teacher data of interest, and the prediction tap By performing learning for each of the degenerate pixel position modes using A learning step of obtaining the tap coefficient for each pixel position mode. 35. At least two-dimensional DCT (Discrete Cosine Transfor) for every predetermined block of image data.
m) A program that causes a computer to perform image processing that learns tap coefficients used to decode encoded data obtained by conversion, encodes image data for learning, and obtains the encoded data obtained as a result. From at least the two-dimensional DC obtained by subjecting the image data to the two-dimensional DCT conversion for each predetermined block.
A student data acquisition step of acquiring a T coefficient as student data to be a student of learning; and each pixel forming the image data for learning as teacher data to be a teacher of learning, of the teacher data,
A prediction tap extraction step of extracting, as a prediction tap, the student data used to predict attention teacher data that is the attention teacher data, and information indicating the position of each pixel in the block as a pixel position mode, A pixel position mode determination step of determining the pixel position mode of the teacher data of interest, and a degenerate pixel position mode which is a pixel position mode in which the pixel position mode of the teacher data of interest is degenerated and the total number is smaller than the number of pixels in the block. A pixel position mode degenerating step of outputting, a code operation step of operating a code of the student data constituting the prediction tap corresponding to the pixel position mode of the teacher data of interest, the teacher data of interest, and the prediction tap By performing learning for each of the degenerate pixel position modes using A learning step of obtaining the tap coefficient for each pixel position mode. 36. Two-dimensional DCT (Discrete Cosine Transfor) for at least predetermined blocks of image data.
m) a recording medium in which a program for causing a computer to perform image processing for learning tap coefficients used for decoding encoded data obtained by conversion is encoded, and for encoding image data for learning, At least two-dimensional DC obtained by performing two-dimensional DCT conversion of the image data for each predetermined block from the encoded data obtained as a result.
A student data acquisition step of acquiring a T coefficient as student data to be a student of learning; and each pixel forming the image data for learning as teacher data to be a teacher of learning, of the teacher data,
A prediction tap extraction step of extracting, as a prediction tap, the student data used to predict attention teacher data that is the attention teacher data, and information indicating the position of each pixel in the block as a pixel position mode, A pixel position mode determination step of determining the pixel position mode of the teacher data of interest, and a degenerate pixel position mode which is a pixel position mode in which the pixel position mode of the teacher data of interest is degenerated and the total number is smaller than the number of pixels in the block. A pixel position mode degenerating step of outputting, a code operation step of operating a code of the student data constituting the prediction tap corresponding to the pixel position mode of the teacher data of interest, the teacher data of interest, and the prediction tap By performing learning for each of the degenerate pixel position modes using And a learning step of obtaining the tap coefficient for each pixel position mode.