JP3107609B2 - Transformation method of visual characteristics to block overlap orthogonal transform domain - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used in an image compression encoding apparatus, and particularly when performing quantization or evaluation in consideration of human visual characteristics, visual characteristics (frequency characteristics).
Is converted into a block overlap orthogonal transform domain.

[0002]

2. Description of the Related Art As a conventional visual characteristic area conversion method, for example, Document 1: "Optimal quantization of DCT coefficients in consideration of visual characteristics in variable length coding" (IEICE Technical Report IE90-101) ) Was disclosed. FIG. 3 is a diagram for explaining this conventional method. Hereinafter, the processing procedure of the conventional method will be described. In FIG. 3, in order to obtain visual characteristics of an orthogonal transform area having an N × N block size, first, block data of an orthogonal transform area in which all N × N block values are A is input.

[0003] The block data is orthogonally transformed to obtain a spatial domain impulse. At this time, the coefficient A is adjusted so that the height of the impulse becomes 1.

[0004] The impulse in the spatial domain is transformed to the spatial frequency domain using a discrete Fourier transform (DFT).

The spatial frequency domain data is also multiplied by the spatial frequency domain visual characteristic H (k, m).

[0006] This data is subjected to an inverse discrete Fourier transform (I
(DFT) to obtain a spatial impulse response based on visual characteristics.

[0007] Finally, the spatial impulse response is transformed into a predetermined orthogonal transform region by performing an inverse orthogonal transform,
Obtain visual characteristics in the orthogonal transform domain. In this case, since the visual characteristics are represented by complex numbers, the norm thereof is obtained and further normalized by the coefficient A to obtain the normalized visual characteristics in the orthogonal transform domain.

Table 1 shows an example of visual characteristics in an 8 × 8 block when the visual distance L is L = 1024 (pixels) and the discrete cosine transform (DCT) is used as the orthogonal transform. is there.

[0009]

[Table 1]

[0010]

However, when the conventional transform method is applied to the block orthogonal transform domain, the spatial impulse response due to the visual characteristic which should originally spread over many blocks is folded back into one block, and an accurate impulse response is obtained. There has been a problem that a response cannot be obtained, and thus accurate conversion of visual characteristics cannot be performed. In addition, there is a problem that the conventional method cannot be directly applied to the block overlap orthogonal transform.

According to the present invention, since the spatial impulse response due to the visual characteristics described above is folded back into one block, an accurate impulse response cannot be obtained, and therefore, the conversion of the visual characteristics cannot be accurately performed. It is an object of the present invention to provide an excellent transform method that removes adverse effects and removes discontinuities at block boundaries by applying a block overlap orthogonal transform.

[0012]

According to the present invention, in order to solve the above-mentioned problem, (a) a spatial region having a spatial impulse having a block size of M × N is expanded to a size of m × M × n × N. (B) a spatial area expanding means for expanding to a spatial area
Converting means for converting the expanded spatial impulse from the expanded spatial domain to the spatial frequency domain; (c) multiplying means for multiplying the spatial impulse converted to the spatial frequency domain by visual characteristics in the spatial frequency domain; Means for inversely transforming the spatial impulse multiplied by the visual characteristic by the means into the extended spatial domain to obtain a spatial impulse response based on the visual characteristic of the spatial impulse. (E) For the spatial impulse and the spatial impulse response, Spatial domain re-expansion means for re-expanding data inside the boundary of the area by folding back, and (f) orthogonal transform means for performing block overlap orthogonal transform on the spatial impulse and the spatial impulse response whose spatial domain has been re-expanded, respectively. , (G) the spatial impulse transformed by the orthogonal transform means Calculation means for calculating the norm of the same component of each block, and (h) the ratio of the norm value of the spatial impulse obtained by the norm calculation means to the norm value of the spatial impulse response. Means for obtaining a visual characteristic of a spatial impulse in a block overlap orthogonal transform region, wherein the visual characteristic is converted into a block overlap orthogonal transform region.

[0013]

According to the present invention, when a spatial area having a spatial impulse having a block size of M × N is extended to an area of m × M × n × N by an expansion magnification m, n which can be set in accordance with the block size. Then, the expansion magnification is selected so that no aliasing occurs in the spatial impulse, and the peak point of the spatial impulse is arranged near the center of the space in the extended area. After transforming the spatial impulse into the spatial frequency domain, the spatial impulse is multiplied by the visual characteristic and inversely transformed into the spatial domain to obtain a response based on the visual characteristic of the spatial impulse. The spatial domain is re-expanded by wrapping the half-block width data inside the area boundaries outward for each said extended area of the spatial impulse and the visual impulse response of the spatial impulse. After the spatial impulse and the spatial impulse response are subjected to block overlap orthogonal transform in the region where the block overlap is performed, a norm for the same component of each block is calculated, and the ratio of this norm provides a visual characteristic in the block overlap orthogonal transform region. Therefore, it is possible to convert a visual characteristic that is accurate and continuity is preserved without causing distortion due to the return of the spatial impulse at the block boundary and without causing discontinuity in the characteristic at the block boundary.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a functional block diagram of an apparatus showing an embodiment of the present invention. In FIG.
Block size (M and N are M, N for convenience of high-speed conversion, etc.)
= Often be set to 2 ^P) is a space impulse. The peak point of the impulse is located near the center of the block in consideration of the characteristics of the impulse response described later.

The spatial region expanding means 1 receives the Mx of the input 101
With the block of N as the center, the spatial area is m · Mxn · N
To the spatial region 102 of Here, m and n are positive integers, and a suitable value according to the block size is selected as described later so that no aliasing occurs in the impulse response.

The discrete Fourier transform (DFT) means 2 converts the expanded m × M × n × N spatial domain 102 into a spatial frequency domain.

The visual characteristic adding means 3 adds the visual characteristic table 9 to the spatial impulse 103 converted into the spatial frequency domain.
Characteristics 111 in the spatial frequency domain output from
Multiply by

The inverse discrete Fourier transform (IDFT) 4 converts the spatial impulse 104, which has been added to the visual characteristics and is converted into the spatial frequency domain, into the spatial domain again. As a result, a spatial impulse response 105 based on visual characteristics is obtained.

The folded space area expanding means 5 is 1/1 of the inside of the boundary of the m × M × n × N space area in order to match the property of the block overlap orthogonal transform means 6 at the next stage.
The two-block width data is turned outward and the space area is re-expanded to a (m + 1) · M × (n + 1) · N folded space area.

Block overlap orthogonal transformation means 6
Converts the spatial impulse 109 and the spatial impulse response 106 of the folded spatial domain into a block overlapping orthogonal transform domain using a block overlap orthogonal transform scheme described later.

The norm calculation means 7 performs a norm calculation on the same components 107 and 110 of each block of the spatial impulse 109 and the spatial impulse response 106 which have been subjected to the block overlap orthogonal transformation.

The comparison normalizing means 8 calculates the ratio between the norm value 108 of the spatial impulse to which the visual characteristic is added after the block overlap orthogonal transform and the norm value 111 of the spatial impulse after the block overlap orthogonal transform, as required. To perform normalization and output as the visual characteristics 112 in the block overlap orthogonal transform domain.

Next, details and procedures of a method of converting visual characteristics into a block overlap orthogonal transform area will be described with reference to FIG.

First, a spatial impulse [X (k, l)] 31 having a block size of M × N is input. However, [・]
Represents a matrix, and k and l take discrete values of 0 ≦ k <M and 0 ≦ l <N.

[0026]

(Equation 1)

This spatial impulse [X (k, 1)]
The space is expanded (m × n) times so that 31 is at the center (where m, n ≧ 1), and the entire expanded area is filled with data “0”. Here, the expansion magnifications m to n are, as described above, such that no aliasing occurs in the impulse response.
This is a value set according to the block size. In general,
If the spread of the impulse response falls within the block,
No wrapping occurs. Conversely, the values of m and n are set appropriately so that the spread of the impulse response falls within the block. As examples of the expansion magnifications m and n, for example, M and N
= 8, m, n ≧ 3, M, N = 16, m, n ≧
If m, n ≧ 1 in the case of 2, M, N = 32, a good result is obtained in which the spatial impulse response is not folded in one block.

The spatial impulse [X (k, 1)] 32 whose region has been expanded in this way is subjected to discrete Fourier transform (DFT). This DFT is represented by the following equation (2).

[0029]

(Equation 2)

[F (u, v)] 33 is a DF of visual characteristics.
Multiply by the T value [HF (u, v)] and the resulting value 3
5 is subjected to an inverse discrete Fourier transform (IDFT) and returned to the spatial domain. This IDFT is represented by the following equation (3). IDFT
[XH (k, 1)] 36 obtained as a result is a response of the spatial impulse [X (k, 1)] 32 whose region has been extended, based on visual characteristics.

[0031]

(Equation 3)

The spatial impulse [X
(K, 1)] 32 and the impulse response [XH (k, 1)] 36 due to its visual characteristics, as shown at 37 and 38 in FIG. The space is extended by wrapping the / 2 block width outward. Impulse [X (k, 1)] 32 in this folded space region
Is [Y (k, 1)] 37 in the following equation (4). The impulse response [XH (k, 1)] 36 is similarly expressed, and becomes [YH (k, 1)] 38 in the folded space area.

[0033]

(Equation 4)

The data [Y (k, l)] 37 whose area has been expanded by the return space expansion and the impulse response [Y _H
(K, l)] 38 is subjected to block overlap orthogonal transformation. Various methods of block overlap orthogonal transformation are conceivable, but in this embodiment, for example, reference 2 shown in FIG.
Using the method of “Overlapping orthogonal transform: transform coding without blocking effect” (Henrique S. et al.
MALVER, David H .; STAELIN, “T
he LOT: Transform Coding W
itout Blocking Effects "I
EEE Trans. Acoustic. , Speech,
Signal Processing, vol. ASS
P-37, no. 4, pp. 553-559, April
1, 1989).

FIG. 4 shows an example of a one-dimensional block overlap orthogonal transform of an 8 × 1 block size.
The expansion of 8 into two dimensions may be performed by performing one-dimensional conversion on each of the rows and columns. In the block overlap orthogonal transformation shown in this example, first, as shown in FIG.
After folding back to the outside by an amount equal to / 2 block size and expanding, the expanded image is divided into blocks, and discrete cosine transform (DCT) is performed on each block. Then, the butterfly operation of the elements shown in FIGS. 4B and 4C is performed on each element after the DCT.

This butterfly operation is an operation for obtaining the sum and difference between the two elements a and b when the angle θ is not attached, as shown in FIG. 4C. , B, the difference (a · Cos θ−b · Sin θ) and the sum (a · Sin θ + b · Cos θ) are calculated.

It is to be noted that no matter what the block overlap orthogonal transform method is used, the property due to the turning back of the edge does not change.

When the block overlap orthogonal transform is represented by [L], the spatial impulse [Y (k, l)] extended to the folded spatial area and the impulse response [Y
_H (k, l)] can be expressed by the following equations (5) and (6), respectively. However,
(I, j) means the (i, j) -th block.

[L _X ^{(i, j)} (u, v)] = [L ^t ] · [Y (k, l)] · [L] (5) [L _H ^{(i, j)} (u, v ⁾ )] = [L ^t ] · [Y _H (k, l)] · [L] (6) [L _X ^{(i, j)} (u, v)] 39 is to the block overlap orthogonal transform domain of the spatial impulse Indicates the conversion value of
[L _H ^{(i, j)} (u, v)] 40 indicates a response of the spatial impulse based on visual characteristics in a block overlap orthogonal transform domain.

Finally, considering the use of the root mean square method when applying visual characteristics to image compression or image quality evaluation, etc., the above-mentioned [L _x ^{(i, j)} (u, v)] and [L _H ^{(i, j)}
(U, v)], the norm of the same component in each block is defined as the square root of the sum of squares of the absolute values of the elements of the target matrix according to the following equations (7) and (8). L
_x (u, v)}] 41 and [{L _H (u, v)}] 42. Then, as shown in equation (9), ‖L _H (u, v)
_Let the ratio H _L (u, v) 43 between ‖ and {L _X (u, v)} be the visual characteristic in the block overlap orthogonal transform domain. This visual characteristic [H _L ] can be normalized or multiplied by k times as needed.

[0041]

(Equation 5)

[0042]

(Equation 6)

H _L (u, v) = { _{L H} (u, v)} / {L _X (u, v)} (9) Next, as an example, L = 240, S = 3, M, N = 8 , M,
Tables 2 to 5 show examples in which the visual characteristics in the case of n = 3 are converted into the block overlap orthogonal transform region. Table 2 shows spatial impulse response [X _H ], and Table 3 shows [L _H ] obtained by subjecting [X _H ] to block overlap orthogonal transform.
Table 4 shows [L _X ] obtained by performing block overlap orthogonal transform on the spatial impulse, and Table 5 shows visual characteristics [H _L ] in the block overlap orthogonal transform region.
From Table 2, it can be seen that the spread of the spatial impulse response within the expanded block is settled.

[0044]

[Table 2]

[0045]

[Table 3]

[0046]

[Table 4]

[0047]

[Table 5]

[0048]

As described above, according to the present invention, it is possible to realize the conversion of the visual characteristic by appropriately taking the property of the block division. In addition, since the visual impulse response is accurately represented and the influence of the spread of the spatial impulse response on the surrounding blocks is appropriately represented, appropriate conversion of visual characteristics can be realized.

Also, in an image compression coding apparatus or the like, since a quantization method or a coding method suitable for the visual characteristics of an image plays an important role, the compression encoding efficiency of the image is greatly increased by using accurate visual characteristics. And the quality of the image can be improved.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of an apparatus showing one embodiment of the present invention.

FIG. 2 is an explanatory diagram of a conversion method according to the present invention.

FIG. 3 is an explanatory diagram of a conventional system.

FIG. 4 is a diagram illustrating an example of a block overlap orthogonal transform.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 spatial domain expanding means 2 DFT means 3 visual characteristic adding means 4 IDFT means 5 folded spatial domain expanding means 6 block overlap orthogonal transform means 7 norm calculating means 8 comparing / normalizing means 9 visual characteristic table

Continuation of the front page (56) References JP-A-1-256888 (JP, A) Saito (2 others), "Optimal Quantization of DCT Coefficient Considering Visual Characteristics in Variable Length Coding", Electronic IEICE Technical Report, IEICE, 1991, Vol. 90, No. 450, P.E. 39-46 (IE90-101) Hotta and Miyahara, "Optimization of Image OTC Quantizer", IEICE Technical Report, IEICE, 1985, Vol. 85, No. 203, p. 13-18 (IE85-92) Harada, "Properties of Convolution of Discrete Cosine Transform", Proc. Of the IEICE Fall National Conference, 1990, IEICE, 1990, p. 1-93 (A-93) NORMAN B. NILL, "A Visual Model Weighted Cosine Transform for Image Compression and Quality Assessment", IEEE TRANSACTIONS ON COMMUNICATION, 1985. COM-33, NO. 6, p. 551-557 (58) Fields surveyed (Int. Cl. ⁷ , DB name) H04N ^7/ 24-7/68 H04N 1/41-1/419 JICST file (JOIS)

Claims (3)

(57) [Claims] 1. A spatial area having a spatial impulse having a block size of M × N is extended to an extended spatial area having a size of m × M × n × N by an expansion magnification m and n which can be set according to the block size. A spatial domain expanding unit that converts the expanded spatial impulse into a spatial frequency domain from the extended spatial domain; and converting the spatial impulse converted into the spatial frequency domain into a visual characteristic in the spatial frequency domain. By multiplying the spatial impulse multiplied by the visual characteristics by the multiplying means,
An inverse transforming means for obtaining a spatial impulse response based on the visual characteristics of the spatial impulse; and a spatial domain re-expanding the spatial impulse and the spatial impulse response by folding back the data inside the boundary of the extended spatial domain to the outside. Expanding means, an orthogonal transforming means for performing block overlap orthogonal transform on the spatial impulse and the spatial impulse response whose space area has been re-expanded, and the spatial impulse and the spatial impulse response converted by the orthogonal transforming means, respectively. A norm calculating means for calculating a norm of the same component of each block, and a block impulse response of the spatial impulse obtained by a ratio of a norm value of the spatial impulse obtained by the norm calculating means to a norm value of the spatial impulse response. Means for obtaining visual characteristics in a burlap orthogonal transform region. A method for converting visual characteristics into a block overlap orthogonal transform region. 2. The spatial domain expanding means selects the expansion factor so that no aliasing occurs in the spatial impulse response, and arranges a peak point of the spatial impulse near the center of the extended spatial domain. 2. The conversion method according to claim 1, wherein: 3. The spatial domain re-expansion means, for each of the spatial impulse and the spatial impulse response, の of a space inside a boundary of the extended spatial domain.
2. The conversion method according to claim 1, wherein the block width data is re-expanded by folding back to the outside.