JP6793499B2 - Image coding device and its control method - Google Patents

Description

The present invention relates to an image data coding technique.

Recently, with the development of digital imaging devices such as digital cameras and digital camcorders, various image data compression coding methods are being studied.

As a typical compression coding method, JPEG (Joint Photograph Experts Group) 2000 can be mentioned. In JPEG2000, the image to be encoded is divided into at least one or more tiles. Then, wavelet transform, quantization, and entropy coding are performed for each tile. For the wavelet transform, for example, a reversible 5/3 tap filter is used. In this filter, one high-pass coefficient is generated from three consecutive pixels, and one low-pass coefficient is generated from five consecutive pixels. Therefore, when filtering the pixels at the boundary of the tile, the pixels outside the tile are required. There is known a proposal to substitute a pixel inside a tile as a pixel outside the tile (Patent Document 1). In Patent Document 1, the position of the pixel at the boundary of the tile of interest is set as the folding position, and the pixel or coefficient data in the tile is used as the pixel or coefficient data outside the tile.

However, it does not filter by referring to the pixels and coefficient data that actually exist outside the tile of interest. Therefore, the coefficient data after the quantization of the tile boundary is different from the coefficient data when actually referring to the pixels outside the tile. This difference means that the continuity of the tiles in the decoded image is lost, resulting in noise at the tile boundaries.

In response to this problem, Patent Document 2 discloses a technique for encoding a tile of interest, including pixels in a tile adjacent to the tile of interest. According to Patent Document 2, in the decoding process of the tile of interest, the pixels of the tile of interest and the pixels in the vicinity of the boundary of the adjacent tiles are obtained. Therefore, the result of removing the pixels of the adjacent tile is used as the image of the decoding result of the tile of interest. According to Patent Document 2, since the continuity of adjacent tiles is maintained, the generation of noise at the tile boundary can be suppressed.

JP-A-2007-142615 Japanese Unexamined Patent Publication No. 2004-56213

However, although the above-mentioned Patent Document 2 can suppress the deterioration of image quality, when encoding the tile of interest, it is necessary to include the pixels of the adjacent tile. Therefore, the amount of coded data generated in Patent Document 2 is not negligible as that in Patent Document 1. That is, in order to use the technique of Patent Document 2 to obtain the same code amount as that of Patent Document 1, the quantization step in Patent Document 2 must be larger than that of Patent Document 1. However, the larger the quantization step, the greater the deterioration of image quality.

The present invention has been made in view of the above points, and is a technique capable of maintaining the continuity of tiles when tiles are divided and generating a code amount much smaller than that of Patent Document 2. Is intended to provide.

In order to solve the above problems, for example, the image coding apparatus of the present invention has the following configuration. That is,
An image coding device that divides image data into a plurality of tiles and encodes each tile.
A conversion means that performs wavelet transform for each tile to obtain conversion coefficient data,
For each tile, a coding means for encoding the conversion coefficient data obtained by the conversion means is provided.
If there are adjacent tiles adjacent to the tile to be encoded,
The conversion means converts by performing the wavelet transform with reference to the pixel data of the tile to be encoded and a plurality of pixel data of the adjacent tiles adjacent to the tile to be encoded. Get the coefficient data,
The coding means itself when acquiring pixel data from the conversion coefficient data by the inverse wavelet transform among the conversion coefficient data corresponding to the adjacent tiles obtained by referring to the plurality of pixel data of the adjacent tiles. The conversion coefficient data required for acquiring the pixel data of the tile is encoded, and a part of the conversion coefficient data corresponding to the adjacent tile is not encoded.

According to the present invention, it is possible to suppress deterioration of image quality at the tile boundary while suppressing an increase in the amount of code due to tile division.

The block block diagram of the image coding part in 1st Embodiment. The figure for demonstrating the tile input method in 1st Embodiment. The figure which shows the wavelet transform processing in 1st Embodiment. The figure for demonstrating the execution method at the time of performing the wavelet transform three times in the 1st Embodiment. The flowchart which shows the procedure of the coding process in 1st Embodiment. The block block diagram of the decoding part in 1st Embodiment. The figure for demonstrating the inverse wavelet transform method in 1st Embodiment. The block block diagram of the image coding part in 2nd Embodiment. The flowchart which shows the processing procedure of the coding apparatus in 2nd Embodiment. The figure which shows the relationship with the number of pixels input overlapping from the adjacent tile when the wavelet transform is performed three times. , , , , , , , The figure for demonstrating the execution method when the wavelet transform is performed three times.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example of a specific embodiment of the present invention, and is one of the specific embodiments of the configuration described in the claims.

[First Embodiment]
The first embodiment describes an example applied to an imaging device such as a digital camera. FIG. 1 is a block configuration diagram of an image coding unit 100 in an imaging device.

As shown in the figure, the image coding unit 100 includes a frame memory 101, a data input control unit 102, a wavelet transform unit 103, a quantization unit 104, an entropy coding unit 105, and a setting unit 106. In the first embodiment, an example using JPEG2000 as an example of the coding method will be described, but the coding method is not particularly limited. Hereinafter, the processing procedure of the coding process will be described based on the configuration of FIG. Here, the setting unit 106 follows an instruction from an operation unit (not shown), how many tiles the image data to be encoded is divided into, and how many times the wavelet transform is performed (how many resolution levels are set). Or) is decided. Then, the setting unit 106 determines the size of one tile and the number of overlapping pixels and the conversion coefficient data to be quantized, although the details will be described later.

In the present embodiment, the number of tile divisions and the number of decomposition levels are determined according to the instructions from the operation unit, but the system control unit (not shown) of the imaging device automatically determines the number of tile divisions and the number of decomposition levels based on predetermined conditions. You may decide.

Further, in the present embodiment, the tile indicates a divided image when one frame image is divided into a plurality of pieces.

When a recording start instruction is given from an operation unit (not shown), the image data captured by the image pickup unit (not shown) is input to and stored in the frame memory 101 in the order of imaging. The data input control unit 102 divides the image data of one frame stored in the frame memory 101 into a plurality of tiles. The number of divisions of the tiles and the size of each tile depend on the setting from the setting unit 106. Hereinafter, the partial image data represented by the divided tiles is referred to as tile image data. The data input control unit 102 supplies each tile image data to the wavelet transform unit 103 in a predetermined order. Although the details will be described later, when the input data control unit 102 supplies a certain attention tile image data to the wavelet transform unit 103, not only the attention tile image data but also within a preset distance from the tile boundary. It also supplies the pixel data in the adjacent tile image data that is located. Hereinafter, pixel data located within a preset distance from the tile boundary in the adjacent tile image data will be referred to as tile peripheral pixel data. As described above, the image data supplied from the input data control unit 102 to the wavelet transform unit 103 overlap each other.

The wavelet transform unit 103 sends the conversion coefficient data obtained by executing the wavelet transform to the quantization unit 104 with respect to the tile image data and the pixel peripheral pixel data input from the frame memory 101. However, the specific execution method of this wavelet transform will be described later.

The quantization unit 104 quantizes the conversion coefficient data sent from the wavelet transform unit 103 using the quantization parameter, and sends it to the entropy coding unit 105. At this time, the quantization parameters given to the same subband are set to the same or close values between the tile images. By doing so, the image quality deterioration between the tile images can be made uniform, and the image quality deterioration at the tile boundary can be made inconspicuous.

The entropy coding unit 105 performs entropy coding on the conversion coefficient data after quantization by the quantization unit 104 for each subband, and outputs the coded data. As the entropy encoding, entropy encoding such as EBCOT (Embedded Block Coding with Optimized Truncation) is performed, and the data is output as encoded data.

Next, the relationship between the tile image data and the tile peripheral pixel data supplied to the wavelet transform unit 103 will be described with reference to FIG.

FIG. 2A shows an example in which nine tile image data A to I are generated from one frame by dividing the image data of one frame into three in the horizontal direction and three in the vertical direction. The data input control unit 102 supplies the tile image data A → B → C → D → ... → I to the wavelet transform unit 103 in the order of raster scan in tile units. Further, as described above, when the wavelet transform unit 102 supplies one tile image data to the wavelet transform unit 103, the wavelet transform unit 102 also supplies the tile peripheral pixel data to the wavelet transform unit 103.

Here, the tile image data A of FIG. 2B will be taken as an example to explain the relationship with the pixel data around the tile. The tile image data A is located at the upper left in the frame image. Therefore, the adjacent tile image data of the tile image data A becomes the tile image data B, D, and E. Therefore, when the data input control unit 102 supplies the tile image data A to the wavelet transform unit 103, the data input control unit 102 inputs the pixel data of the portion indicated by the shaded line P in FIG. 2B to the wavelet transform unit 102 as the pixel peripheral pixel data of the tile. Applying the same idea to all tiles of tile image data A to I, the pixel data around the tile for each tile image data is indicated by the diagonal line Q in FIG. 2 (c). The horizontal and vertical distances (number of pixels) of the pixel data P around the tile differ depending on the number of wavelet transforms, the size of the tile image data of interest, and the position of the adjacent tile image. Further, the number of pixels in the horizontal and vertical directions of the tile peripheral pixel data P is referred to as the number of overlapping pixels.

The relationship between the number of overlapping pixels and the number of conversion coefficient data of adjacent tiles required to decode the pixels at the edge of the tile of interest when the wavelet transform is performed up to 3 times is shown in FIGS. 10 (a) and 10 (b). Shown in.

10 (a) and 10 (b) show that among the conversion coefficient data of the adjacent tiles acquired when the pixels of the adjacent tiles are added and the wavelet transform is performed, it is necessary to decode the pixels of the tile of interest to be encoded. The number of conversion coefficient data is described for each sub-sub band. In the present embodiment, when the wavelet transform is performed once, the conversion coefficient data of the subbands 1H and 1L is generated, and the conversion coefficient data of the subbands 1H and 1L are encoded. When the wavelet transform is performed twice, the conversion coefficient data of the subbands 1H, 1L, 2H and 2L is generated, and the conversion coefficient data of the subbands 1H, 2H and 2L are encoded. When the wavelet transform is performed three times, the conversion coefficient data of the subbands 1H, 1L, 2H, 2L, 3H and 3L is generated, and the conversion coefficient data of the subbands 1H, 2H, 3H and 3L are encoded. Therefore, the subbands that are not generated and the subbands that are not encoded are grayed out without describing the number of data.

FIG. 10A shows the number of wavelet transforms, the number of overlapping pixels, and the upper end or leftmost pixel of the tile of interest when the adjacent tile is located on or to the left of the tile of interest. It shows the relationship with the number of conversion coefficient data of the required adjacent tiles.

In this embodiment, the wavelet transform uses an integer type 5/3 tap filter. When wavelet transforming one tile image data, the conversion is started with the upper left corner of the tile image data as the start position. Therefore, regardless of the size of the tile image data of interest, the tile image data adjacent to the upper and left sides refers to pixels having a fixed number of overlapping pixels according to the number of web rate conversions. As shown in the figure, when the wavelet transform is performed only once, the number of overlapping pixels of the tile image data adjacent to the upper or left side is “2”. Further, when the wavelet transform is performed twice, the number of overlapping pixels with the adjacent tile is "8". Then, when the wavelet transform is performed three times, the number of overlapping pixels with the adjacent tile is "16".

FIG. 10B shows the number of wavelet transforms, the number of overlapping pixels, and the adjacent tiles required to decode the pixels of the tile of interest when the adjacent tiles are located below or to the right of the tile image of interest. The relationship with the number of conversion coefficient data is shown. When the adjacent tile image data is located below or to the right of the tile image data of interest, the number of overlapping pixels depends on the size of the tile image data of interest. To be precise, it depends on the remainder obtained by dividing the number of pixels in the horizontal and vertical directions of the tile image data of interest by "8". For example, it is assumed that the remainder obtained by dividing the number of pixels in the horizontal or vertical direction of the tile image data of interest by "8" is 0. In this case, when the wavelet transform is performed only once, the number of overlapping pixels of the tile image data adjacent to the bottom or the right is "3". Further, when the wavelet transform is performed three times, the number of overlapping pixels is "15".

FIG. 3 shows a lifting structure for executing the wavelet transform once by the wavelet transform unit 103 using the above-mentioned reversible 5/3 tap filter. Since the wavelet transform is executed once, the level of the obtained transform coefficient data is 1.

Reference numerals a to i in FIG. 3 indicate one line of pixel data near the tile boundary of the image data. The illustration shows an example in which there is a tile boundary between pixels d and e, pixels e to i belong to the tile image data of interest to be wavelet-transformed, and pixels a to d belong to adjacent tile image data located on the left. This is an example. When wavelet transform is performed using a 5/3 tap filter, conversion coefficient data indicating a high frequency component is calculated with reference to three consecutive pixels. In the figure, b', d', f', and h'indicate the conversion coefficient (1H) of the high frequency component of the decomposition level 1. In addition, conversion coefficient data indicating a low frequency component is calculated with reference to five consecutive pixels. The c ", e", and g "in the figure indicate the conversion coefficient data (1 L) indicating the low frequency component of the decomposition level 1.

Here, a case where the pixel e closest to the tile boundary in the tile image data of interest is decoded will be considered. In order to decode the pixel e, as shown in the figure, there are only three high-frequency conversion coefficient data d'and f'and low-frequency conversion coefficient data e ". Of these, tiles adjacent to the tile image data of interest The high-frequency conversion coefficient data d'belongs to. The high-frequency conversion coefficient d'is calculated from a total of three pixels e in the tile of interest and pixels c and d of the adjacent tiles. Therefore, when the wavelet conversion is performed once, the number of overlapping pixels of the tile image adjacent to the left is “2”.

As described above, when the wavelet transform unit 103 performs the wavelet transform only once when the position of the adjacent tile is on the left, the number of overlapping pixels is “2” from the adjacent tile image data in addition to the attention tile image data. The pixels c and d corresponding to are input to perform wavelet transform. Then, the wavelet transform unit 103 supplies the transform coefficient data d'obtained from the adjacent tile image data to the quantization unit 104 in addition to the transform coefficient data obtained from the tile image data of interest.

According to the technique of Patent Document 2 described above, in order to generate the coded data of the tile image data of interest, the coded data necessary for restoring the c and d of the adjacent tile image is generated. That is, in the technique disclosed in Patent Document 2, when the tile image of interest is encoded, the conversion coefficients b', c ", and d'of the adjacent tiles are included in the quantization and the coding is performed. On the other hand, in the present embodiment, the conversion coefficient data required for the adjacent tiles is only d', and the coding efficiency can be further increased.

When there are no adjacent tiles on or to the left as in the tile image data A in FIG. 2A, mirror processing is performed in the same manner as in conventional JPEG2000 to generate pixels outside the tiles, and wavelets are generated. Perform the conversion.

Further, as the number of wavelet transforms increases (as the decomposition level increases), the amount of code that can be reduced increases as compared with the case of Patent Document 2, so that the effect of reducing the amount of generated code by the first embodiment is greatly exhibited. FIG. 4 shows a lifting structure for obtaining conversion coefficients up to decomposition level 3 using a reversible 5/3 tap filter.

In the case of FIG. 3 in which the wavelet transform is executed once, the pixels to be input overlapping from the adjacent tiles in order to generate the conversion coefficient required to decode the leftmost pixel in the tile of interest are already shown in FIG. It was 2 pixels as explained using. On the other hand, when the wavelet transform is performed three times, it is necessary to input 16 pixels from the adjacent tile image data as shown in FIG.
In the case of Patent Document 2, conversion coefficient data for 16 pixels, 8 coefficients of conversion coefficient data (1H), 4 coefficients of conversion coefficient data (2H), 2 coefficients of conversion coefficient data (3H), conversion coefficient data ( It is necessary to quantize and encode the 2 coefficients) of 3L). Therefore, the amount of code becomes so large that it cannot be ignored.

On the other hand, in the case of the first embodiment, the conversion coefficient data generated from the adjacent tile image data required to decode the pixel located at the end of the tile image data of interest is the conversion coefficient circled in the figure. It becomes. That is, there are only three coefficients in total: one coefficient of the conversion coefficient data (1H) of the decomposition level 1, one coefficient of the conversion coefficient data (2H) of the decomposition level 2, and one coefficient of the conversion coefficient data (3H) of the decomposition level 3. .. Therefore, of the conversion coefficient data obtained from the adjacent tile image data, these three coefficients may be quantized and encoded. That is, as compared with Patent Document 2, the number of conversion coefficients to be encoded can be significantly reduced.

As can also be seen from FIG. 4, when the position of the adjacent tile is up or left, it was generated from the adjacent tile in order to restore the number of wavelet transforms and the pixels at the edges in the tile of interest in the first embodiment. The relationship between the conversion coefficient data to be used and the conversion coefficient data to be used is as follows.
Wavelet transform count is 1 time: 1-coefficient wavelet transform count of conversion coefficient data (1H) is 2 times: 2-coefficient wavelet transform count of conversion factor data (1H) and (2H) is 3 times: Conversion factor data (1H), 3 coefficients of (2H) and (3H):
Here, let N be the number of times the wavelet transform is executed. In this case, the wavelet transform unit 103 quantizes N of the conversion coefficient data 1H, 2H, ..., NH closest to the tile boundary among the high-frequency conversion coefficient data of each decomposition level generated from the adjacent tiles. It can be said that it should be supplied to.

Next, when the position of the adjacent tile is down or right and the wavelet transform is performed three times, it is necessary to restore the pixel at the end of the tile of interest in the conversion coefficient data generated from the adjacent tile. The conversion coefficient data will be described with reference to FIGS. 11A to 11H.

When the wavelet transform is performed three times, the conversion coefficient data 1H, the conversion coefficient data 2H, the conversion coefficient data 3H, and the conversion coefficient data 3L are quantized and encoded. That is, the conversion coefficient data 1L and the conversion coefficient data 2L are not encoded. This is because the conversion coefficient data 1L can be generated from the conversion coefficient data 2H and the conversion coefficient data 2L, and the conversion coefficient data 2L can be generated from the conversion coefficient data 3H and the conversion coefficient data 3L. That is, when the wavelet transform is performed N times, the conversion coefficient data 1H, 2H, ..., NH and the conversion coefficient data NL are encoded, and the conversion coefficient data 1L, 2L, ..., (N-1) L are codes. It will not be converted and will not be recorded.

The above is common to the tile of interest and the adjacent tile. Hereinafter, based on this point, the conversion coefficient data to be encoded in the conversion coefficient data generated in the adjacent tile will be described. FIG. 11A shows a case where the wavelet transform is performed three times when the remainder of dividing the number of pixels (tile width of interest) in the horizontal or vertical direction of the tile of interest to be encoded by “8” is “0”. There is. That is, the number of pixels in the horizontal direction or the vertical direction of the tile of interest is an integral multiple of 8.

In order to decode the pixel d at the end of the tile of interest to be encoded, the data of the pixel c, the conversion coefficient data 1Hd, and the pixel e are required. However, the pixel e can be generated from the conversion coefficient data 1Hd, the conversion coefficient data 1Le, and the conversion coefficient data 1Hf. Here, the conversion coefficient data 1Le is 1L of conversion coefficient data, which is not recorded, and can be generated from the conversion coefficient data 2Hc, the conversion coefficient data 2Le, and the conversion coefficient data 2Hg. Further, the conversion coefficient data 2Le is data that is not recorded, and can be generated from the conversion coefficient data 3Ha, the conversion coefficient data 3Le, and the conversion coefficient data 3Hi. That is, in order to decode the pixel d at the end of the tile of interest to be encoded, the conversion coefficient data 1Hf, the conversion coefficient data 2Hg, the conversion coefficient data 3Hi, and the conversion coefficient data surrounded by the solid line shown in the figure near the boundary in the adjacent tile. 3Le is enough. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Therefore, in addition to the conversion coefficient data of 1H, 2H, 3H, and 3L in the tile of interest to be encoded, the conversion coefficient data {1Hf, 2Hg, 3Hi, 3Le} of the adjacent tile is encoded.

Further, in order to calculate these conversion coefficient data, 15 pixels of adjacent tiles are required. Therefore, in addition to the tile of interest, at least 15 pixels of adjacent tiles are input and the wavelet transform is performed three times. Then, when encoding, the conversion coefficient required to decode the data at the end of the tile of interest to be encoded without encoding all the conversion coefficient data of the adjacent tiles obtained by the wavelet transform. Encode the data. Of course, all the conversion coefficient data of 1H, 2H, 3H, and 3L are encoded in the tile of interest to be encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 1 coefficient, and subband 3H. There are a total of 4 coefficients, 1 coefficient and 3L of subband: 1 coefficient.

FIG. 11B shows a case where the wavelet transform is performed three times when the remainder of dividing the number of pixels (tile width of interest) in the horizontal or vertical direction of the tile of interest to be encoded by “8” is “1”. There is.

In order to decode the pixel e at the end of the tile of interest to be encoded, the conversion coefficient data 1Le of 1L at the end of the tile of interest, the conversion coefficient data 1Hd of 1H in the tile of interest, and the conversion of 1H of the adjacent tile Coefficient data 1Hf is required. The conversion coefficient 1Le can be calculated from the conversion coefficients 2Hc and 2Le in the tile of interest and the conversion coefficient data 2Hg in the adjacent tile. Further, the conversion coefficient 2Le is the conversion coefficient 3Ha, 3Le. In the tile of interest. And, it can be calculated from the conversion coefficient data 3Hi of the adjacent tile. Therefore, after all, as shown in FIG. 11B, in addition to the data in the tile of interest, the conversion coefficient data 1Hf, the conversion coefficient data 2Hg, and the conversion coefficient data 3Hi in the adjacent tile are sufficient. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 14 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 1, at least 14 pixels of adjacent tiles are input in addition to the tile of interest, and wavelet transform is performed three times. Then, when encoding, the conversion coefficient data of the adjacent tiles is not all the conversion coefficients obtained by the wavelet transform, but the pixel e at the end of the tile of interest and the conversion coefficient data at the end are decoded. The conversion coefficient data {1Hf, 2Hg, 3Hi} required for the above is encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 1 coefficient, and subband 3H. There are a total of 3 coefficients, 1 coefficient and 3L of subband: 0 coefficient.

FIG. 11C shows a case where the wavelet transform is performed three times when the remainder of dividing the number of pixels (tile width of interest) in the horizontal or vertical direction of the tile of interest to be encoded by “8” is “2”. There is.

In order to decode the pixel f at the edge of the tile of interest, the pixel e of the tile of interest, the conversion coefficient data 1Hf of 1H of the tile of interest, and the pixel g are required. In order to decode the pixel g, 1H conversion coefficient data 1Hf in the tile of interest, 1L conversion coefficient data 1Lg in the adjacent tile, and 1H conversion coefficient data 1Hf in the adjacent tile are required. Of these, 1 L of conversion coefficient data 1 Lg is not recorded, so it is necessary to generate it. The conversion coefficient data 1Lg can be calculated from the conversion coefficient 1Le in the tile of interest and the conversion coefficient data 2Hg and 1Li in the adjacent tile. Since 1L of conversion coefficient data 1Le and 1Li are not recorded, they need to be calculated. The conversion coefficient data 1Le can be calculated from the conversion coefficient data 2Hc and 2Le in the tile of interest and 2Hg in the adjacent tile. The 2L conversion coefficient data that is not recorded, 2Le, can be calculated from 2Ha and 3Le in the tile of interest and 3Hi in the adjacent tile. Further, the conversion coefficient data 1Li can be calculated from the conversion coefficient data 2Hg, 2Li, 2Hk in the adjacent tile. The 2L conversion coefficient data that is not recorded, 2Li, can be calculated from the conversion coefficient data 2Le in the tile of interest and 3Hi and 2Lm in the adjacent tile. Further, 2Lm, which is 2L of conversion coefficient data that is not recorded, can be calculated from the conversion coefficient data 3Hi, 3Lm, 3Hq in the adjacent tile. Therefore, after all, as shown in FIG. 11C, in addition to the data in the tile of interest to be encoded, the conversion coefficient data 1Hh of the adjacent tile, the conversion coefficient data 2Hg, the conversion coefficient data 2Hk, the conversion coefficient data 3Hi, and the conversion Coefficient data 3Hq and conversion coefficient data 3Lm are sufficient. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 21 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 2, at least 21 pixels of adjacent tiles are input in addition to the tile of interest to perform wavelet transform. Then, when encoding, the conversion coefficient data {1Hh, 2Hg, 2Hk, 3Hi, 3Hq, 3Lm} is encoded instead of all the conversion coefficients obtained by the wavelet transform for the conversion coefficient data of the adjacent tiles. To do.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 2 coefficient, and subband 3H. : 2 coefficients, subband 3L: 1 coefficient, total 6 coefficients.

FIG. 11D shows a case where the wavelet transform is performed three times when the remainder of dividing the number of pixels (tile width of interest) in the horizontal or vertical direction of the tile of interest to be encoded by “8” is “3”. There is.

In order to decode the pixel g at the end of the tile of interest, the conversion coefficient data 1Hf of 1H at the end of the tile of interest, the conversion coefficient data 1Lg of 1L, and the conversion coefficient data 1Hh of 1H in the adjacent tile are required. Of these, 1 L of conversion coefficient data 1 Lg is not recorded, so it is necessary to calculate from other conversion coefficient data. The conversion coefficient data 1Lg can be calculated from 1Le and 2Hg in the tile of interest and 1Li in the adjacent tile. Since 1Le and 1Li, which are 1L conversion coefficient data, are not recorded, it is necessary to calculate from other conversion coefficient data. The conversion coefficient data 1Le can be calculated from the conversion coefficient data 2Hc, 2Le, and 2Hg in the tile of interest. The 2L conversion coefficient data that is not recorded, 2Le, can be calculated from the conversion coefficient data 3Ha and 3Le in the tile of interest and the conversion coefficient data 3Hi in the adjacent tile. Further, the conversion coefficient data 1Li can be calculated from the conversion coefficient data 2Hg in the tile of interest and the conversion coefficient data 2Li, 2Hk in the adjacent tile. 2Li, which is the conversion coefficient data of 2L that is not recorded, is the conversion coefficient data 2Le and the conversion coefficient data in the tile of interest. It can be calculated from 3Hi and 2Lm in the adjacent tile. 2Lm, which is 2L of conversion coefficient data that is not recorded, can be calculated from the conversion coefficient data 3Hi, 3Lm, 3Hq in the adjacent tile. Therefore, after all, as shown in FIG. 11D, in addition to the data in the tile of interest to be encoded, the conversion coefficient data 1Hh of the adjacent tile, the conversion coefficient data 2Hk, the conversion coefficient data 3Hi, the conversion coefficient data 3Hq, and the conversion A coefficient data of 3 Lm is sufficient. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 20 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 3, at least 20 pixels of adjacent tiles are input in addition to the tile of interest to perform wavelet transform. Then, when encoding, the conversion coefficient data of the adjacent tiles is not all the conversion coefficients obtained by the wavelet transform, but the pixel g at the end of the tile of interest and the conversion coefficient data at the end are decoded. The conversion coefficient data {1Hh, 2Hk, 3Hi, 3Hq, 3Lm} required for the above is encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 1 coefficient, and subband 3H. : 2 coefficients, subband 3L: 1 coefficient, total 5 coefficients.

FIG. 11E shows a case where the wavelet transform is performed three times when the remainder of dividing the number of pixels (tile width of interest) in the horizontal or vertical direction of the tile of interest to be encoded by “8” is “4”. There is.

In order to decode the pixel h at the edge of the tile of interest, the pixel g in the tile of interest, the conversion coefficient data 1Hh in the tile of interest, and the pixel i in the adjacent tile are required. In order to decode the pixel i, the conversion coefficient data 1Hh in the tile of interest and the conversion coefficient data 1Li, 1Hj in the adjacent tile are required. The 1L conversion coefficient data that is not recorded, 1Li, can be calculated from the conversion coefficient data 2Hg in the tile of interest and the conversion coefficient data 2Li, 2Hk in the adjacent tile. The 2L conversion coefficient data that is not recorded, 2Li, can be calculated from the conversion coefficient data 2Le in the tile of interest and the conversion coefficient data 3Hi and 2Lm in the adjacent tile. The conversion coefficient data 2Le can be calculated from the conversion coefficient data 3Ha, 3Le, and 3Hi. The 2L conversion coefficient data that is not recorded, 2Lm, can be calculated from the conversion coefficient data 3Hi, 3Lm, and 3Hq in the adjacent tile. Therefore, after all, as shown in FIG. 11E, in addition to the data in the tile of interest to be encoded, the conversion coefficient data 1Hj of the adjacent tile, the conversion coefficient data 2Hk, the conversion coefficient data 3Hi, the conversion coefficient data 3Hq, and the conversion A coefficient data of 3 Lm is sufficient. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 19 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 4, at least 19 pixels of adjacent tiles are input in addition to the tile of interest to perform wavelet transform. Then, when encoding, the conversion coefficient data of the adjacent tiles is not all the conversion coefficients obtained by the wavelet transform, but the pixel h at the end of the tile of interest and the conversion coefficient data at the end are decoded. The conversion coefficient data {1Hj, 2Hk, 3Hi, 3Hq, 3Lm} required for the above is encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 1 coefficient, and subband 3H. : 2 coefficients, subband 3L: 1 coefficient, total 5 coefficients.

FIG. 11F shows a case where the wavelet transform is performed three times when the remainder of dividing the number of pixels (tile width of interest) in the horizontal or vertical direction of the tile of interest to be encoded by “8” is “5”. There is.

In order to decode the pixel i at the edge of the tile of interest, the conversion coefficient data 1Hh and 1Li of the tile of interest and the conversion coefficient data 1Hj of the adjacent tile are required. The conversion coefficient data 1Li can be calculated from the conversion coefficient data 2Hg and 2Li of the tile of interest and the conversion coefficient data 2Hk of the adjacent tile. The conversion coefficient data 2Li can be calculated from the conversion coefficient data 2Le and 3Hi of the tile of interest and the conversion coefficient data 2Lm of the adjacent tile. The conversion coefficient data 2Lm requires the conversion coefficient data 3Hi of the tile of interest and the conversion coefficient data 3Lm and 3Hq of the adjacent tiles. Therefore, after all, as shown in FIG. 11F, in addition to the data in the tile of interest to be encoded, there are conversion coefficient data 1Hj, conversion coefficient data 2Hk, conversion coefficient data 3Hq, and conversion coefficient data 3Lm of the adjacent tile. Is enough. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 18 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 5, at least 18 pixels of adjacent tiles are input in addition to the tile of interest to perform wavelet transform. Then, when encoding, the conversion coefficient data of the adjacent tiles is not all the conversion coefficients obtained by the wavelet transform, but the pixel i at the end of the tile of interest and the conversion coefficient data at the end are decoded. The conversion coefficient data {1Hj, 2Hk, 3Hq, 3Lm} required for the above is encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 1 coefficient, and subband 3H. In FIG. 11G, the number of pixels in the horizontal or vertical direction of the tile of interest to be encoded (the width of the tile of interest) is divided by "8", which is a total of four coefficients, one coefficient of one coefficient and one coefficient of subband 3L: one coefficient. It shows the case where the wavelet transform is performed three times when the remainder is "6".

In order to decode the pixel j at the end of the tile of interest, the pixel i in the tile of interest, the conversion coefficient data 1Hj, and the pixel k of the adjacent tile are required. In order to decode the pixel k of the adjacent tile, the conversion coefficient data 1Hj in the tile of interest and the conversion coefficient data 1Lk and 1Hl in the adjacent tile are required. The conversion coefficient data 1Lk can be calculated from the conversion coefficient data 1Li in the tile of interest and the conversion coefficient data 2Hk, 1Lm in the adjacent tile. The conversion coefficient data 1Lm can be calculated from the conversion coefficient data 2Hk, 2Lm, 2Ho in the adjacent tile. The conversion coefficient data 2Lm can be calculated from the conversion coefficient data 3Hi in the tile of interest and the conversion coefficient data 3Lm, 3Hq in the adjacent tile. The conversion coefficient data 1Li and 2Li that are not recorded in the tile of interest can be calculated from the conversion coefficient recorded in the tile of interest and the conversion coefficients 2Hk and 2Lm in the adjacent tile. Therefore, after all, as shown in FIG. 11G, in addition to the data in the tile of interest to be encoded, the conversion coefficient data 1Hl, the conversion coefficient data 2Hk, the conversion coefficient data 2Ho, the conversion coefficient data 3Hq, and the conversion of the adjacent tiles are converted. A coefficient data of 3 Lm is sufficient. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 17 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 6, at least 17 pixels of adjacent tiles are input in addition to the tile of interest to perform wavelet transform. Then, when encoding, the conversion coefficient data of the adjacent tiles is not all the conversion coefficients obtained by the wavelet transform, but the pixel j at the end of the tile of interest and the conversion coefficient data at the end are decoded. The conversion coefficient data {1Hl, 2Hk, 2Ho, 3Hq, 3Lm} required for the above is encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 2 coefficient, and subband 3H. In FIG. 11H, which is a total of 5 coefficients of 1 coefficient and 3L of subbands: 1 coefficient, the remainder obtained by dividing the number of pixels in the horizontal or vertical direction (width of the tile of interest) of the tile to be encoded by “8” is “ In the case of 7 ”, the case where the wavelet transform is performed three times is shown.

In order to decode the pixel k at the edge of the tile of interest, the conversion coefficient data 1Hj, 1Lk in the tile of interest and the conversion coefficient data 1Hl in the adjacent tile are required. The conversion coefficient data 1Lk can be calculated from the conversion coefficient data 1Li, 2Hk in the tile of interest and the conversion coefficient data 1Lm in the adjacent tile. The conversion coefficient data 1Lm can be calculated from the conversion coefficient data 2Hk in the tile of interest and the conversion coefficient data 2Lm, 2Ho in the adjacent tile. The conversion coefficient data 2Lm can be calculated from the conversion coefficient data 3Hi in the tile of interest and the conversion coefficient data 3Lm and 3Hq in the adjacent tile. Further, although detailed description is omitted, regarding the conversion coefficient data of the ends of 1L and 2L that are not recorded in the tile of interest, the conversion coefficient data recorded in the tile of interest and the conversion coefficient data 1Lm and 2Lm in the adjacent tile are also obtained. It can be calculated from. Therefore, after all, as shown in FIG. 11H, in addition to the data in the tile of interest to be encoded, the conversion coefficient data 1Hl of the adjacent tile, the conversion coefficient data 2Ho, the conversion coefficient data 3Hi, the conversion coefficient data 3Hq, and the conversion A coefficient data of 3 Lm is sufficient. With this conversion coefficient data, it is possible to decode up to the pixel at the end on the adjacent tile side in the tile of interest. Further, in order to calculate these conversion coefficient data, 16 pixels of adjacent tiles are required. Therefore, when the remainder of "width of interest tile / 8" is 7, at least 16 pixels of adjacent tiles are input in addition to the tile of interest to perform wavelet transform. Then, when encoding, the conversion coefficient data of the adjacent tiles is not all the conversion coefficients obtained by the wavelet transform, but the pixel k at the end of the tile of interest and the conversion coefficient data at the end are decoded. The conversion coefficient data {1Hl, 2Ho, 3Hq, 3Lm} required for the above is encoded.

That is, the conversion coefficient data used among the conversion coefficient data generated from the adjacent tiles for restoring the edge pixels in the tile of interest are subband 1H: 1 coefficient, subband 2H: 1 coefficient, and subband 3H. A total of 4 coefficients, 1 coefficient and 3L of subbands: 1 coefficient. Thus, of the conversion coefficient data generated from the adjacent tiles, the conversion coefficient required to restore the edge pixels in the tile of interest. When the data is obtained, it becomes as shown in FIGS. 10 (a) and 10 (b). It is shown that among the conversion coefficient data in the adjacent tiles, the conversion coefficient data for a few minutes shown in FIG. 10 from the tile of interest is required to restore the pixels at the edges in the tile of interest.

One or more conversion coefficient data of the subband of the high frequency component is always required, and the conversion coefficient data 1H is always 1 regardless of the number of wavelet transforms. The number of conversion coefficient data other than 1H differs depending on the tile width of interest. Further, the conversion coefficient data of the subband of the low frequency component encoded when the wavelet conversion number is N times, that is, the conversion coefficient data (NL) is necessary to restore the pixel at the end of the tile of interest. The number of conversion coefficient data (NL) is 0 or 1, and is determined by the tile width of interest.

As shown in the flow described later, the wavelet transform is performed by the number of overlapping pixels with the numerical values shown in FIGS. 10 (a) and 10 (b), and further, the transform required for decoding the attention tile from the attention tile side in the adjacent tile. By encoding only the conversion coefficient data for the number of coefficient data, the processing speed can be increased and the code amount can be reduced.

Further, in the above description, the conversion coefficient data in the adjacent tile is the conversion coefficient data obtained by referring to the pixels of the adjacent tile, and the tile boundary between the pixel of the tile of interest and the pixel of the adjacent tile. When the conversion coefficient is divided into the tile side of interest and the adjacent tile side by this tile boundary, the conversion coefficient data included in the adjacent tile side is shown.

The above is an example using a reversible 5/3 tap filter, but for example, when wavelet transform is performed using a lossy 9/7 tap filter, the difference from Patent Document 2 can be further increased. ..

Here, the processing of the image coding unit 100 of the first embodiment described above will be described with reference to the flowchart of FIG.

In steps S1 and S2, the setting unit 106 sets the number of divided tiles M and the number of wavelet transforms N (or the number of decomposition levels N) of the one-frame image. Then, in step S3, the setting unit 106 determines the number of overlapping pixels representing the number of pixels around the tile when wavelet transforming each tile. The setting unit 106 holds a table as shown in FIGS. 10A and 10B, and the number of overlapping pixels is set in FIG. 10 according to the wavelet transform number N set in step S2 and the tile width. The number of overlapping pixels is determined as in (a) and (b) B, and the number of overlapping pixels is determined depending on whether the adjacent tile is on the top or left and on the bottom or right. Further, since the number of overlapping pixels when the adjacent tile is below or to the right differs depending on the width of the tile of interest to be encoded, the number of overlapping pixels is not determined collectively for all tiles as shown in FIG. The number of overlapping pixels may be determined. Further, if all the tile widths are the same, the processing may be switched collectively so that if the tile widths are different for each tile, the tiles of interest are determined.

After that, in step S4, the input data control unit 102 receives the attention tile image data and the tile peripheral pixel data (the end of the adjacent tile on the attention tile side) from the frame memory 101 based on the information determined in steps S1 and S3. The pixel data) is read out and supplied to the wavelet transform unit 103. In step S5, the wavelet transform unit 103 executes the wavelet transform a set number of times to generate conversion coefficient data. Then, in step S6, the wavelet transform unit 103 supplies the quantization coefficient data of all the subbands to be encoded, which is obtained from the tile image data of interest, to the quantization unit 104. Further, the wavelet transform unit 103 converts the conversion coefficient data obtained from the pixel data around the tile from the end on the tile boundary side to the number required for decoding the pixels at the end of the tile of interest shown in FIGS. 10A and 10B. The coefficient data is supplied to the decomposition quantization unit 104. The quantization unit 104 receives this and executes the quantization process. As shown in FIGS. 10A and 10B, the number of conversion coefficient data supplied to the quantization unit 104 differs depending on the tile width of the tile of interest and the number of wavelet transforms, and quantization is performed for each subband. The number of conversion coefficient data supplied to unit 104 is defined.

After that, in step S7, the entropy coding unit 105 entropy-encodes the quantized conversion coefficient data to generate the coded data. Then, in step S8, the processing by the data input control unit 102 wavelet transform unit 103, the quantization unit 104, and the entropy coding unit 105 is repeated until the coding of all the tiles is completed.

When encoding a moving image, the processes after step S4 may be repeated after the second frame.

In this embodiment, the tiles of interest are coded using the relationships shown in FIGS. 10 (a) and 10 (b).

That is, when the position of the adjacent tile is lower or right, the number of overlapping pixels and the conversion coefficient data to be encoded among the conversion coefficient data of the adjacent tile are determined according to the tile width of interest and the number of wavelet transforms.

However, in the following modification, the number of overlapping pixels and the conversion coefficient data to be encoded among the conversion coefficient data of the adjacent tiles may be determined regardless of the tile width of interest.

According to FIGS. 10 (a) and 10 (b), when the number of wavelets is 1, the transform required to restore the pixel at the end of the subband (1H, 1L) to be encoded in the tile of interest. The number of coefficient data is 0 to 1, and the number of overlapping pixels is 2 to 3. Therefore, when the number of wavelets is one, the number of overlapping pixels is set to 3, and the number of conversion coefficient data of adjacent tiles to be encoded is set to 1.

When the number of wavelets is 2, the number of conversion coefficient data required to restore the end pixels in the tile of interest in the coded subbands (1H, 2H, 2L) is 1 for 1H. 2H is 1 to 2, 2L is 0 to 1, and the number of overlapping pixels is 6 to 9. Therefore, when the number of wavelets is 2, the number of overlapping pixels is 9, and the number of conversion coefficient data of adjacent tiles to be encoded is 1H: 1, 2H: 2, 2L: 1. Alternatively, the number of conversion coefficient data of adjacent tiles to be encoded may be 2 for all the subbands (1H, 2H, 2L) to be encoded.

When the number of wavelets is 3, the number of conversion coefficient data required to restore the pixel at the end in the tile of interest in the coded subbands (1H, 2H, 3H, 3L) is 1H. 1, 2H is 1 to 2, 3H is 1 to 2, 3L is 0 to 1, and the number of overlapping pixels is 14 to 21. Therefore, when the number of wavelets is 3, the number of overlapping pixels is 21, and the number of conversion coefficient data of adjacent tiles to be encoded is 1H: 1, 2H: 2, 3H: 2, 3L: 1. Alternatively, the number of conversion coefficient data of adjacent tiles to be encoded may be 2 for all the subbands (1H, 2H, 3H, 3L) to be encoded.

In this modification, the amount of coding is larger than that in FIGS. 10A and 10B of the first embodiment, but even if the conversion coefficient data of the adjacent tiles obtained by the wavelet transform is obtained, There is unencoded coefficient data. Therefore, the amount of data can be reduced as compared with encoding all the conversion coefficient data of the adjacent tiles obtained by the wavelet transform.

Next, the decoding of the coded data obtained by using the first embodiment will be described with reference to FIG. FIG. 6 is a block configuration diagram of the decoding unit 600 according to the first embodiment. The decoding unit 600 will be provided in the image pickup apparatus.

The decoding unit 600 includes an entropy decoding unit 601, an inverse quantization unit 602, and an inverse wavelet transform unit 603.

The entropy decoding unit 601 inputs and decodes entropy-encoded data in tile units by EBCOT (Embedded Block Coding with Optimized Truncation) or the like. Then, the entropy decoding unit 601 supplies the quantized conversion coefficient data and the quantization parameter obtained by decoding to the inverse quantization unit 602.

The inverse quantization unit 602 calculates the conversion coefficient data before quantization by multiplying the conversion coefficient data sent from the entropy decoding unit 601 by the quantization parameter obtained by decoding. Then, the inverse quantization unit 602 supplies the calculated conversion coefficient data to the inverse wavelet conversion unit 603.

The inverse wavelet transform unit 603 executes the inverse wavelet transform on the conversion coefficient data sent from the inverse quantize unit 602, and restores the tile image data.

Here, the decoding process of the decoding unit 600 when the wavelet transform is encoded by using the lifting structure using the above-mentioned reversible 5/3 tap filter will be described with reference to FIG. 7.

FIG. 7 is a diagram showing how the coefficients generated by the wavelet transform described with reference to FIG. 3 are subjected to the inverse wavelet transform using the lifting structure. The d', e ", f', g" in FIG. 7 correspond to the d', e ", f', g" in FIG. That is, d'and f'are high frequency conversion coefficient data (1H) of decomposition level 1, and e "and g" are low frequency conversion coefficient data (1L) of decomposition level 1. Of these, the conversion coefficient data d'belongs to the adjacent tile image. From the coefficient data d', e ", f', the pixel e at the end of the tile of interest can be generated.

As described above, in the first embodiment, when the pixel data e of the tile boundary is generated, the wavelet coefficient d'belonging to the adjacent tiles that are not subject to the decoding tile, which is pre-coded as shown in FIG. Decrypt. Therefore, the continuity between adjacent tiles is maintained, and it is possible to prevent the occurrence of image quality deterioration at the tile boundary.

[Second Embodiment]
FIG. 8 is a block configuration diagram of the image coding unit 800 according to the second embodiment. The image coding unit 800 in the second embodiment is different in that the setting unit 801 is provided instead of the setting unit 106 in FIG. 1 in the first embodiment, and the code amount adjusting unit 802 is provided. Further, the operations of the data input control unit 102 and the wavelet transform unit 103 are different from those in FIG. In addition to the function of the setting unit 106 in FIG. 1, the setting unit 801 in the second embodiment inputs a compression rate according to an instruction from an operation unit (not shown), and the data input control unit 102 accordingly. , Controls the code amount adjusting unit 802. FIG. 9 is a flowchart showing a processing procedure according to the second embodiment. Hereinafter, the processing of each component of FIG. 8 will be described with reference to the flowchart of FIG.

When the input compression ratio is smaller than the preset threshold value, the setting unit 801 instructs the data input control unit 102 to perform the wavelet transform of the tile boundary by the mirror processing (S1001, S1002). On the other hand, when the compression rate is equal to or higher than the threshold value, the data input control unit 102 is instructed to perform the wavelet transform of the tile boundary so as to overlap the pixels of the adjacent tiles (S1001, S1003). In addition, a flag indicating which method was encoded (a flag indicating the magnitude determination result of the compression rate and the threshold value) is stored in the header of the encoded stream. The reason why the processing is switched according to the compression rate is as follows.

When the compression rate is smaller than the threshold value, the code amount is large, but the quantization step is small, so that the quantization error is small and the image quality deterioration at the tile boundary is unlikely to occur. Therefore, even if mirror processing is selected, control is performed so as to prevent an increase in the extra coding target coefficient due to overlap.

On the other hand, when the compression ratio is large, the quantization error cannot be ignored, and the image quality deterioration at the tile boundary is conspicuous. Therefore, as in the first embodiment, the coding process with reference to the pixel data around the tile is executed. The actual code amount adjustment is performed by the code amount adjustment unit 802, the details of which will be described later.

The data input control unit 102 divides the image data into tiles of a predetermined size according to the wavelet transform processing method specified by the setting unit 801 and reads the tiles from the frame memory 101 in a predetermined order to the wavelet transform unit 103. Supply.

For example, when the method of performing the wavelet transform of the tile boundary by mirror processing is selected, only the pixels of the tile to be encoded are input without inputting the pixels of the adjacent tile (S1004). On the other hand, if a method of performing the wavelet transform overlapping the pixels of adjacent tiles is selected, the tile input method shown in the first embodiment described above is carried out (S1005). At that time, the wavelet transform processing method specified by the setting unit 801 is also transmitted to the wavelet transform unit 103.

The wavelet transform unit 103 executes the wavelet transform according to the wavelet transform method transmitted from the data control input unit 102. For example, when the method of performing the wavelet transform of the tile boundary by the mirror processing is selected, the pixels of the tile boundary are generated by the mirror processing and the wavelet transform is performed (S1006).

On the other hand, when the method of performing the wavelet transform by overlapping the pixels of the adjacent tiles is selected, the wavelet transform unit 103 executes the wavelet transform using the tile image data of interest and the pixel data around the tile (S1007). ). At this time, the selection of the conversion coefficient data supplied to the quantization unit 104 is the same as that of the first embodiment.

Next, the processing of the code amount adjusting unit 802 in the second embodiment will be described.

The target code amount per frame image is determined by the compression rate. Therefore, let this target code amount be A_Target. The entropy encoding unit 105 entropy-encodes a bit plane composed of the same bit positions of each conversion coefficient data after quantization. The coded data of the plane of bit i is represented by Ci, and the code amount thereof is represented by A (Ci). At this time, the total code amount C_Total of one frame of the coded data generated by the entropy coding unit 105 is as follows.
C_Total = ΣA (Ci) (i = 0, 1, ..., MSB)
Therefore, in order to make the code amount of the frame of interest equal to or less than the target code amount A_Target, the minimum value of k that satisfies the following equation is obtained.
C_total-ΣA (Ck) ≤ A_Target
If k is obtained, the coded data of the bit plane from bit 0 to bit k may be discarded.

When decoding the stream encoded in the second embodiment, it is sufficient to refer to the flag indicating which method recorded in the stream header is selected. By referring to the flag, it can be determined whether the coded data is obtained by overlapping the tile boundaries to the coefficients of adjacent tiles or by mirroring the tile boundaries. ..

As described above, according to the second embodiment, it is selected whether to perform mirror processing or to refer to the tile peripheral pixel data at the time of coding processing of the tile image data, depending on the compression rate. As a result, the generation of noise at the tile boundary can be suppressed more than ever, especially when a high compression ratio is set.

(Other Examples)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

100 ... image coding unit, 101 ... frame memory, 102 ... data input control unit, 103 ... wavelet transforming unit, 104 ... quantization unit, 105 ... entropy coding unit, 106 ... setting unit

Claims (14)

An image coding device that divides image data into a plurality of tiles and encodes each tile.
A conversion means that performs wavelet transform for each tile to obtain conversion coefficient data,
For each tile, a coding means for encoding the conversion coefficient data obtained by the conversion means is provided.
If there are adjacent tiles adjacent to the tile to be encoded,
The conversion means converts by performing the wavelet transform with reference to the pixel data of the tile to be encoded and a plurality of pixel data of the adjacent tiles adjacent to the tile to be encoded. Get the coefficient data,
The coding means itself when acquiring pixel data from the conversion coefficient data by the inverse wavelet transform among the conversion coefficient data corresponding to the adjacent tiles obtained by referring to the plurality of pixel data of the adjacent tiles. An image coding device characterized in that the conversion coefficient data required for acquiring the pixel data of a tile is encoded, and a part of the conversion coefficient data corresponding to the adjacent tile is not encoded. The image coding apparatus according to claim 1, wherein the coding means quantizes the conversion coefficient data obtained by the conversion means and encodes the quantized coefficient data. The conversion means performs wavelet transform a plurality of times to acquire conversion coefficient data of a plurality of subbands, and obtains the conversion coefficient data.
The image according to claim 1, wherein the coding means encodes a subband of a low frequency component having the highest decomposition level and a subband of a high frequency component among the plurality of subbands. Encoding device. Among the conversion coefficient data in the adjacent tile, the coding means converts the conversion coefficient data adjacent to the tile boundary with the coded target tile into the number of pixels in the vertical or horizontal direction of the coded target tile. The image coding apparatus according to claim 1, wherein the image coding apparatus is encoded by a corresponding number. Among the conversion coefficient data in the adjacent tiles, the coding means encodes the conversion coefficient data adjacent to the tile boundary with the tile to be coded by the number corresponding to the number of wavelet transforms by the conversion means. The image coding apparatus according to claim 1, wherein the image coding apparatus is used. The claim is characterized in that the number of pixels of the adjacent tile for which wavelet transform is performed by the conversion means together with the tile to be encoded is determined according to the number of pixels in the horizontal or vertical direction of the tile to be encoded. The image coding apparatus according to 1. The number of pixels of the adjacent tile to be wavelet-transformed by the conversion means together with the tile to be encoded is determined according to the remainder when the number of pixels in the horizontal or vertical direction of the tile to be encoded is divided by a predetermined number. The image coding apparatus according to claim 6 , wherein the image coding apparatus is used. The image according to claim 1, wherein the conversion coefficient data in the adjacent tile to be encoded by the coding means is determined according to the number of pixels in the horizontal or vertical direction of the tile to be encoded. Encoding device. The conversion coefficient data in the adjacent tile to be encoded by the coding means is determined according to the remainder when the number of pixels in the horizontal or vertical direction of the tile to be coded is divided by a predetermined number. The image coding apparatus according to claim 8 . Further, it has a setting means for setting the number of resolution levels of the conversion coefficient data obtained by the wavelet transform and the compression rate.
When the compression ratio set by the setting means is smaller than the preset threshold value, the conversion means is said to be outside the coding target tile when performing wavelet transform on the coding target tile. Assuming that the pixels in the tile to be encoded are folded back, wavelet transform is performed to obtain the conversion coefficient data of the resolution level set by the setting means, and the conversion coefficient data generated from the tile to be encoded is performed. To the quantization means,
When the compression rate set by the setting means is equal to or higher than a preset threshold value, the tile to be encoded and the tile to be encoded are adjacent to each other in order to obtain the conversion coefficient data of the resolution level set by the setting means. Wavelet transform is performed with reference to the pixel data of adjacent tiles.
Among the conversion coefficient data obtained from the adjacent tiles, the conversion coefficient data of each resolution level closest to the boundary between the tile to be encoded and the adjacent tile, and the conversion coefficient data obtained from the tile to be encoded. Is supplied to the quantization means as conversion coefficient data of the quantization target corresponding to the tile to be encoded.
The image coding apparatus according to claim 1. The image coding apparatus according to claim 10 , wherein information indicating a result of comparison between the compression rate set by the setting means and the threshold value is stored in a header of a coded stream and output. The conversion means performs wavelet transform using a 5/3 tap filter.
The image coding apparatus according to claim 10 , wherein among the conversion coefficient data obtained from the adjacent tiles, high-frequency conversion coefficient data of each resolution level is supplied to the quantization means. It is a control method of an image coding device that divides image data into a plurality of tiles and encodes each tile.
The conversion process performs wavelet transform for each tile to obtain conversion coefficient data, and
The coding means includes, for each tile, a coding step of coding the conversion coefficient data obtained by the conversion means.
If there are adjacent tiles adjacent to the tile to be encoded,
In the conversion step, conversion is performed by performing the wavelet transform with reference to the pixel data of the tile to be encoded and a plurality of pixel data of the adjacent tiles adjacent to the tile to be encoded. Get the coefficient data,
In the coding step, among the conversion coefficient data corresponding to the adjacent tiles obtained by referring to the plurality of pixel data of the adjacent tiles, the pixel data is acquired from the conversion coefficient data by the inverse wavelet transform. A control method of an image coding apparatus, characterized in that the conversion coefficient data required for acquiring the pixel data of a tile is encoded, and a part of the conversion coefficient data corresponding to the adjacent tile is not encoded. A program for causing the computer to function as each means of the image coding apparatus according to any one of claims 1 to 12 , when the computer reads and executes the computer.