JP4014085B2 - Image processing apparatus, program, and storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, a program, and a storage medium.
[0002]
[Prior art]
Due to advances in image input technology and output technology, the demand for higher definition of images has increased greatly in recent years. For example, taking a digital camera as an example of an image input device, the price of a high-performance charge-coupled device (CCD) having a number of pixels of 3 million or more has been reduced, and the price range has become widespread. It has come to be widely used in products. A product with 5 million pixels is also close. And it is said that this increasing tendency of the number of pixels will continue for a while.
[0003]
On the other hand, for image output / display devices, for example, products in the hard copy field such as laser printers, ink jet printers, sublimation printers, and flats such as CRTs, LCDs (liquid crystal display devices), and PDPs (plasma display devices). The high definition and low price of products in the soft copy field of panel displays are remarkable.
[0004]
Due to the market launch of these high-performance, low-priced image input / output products, high-definition images have become popular, and it is expected that demand for high-definition images will increase in all situations. In fact, the development of technology related to networks such as personal computers and the Internet is accelerating these trends. In particular, recently, mobile devices such as mobile phones and notebook personal computers have become very popular, and opportunities for transmitting or receiving high-definition images from any point using communication means are rapidly increasing.
[0005]
Against this background, it is inevitable that the demand for higher performance or higher functionality for image compression / decompression technology that facilitates the handling of high-definition images will become stronger in the future.
[0006]
Therefore, in recent years, a new method called JPEG2000, which can restore high-quality images even at a high compression rate, is being standardized as one of image compression methods that satisfy these requirements. In JPEG2000, it is possible to perform compression / decompression processing in a small memory environment by dividing an image into rectangular regions (tiles). That is, each tile becomes a basic unit for executing the compression / decompression process, and the compression / decompression operation can be performed independently for each tile.
[0007]
[Problems to be solved by the invention]
By the way, conventionally, with respect to correction of an encoded image, image correction processing such as skew correction and color correction is performed on an image obtained by expanding the entire encoded data, and then the entire image is compressed again and encoded. The method of making data is used.
[0008]
However, in this method, since the entire image is expanded in the memory, particularly in an apparatus or system with a small amount of mounted memory, the memory becomes insufficient, and various restrictions may occur.
[0009]
The purpose of the present invention is to In particular, skew correction of encoded images At the same time, an object is to provide an image processing apparatus, a program, and a storage medium that can avoid a memory shortage even in an apparatus or system with a small amount of mounted memory.
[0010]
[Means for Solving the Problems]
An image processing apparatus according to a first aspect of the present invention comprises: means for compressing a pixel value by a procedure of discrete wavelet transform, quantization, and encoding for each tile obtained by dividing an image into a plurality of units; A skew correction process for decompressing the compression code data and correcting the inclination of the image with respect to the decompressed image. And a series of processes for recompressing the skew-corrected image is executed in tile units.
[0011]
Therefore, when the skew correction for correcting the inclination of the image is executed based on the compressed image data, the skew correction process for the expanded image is executed for each tile. As a result, since it is sufficient to develop an image in memory in units of tiles, it is possible to avoid a memory shortage even in an apparatus or system with a small amount of mounted memory, and there are no restrictions.
[0012]
Claim 2 The image processing apparatus according to claim 1, wherein the skew correction processing includes a skew correction angle detection unit that detects a skew correction angle based on an image obtained by extending one tile, and the skew correction angle detection unit. And image rotation means for sequentially rotating each tile in accordance with the skew correction angle detected by the above.
[0013]
Accordingly, the tiles are sequentially rotated according to the skew correction angle detected based on the image obtained by extending one tile. Thereby, it becomes possible to correct the inclination of the image quickly and reliably.
[0014]
Claim 3 The described invention Claim 2 In the described image processing apparatus, the skew correction angle detection unit detects a skew correction angle based on a low-resolution image.
[0015]
Accordingly, it is possible to reduce the skew correction angle detection time and reduce the memory used.
[0016]
Claim 4 The described invention Claim 2 or 3 In the image processing apparatus described above, the skew correction angle detection unit detects the skew correction angle using any one of the color spaces of the image.
[0017]
Accordingly, it is possible to reduce the skew correction angle detection time and reduce the memory used.
[0018]
Claim 5 The described invention Claim 2 or 3 In the described image processing apparatus, the skew correction angle detection unit detects the skew correction angle using only the upper bit plane.
[0019]
Accordingly, it is possible to reduce the skew correction angle detection time and reduce the memory used.
[0020]
Claim 6 The described invention Claims 3 to 5 In the image processing apparatus according to any one of the above, the mode selection unit that enables selection of an image quality priority mode that prioritizes image quality and a speed priority mode that prioritizes processing speed for the skew correction processing, and the skew correction processing, Mode switching means for switching between the image quality priority mode and the speed priority mode according to the selection by the mode selection means.
[0021]
Therefore, the image quality priority mode and the speed priority mode are switched according to the selection by the mode selection means, and the skew correction process is executed. Thus, when the speed priority mode is designated, the resolution may be lowered, or a specific color space or a part of bit planes may be used. Further, when the image quality priority mode is designated, the resolution may be increased, or all color spaces or all bit planes may be displayed.
[0022]
Claim 7 The described invention Claims 2 to 6 In the image processing apparatus according to any one of the above, the skew correction angle detection unit detects a skew correction angle based on a timing mark preset in the image.
[0023]
Therefore, the skew correction angle can be detected with high accuracy and at high speed.
[0024]
Claim 8 The described invention Claim 7 The described image processing apparatus is characterized in that only tiles that are highly likely to contain timing marks are expanded.
[0025]
Therefore, it is possible to increase the processing speed and save memory.
[0026]
Claim 9 The described invention Claim 7 In the described image processing apparatus, when a ROI (Region Of Interest) area is designated, only a tile that is highly likely to include a timing mark in the ROI area is expanded.
[0027]
Therefore, it is possible to further increase the processing speed and save memory.
[0028]
Claim 10 The described invention Claims 2 to 6 In the image processing apparatus according to any one of the above, the skew correction angle detection unit displays a grid and detects a skew correction angle based on an image rotated according to the grid.
[0029]
Therefore, the user can visually confirm the skew correction angle, and the skew correction angle is detected by a simple method.
[0030]
Claim 11 The described invention Claim 2 In the image processing apparatus described above, the image rotation unit complements white pixels for invalid areas generated by rotation in each tile.
[0031]
Therefore, it is possible to supplement the pixels by a simple method with respect to the invalid area generated by rotation in each tile. Here, the invalid area generated by rotation in each tile is an area that is generated with rotation in each tile and discarded.
[0032]
Claim 12 The described invention Claim 11 In the described image processing apparatus, a low-pass filter process is performed on a region supplemented with white pixels.
[0033]
Therefore, it is possible to make the region supplemented with white pixels inconspicuous.
[0034]
Claim 13 The described invention Claim 2 In the image processing apparatus described above, the image rotation unit forms a tile group by a desired tile and three peripheral tiles adjacent to each other at one corner of the tile, and the tile group is formed using one corner of the tile group as a starting point. It is characterized by being rotated by a skew correction angle.
[0035]
Therefore, it becomes possible to supplement the invalid area caused by the rotation.
[0036]
Claim 14 The described invention Claim 2 In the image processing apparatus described above, the image rotation unit forms a tile group by a desired tile and four peripheral tiles adjacent to the right, left, top, and bottom of the tile, and starts from the center of the desired tile. The tile group is rotated by a skew correction angle.
[0037]
Therefore, it becomes possible to supplement the invalid area caused by the rotation.
[0038]
Claim 15 The described invention Claim 2 In the image processing apparatus described above, the image rotation unit sets a rotation start point corresponding to a skew correction angle for all tiles at one position of the image.
[0039]
Therefore, it is possible to eliminate the rotation error.
[0040]
Claim 16 The described invention Claims 1 to 15 In the image processing apparatus according to any one of the above, the moving image that continuously displays images at a predetermined frame rate is based on the image expanded in units of tiles in the first frame. Skew A correction condition related to the correction process is obtained, and the correction condition is applied to other frames.
[0041]
Therefore, it is possible to shorten the detection time of the correction condition.
[0042]
Claim 17 The program of the described invention includes a function of generating compressed code data by compressing a pixel value for each tile obtained by dividing an image into a plurality of steps by a procedure of discrete wavelet transform, quantization and encoding, and compressing code data of the above procedure. A program that causes a computer to execute a function of decompressing and generating an image in the reverse procedure, and decompressing the compressed code data to the computer and correcting skew of the image with respect to the decompressed image A series of processes for performing processing and re-compressing the skew-corrected image is executed in units of tiles.
[0043]
Therefore, when the skew correction for correcting the tilt of the image is executed based on the compressed image data, the skew correction process for the expanded image is executed on a tile basis. As a result, since it is sufficient to develop an image in memory in units of tiles, it is possible to avoid a memory shortage even in an apparatus or system with a small amount of mounted memory, and there are no restrictions.
[0044]
Claim 18 The storage medium of the described invention is Claim 17 The described program is stored.
[0045]
Therefore, by having the computer read the program stored in this storage medium, Claim 17 It is possible to obtain the same operation as the described invention.
[0046]
DETAILED DESCRIPTION OF THE INVENTION
First, an outline of the “hierarchical encoding algorithm” and the “JPEG2000 algorithm” which are the premise of the embodiment of the present invention will be described.
[0047]
FIG. 1 is a functional block diagram of a system that realizes a hierarchical encoding algorithm that is the basis of the JPEG2000 system. This system includes color space transform / inverse transform unit 101, two-dimensional wavelet transform / inverse transform unit 102, quantization / inverse quantization unit 103, entropy encoding / decoding unit 104, and tag processing unit 105. It is configured.
[0048]
One of the biggest differences between this system and the conventional JPEG algorithm is the conversion method. In JPEG, discrete cosine transform (DCT) is used. In this hierarchical coding algorithm, the secondary wavelet transform / inverse transform unit 102 uses discrete wavelet transform (DWT). ing. DWT has the advantage that the image quality in the high compression region is better than DCT, and this is one of the main reasons why DWT is adopted in JPEG2000, which is a successor algorithm of JPEG.
[0049]
Another major difference is that in this hierarchical encoding algorithm, a functional block of the tag processing unit 105 is added in order to perform code formation at the final stage of the system. The tag processing unit 105 generates compressed data as code string data during an image compression operation, and interprets code string data necessary for decompression during the decompression operation. With the code string data, JPEG2000 can realize various convenient functions. For example, the compression / decompression operation of a still image can be freely stopped at an arbitrary layer (decomposition level) corresponding to octave division in block-based DWT (see FIG. 3 described later). Further, a low resolution image (reduced image) can be extracted from one file, or a part of the image (tiling image) can be extracted.
[0050]
In many cases, color space conversion / inverse conversion 101 is connected to the input / output portion of the original image. For example, the RGB color system composed of R (red) / G (green) / B (blue) components of the primary color system and the Y (yellow) / M (magenta) / C (cyan) components of the complementary color system This corresponds to the part that performs conversion or reverse conversion from the YMC color system consisting of the above to the YUV or YCbCr color system.
[0051]
Next, the JPEG2000 algorithm will be described.
[0052]
As shown in FIG. 2, in a color image, each component 111 (RGB primary color system here) of an original image is generally divided by a rectangular area. This divided rectangular area is generally called a block or a tile, but in JPEG2000, it is generally called a tile. Therefore, such a divided rectangular area is hereinafter referred to as a tile. (In the example of FIG. 2, each component 111 is divided into a total of 16 rectangular tiles 112, 4 × 4 in length and breadth). When such individual tiles 112 (R00, R01,..., R15 / G00, G01,..., G15 / B00, B01,..., B15 in the example of FIG. 2) execute the image data compression / decompression process. It becomes the basic unit. Therefore, the compression / decompression operation of the image data is performed independently for each component and for each tile 112.
[0053]
At the time of encoding image data, the data of each tile 112 of each component 111 is input to the color space conversion / inverse conversion unit 101 of FIG. 1 and subjected to color space conversion, and then the two-dimensional wavelet conversion / inverse conversion unit. A two-dimensional wavelet transform (forward transform) is applied at 102 to divide the space into frequency bands.
[0054]
FIG. 3 shows subbands at each decomposition level when the number of decomposition levels is three. In other words, the tile original image (0LL) (decomposition level 0) obtained by tile division of the original image is subjected to two-dimensional wavelet transform, and the subbands (1LL, 1HL, 1LH shown in the decomposition level 1) , 1HH). Subsequently, the low-frequency component 1LL in this hierarchy is subjected to two-dimensional wavelet transformation to separate the subbands (2LL, 2HL, 2LH, 2HH) indicated by the decomposition level 2. Similarly, the low-frequency component 2LL is also subjected to two-dimensional wavelet transform to separate subbands (3LL, 3HL, 3LH, 3HH) shown in the decomposition level 3. In FIG. 3, the subbands to be encoded at each decomposition level are indicated by shading. For example, when the number of decomposition levels is 3, the subbands (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, 1HH) indicated by shading are the encoding targets, and the 3LL subband is encoded. It is not converted.
[0055]
Next, the bits to be encoded are determined in the specified encoding order, and the context is generated from the bits around the target bits by the quantization / inverse quantization unit 103 shown in FIG.
[0056]
The wavelet coefficients that have undergone the quantization process are divided into non-overlapping rectangles called “precincts” for each subband. This was introduced to use memory efficiently in implementation. As shown in FIG. 4, one precinct consists of three rectangular regions that are spatially coincident. Further, each precinct is divided into non-overlapping rectangular “code blocks”. This is the basic unit for entropy coding.
[0057]
The coefficient values after the wavelet transform can be quantized and encoded as they are, but in JPEG2000, in order to increase the encoding efficiency, the coefficient values are decomposed into “bit plane” units, and each pixel or code block is divided. Ranking can be performed on “bitplanes”.
[0058]
Here, FIG. 5 is an explanatory diagram showing an example of a procedure for ranking the bit planes. As shown in FIG. 5, this example is a case where the original image (32 × 32 pixels) is divided into four 16 × 16 pixel tiles, and the size of the precinct and code block at the composition level 1 is Each is 8 × 8 pixels and 4 × 4 pixels. The numbers of the precinct and the code block are assigned in raster order. In this example, the number of assigns is assigned from numbers 0 to 3, and the code block is assigned from numbers 0 to 3. A mirroring method is used for pixel expansion outside the tile boundary, wavelet transform is performed with a reversible (5, 3) filter, and a wavelet coefficient value of decomposition level 1 is obtained.
[0059]
An explanatory diagram showing an example of the concept of a typical “layer” configuration for tile 0 / precinct 3 / code block 3 is also shown in FIG. The converted code block is divided into subbands (1LL, 1HL, 1LH, 1HH), and wavelet coefficient values are assigned to the subbands.
[0060]
The layer structure is easy to understand when the wavelet coefficient values are viewed from the horizontal direction (bit plane direction). One layer is composed of an arbitrary number of bit planes. In this example, layers 0, 1, 2, and 3 are made up of bit planes of 1, 3, 1, and 3, respectively. A layer including a bit plane closer to the LSB (Least Significant Bit) is first subjected to quantization, and conversely, a layer close to the MSB (Most Significant Bit: most significant bit) is quantized to the end. It will remain without being. A method of discarding from a layer close to the LSB is called truncation, and the quantization rate can be finely controlled.
[0061]
The entropy encoding / decoding unit 104 illustrated in FIG. 1 performs encoding on the tile 112 of each component 111 by probability estimation from the context and the target bit. In this way, encoding processing is performed in units of tiles 112 for all components 111 of the original image. Finally, the tag processing unit 105 performs a process of combining all the encoded data from the entropy encoding / decoding unit 104 into one code string data and adding a tag thereto.
[0062]
FIG. 6 shows a schematic configuration of this code string data. Tag information called a header (a main header (tile header) that is tile header position information) is provided at the head of the code string data and the head of the code data (bit stream) of each tile. Appended, followed by the encoded data for each tile. Note that the main header (Main header) describes coding parameters and quantization parameters. A tag (end of codestream) is placed again at the end of the code string data.
[0063]
On the other hand, when the encoded data is decoded, the image data is generated from the code string data of each tile 112 of each component 111, contrary to the case of encoding the image data. In this case, the tag processing unit 105 interprets tag information added to the code string data input from the outside, decomposes the code string data into code string data of each tile 112 of each component 111, and Decoding processing (decompression processing) is performed for each code string data of each tile 112. At this time, the positions of the bits to be decoded are determined in the order based on the tag information in the code string data, and the quantization / inverse quantization unit 103 determines the peripheral bits of the target bit position (already decoded). Context is generated from the sequence of The entropy encoding / decoding unit 104 performs decoding by probability estimation from the context and code string data, generates a target bit, and writes it in the position of the target bit. Since the data decoded in this way is spatially divided for each frequency band, the two-dimensional wavelet transform / inverse transform unit 102 performs two-dimensional wavelet inverse transform on each of the components of the image data. The tile is restored. The restored data is converted to original color system image data by the color space conversion / inverse conversion unit 101.
[0064]
The above is the outline of the “JPEG2000 algorithm”.
[0065]
Hereinafter, a first embodiment of the present invention will be described.
[0066]
FIG. 7 is a system configuration diagram showing a system including the image processing apparatus 1 to which the present invention is applied. As shown in FIG. 7, an image processing apparatus 1 to which the present invention is applied is, for example, a personal computer, and can be connected to a server computer S that stores various image data via a network 5 that is the Internet. .
[0067]
In the present embodiment, the image data stored and held in the server computer S is a compressed code generated according to the “JPEG2000 algorithm”. More specifically, the compression code is configured as shown in FIG. 9 by compressing and coding the divided images divided into rectangular areas (tiles) as shown in FIG. 8 and arranging them in one dimension. In FIG. 9, SOC is a marker segment indicating the start of a code stream. MH is a main header, and stores a value common to the entire code stream. As values common to the entire code stream, for example, a tile width, a tile height, an image width, an image height, and the like are recorded. The data following MH is data obtained by encoding each tile. In FIG. 9, data obtained by compressing each tile in the main scanning direction / sub-scanning direction is arranged according to the tile number shown in FIG. The EOC marker at the end of the compression code is a marker segment indicating the end of the compression code.
[0068]
FIG. 10 is an explanatory diagram showing a resolution model of a compression code generated according to the “JPEG2000 algorithm”. As shown in FIG. 10, in the compression code generated according to the “JPEG2000 algorithm”, the low resolution data and the high resolution data are separated in one image file. Although only two types of resolution are shown in FIG. 10, in reality, if all data are set to 1, according to an arbitrary hierarchy (decomposition level) corresponding to octave division in DWT, 1 / It is possible to extract compression codes related to a plurality of low resolution images such as 2, 1/4, 1/8, and 1/16.
[0069]
FIG. 11 is a block diagram schematically showing a hardware configuration of the image processing apparatus 1. As shown in FIG. 11, the image processing apparatus 1 includes a CPU (Central Processing Unit) 6 that performs information processing, a ROM (Read Only Memory) 7 that stores information, a RAM (Random Access Memory) 8 and other primary storage devices, HDD (Hard Disk Drive) 10, which is a storage unit for storing compressed codes (see FIG. 9) downloaded from the server computer S via the network 5, stores information, distributes information to the outside, and obtains information from the outside A CD-ROM drive 12 for communication, a communication control device 13 for communicating information with other external computers via the network 5, and a CRT (Cathode Ray Tube) for displaying processing progress and results to the operator And LCD (Liquid a display device 15 such as a crystal display) and an input device 14 such as a keyboard and a mouse for an operator to input commands and information to the CPU 6, and buses data transmitted and received between these components. The controller 9 operates by arbitrating.
[0070]
Since the RAM 8 has a property of storing various data in a rewritable manner, the RAM 8 functions as a work area for the CPU 6 and plays a role of, for example, a buffer memory in image correction processing described later.
[0071]
In such an image processing apparatus 1, when the user turns on the power, the CPU 6 activates a program called a loader in the ROM 7, loads a program for managing the hardware and software of the computer called the operating system from the HDD 10 into the RAM 8, and Start the system. Such an operating system starts a program, reads information, and performs storage according to a user operation. As typical operating systems, Windows (registered trademark), UNIX (registered trademark), and the like are known. An operation program running on these operating systems is called an application program.
[0072]
Here, the image processing apparatus 1 stores an image processing program in the HDD 10 as an application program. In this sense, the HDD 10 functions as a storage medium that stores an image processing program.
[0073]
In general, an operation program installed in the HDD 10 of the image processing apparatus 1 is recorded on an optical information recording medium such as a CD-ROM 11 or DVD-ROM, a magnetic medium such as an FD, and the like. A program is installed in the HDD 10. For this reason, portable storage media such as optical information recording media such as the CD-ROM 11 and magnetic media such as the FD can also be storage media for storing the image processing program. Furthermore, the image processing program may be imported from the outside via, for example, the communication control device 13 and installed in the HDD 10.
[0074]
In the image processing apparatus 1, when an image processing program that operates on an operating system is started, the CPU 6 executes various arithmetic processes according to the image processing program and controls each unit intensively. Of the various types of arithmetic processing executed by the CPU 6 of the image processing apparatus 1, characteristic processing of the present embodiment will be described below.
[0075]
FIG. 12 is a flowchart showing the flow of image correction processing executed by the CPU 6 in accordance with the image processing program. Here, as the image correction process, a skew correction process for correcting an image tilted (skewed) during image input will be mainly described.
[0076]
In the image correction processing according to the present embodiment, as shown in FIG. 12, first, a low-resolution image is obtained by decoding a compression code related to a low-resolution image (step S1), and then the low-resolution image is processed. Binarization processing is executed (step S2). As described above, since only the low resolution image can be extracted based on the compression code generated according to the “JPEG2000 algorithm”, in this embodiment, the low resolution image is eliminated in order to eliminate the shortage of the memory area. The binarization process is executed for. In the binarization process in the case of a color multi-valued image, for example, attention is paid to any one of RGB components and YCbCr components (for example, G component), and black pixels that are larger than a predetermined density threshold of the G component are black pixels And a pixel smaller than a predetermined density threshold of the G component may be a white pixel. Alternatively, RGB may be color-converted to be divided into a luminance component and a color difference component, and threshold processing may be performed with the luminance component. In this way, by using one of the color spaces of the image file for detecting the skew correction angle, it is possible to shorten the skew correction angle detection time and reduce the memory used in the RAM 8.
[0077]
Note that the decompression process (decoding process) according to the “JPEG2000 algorithm” of the compression code is the spatial transform / inverse transform unit 101, the two-dimensional wavelet transform / inverse transform unit 102, the quantization / inverse quantum shown in FIG. Since description is made in the description of the encoding unit 103, the entropy encoding / decoding unit 104, and the tag processing unit 105, description thereof is omitted here (the same applies hereinafter). In this way, by detecting a skew correction angle, which will be described later, based on an image encoded / decoded according to the “JPEG2000 algorithm”, there are very few defects in the image encoded / decoded according to the “JPEG2000 algorithm”. For this reason (for example, there is no break in ruled lines and surface lines), it becomes possible to improve the accuracy of skew angle detection.
[0078]
In the subsequent step S3, a skew correction angle is detected based on the low-resolution image on which the binarization process has been executed.
[0079]
Here, the skew correction angle detection process will be described in detail. The skew correction angle detection process is performed based on detection of timing marks as shown in FIG. 13, for example. The timing mark is a reference point used for correcting that the image is tilted (skewed), the image is shifted, or stretched. More specifically, three timing marks (upper left, upper right, lower left or lower right) are provided on one document. The two timing marks on the upper left and upper right can detect how much the input image is tilted (skewed), how much it is shifted in the X direction, and is expanded and contracted. By using another timing mark, it is possible to detect how much the Y-direction is shifted and expanded / contracted. That is, here, a skew correction angle detection process for an image obtained by reading an original on which such a timing mark is printed will be described.
[0080]
FIG. 14 is a flowchart showing the flow of timing mark detection processing. First, for a low-resolution image that has been subjected to binarization processing, a timing mark search range is obtained based on a preset timing mark position (that is, a position of a timing mark with no skew) (step S101). ). The search range is determined by how much skew, deviation, etc. are allowed. That is, as shown in FIG. 15, a coordinate value obtained by adding / subtracting an allowable amount (± α) to a preset timing mark position becomes a timing mark search range, and the timing mark is included in this search range. The range should be decided. This range is also
The range may be determined dynamically and dynamically, or the range may be designated every time the user inputs an image. Accordingly, since a part of the image (tiling image) can be extracted based on the compression code generated according to the “JPEG2000 algorithm”, only a tile having a high possibility of including a timing mark (for example, skew occurs). If only the tiles at the position where the tile having the timing mark exists and the surrounding tiles are decoded), it is possible to increase the processing speed and save the memory.
[0081]
When a ROI (Region Of Interest) area is designated, only tiles that are likely to include a timing mark in the ROI area (for example, when there is no skew, the timing mark is included). If only the tiles at the position where the tiles exist and the surrounding tiles are decoded, it is possible to further increase the processing speed and memory saving. Note that the ROI region is a part of the entire image when the image is cut out and enlarged from the entire image or emphasized as compared with other parts.
[0082]
When the timing mark search range is determined (step S101), a rectangle (circumscribed rectangle) is extracted within the timing mark search range (step S102). As this rectangle extraction method, a known method for extracting connected components of black pixels is used. In general, a plurality of rectangles are extracted by the rectangle extraction process. That is, the timing mark detection range may include not only the timing mark to be detected but also a character rectangle, a noise rectangle, and the like, and the plurality of rectangles refer to these rectangles.
[0083]
However, as shown in FIG. 15, the rectangle in contact with the search range is deleted (ignored) as noise. Further, when the rectangle extraction process is completed and no rectangle is extracted (N in step S103), the timing mark cannot be detected (step S109).
[0084]
Next, when a plurality of rectangles are extracted (Y in step S103), a feature extraction process for extracting feature amounts from each rectangle is performed (step S104). After obtaining the feature amount of each rectangle, the distance between the preset feature amount of the timing mark and the feature amount of each rectangle is calculated to obtain timing mark candidates (steps S105 and S106). Here, the distance is, for example, a dissimilarity that takes a difference between feature amounts. The smaller the distance, the smaller the dissimilarity, and the timing mark candidate rank is higher.
[0085]
When the distance calculation is performed in this way and the timing mark candidate rank is determined from the first candidate to the nth complement (Y in step S106), the calculated first complement distance is set in advance. It is determined whether or not the threshold value is satisfied, and if it is equal to or less than the threshold value (Y in step S107), the first complement is output as a timing mark (step S108). By performing the comparison process with the threshold value in this way, it is possible to prevent a part such as a character from being erroneously extracted as a timing mark. In other words, when the first complement calculated as a distance is output as it is as a timing mark, when an image without a timing mark is processed, a part such as a character is erroneously extracted as a timing mark. Therefore, the timing mark is detected accurately when there is a timing mark, and when there is no timing mark (N in step S107), the result is output as “not present (cannot be detected)” (step S109).
[0086]
When the timing mark detection process is completed, a skew correction angle is detected according to the detected timing mark. There are various known methods for detecting the skew correction angle. For example, there is a method for detecting the skew correction angle from the difference between the coordinate value of the preset timing mark and the coordinate value of the detected timing mark. .
[0087]
Therefore, in steps S1 to S3, the function of the skew correction angle detection means is executed.
[0088]
When the skew correction angle is detected as described above, tiles (encoded data of each tile shown in FIG. 9) are sequentially taken out, decompressed in the RAM 8, and subjected to rotation processing according to the detected skew correction angle, and then compressed. By performing processing and writing back the code string data shown in FIG. 9, skew correction of the entire image is performed (steps S4 to S7: image rotation means).
[0089]
A rotation process according to the skew correction angle will be described. FIG. 16A shows a state where the tile is rotated starting from the upper left corner, and FIG. 16B shows a state where the tile is rotated starting from the center of the tile. As shown in FIG. 16A, when the skew correction angle is rotated starting from the upper left corner of the tile, the area corrected by rotation (effective area) and the area discarded by rotation in each tile (invalid area) ) Occurs. Such invalid areas are complemented with white pixels. On the other hand, as shown in FIG. 16B, when the skew correction angle is rotated starting from the center of the tile, similarly, the area corrected by the rotation (effective area) is discarded along with the rotation of each tile. Area (invalid area) occurs. Such invalid areas are complemented with white pixels. Therefore, it is possible to supplement the pixels by a simple method with respect to the invalid area generated by rotation in each tile.
[0090]
However, as described above, if the tiles are rotated one by one, there is a merit that the skew correction processing can be executed even with a small amount of memory. However, an invalid area is generated in each tile, and the image is in a tooth missing state ( There is a problem that a gap is generated.
[0091]
Therefore, when the image is in a missing state (a state in which a gap is generated) as described above, the gap can be made inconspicuous by performing a low-pass filter process on the gap. Since the low-pass filter processing is well known, the description thereof is omitted. Note that, in order to solve the problem that the tile boundary becomes discontinuous during the tile expansion process (step S5), the tile boundary is made inconspicuous by performing low-pass filter processing only in the vicinity of the tile boundary. Therefore, it is not necessary to perform low-pass filter processing on the gap.
[0092]
Further, as shown in FIG. 17A, a tile group is formed by a desired tile and three peripheral tiles adjacent at one corner of the tile, and the tile group is skewed starting from one corner of the tile group. You may make it complement the invalid area | region which arises by rotation by rotating only a correction | amendment angle | corner. Here, the three peripheral tiles that are rotated together with the target tile are tiles that are located at the left, lower left, and lower of the target tile when the tile group starts from the upper left and rotates left (clockwise). is there. In addition, when the tile group starts from the upper left and rotates right (counterclockwise), the tile is located on the right, upper right, and upper side of the target tile. On the other hand, as shown in FIG. 17B, a tile group is formed by a desired tile and four peripheral tiles adjacent to the right, left, top, and bottom of the tile, and the center of the target tile is used as a starting point. By rotating the tile group by the skew correction angle, the invalid area generated by the rotation may be complemented.
[0093]
Furthermore, a modified example of the rotation process according to the skew correction angle will be described. FIG. 18 shows a state where the tile is rotated from one place of the image. As shown in FIG. 18, when the starting point of rotation corresponding to the skew correction angle for all tiles is set at one place in the image and all tiles are rotated according to this skew correction angle, the tiles are rotated. The moving distance increases as the distance from the center increases, but the rotation error can be eliminated. More specifically, as shown in (1) to (2) in FIG. 19, the tile to be rotated is expanded and rotated together with the surrounding tiles (see FIGS. 17A and 17B). Further, as shown in (3) of FIG. 19, a work memory for one document is secured in the RAM 8 separately from the original image. Then, as shown in (4) of FIG. 19, the rotated result is copied to the work memory of the RAM 8. At this time, the copy position is determined based on the distance from the center of rotation (upper left). That is, the copy position differs depending on the distance from the center of rotation (upper left). Thereby, the rotation error of each tile can be eliminated.
[0094]
In a format capable of acquiring data for each bit plane such as JPEG2000, there are few bit planes in which only the upper plane is extracted without using all bit planes for skew correction as in this embodiment. The skew correction angle may be detected using an image. As a result, it is possible to further reduce the skew correction angle detection time and reduce the memory used in the RAM 8.
[0095]
Further, a correction mode designation dialog X as shown in FIG. 20 may be displayed on the display device 15 to allow the user to designate speed priority or image quality priority (mode selection means). In this case, when the speed priority mode is designated, the resolution may be lowered, or a specific color space or a part of bit planes may be used (mode switching means). Further, when the image quality priority mode is designated, the resolution may be increased or all color spaces or all bit planes may be displayed (mode switching means).
[0096]
Here, when the skew correction for correcting the inclination of the image is executed based on the compressed image data, the skew correction process for the expanded image is executed for each tile. Thus, since it is sufficient to develop the image in the memory (RAM 8) in units of tiles, it is possible to avoid a memory shortage even in an apparatus or system with a small amount of mounted memory, and it is not subject to various restrictions.
[0097]
In the present embodiment, the skew correction angle is detected based on the timing mark, but the present invention is not limited to this. For example, as shown in FIG. 21, when the original image I is displayed on the display device 15, the grid line G is also displayed. As a result, when the original image I has an inclination (skew), the grid lines G and the ruled lines of the original image I are displayed in a shifted manner, so that the user can visually recognize the inclination (skew) of the original image I. Can be confirmed. Then, when the user operates the rotation button B (right rotation button B1, left rotation button B2), the original image I is actually rotated, and the actually rotated angle can be detected as the skew correction angle. . As a result, the user can visually check the skew correction angle, and the skew correction angle is detected by a simple method. When the skew correction angle is detected in this way, the processing of steps S4 to S7 described above is executed on condition that the user operates the skew correction button A, thereby correcting the skew of the entire image according to the skew correction angle. Executed.
[0098]
next, Second embodiment Will be described. The same parts as those described in the first embodiment are denoted by the same reference numerals, and description thereof is also omitted.
[0099]
FIG. 22 is a flowchart showing the flow of image correction processing executed by the CPU 6 in accordance with the image processing program. Here, as the image correction process, a display condition correction process for correcting the image display conditions (color correction, γ correction, edge enhancement, etc.) will be mainly described.
[0100]
In the image correction processing of the present embodiment, as shown in FIG. 22, first, after a compression code related to a low resolution image is decoded to obtain a low resolution image (step S11), this low resolution image is displayed on the display device. Is displayed on the display device 15 (step S12). Note that the resolution in this case matches the resolution of the display device 15.
[0101]
Subsequently, correction parameters (color correction, γ correction, edge enhancement, etc.) for the image are determined (steps S13 to S15). Here, the reception of the correction parameter from the user (step S13) and the display of the correction result image on the display device 15 for confirmation (step S14) until the correction parameter is determined (Y in step S15). repeat.
[0102]
Therefore, in steps S11 to S15, the function of the display condition acquisition unit is executed.
[0103]
When the correction parameters (color correction, γ correction, edge enhancement, etc.) are determined (Y in step S15), tiles (encoded data of each tile shown in FIG. 9) are sequentially extracted and expanded in the RAM 8, and the correction parameters are used. After executing the correction process, the compression process is performed to write back the code string data shown in FIG. 9, thereby correcting the display conditions of the entire image (steps S16 to S19: display condition changing means).
[0104]
Here, when the display condition correction for correcting the display condition of the image is executed based on the compressed image data, the display condition correction process for the expanded image is executed for each tile. Thus, since it is sufficient to develop the image in the memory (RAM 8) in units of tiles, it is possible to avoid a memory shortage even in an apparatus or system with a small amount of mounted memory, and it is not subject to various restrictions.
[0105]
In each embodiment, a still image has been described, but the present invention can also be applied to a moving image. The “Motion JPEG2000 algorithm” is a still image as described in each embodiment, that is, a moving image obtained by extending the method for a single frame to a plurality of frames. That is, “Motion JPEG2000” is a moving image by continuously displaying a JPEG2000 image of one frame at a predetermined frame rate (the number of frames reproduced per unit time) as shown in FIG. . Therefore, correction conditions (skew correction angles and correction parameters) are obtained for such moving images based on the first frame, and the correction conditions (skew correction angles and correction parameters) are applied to the other frames. Just do it. This is because the same correction is considered necessary for all frames. As a result, it is possible to shorten the detection time of the correction condition (skew correction angle and correction parameter).
[0106]
【The invention's effect】
According to the image processing apparatus of the first aspect, when the skew correction for correcting the inclination of the image is executed based on the compressed image data, the skew correction process for the expanded image is executed in units of tiles. As a result, since it is sufficient to develop an image in memory in units of tiles, a memory shortage can be avoided even in a device or system with a small amount of mounted memory, and various restrictions are not imposed.
[0107]
Claim 2 According to the invention described above, in the image processing apparatus according to claim 1, each tile is sequentially rotated in accordance with a skew correction angle detected based on an image obtained by expanding one tile, thereby quickly and reliably providing an image. Can be corrected.
[0108]
Claim 3 According to the described invention, Claim 2 In the described image processing apparatus, by detecting the skew correction angle based on the low-resolution image, it is possible to reduce the skew correction angle detection time and the memory used.
[0109]
Claim 4 According to the described invention, Claim 2 or 3 In the described image processing apparatus, the skew correction angle detection time can be reduced and the memory used can be reduced by detecting the skew correction angle using any one of the color spaces of the image. .
[0110]
Claim 5 According to the described invention, Claim 2 or 3 In the described image processing apparatus, the skew correction angle detection time can be shortened and the memory used can be reduced by detecting the skew correction angle using only the upper bit plane.
[0111]
Claim 6 According to the described invention, Claims 3 to 5 When the speed priority mode is designated by switching between the image quality priority mode and the speed priority mode according to the selection by the mode selection unit and executing the skew correction process, the resolution is reduced. Alternatively, a specific color space or a part of bit planes may be used. Further, when the image quality priority mode is designated, the resolution may be increased, or all color spaces or all bit planes may be displayed.
[0112]
Claim 7 According to the described invention, Claims 2 to 6 In the image processing apparatus according to any one of the above, the skew correction angle can be detected with high accuracy and at high speed by detecting the skew correction angle based on a timing mark preset in the image.
[0113]
Claim 8 According to the described invention, Claim 7 In the described image processing apparatus, by expanding only tiles that are highly likely to contain timing marks, it is possible to increase the processing speed and save memory.
[0114]
Claim 9 According to the described invention, Claim 7 In the described image processing apparatus, when a ROI (Region Of Interest) area is specified, only tiles that are highly likely to contain timing marks in the ROI area are expanded to further process. Speeding up and memory saving can be achieved.
[0115]
Claim 10 According to the described invention, Claims 2 to 6 In the image processing apparatus according to any one of the above, the user visually confirms the skew correction angle by displaying a grid and detecting the skew correction angle based on an image rotated according to the grid. In other words, the skew correction angle can be detected by a simple method.
[0116]
Claim 11 According to the described invention, Claim 2 In the described image processing apparatus, pixels can be complemented by a simple method with respect to the invalid area generated by rotation in each tile by complementing the invalid area generated by rotation in each tile. .
[0117]
Claim 12 According to the described invention, Claim 11 In the described image processing apparatus, by performing low-pass filter processing on a region where white pixels are completed, a region where white pixels are complemented can be made inconspicuous.
[0118]
Claim 13 According to the described invention, Claim 2 In the described image processing apparatus, a tile group is formed by a desired tile and three neighboring tiles adjacent at one corner of the tile, and the tile group is rotated by a skew correction angle from one corner of the tile group as a starting point. Thus, it is possible to supplement the invalid area caused by the rotation.
[0119]
Claim 14 According to the described invention, Claim 2 In the described image processing apparatus, a tile group is formed by a desired tile and four peripheral tiles adjacent to the right, left, top, and bottom of the tile, and the tile group is skew-corrected starting from the center of the desired tile. By rotating only the corner, the invalid area caused by the rotation can be complemented.
[0120]
Claim 15 According to the described invention, Claim 2 In the described image processing apparatus, the rotation error can be eliminated by setting the starting point of the rotation according to the skew correction angle for all the tiles at one place in the image.
[0121]
Claim 16 According to the described invention, Claims 1 to 15 In the image processing apparatus according to any one of the above, the moving image that continuously displays images at a predetermined frame rate is based on the image expanded in units of tiles in the first frame. Skew By obtaining the correction condition for the correction process and applying the correction condition to other frames, the detection time of the correction condition can be shortened.
[0122]
Claim 17 According to the program of the invention described above, when the skew correction for correcting the inclination of the image is executed based on the compressed image data, the skew correction process for the expanded image is executed for each tile, Therefore, even if the device or system has a small amount of mounted memory, it is possible to avoid a memory shortage and avoid various restrictions.
[0123]
Claim 18 According to the storage medium of the described invention, Claim 17 By storing the program described above, by causing the computer to read the program stored in this storage medium, Claim 17 Effects similar to those of the described invention can be obtained.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of a system that implements a hierarchical encoding algorithm that is the basis of a JPEG2000 system that is a premise of an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing a divided rectangular area of each component of the original image.
FIG. 3 is an explanatory diagram showing subbands at each decomposition level when the number of decomposition levels is 3. FIG.
FIG. 4 is an explanatory diagram showing a precinct.
FIG. 5 is an explanatory diagram showing an example of a procedure for ranking bit planes;
FIG. 6 is an explanatory diagram showing a schematic configuration of one frame of code string data.
FIG. 7 is a system configuration diagram showing a system including the image processing apparatus according to the first embodiment of the present invention.
FIG. 8 is an explanatory diagram illustrating an example of a divided image divided two-dimensionally.
FIG. 9 is an explanatory diagram showing a compression code generated according to the “JPEG2000 algorithm” based on the divided image.
FIG. 10 is an explanatory diagram showing a resolution model of a compression code generated according to the “JPEG2000 algorithm”.
FIG. 11 is a block diagram schematically illustrating a hardware configuration of the image processing apparatus.
FIG. 12 is a flowchart showing a flow of image correction processing.
FIG. 13 is an explanatory diagram showing timing marks.
FIG. 14 is a flowchart showing a flow of timing mark detection processing;
FIG. 15 is an explanatory diagram showing a search range of timing marks.
FIGS. 16A and 16B are diagrams illustrating a state where the tile is rotated starting from the upper left corner, and FIG. 16B is an explanatory diagram illustrating a state where the tile is rotated starting from the center of the tile.
FIG. 17A shows a case where a tile group is formed by a desired tile and three neighboring tiles adjacent to one corner of the tile, and the tile group is rotated with one corner of the tile group as a starting point. (B) shows a case where a tile group is formed by a desired tile and four peripheral tiles adjacent to the right, left, top, and bottom of the tile, and the tile group is rotated from the center of the desired tile. It is explanatory drawing shown.
FIG. 18 is an explanatory diagram showing a state in which the tile is rotated from one place of the image as a starting point.
FIG. 19 is a detailed explanatory diagram thereof.
FIG. 20 is a front view showing a correction mode designation dialog displayed on the display device.
FIG. 21 is a front view showing grid lines displayed together with the original image on the display device.
FIG. 22 Second embodiment It is a flowchart which shows the flow of this image correction process.
FIG. 23 is an explanatory diagram showing the concept of Motion JPEG2000.
[Explanation of symbols]
1 Image processing device
10,11 Storage media

Claims (18)

Means for generating compressed code data by compressing the pixel values for each tile obtained by dividing the image into a plurality of steps, such as discrete wavelet transform, quantization, and encoding; and decompressing the compressed code data by reversing the above procedure In an image processing apparatus having means for generating an image,
A process of decompressing the compression code data, performing skew correction processing for correcting the inclination of the image on the expanded image, and executing a series of processing for re-compressing the image subjected to the skew correction processing in tile units. A featured image processing apparatus. The skew correction process includes
A skew correction angle detecting means for detecting a skew correction angle based on an image obtained by extending one tile;
Image rotating means for sequentially rotating each tile according to the skew correction angle detected by the skew correction angle detecting means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is executed by: The image processing apparatus according to claim 2, wherein the skew correction angle detection unit detects a skew correction angle based on a low-resolution image. The image processing apparatus according to claim 2, wherein the skew correction angle detection unit detects the skew correction angle using any one of color spaces of the image. 4. The image processing apparatus according to claim 2, wherein the skew correction angle detection unit detects the skew correction angle using only the upper bit plane. Mode selection means for enabling selection of an image quality priority mode that prioritizes image quality and a speed priority mode that prioritizes processing speed for the skew correction processing;
About the skew correction processing, mode switching means for switching between the image quality priority mode and the speed priority mode according to the selection by the mode selection means,
The image processing apparatus according to claim 3, further comprising: The image processing apparatus according to claim 2, wherein the skew correction angle detection unit detects a skew correction angle based on a timing mark preset in the image. The image processing apparatus according to claim 7, wherein only a tile that is highly likely to include a timing mark is expanded. 8. The image processing apparatus according to claim 7, wherein when a ROI (Region Of Interest) area is designated, only a tile that is likely to contain a timing mark in the ROI area is expanded. . 7. The image processing according to claim 2, wherein the skew correction angle detection unit displays a grid and detects a skew correction angle based on an image rotated in accordance with the grid. apparatus. The image processing apparatus according to claim 2, wherein the image rotation unit complements white pixels for an invalid area generated by rotation in each tile. The image processing apparatus according to claim 11, wherein a low-pass filter process is performed on a region supplemented with white pixels. The image rotation means forms a tile group by a desired tile and three peripheral tiles adjacent at one corner of the tile, and rotates the tile group by a skew correction angle starting from one corner of the tile group. The image processing apparatus according to claim 2. The image rotation means forms a tile group by a desired tile and four peripheral tiles adjacent to the right, left, top, and bottom of the tile, and the skew correction angle of the tile group starts from the center of the desired tile. The image processing apparatus according to claim 2, wherein the image processing apparatus is rotated only by the amount of rotation. The image processing apparatus according to claim 2, wherein the image rotation unit sets a rotation start point corresponding to a skew correction angle for all tiles at one position of the image. For a moving image that displays images continuously at a predetermined frame rate, a correction condition related to skew correction processing is obtained based on an image expanded in tile units in the first frame, and the correction condition is set for other frames. The image processing apparatus according to claim 1, wherein the image processing apparatus is applied. A function for generating compressed code data by compressing the pixel values for each tile obtained by dividing the image into a plurality of steps by discrete wavelet transform, quantization and coding, and decompressing the compressed code data by the reverse procedure of the above procedure A program for causing a computer to execute a function of generating an image, wherein the computer
A process of decompressing the compression code data, performing skew correction processing for correcting the inclination of the image on the expanded image, and executing a series of processing for re-compressing the image subjected to the skew correction processing in tile units. A featured program. A computer-readable storage medium storing the program according to claim 17.

Images