JP4569697B2 - Image processing apparatus, image display apparatus, and image processing method - Google Patents

Description

  The present invention relates to an image processing device, an image display device, and an image processing method, and particularly to an image processing device, an image display device, and an image processing method to which an image having a high spatial frequency is input.

  In a projection display device such as a rear projection television, image distortion (for example, keystone distortion) due to the positional relationship between the light source to be projected and the screen, or image distortion due to aberrations of the optical system occurs. In order to correct this distortion, a method of projecting an image that has been subjected to conversion of distortion and reverse characteristics is known.

  As one of the methods, there is a method of generating an image with reverse characteristics by image signal processing, and a liquid crystal that corrects keystone distortion by changing the number of pixels in a scanning line in an input image in units of a predetermined number of scanning lines. A projector device has been proposed (see, for example, Patent Document 1).

JP-A-8-102900 (paragraphs 0021, 0027, FIGS. 2 and 3)

  However, for example, when enlargement / reduction is performed by changing the number of pixels as described above, if a high-frequency component is included in the input image, aliasing distortion caused by resampling the original image at the converted pixel position. Therefore, there is a problem that moire occurs in the output image and the image quality deteriorates.

  In general, a method of preventing moire by smoothing an image signal without considering the distribution of frequency components included in the image data may be employed. In this case, there is no relation to moire. There is a problem in that even fine patterns are blurred and image quality is deteriorated.

  The present invention has been made to solve the above-described problems. For example, even if there is a portion containing a high-frequency component such as a checkered pattern, moiré can be generated without degrading the image quality. An object is to obtain an image processing apparatus, an image display apparatus, and an image processing method capable of providing a reduced image.

An image processing apparatus according to the present invention includes a high-frequency component detection unit that performs a convolution operation on a pixel or a pixel group of image data to detect a high-frequency component of the image data and outputs a detection result, and smoothing that smoothes the image data image processing, comprising: a smoothing processing section for outputting image data, the image data and the smoothed image data image data processing unit and outputs the combined synthetic synthetic data combination ratio determined based on the detection result In the apparatus, the smoothing processing unit performs smoothed image data from the data of each pixel included in the smoothing target area including the target pixel and pixels in the vicinity of the target pixel for each target pixel in the screen area of the image data. The number of pixels constituting the smoothing target area is smaller than the number of pixels in the pixel group that is the calculation target of the convolution calculation. .
An image processing apparatus according to the present invention includes a high-frequency component detection unit that performs a convolution operation on a pixel or a group of pixels of image data, detects a high-frequency component of the image data, and outputs a detection result; and smoothes the image data A smoothing processing unit for outputting the smoothed image data, and an image data processing unit for combining the image data and the smoothed image data at a composition ratio determined based on the detection result and outputting the synthesized data. In the image processing apparatus, the high-frequency component detection unit includes a convolution pattern generation unit that generates convolution pattern data for performing a convolution operation, and image data and the convolution pattern data for each region of interest including pixels or pixel groups of image data. The convolution operation unit that outputs the result of the convolution operation and the absolute value of the convolution operation result is output as the high-frequency component detection result A high-frequency component amount calculation unit, the convolution operation is a pixel group consisting of a plurality of pixels including at least a region of interest, and the convolution pattern data is for each pixel in the pixel group to be calculated, It consists of the coefficient which replaced positive and negative for every continuous pixel .

  An image display apparatus according to the present invention includes the above-described image processing apparatus and an image display unit that displays an image based on image data output from the image processing apparatus.

The image processing method according to the present invention includes a high frequency component detection step of performing a convolution operation on a pixel or a pixel group of image data to detect a high frequency component of the pixel data and outputting a detection result, and smoothing the image data and smoothing processing step of outputting the smoothed image data, the image data and the image data processing step and the including images and outputs the synthesized synthesized synthetic data combination ratio determined based on the smoothed image data and a detection result In the processing method, the smoothing processing step includes, for each target pixel in the screen area of the image data, a smoothed image from the data of each pixel included in the smoothing target region including the target pixel and pixels in the vicinity of the target pixel. The number of pixels that generate data and configure the smoothing target region is smaller than the number of pixels in the pixel group that is the calculation target of the convolution calculation. It is intended.
The image processing method according to the present invention includes a high frequency component detection step of performing a convolution operation on a pixel or a pixel group of image data to detect a high frequency component of the image data and outputting a detection result, and smoothing the image data Image processing comprising: a smoothing processing step for outputting smoothed image data; and an image data processing step for combining image data and smoothed image data at a combining ratio determined based on the detection result and outputting the combined data. In the method, the high-frequency component detection step includes a convolution pattern generation step for generating convolution pattern data for performing a convolution operation, and convolution of image data and convolution pattern data for each region of interest consisting of pixels or pixel groups of image data. The convolution operation step that outputs the operation result and the absolute value of the convolution operation result are increased. A high-frequency component amount calculation step to output as a wave component detection result, and the convolution calculation is performed on a pixel group including a plurality of pixels including at least a region of interest, and the convolution pattern data is included in the pixel group to be calculated. Each pixel is composed of a coefficient in which positive and negative are switched for each successive pixel.

  According to the present invention, the high frequency component of the image data is detected, and the composition ratio of the smoothed image data is increased for pixels or pixel groups having a high high frequency component. Even if there is a portion containing a high-frequency component such as a checkered pattern, an image with reduced moiré can be provided without degrading the image quality.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating the configuration of an image processing apparatus and an image processing method according to a first embodiment of the present invention, and is used as a part of an image display apparatus typified by a projection display apparatus such as a rear projection television. Can do.

  The image processing apparatus according to the first embodiment includes an input terminal 1, a high frequency component processing unit 2, and a partial enlargement / reduction processing unit 3. The input image data Da is input from the input terminal 1 to the high frequency component processing unit 2. The high frequency component processing unit 2 outputs high frequency component smoothed image data Db obtained by smoothing a portion including the high frequency component of the input image data Da by a method described later. The partial enlargement / reduction processing unit 3 outputs partial enlarged / reduced image data Dc obtained by enlarging or reducing the smoothed image data Db for each region. Here, an example is shown in which the high-frequency component processing unit 2 and the partial enlargement / reduction processing unit 3 are provided in the same image processing apparatus, but this is not particularly limited, and a plurality of image processing apparatuses are provided, The processing units 2 and 3 may be provided in separate apparatuses.

The internal configuration of the high frequency component processing unit 2 will be described in detail. FIG. 2 is a diagram showing an internal configuration of the high-frequency component processing unit 2. The high frequency component processing unit 2 includes a smoothing processing unit 11, a high frequency component detection unit 12, and an image data processing unit 13. The smoothing processing unit 11 outputs smoothed image data Dd obtained by smoothing the input image data Da. The high frequency component detector 12 detects a high frequency component included in each pixel of the input image data Da, and outputs a high frequency component detection result Sb for each pixel. The image data processing unit 13 outputs high-frequency component smoothed image data Db as synthesized data obtained by synthesizing the input image data Da and the smoothed image data Dd based on the high-frequency component detection result Sb.
Hereinafter, the smoothing processing unit 11, the high frequency component detection unit 12, and the image data processing unit 13 will be described.

  First, the smoothing processing unit 11 will be described. In the first embodiment, the smoothing processing unit 11 includes an average value filter that outputs an average value of pixel values of a pixel of interest (hereinafter referred to as a pixel of interest) and pixels existing in the vicinity thereof. That is, the smoothed image data Dd is an image in which an average value filter is applied to the input image data Da.

  Next, the high frequency component detection unit 12 will be described. FIG. 3 is a diagram illustrating an internal configuration of the high-frequency component detection unit 12. The high frequency component detection unit 12 includes a convolution pattern generation unit 21, a convolution operation unit 22, and a high frequency component amount calculation unit 23. The convolution pattern generation unit 21 outputs convolution pattern data Ta. The convolution operation unit 22 uses the convolution pattern data Ta output from the convolution pattern generation unit 21 and, for the entire screen area of the input image data Da, the target pixel and the surrounding of the target pixel for each target pixel that is a part of the screen area. A convolution calculation result De obtained by performing a convolution operation on the pixel value of the pixel is output. The high-frequency component amount calculation unit 23 calculates a high-frequency component included in the input image data Da based on the convolution calculation result De, and calculates a high-frequency component detection result Sb that is a ratio of the high-frequency component for each pixel of interest in the entire screen area. Output.

  Next, the image data processing unit 13 will be described. FIG. 4 is a diagram showing an internal configuration of the image data processing unit 13. The image data processing unit 13 includes a composition ratio generation unit 31 and a weighted addition unit 32. The composition ratio generation unit 31 outputs the composition ratio Sc based on the high frequency component detection result Sb. The composition ratio Sc is a numerical value having a value of 0 or more. The weighted addition unit 32 performs weighted addition of the input image data Da and the smoothed image data Dd based on the synthesis ratio Sc, and outputs the result as high frequency component smoothed image data Db.

Assuming that the maximum value that the composition ratio Sc can take is Scmax, the relationship among the input image data Da, the smoothed image data Dd, and the high-frequency component smoothed image data Db is expressed by the following mathematical formula.

  Here, since the smoothed image data Dd is an image in which an average value filter is applied to the input image data Da, it loses detailed information and becomes a blurred image as compared with the input image data Da. The image does not contain a high-frequency component that causes moire. That is, in the composition ratio generation unit 31, when the attention area (the attention pixel in the present embodiment) that is a part of the screen area of the input image data Da contains a lot of high-frequency components that cause moire, the attention area ( When the value of the composition ratio Sc in the target pixel) is increased and a high-frequency component that causes moire is not included so much, the value of the composition ratio Sc in the target region (target pixel) is decreased, thereby reducing the image. It is possible to obtain high-frequency component smoothed image data Db that does not include high-frequency components that cause moire while leaving detailed information.

  In other words, the composition ratio Sc is a coefficient for controlling the pass frequency band of the high frequency component processing unit 2 with respect to the input image data Da. In other words, the pass frequency band decreases as the value of the combination ratio Sc approaches Scmax, and conversely, the pass frequency band increases as the value of the combination ratio Sc approaches zero. In the high-frequency component processing unit 2 according to Embodiment 1 of the present invention, the value of the combination ratio Sc approaches Scmax as the number of high-frequency components that cause moire is increased for each region of interest (target pixel) in the input image data Da. Therefore, the pass frequency band of the high frequency component processing unit 2 is locally controlled. As a result, it is possible to remove only high-frequency components that cause moire while leaving information on image details.

  Next, a method of calculating the high frequency component detection result Sb that quantitatively indicates the ratio of high frequency components included in each region of interest (target pixel) as described above will be described. By appropriately setting the convolution pattern data Ta in the high-frequency component detection unit 12, the high-frequency component detection result Sb quantitatively indicates how much high-frequency components that cause moire are included in the input image data Da. The value to represent. Then, the composition ratio generation unit 31 obtains the composition ratio Sc based on the high-frequency component detection result Sb, so that the more the high-frequency components that cause moire are included, the closer the value of the composition ratio Sc is to Scmax and vice versa. When the high-frequency component causing the above is not included so much, the value of the composition ratio Sc can be made close to zero.

A case where a portion including a checkered pattern is detected among images including high-frequency components that cause moire will be described as an example. Note that the checkered pattern represents a state in which pixels with different gradations are arranged periodically on the top, bottom, left, and right, and expresses the difference in gradation on a pixel-by-pixel basis. It is visually recognized in such a state.

  FIG. 5 is a diagram showing the convolution pattern data Ta generated by the convolution pattern generation unit 21 when a portion including a checkerboard pattern is detected. In the first embodiment, since one pixel in the image data Da is one area, the portion represented by a square in FIG. 5 corresponds to each pixel in the input image data Da, and pixels indicated by halftone dots are Become a pixel of interest. The number written in the square is a coefficient to be multiplied to each pixel value in the convolution operation.

  That is, the pixel value of the target pixel is V (x), the pixel value of the pixel adjacent to the left of the target pixel is V (x−1), the pixel value of the pixel adjacent to the right of the target pixel is V (x + 1), When the pixel value further to the right of the pixel is V (x + 2), the convolution calculation result De is expressed by the following formula using four multiplications.

  In other words, the convolution calculation targets at least a target pixel and a pixel group including pixels in the vicinity of the target pixel, and the convolution pattern data Ta is correct for each successive pixel for each pixel in the target pixel group. And negative (inverted addition and subtraction) coefficients. In addition, since the coefficient to be multiplied to each pixel value in the convolution operation is either −1 or 1, the multiplication part of the calculation shown in Equation 2 can be expressed by addition or subtraction. Equivalent to mathematical formula.

  While Equation 2 requires multiplication, Equation 3 can be realized only by addition and subtraction, which is advantageous in terms of hardware. In other words, by setting the convolution pattern data to be a coefficient consisting of 1 or -1, it is possible to suppress the mounting scale at the time of hardware implementation.

  FIG. 6 is a diagram showing the relationship between the input image data Da and the convolution calculation result De. FIG. 6A shows a certain input image data Da, and FIG. 6B shows a convolution operation using the input image data Da shown in FIG. 6A and the convolution pattern data Ta shown in FIG. The resulting convolution operation result De is shown. The input image is a monochrome image. In FIG. 6A, a portion represented by a square corresponds to each pixel, and a number written in the square corresponds to a pixel value of each pixel. In FIG. 6B, a portion represented by a square corresponds to each pixel, and a number written in the square represents a value obtained as a result of the convolution operation. Note that when performing a convolution operation paying attention to pixels in the vicinity of the boundary of image data, a part of the coefficient given by the convolution pattern data Ta does not have a pixel that provides a pixel value to be multiplied with the coefficient. In such a case, 0 was assumed as the pixel value.

  In FIG. 6A, the left side of the image has a so-called checkered pattern in which pixel values of pixels adjacent vertically and horizontally change periodically, and has a high-frequency component that causes moiré. Hereinafter, this area is referred to as a checkered pattern area. On the other hand, on the right side of the image, the pixel value of the adjacent pixel does not change and does not include a high frequency component that causes moiré. Hereinafter, this area is referred to as a solid area. As is apparent from the convolution operation De shown in FIG. 6B, the absolute value of the convolution operation result De is a large value in the checkered region, a small value in the solid region, and the checkered region and An intermediate value is taken near the boundary of the solid region.

  Therefore, if the high-frequency component amount calculation unit 23 outputs the absolute value of the convolution calculation result De as the high-frequency component detection result Sb, the value of the high-frequency component detection result Sb is large for each pixel of interest in the checkered pattern area. The value is small for each pixel of interest in the solid area, and is an intermediate value for each pixel of interest near the boundary between the checkered pattern area and the solid area.

  In other words, by using the convolution operation, it is possible to quantitatively grasp whether a specific pattern (or convolution pattern data Ta) is included in the input image data Da. In addition, as described above, by making the convolution pattern data Ta into periodic data, it is quantitatively determined how much high-frequency components (a checkered pattern in the above description) are included in the input image Da. It becomes possible to grasp.

  That is, the value of the high-frequency component detection result Sb changes continuously according to how much each pixel of interest in the input image data Da contains a checkered pattern (high-frequency component), and an area composed of the checkered pattern If it becomes larger than a certain level, the value becomes larger. Further, the value of the high frequency component detection result Sb is clearly a numerical value of 0 or more. That is, the high frequency component detection result Sb is a value of 0 or more that quantitatively represents how much of the high frequency component in question is included in the input image data Da, and the larger the value, the higher the input image data Da. This indicates that each target pixel (region) in the center contains a lot of problematic high-frequency components.

  If the value of the composition ratio Sc is increased in the weighted addition according to Equation 1, the high-frequency component included in the input image data Da is removed, so that it becomes possible to remove the checkered pattern in question. However, if a high frequency component is removed more than necessary, a fine pattern (hereinafter referred to as texture) in the input image data Da is removed even if it is not desired to be removed.

  Therefore, in order to remove the checkered pattern including the high-frequency component that causes moire while leaving the texture portion, the larger the value of the high-frequency component detection result Sb, the larger the value of the combination ratio Sc. It was determined as follows. The calculation method is shown below.

  1) When the value of the high frequency component detection result Sb is less than the threshold value Tb (Sb <Tb), the synthesis ratio Sc is set to 0. Thereby, it is possible to suppress the influence of high frequency component smoothing on a texture portion in which the value of the high frequency component detection result Sb outputs a value less than the threshold value Tb. That is, by appropriately setting the threshold value Tb, it is possible to perform processing such as removing the checkered pattern while leaving the texture.

  2) On the other hand, when the target pixel in the screen area of the input image data Da contains a lot of high frequency components such as a checkered pattern, the value of the high frequency component detection result Sb becomes a large value. Therefore, when the threshold value Tmax is set and the value of the high-frequency component detection result Sb is larger than the threshold value Tmax (Sb> Tmax), the target pixel contains a lot of high-frequency components such as a checkered pattern that causes moiré. It is possible to effectively remove the checkerboard pattern by determining and setting the value of the composition ratio Sc to the maximum value Scmax.

  3) When the value of the high-frequency component detection result Sb is greater than or equal to Tb and less than or equal to Tmax (Tb ≦ Sb ≦ Tmax), it is difficult to determine whether the pixel of interest includes a high-frequency component such as a checkered pattern or a texture It is thought that. For example, the vicinity of the boundary between the checkered pattern and the texture (hereinafter referred to as the boundary portion). If a binary process of whether or not smoothing is performed at such a location (or if only Sc or 0 is allowed as the Sc value), discontinuity occurs in the high-frequency component smoothed image data Db. That is, when Sc = 0 is selected, the input image data Da is output as it is, but when Sc = Scmax is selected, the smoothed image data Dd is output. Since the input image data Da includes a high-frequency component, the input image data Da is recognized as a clear image by human eyes, but the smoothed image data Dd does not include a high-frequency component, and thus is recognized as a blurred image by human eyes. . If only 0 or Scmax is allowed as the value of Sc, a clear image and a blurred image appear discontinuously in the vicinity of the boundary portion, which is unnatural to human eyes. In addition, when such an unnatural part (hereinafter, false contour) appears in a moving image, it is reflected in the human eye, and the unnaturalness increases.

  Therefore, when the value of the high frequency component detection result Sb is greater than or equal to Tb and less than or equal to Tmax, the value of the synthesis ratio Sc is gradually increased from 0 to Scmax as the high frequency component detection result Sb increases. Therefore, the high-frequency component smoothed image data Db becomes an intermediate image between the input image data Da and the smoothed image data Dd at the boundary portion, so that the boundary portion can be blurred. In other words, the change at the boundary can be smoothed. Since the human eye is insensitive to gentle changes, it does not see a false contour that occurs when only Sc and 0 are allowed as Sc values. That is, when the value of the high-frequency component detection result Sb is Tb or more and Tmax or less, the false contour can be prevented from occurring by monotonically increasing the composite ratio Sc with respect to the increase in the high-frequency component detection result Sb.

  When the value of the slope Ka in Equation 4 is the value shown in Equation 5 below, Sc = Smax when the value of the high-frequency component detection result Sb is equal to or greater than the threshold value Tmax, so that Sb exceeds 0 to Tmax. The value of Sc when it changes changes continuously.

  Therefore, the composition ratio Sc changes smoothly with respect to the change in the high-frequency component detection result Sb including the boundary portion, and the checkered pattern is removed while suppressing the influence of the high-frequency component smoothing on the texture portion and leaving the texture. Such processing is possible.

  FIG. 7 is a diagram showing the relationship between the high-frequency component detection result Sb and the composition ratio Sc when the composition ratio Sc is calculated based on the above equations 4 and 5. In FIG. 7, the horizontal axis (high-frequency component detection result Sb) is an amount representing how much of the high-frequency component in question is included in the attention area (pixel) of the input image data Da, and the higher the high-frequency component in question is, The value increases. On the other hand, when many components such as texture are desired to remain, the value of the high frequency component detection result Sb is considered to have a small value. As is clear from FIG. 7, the value of the composition ratio Sc becomes closer to Scmax as the input image data Da contains more high-frequency components that cause moire, and conversely, the high-frequency components that cause moire are less. If it is not included, it is close to 0. Therefore, the weighted addition unit 32 can obtain high-frequency component smoothed image data Db that does not include high-frequency components that cause a moire that causes a problem while leaving detailed information such as texture.

  Note that when the value of the high frequency component detection result Sb is equal to or greater than Tb and equal to or less than Tmax, the formula for monotonically increasing the composite ratio Sc with respect to the increase in the high frequency component detection result Sb is not limited to Equation 4.

  In the first embodiment, the high-frequency component detection result Sb is calculated for each pixel. However, the present invention is not limited to this, and an area composed of a plurality of pixels that are part of the screen area of the image data Da. May be set as the attention area. In this case, the pixel group to be subjected to the convolution calculation may be a pixel group including at least a plurality of pixels including the attention area, and only the pixel group in the attention area may be the calculation target.

  As described above, the high-frequency component smoothed data Db output from the high-frequency component processing unit 2 does not include high-frequency components that cause moire while leaving detailed information such as texture. Since the partial enlargement / reduction processing unit 3 outputs the partial enlarged / reduced image data Dc obtained by enlarging or reducing the smoothed image data Db that does not include the high-frequency component causing the moire for each region, the image according to the first embodiment is output. The processing apparatus can generate a good image with reduced moire.

  FIG. 8 is a diagram showing a configuration of an image display device typified by a projection display device incorporating the image processing device according to the first embodiment of the present invention. As shown in FIG. 8, in the image display device, light emitted from a light source 81 is incident on a modulator 83 via various optical elements 82, and light modulated by the modulator 83 in accordance with a control signal is projected lens. The light is projected onto the screen 85 via an optical system 84 or the like. The image processing apparatus described above is arranged as a control unit, and modulation (control of output pixels) is performed by the modulator 83 in accordance with a control signal generated based on the composite data output from the image processing apparatus. It has been made.

  As described above, the image display device includes the image processing device described above, the light source 81, the projection unit including the modulator 82, the projection lens 83 used for projecting the image of the modulator 82 onto the screen 85, and the like. An optical system, a screen 85, and the like are included. Note that the projection display device is not limited to that shown in FIG. 8, and the image processing device described above may be applied to other projection display devices.

  As described above, the partial enlargement / reduction processing unit 3 is included in the image processing apparatus, and the partial enlargement / reduction processing unit 3 enlarges or reduces the input image for each region. Here, it is desirable that the enlargement or reduction of each region is performed with characteristics opposite to the distortion caused by the optical system constituting the image display device, for example. By doing so, for example, it is possible to correct image distortion generated in the optical system constituting the image display device, and it is possible to display an image without distortion on the screen 85. Note that enlargement or reduction for each region is realized by re-sampling the input image at a sampling period different from that of the input image. For example, in an area where the image is to be partially enlarged, the input image may be resampled at a sampling period shorter than the sampling period (or pixel interval) of the input image. In an area where the image is to be partially reduced, the input image is input. Re-sampling may be performed at a sampling period longer than the image sampling period (or pixel interval).

  When the partial enlargement / reduction processing unit 3 resamples an image, if the input image to the partial enlargement / reduction processing unit 3 includes a periodic pattern with a period different from the re-sampling period described above, In some cases, interference occurs with the sampling period and moire occurs. Further, it is known that moire is likely to occur between different high-frequency components.

  In the image processing apparatus according to this embodiment, the high frequency component detection unit 12 in the image processing apparatus detects the degree to which a high frequency component causing moire is included as the high frequency component detection result Sb, and the image data processing unit 13 detects the high frequency component. Based on the detection result Sb, smoothed image data Db that does not include a high-frequency component that causes moire is generated. In the partial enlargement / reduction processing unit 3, the smoothed image data Db is partially converted with characteristics opposite to those of the optical system. Then, after enlarging or reducing the image, the image is projected onto the screen 85 via the image projection device, so that an image without moire can be projected onto the screen 85.

  In the description relating to the internal configuration of the image processing apparatus, the smoothing processing unit 11 is configured by an average value filter that outputs an average value of pixel values of a pixel of interest and pixels around it. The means is not limited to this, and any means that outputs an average value of pixel values in the vicinity of the target pixel may be used. In other words, any pixel may be output as long as the pixel values of a plurality of pixels in the vicinity of the target pixel are weighted and added.

  Here, the number of pixels used for the weighted addition is limited to the vicinity of the target pixel, and is preferably smaller than the number of pixels used for the convolution operation in the convolution operation unit 22.

  FIG. 9 is a diagram illustrating the configuration of two different average value filters. The smoothing coefficient A represents the case where the average value is calculated using two pixels near the target pixel, and the smoothing coefficient B is the case where the average value is calculated using four pixels near the target pixel. Represents. Subsequently, assuming that the convolution pattern data Ta requires pixel values for four pixels for the convolution calculation as shown in FIG. 5, the smoothing coefficient A is used for the calculation in the smoothing processing unit 11 and the smoothing. Differences in the case where the use coefficient B is used will be described with reference to FIGS.

  FIG. 10 shows two types of image data A and B. FIG. 11A shows a convolution calculation result De and smoothed image data Dd obtained when the image data A is input as the input image data Da. 11 (b) represents a convolution calculation result De and smoothed image data Dd obtained when image data B is input as input image data Da. The smoothed image data Dd obtained when the smoothing coefficient A is used is defined as the smoothed image data A, and the smoothed image data Dd obtained when the smoothing coefficient B is used as the smoothed image data B. As shown. Further, when performing the convolution operation or calculating the smoothed image data, if the pixel of interest is in the vicinity of the boundary of the image data, there is no pixel that provides a pixel value necessary for the operation for some pixel values. In such a case, 0 was assumed as the pixel value.

  In FIG. 10, image data A represents a checkered pattern, and image data B represents a single vertical line. Since the image data A causes moire, it must be removed. However, the vertical lines of the image data B have a high frequency but do not have periodicity (repetitiveness), so they do not cause moire and do not need to be removed.

  First, consider a case where image data A (checkered pattern) is input as input image data Da. The convolution calculation result De obtained at this time has a large absolute value as shown in FIG. Therefore, the value of the synthesis ratio Sc is also increased, and the high-frequency component smoothed image data Db is obtained by increasing the ratio of the smoothed image data Dd to the input image data Da and the smoothed image data Dd.

  Here, as shown in FIG. 11A, the smoothed image data A generated using the smoothing coefficient A and the smoothed image data B generated using the smoothing coefficient B are both image data. A (checkered pattern) is sufficiently smoothed. Since the same data can be obtained as the smoothed image data Dd using either of the smoothing coefficients A and B, both the smoothing coefficients A and B have the same effect in the sense of processing a checkered pattern.

  Next, consider a case where image data B (vertical line) is input as input image data Da. As shown in FIG. 11B, the convolution calculation result De has an absolute value with a certain size within the same range (4 pixels) as the convolution pattern data Ta including the position of the vertical line. . Therefore, the value of the synthesis ratio Sc is also a certain size within the four pixels including the vertical line, and the high-frequency component smoothed image data Db includes the smoothed image data Dd to some extent.

  Here, as shown in FIG. 11B, the width of the vertical line in the smoothed image data Dd is 2 pixels in the smoothed image data A, but 4 pixels in the smoothed image data B. Considering the high-frequency component smoothed image data Db obtained by weighted addition of the image data B and the smoothed image data Dd, when the smoothed image A is used as the smoothed image data Dd, the high-frequency component processed image data Db While the spread of the line is suppressed to 2 pixels, when the smoothed image B is used as the smoothed image data Dd, the width of the straight line is expanded to 4 pixels in the high-frequency component smoothed image data Db, and the vertical line is blurred. Give an impression.

  In other words, both the smoothing coefficients A and B have the same effect in terms of processing image data including periodic high-frequency components that cause moiré, such as a checkerboard pattern, but even if high-frequency components are included. The smoothing coefficient A is superior in that it does not cause the generation of moiré and maintains image detail information represented by vertical lines. This is not periodic and does not cause moiré, as in the case where the target pixel corresponds to a line portion when a character is displayed, but is regarded as a high frequency, and thus the value of the high frequency component detection result Sb becomes high. In this case, it is shown that blurring of characters can be reduced by using the smoothing coefficient A.

  That is, even when image data Da that may output a high-frequency detection component detection result Sb having a high value is input by the high-frequency component detection unit 12, the pixels used when calculating the smoothed image data Dd are smoothed. By limiting to the target pixel to be smoothed and the pixels adjacent to the target pixel to be smoothed, local information can be left in the smoothed image data Dd. Actually, the number of pixels used when calculating the smoothed image data Dd is smaller than the number of pixels used for the convolution operation, thereby limiting the pixels used for calculating the smoothed image data Dd to those near the target pixel. It becomes possible to do. Further, in the smoothing coefficient A, the weight of the target pixel and the adjacent pixel is calculated to be the same, but the weight of the target pixel is calculated to be larger than that of the adjacent pixel, that is, as the weight is closer to the target pixel. Even the effect of leaving local information can be obtained. By devising these smoothing coefficients, it is possible to generate high-frequency component smoothed data Db from which only high-frequency components that cause problems are removed while maintaining detailed image information. From the viewpoint of calculating the average value, the minimum number of pixels necessary for calculating the smoothed image data Dd is two.

  As described above, according to the image processing apparatus according to the first embodiment, the convolution calculation is performed for each attention area formed of a part of the screen area of the input image data Da, and the high-frequency component detection result Sb for each attention area. , The smoothing processing unit 11 that outputs the smoothed image data Dd obtained by smoothing the image data Da, and the image data Da for each region of interest based on the high-frequency component detection result Sb. An image data processing unit 13 that outputs high-frequency component processed image data Db obtained by synthesizing the smoothed image data Dd, and the image data processing unit 13 determines the smoothed image based on the value of the high-frequency component detection result Sb. Since the data composition ratio Sc is changed (the higher the high-frequency component detection result Sb, the higher the composition ratio Sc of the smoothed image data Dd). Even if there is a portion containing high frequency components such as a checkered pattern in the data Da, the display of the texture portion is not blurred and the image quality is not deteriorated. It is possible to display an image in which the occurrence of noise is reduced.

  In particular, since the attention area is composed of one pixel, the composition ratio of the image data Da and the smoothed image data Dd can be changed on a pixel-by-pixel basis, and the moire can be reduced more efficiently without degrading the image quality. An image with reduced generation can be displayed.

  The high-frequency component detection unit 12 outputs a convolution calculation result De of the image data Da and the convolution pattern data Ta for each region of interest and the convolution pattern generation unit 21 that generates the convolution pattern data Ta for performing the convolution operation. A convolution operation unit 22 and a high-frequency component amount calculation unit 23 that outputs the absolute value of the convolution operation result De as a high-frequency component detection result Sb for each region of interest. The convolution operation includes a plurality of pixels including at least the region of interest. And the convolution pattern data Ta is composed of coefficients obtained by switching positive and negative for each successive pixel with respect to each pixel in the pixel group to be calculated. A region including a periodic high-frequency component that causes moiré, such as a pattern, can be reliably detected.

  Furthermore, since the convolution pattern data Ta is composed of data consisting of “1” and “−1”, multiplication is not required and the circuit configuration can be simplified.

  For each target pixel in the screen area of the image data Da, the smoothing processing unit 11 performs smoothed image data Dd from the data of each pixel included in the smoothing target region configured by the target pixel and pixels near the target pixel. Since the number of pixels constituting the smoothing target area is smaller than the number of pixels in the pixel group to be subjected to the convolution operation, the smoothing target area is a pixel in the vicinity of the target pixel. Even if a high-frequency component having no periodicity that is not related to moire is detected, blurring of the image at that portion can be reduced.

  Since the smoothing target area is composed of the target pixel and the pixel adjacent to the target pixel, the smoothing target area is limited to the pixel adjacent to the target pixel, and there is no periodicity that does not cause moire. Even if a high-frequency component is detected, blurring of the image at that portion can be more effectively reduced.

  Further, according to the image processing apparatus according to the first embodiment, the high-frequency component processing unit 2 and the partial enlargement for enlarging or reducing at least part of the high-frequency component processing image data Db output from the high-frequency component processing unit 2. Therefore, for example, if the partial enlargement / reduction processing unit 3 has a function of converting the inverse characteristics of the keystone distortion, the keystone distortion can be corrected and the image data Da is included in the image data Da. Even if there is a part containing a high frequency component such as a checkered pattern, the image of the moiré is reduced when the image is reduced or enlarged without blurring the display of the texture part and reducing the image quality. it can. If the partial enlargement / reduction processing unit 3 has a conversion function for enlarging or reducing a part of the screen, the screen layout can be changed in various ways, and a checkerboard pattern can be displayed in the image data Da. Even if there is a portion containing a high-frequency component, an image with reduced moiré can be displayed when the image is reduced or enlarged without blurring the display of the texture portion and reducing the image quality.

  The image display apparatus according to the first embodiment includes an image processing apparatus and an image projection apparatus having an optical system that projects image data output from the image processing apparatus. A high-frequency component detection unit that performs a convolution operation on the input image data pixel or pixel group to detect a high-frequency component of the image data and outputs a detection result; and smoothing that outputs smoothed image data obtained by smoothing the image data Image processing unit, an image data processing unit that combines image data and smoothed image data at a combining ratio determined based on the detection result, and outputs the combined image data; and image data output from the image data processing unit Is provided with a partial enlargement / reduction processing unit for enlarging or reducing each region with a characteristic opposite to that of the optical system included in the image projection apparatus. Therefore, for example, video light is projected obliquely to the projected screen by the optical system provided in the image projection apparatus, and even when keystone distortion occurs, the partial enlargement / reduction processing unit 3 has an optical system or If you have a function to convert the inverse characteristics of keystone distortion, you can correct keystone distortion caused by oblique projection, and even if there are parts that contain high frequency components such as checkered patterns in the input image, It is possible to display an image in which the moire generated when the keystone distortion is corrected is reduced without blurring the display of the textured portion and degrading the image quality.

Embodiment 2. FIG.
In the description of the first embodiment, the high-frequency component processing unit 2 performs processing using pixels that are close to the target pixel in the horizontal direction, but the pixels used for processing are not limited to this. For example, an image holding unit may be provided inside the high frequency component processing unit, and processing using pixels adjacent in the vertical direction in addition to pixels adjacent in the horizontal direction may be performed. Since the components other than the high frequency component processing unit are the same as those in the first embodiment, the description thereof is omitted.

  FIG. 12 is a diagram illustrating the configuration of the high frequency component processing unit 102 of the image processing apparatus according to the second embodiment of the present invention. The image holding unit 14 is configured by a memory that holds image data for one line of input image data Da or image data for a plurality of lines, and outputs the image data for a plurality of lines as a plurality of line image data Df.

  The smoothing processing unit 111 performs a smoothing process with reference to pixels close to each other in the vertical direction and the horizontal direction. The detailed description is omitted because it is the same as in the first embodiment except that not only the pixels adjacent in the horizontal direction but also the pixels adjacent in the vertical direction are referred to. Further, the operation of the image data processing unit 113 is also the same as that of the first embodiment, and thus description thereof is omitted.

  Next, the high frequency component detection unit 112 according to the second embodiment will be described. FIG. 13 shows an internal configuration of the high-frequency component detection unit 112. The high frequency component detection unit 112 includes a convolution pattern generation unit 121, a convolution operation unit 122, and a high frequency component detection unit 123. The convolution pattern generation unit 121 includes a convolution pattern A generation unit 121A that generates convolution pattern data TaA, a convolution pattern B generation unit 121B that generates convolution pattern data TaB, and a convolution pattern C generation unit 121C that generates convolution pattern data TaC. A convolution operation unit 122 generates a convolution operation result DeA obtained by performing a convolution operation on convolution pattern data TaA and input image data Da, and a convolution obtained by performing a convolution operation on convolution pattern data TaB and input image data Da. A convolution operation B operation unit 122B that generates an operation result DeB and a convolution operation C operation unit 122C that generates a convolution operation result DeC obtained by performing a convolution operation of the convolution pattern data TaC and the input image data Da are included.

  In the high-frequency component detection unit 112 according to the second embodiment, the convolution pattern generation unit 121 generates a plurality of convolution pattern data TaA, TaB, and TaC. Various high frequency components can be detected.

  FIG. 14 (a) shows a typical checkered pattern as a periodic high-frequency component that causes moiré, vertical stripes (a state in which pixels having different gradations are periodically arranged on the left and right, and horizontal stripes). FIG. 14B shows image data in a state in which pixels with different gradations are periodically arranged at the upper limit). FIG. 14B shows convolution pattern data TaA for checkered pattern detection, convolution pattern data TaB for vertical stripe detection, and convolution pattern data for horizontal stripe detection. TaC is shown.

  As the convolution pattern data for checkerboard, vertical stripes, and horizontal stripes, the signs of + − (coefficients for determining whether to add or subtract) are arranged like a checkerboard pattern in the checkerboard pattern detection convolution pattern data TaA, In the convolution pattern data TaB for detecting vertical stripes, the convolution pattern data TaC for detecting horizontal stripes are arranged in a vertical stripe pattern.

  FIG. 15 shows a convolution calculation result using the image data shown in FIG. 14A and the convolution pattern data shown in FIG. For checkered image data, the absolute value of the result of convolution calculation with checkered pattern detection convolution pattern data TaA is large, and for vertical stripe image data with convolution pattern data TaB for detection of vertical stripes. It can be seen that the absolute value of the result is large, and the absolute value of the convolution calculation result with the convolution pattern data TaC for detecting horizontal stripes is large for the horizontal stripe image data.

  For example, if convolution pattern data for checkerboard pattern detection is output as convolution pattern data TaA, convolution pattern data for detection of vertical stripes is output as convolution pattern data TaB, and convolution pattern data for detection of horizontal stripes is output as convolution pattern data TaC. When the input image data Da includes many checkered patterns, the absolute value of the convolution calculation result DeA is large, and when the input image data Da includes many vertical stripes, the absolute value of the convolution calculation result DeB is large. Thus, when the input image data Da contains many horizontal stripes, the absolute value of the convolution calculation result DeC becomes large.

  In this way, by using the convolution calculation pattern data TaA, TaB, and TaC that simulate the high-frequency pattern that needs to be detected, the high-frequency component amount calculation unit 123 uses the values of the convolution calculation results DeA, DeB, and DeC. Thus, it is possible to generate a high-frequency component detection result Sb that quantitatively represents how much high-frequency components in question are included. For example, the absolute value of the convolution calculation result DeA, the absolute value of the convolution calculation result DeB, and the absolute value of the convolution calculation result DeC are output as the high frequency component detection result Sb, or the convolution calculation results DeA, DeB, A method of outputting the square sum of DeC as the high frequency component detection result Sb is conceivable. Further, the convolution calculation results DeA, DeB, and DeC are compared. For example, when the enlargement / reduction direction in the subsequent partial enlargement / reduction unit 3 is only the horizontal direction, the influence on the moire with respect to the enlargement / reduction in the horizontal direction is small. The processing may be changed according to the detected pattern of the high frequency component, for example, the value of DeC indicating the detection of the horizontal stripe is not applied to Sb.

  As described above, according to the image processing apparatus of the second embodiment, the high frequency component processing unit 102 includes the image holding unit 14 that holds and outputs the image data Df for a plurality of lines of the input image data Da. The convolution pattern generation unit 112 generates two-dimensional convolution pattern data (TaA, TaB, TaC) corresponding to each of a plurality of types of high-frequency components (checkered pattern, vertical stripe, horizontal stripe), and the convolution operation unit 122. Is configured to output a convolution calculation result (DeA, DeB, DeC) of each of the image data Df for a plurality of lines and each of the two-dimensional convolution pattern data (TaA, TaB, TaC). Multiple types of high-frequency components included can be detected accurately, and image processing corresponding to the types can be performed for more efficiency. In without reducing image quality blur the display of texture portions, yet can display an image with reduced moire upon shrink or enlarge.

Embodiment 3 FIG.
In the description of the first and second embodiments, it is assumed that the number of channels of the input image data Da is 1. However, what is considered as the input image data Da is not limited to this. For example, when the input image data Da is a color image, the input image data Da is divided into three components of red, green, and blue (or R, G, and B), so the number of channels is three. In the image processing apparatus according to the third embodiment of the present invention, processing when the number of channels of the input image data Da is 3 will be described as an example. Even when the number of channels is other than 3, it can be realized by the same processing as in the third embodiment if each part is configured corresponding to the number of channels.
Further, since the components other than the high frequency component processing unit 302 of the image processing apparatus are the same as those in the first and second embodiments, the description thereof is omitted.

  FIG. 16 is a diagram showing a configuration of the high frequency component processing unit 302 of the image processing apparatus according to the third embodiment of the present invention. The high frequency component processing unit 302 includes a smoothing processing unit 311, a high frequency component detection unit 312, and an image data processing unit 313. The configuration of the image processing apparatus 302 is not limited to this, and the image holding unit 14 may be provided as in the second embodiment.

  The input image data Da includes input red image data DaR representing a red component, input green image data DaG representing a green component, input blue image data DaB representing a blue component, that is, image data for each color component. . The smoothing processing unit 311 performs smoothed image data DdR obtained by performing the smoothing process on the input red image data DaR, smoothed image data DdG obtained by performing the smoothing process on the input green image data DaG, and the input blue image. The smoothed image data DdB obtained by performing the smoothing process on the data DaB, that is, the color-specific smoothed image data for each color component is output. The high frequency component detection unit 312 detects the high frequency component included in the input red image data DaR, and outputs the result as the high frequency component detection result SbR. The high frequency component detection unit 312R and the high frequency component included in the input green image data DaG. G high-frequency component detection unit 312G that detects and outputs the result as a high-frequency component detection result SbG, detects a high-frequency component included in the input blue image data DaB, and outputs the result as a high-frequency component detection result SbB Part 312B. That is, the high frequency component detection unit 312 generates a color-specific high frequency component detection result from the color-specific image data for each of a plurality of color components. The individual operations of the R high frequency component detector 312R, the G high frequency component detector 312G, and the B high frequency component detector 312B are the same as those of the high frequency component detector 12 or 112.

  FIG. 17 is a diagram illustrating the configuration of the image data processing unit 313. The image data processing unit 313 includes a composition ratio generation unit 331 and a weighted addition unit 332. The composition ratio generation unit 331 includes an R composition ratio generation section 331R, a G composition ratio generation section 331G, and a B composition ratio generation section 331B, and each operation is the same as that of the composition ratio generation section 31, and the composition ratios ScR and ScG, respectively. , ScB is output. The weighted addition unit 332 includes an R weighted addition unit 332R, a G weighted addition unit 332G, and a B weighted addition unit 332B, and each operation is the same as that of the weighted addition unit 32. That is, the R weighted addition unit 332R outputs the result of weighted addition of the input red image data DaR and the smoothed image data DdR based on the composite ratio ScR as the R high frequency component smoothed image data DbR, and the G weighted addition unit 332G The result of weighted addition of the input green image data DaG and the smoothed image data DdG based on the composition ratio ScG is output as G high frequency component smoothed image data DbG, and the B weighted addition unit 332B is input based on the composition ratio ScB. The result of weighted addition of the blue image data DaB and the smoothed image data DdB is output as B high frequency component smoothed image data DbB. That is, the image data processing unit 313 generates color-specific high-frequency component processed image data obtained by combining color-specific image data and color-specific smoothed image data based on the color-specific high-frequency component detection results output for each of the plurality of color components. Output. By operating in this way, even when the input image data Da is composed of a plurality of channels like a color image, it is possible to remove high frequency components that cause moire.

  As described above, according to the image processing apparatus according to the third embodiment, the image data Da is made up of color-specific image data DaR, DaG, DaB for each of a plurality of color components (red, green, blue), and high-frequency components. The detection unit 312 generates color-specific high-frequency component detection results SbR, SbG, and SbB obtained from the color-specific image data DaR, DaG, and DaB for each of the plurality of color components (red, green, and blue). The smoothing processing unit 311 outputs color-specific smoothed image data DdR, DdG, DdB obtained by smoothing the color-specific image data DaR, DaG, DaB for each of a plurality of color components (red, green, blue), and the image data The processing unit 313 performs color-specific image data DaR, DaG, DaB and color-specific smoothed image data DdR based on the color-specific high-frequency component detection results SbR, SbG, SbB for each of a plurality of color components (red, green, blue). , DdG Since the high-frequency component processed image data DbR, DbG, and DbB, which are combined with DdB, are output, the texture portion can be displayed even when the input image data Da is composed of a plurality of channels like a color image. It is possible to display an image with reduced generation of moire when the image is reduced or enlarged without blurring and reducing the image quality.

Embodiment 4 FIG.
FIG. 18 is a diagram illustrating the image data processing unit 313b according to the fourth embodiment. The fourth embodiment is different from the third embodiment (FIG. 17) in the configuration of the composition ratio generation unit 331b, and the others are the same. The composition ratio generation unit 331b includes a representative value generation unit 333 and an RGB composition ratio generation unit 331RGB. The representative value generation unit 333 generates common high-frequency component detection results common to the three color components (red, green, and blue) from the high-frequency component detection results SbR, SbG, and SbB for each of the three color components (red, green, and blue). SbC is output. As the common high-frequency component detection result SbC, for example, the maximum value, the minimum value, or the average value for each pixel of the color-specific high-frequency component detection results SbR, SbG, and SbB can be considered. The RGB composition ratio generation unit 331RGB generates a common composition ratio ScC common to the three color components (red, green, and blue) based on the common high-frequency component detection result SbC, and sets the weight addition unit 332 for each color component. Output to each of the weighted addition units 332R, 332G, and 332B. That is, the image data processing unit 313b calculates a common high-frequency component detection result ScC common to a plurality of color components (red, green, and blue) from the color-specific high-frequency component detection results SbR, SbG, and SbB, and calculates the calculated common high-frequency component detection Color-specific high-frequency component processed images obtained by synthesizing color-specific image data DaR, DaG, DaB and color-specific smoothed image data DdR, DdG, DdB for each of a plurality of color components (red, green, blue) based on the result ScC Data DbR, DbG, DbB are output. The method for obtaining the composition ratio ScC is the same as that for the composition ratio generator 31.

  A common composition ratio ScC having the same value is output to the color-specific weighted addition units 332R, 332G, and 332B. The common composition ratio ScC is a coefficient for determining the pass frequency band for the input red image data DaR, the input green image data DaG, and the input blue image data DaB of the high frequency component processing unit 302, and therefore the composition ratio value is the same for each color. Thus, the high frequency component processing unit 302 is controlled so that the pass frequency bands for the input red image data DaR, the input green image data DaG, and the input blue image data DaB are always the same. By controlling in this way, it is possible to prevent unnecessary coloring from occurring near the edge of the image.

  For example, consider a case where the value of the high-frequency component detection result SbG out of the high-frequency component detection results SbR, SbG, and SbB is larger in the vicinity of an edge included in the input image data Da. At this time, when the values of the synthesis ratios ScR, ScG, and ScB are obtained individually as in the third embodiment, the value of the synthesis ratio ScG is larger than the synthesis ratios ScR and ScB. As a result, since only the pass frequency band for the input green image data DaG is lowered, only the signal level of the high-frequency component smoothed image data DbG is present in the vicinity of the focused edge compared to the high-frequency component smoothed image data DbR and DbB. As a result, the vicinity of the edge is colored purple (magenta) which is a complementary color of green.

  On the other hand, if the values of the composition ratios ScR, ScG, and ScB are the same as in the fourth embodiment, the input red image data DaR, the input green image data DaG, and the input blue image data are near the target edge. Since the pass frequency bands for DaB are the same, the signal levels of the color-specific high-frequency component smoothed image data DbR, DbG, and DbB are also aligned. That is, the vicinity of the edge is not colored.

  As described above, according to the image processing apparatus according to the fourth embodiment, the image data Da is made up of the color-specific image data DaR, DaG, DaB for each of the plurality of color components (red, green, blue), and the high frequency component The detection unit 312 generates color-specific high-frequency component detection results SbR, SbG, and SbB obtained from the color-specific image data DaR, DaG, and DaB for each of the plurality of color components (red, green, and blue). The smoothing processing unit 311 outputs color-specific smoothed image data DdR, DdG, DdB obtained by smoothing the color-specific image data DaR, DaG, DaB for each of a plurality of color components (red, green, blue), and the image data The processing unit 313b calculates a representative high-frequency component detection result ScC common to a plurality of color components (red, green, and blue) from the color-specific high-frequency component detection results SbR, SbG, and SbB, and calculates the calculated representative high-frequency component detection result Sc. The color-specific high-frequency component processed image data DbR obtained by combining the color-specific image data DaR, DaG, DaB and the color-specific smoothed image data DdR, DdG, DdB for each of a plurality of color components (red, green, blue). , DbG and DbB are output so that even if the input image data Da is composed of a plurality of channels like a color image, the image quality is reduced by blurring the display of the texture portion without causing color misregistration. Thus, an image with reduced moiré can be displayed when the image is reduced or enlarged.

Embodiment 5 FIG.
In the first to fourth embodiments, processing such as high-frequency components that cause moire and partial enlargement / reduction processing for correcting image distortion are performed by the image processing apparatus. However, the present invention is not limited to this, and for example, part or all of it can be performed by software processing. FIG. 19 shows a flowchart when the present invention is performed by software processing. Processing by software of the image conversion processing according to the fifth embodiment of the present invention includes a high frequency component processing step ST2 and a partial enlargement / reduction step ST3.

  Then, the high frequency component processing step ST2 smoothes a portion including the high frequency component with respect to the input image data Da read in the input image data reading step (not shown) by a method described later, and the obtained image data is converted into the high frequency component. Output as smoothed image data Db. The partial enlargement / reduction step ST3 outputs partial enlargement / reduction image data obtained by enlarging or reducing the high-frequency component smoothed image data Db for each region. Then, the output partial enlarged / reduced image data is displayed on a screen of a rear projection television, for example, through an image display step (not shown).

FIG. 20 is a flowchart showing details of the high frequency component processing step ST2. The high frequency component processing step ST2 includes a high frequency component detection step ST11, a smoothing processing step ST12, and an image data processing step ST13.
In the high-frequency component detection step ST11, when the number of channels of the input image data Da is 1, the same calculation as the high-frequency component detection unit 12 or 112 described in the first or second embodiment is performed to calculate the high-frequency component detection result Sb. When the number of channels of the input image data Da is plural, the same calculation as that of the high-frequency component detection unit 312 described in the third or fourth embodiment is performed. For example, when the number of channels of the input image data Da is 3, corresponding to red, blue, and green, the high frequency component detection results SbR, SbG, and SbB are calculated.

  The smoothing processing step ST12 outputs smoothed image data Dd obtained by smoothing the input image data Da. The calculation here is also the same as the smoothing processing unit 11 or 111 described in the first or second embodiment according to the number of channels of the input image data Da, or the smoothing processing unit described in the third or fourth embodiment. What is necessary is just like 311.

  In the image data processing step ST13, depending on the number of channels of the input image data Da, the image data processing unit 13 or 113 described in the first or second embodiment, the image data processing unit 313 described in the third or fourth embodiment, or The same calculation as 313b is performed to calculate high-frequency component smoothed image data Db.

  FIG. 21 is a flowchart showing details of the high-frequency component detection step ST11. The high-frequency component detection step ST11 includes a convolution pattern generation step ST21 for generating convolution pattern data for performing convolution operations with the input image data Da, a convolution operation step ST22 for performing convolution operations of the input image data Da and the convolution pattern data, and convolution. Based on the result of the convolution calculation performed in the calculation step ST22, the method includes a high-frequency component amount calculation step ST23 for quantitatively determining how much high-frequency components that cause moire are included in the input image data Da. The calculation to be performed in each step ST21, ST22, ST23 is the same as that of the high frequency component detection unit 12 or 112 described in the first or second embodiment or the high frequency component detection unit 312 described in the third or fourth embodiment.

  FIG. 22 is a flowchart showing details of the image data processing step ST13. The image data processing step ST13 includes a composition ratio generation step ST31 and a weighted addition step ST32. The composition ratio generation step ST31 performs the same operation as the composition ratio generation section 31 described in the first or second embodiment, or the composition ratio generation section 331 or 331b described in the third or fourth embodiment, and the weighted addition step is performed. The same calculation as the weighted addition unit 32 described in the first or second embodiment or the weighted addition unit 332 described in the third or fourth embodiment is performed. Here, the calculation in the weighted addition step may be determined according to the number of channels of the image data, like the weighted addition unit 32 or the same as the weighted addition unit 332. When there is data for each color component, the same calculation as that performed by the combination ratio generation unit 331 is performed when calculating with a different combination ratio for each color component as in the third embodiment. Alternatively, when the calculation is performed at a combination ratio common to the respective color components as in the fourth embodiment, the same calculation as that performed by the combination ratio generation unit 331b may be performed. In that case, unnecessary coloring does not occur in the high-frequency component smoothed image data Db.

  As described above, according to the image processing method according to the fifth embodiment, the convolution operation is performed for each attention area formed of a part of the screen area of the input image data Da, and the high-frequency component detection result Sb for each attention area. The high-frequency component detection step ST11 for outputting the image data, the smoothing processing step ST12 for outputting the smoothed image data Dd obtained by smoothing the image data Da, and the image data Da for each region of interest based on the high-frequency component detection result Sb. An image data processing step ST13 for outputting high-frequency component processed image data Db obtained by synthesizing the smoothed image data Dd. In the image data processing step ST13, the smoothed image data is based on the value of the high-frequency component detection result Sb. The synthesis ratio Sc of Dd is changed (the greater the value of the high frequency component detection result Sb, the greater the synthesis of the smoothed image data Dd. Therefore, even if there is a part in the image data Da that contains periodic high-frequency components that cause moire, the display of the texture part is blurred and the image quality is lowered. In addition, it is possible to display an image in which the occurrence of moire is reduced when the image is reduced or enlarged.

  According to the software processing described above, an operation similar to that of the image processing apparatus described in any of Embodiments 1 to 4 is realized. Therefore, the same operation and effect as the image processing apparatus described in any of Embodiments 1 to 4 can be obtained. Further, the software processing described above can be used as a part of the image display device as in the case of the image processing device described in any of the first to fourth embodiments, and in that case also in any of the first to fourth embodiments. The same operations and effects as the image display apparatus described are exhibited.

1 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention. 1 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention. It is a block diagram which shows the part of the image processing apparatus concerning Embodiment 1 of this invention. It is a block diagram which shows the part of the image processing apparatus concerning Embodiment 1 of this invention. It is a figure which shows an example of the convolution pattern data used with the image processing apparatus concerning Embodiment 1 of this invention. It is a figure which shows an example of the input image data and the convolution calculation result in the image processing apparatus concerning Embodiment 1 of this invention. It is a figure which shows the relationship between the high frequency component detection result in the image processing apparatus concerning Embodiment 1 of this invention, and a synthetic | combination ratio. 1 is a diagram illustrating an image processing apparatus according to a first embodiment of the present invention. It is a figure which shows the example of a setting of the coefficient for smoothing used with the image processing apparatus concerning Embodiment 1 of this invention. It is a figure which shows the input image data processed with the image processing apparatus concerning Embodiment 1 of this invention. It is a figure which shows the calculation result of the convolution calculation result with respect to different input image data and a different smoothing coefficient, and the smoothed image data processed with the image processing apparatus concerning Embodiment 1 of this invention. It is a block diagram which shows the image processing apparatus concerning Embodiment 2 of this invention. It is a block diagram which shows the part of the image processing apparatus concerning Embodiment 2 of this invention. It is a figure which shows the example of the high frequency component which causes the moire processed with the image processing apparatus concerning Embodiment 2 of this invention, and the example of the convolution pattern data for detecting it. It is a figure which shows the convolution calculation result processed with the image processing apparatus concerning Embodiment 2 of this invention. It is a block diagram which shows the image processing apparatus concerning Embodiment 3 of this invention. It is a block diagram which shows the part of the image processing apparatus concerning Embodiment 3 of this invention. It is a block diagram which shows the part of the image processing apparatus concerning Embodiment 4 of this invention. It is a flowchart which shows the image processing method concerning Embodiment 5 of this invention. It is a flowchart which shows the image processing method concerning Embodiment 5 of this invention. It is a flowchart which shows the part of the image processing method concerning Embodiment 5 of this invention. It is a flowchart which shows the part of the image processing method concerning Embodiment 5 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input terminal, 2, 102, 302 High frequency component processing part, 3 Partial expansion / contraction processing part, 11, 111, 311 Smoothing processing part, 12, 112, 312 High frequency detection part, 13, 113, 313, 313b Image data processing , 21, 121 convolution pattern generation unit, 22, 122 convolution operation unit, 23, 123 high frequency component amount calculation unit,
Da image data, Db high-frequency component smoothed image data, Dc partially enlarged / reduced image data, Dd smoothed image data, Sb high-frequency component component detection result, Sc composition ratio, Ta convolution pattern data

Claims (12)

A high-frequency component detector that performs a convolution operation on a pixel or pixel group of image data to detect a high-frequency component of the image data and outputs a detection result; and smoothing that outputs smoothed image data obtained by smoothing the image data An image processing apparatus comprising: a processing unit; and an image data processing unit that synthesizes the image data and the smoothed image data at a synthesis ratio determined based on the detection result and outputs synthesized data .
For each pixel of interest in the screen area of the image data, the smoothing processing unit performs the smoothing from the data of each pixel included in the smoothing target region including the pixel of interest and pixels in the vicinity of the pixel of interest. To generate image data,
The image processing apparatus according to claim 1, wherein the number of pixels constituting the smoothing target area is smaller than the number of pixels of a pixel group that is a calculation target of the convolution calculation . The high-frequency component detection unit includes a convolution pattern generation unit that generates convolution pattern data for performing a convolution operation, and the image data and the convolution pattern data for each region of interest including pixels or pixel groups of the image data. A convolution operation unit that outputs a convolution operation result; and a high-frequency component amount calculation unit that outputs an absolute value of the convolution operation result as the high-frequency component detection result, wherein the convolution operation includes at least a plurality of regions of interest. A pixel group composed of pixels is set as a calculation target, and the convolution pattern data includes coefficients obtained by switching positive and negative for each successive pixel with respect to each pixel in the pixel group as the calculation target. The image processing apparatus according to claim 1. A high-frequency component detector that performs a convolution operation on a pixel or pixel group of image data to detect a high-frequency component of the image data and outputs a detection result; and smoothing that outputs smoothed image data obtained by smoothing the image data An image processing apparatus comprising: a processing unit; and an image data processing unit that synthesizes the image data and the smoothed image data at a synthesis ratio determined based on the detection result and outputs synthesized data.
The high-frequency component detection unit includes a convolution pattern generation unit that generates convolution pattern data for performing a convolution operation, and the image data and the convolution pattern data for each region of interest including pixels or pixel groups of the image data. A convolution operation unit that outputs a convolution operation result; and a high-frequency component amount calculation unit that outputs an absolute value of the convolution operation result as the high-frequency component detection result, wherein the convolution operation includes at least a plurality of regions of interest. A pixel group composed of pixels is set as a calculation target, and the convolution pattern data includes coefficients obtained by switching positive and negative for each successive pixel with respect to each pixel in the pixel group as the calculation target. Image processing device. The smoothing target area, the image processing apparatus according to claim 1 or claim 2, characterized in that they are composed of pixels adjacent to the pixel of interest and the pixel of interest. The image data processing unit includes an image holding unit that holds and outputs image data for a plurality of lines of the image data,
The convolution pattern generation unit generates two-dimensional convolution pattern data corresponding to each of a plurality of types of high frequency components,
The convolution operation unit outputs a convolution operation result of the image data for the plurality of lines and each of the two-dimensional convolution pattern data;
The image processing apparatus according to claim 2 , wherein the image processing apparatus is an image processing apparatus. The image data comprises color-specific image data for each of a plurality of color components,
The high-frequency component detection unit generates a color-specific high-frequency component detection result obtained from the color-specific image data for each of the plurality of color components to obtain the high-frequency component detection result,
The smoothing processing unit outputs color-specific smoothed image data obtained by smoothing the color-specific image data for each of the plurality of color components,
The image data processing unit outputs combined data obtained by combining the color-specific image data and the color-specific smoothed image data based on the color-specific high-frequency component detection result for each of the plurality of color components;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The image data comprises color-specific image data for each of a plurality of color components,
The high-frequency component detection unit generates a color-specific high-frequency component detection result obtained from the color-specific image data for each of the plurality of color components to obtain the high-frequency component detection result,
The smoothing processing unit outputs color-specific smoothed image data obtained by smoothing the color-specific image data for each of the plurality of color components,
The image data processing unit calculates a common high-frequency component detection result common to the plurality of color components from the color-specific high-frequency component detection result, and for each of the plurality of color components based on the calculated common high-frequency component detection result Outputting synthesized data obtained by synthesizing the color-specific image data and the color-specific smoothed image data;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The image processing apparatus according to claim 1, further comprising an enlargement / reduction processing unit that enlarges or reduces the combined data for each region. The image processing apparatus according to claim 1, further comprising a processing unit that performs keystone distortion correction on the synthesized data. The image processing apparatus according to any one of claims 1 to 9,
An image display unit for displaying an image based on the composite data output from the image processing device;
An image display device comprising: A high-frequency component detection step for performing a convolution operation on a pixel or a pixel group of image data to detect a high-frequency component of the image data and outputting a detection result; and smoothing for outputting smoothed image data obtained by smoothing the image data In an image processing method comprising: a processing step; and an image data processing step of combining the image data and the smoothed image data at a combining ratio determined based on the detection result and outputting combined data .
The smoothing processing step includes, for each target pixel in the screen area of the image data, smoothing from the data of each pixel included in the smoothing target region including the target pixel and pixels in the vicinity of the target pixel. Generate image data,
An image processing method, wherein the number of pixels constituting the smoothing target area is smaller than the number of pixels in a pixel group to be calculated by the convolution calculation . A high-frequency component detection step for performing a convolution operation on a pixel or a pixel group of image data to detect a high-frequency component of the image data and outputting a detection result; and smoothing for outputting smoothed image data obtained by smoothing the image data In an image processing method comprising: a processing step; and an image data processing step of combining the image data and the smoothed image data at a combining ratio determined based on the detection result and outputting combined data.
The high-frequency component detection step includes a convolution pattern generation step for generating convolution pattern data for performing a convolution operation, and the image data and the convolution pattern data for each region of interest consisting of pixels or pixel groups of the image data. A convolution operation step for outputting a convolution operation result; and a high-frequency component amount calculation step for outputting an absolute value of the convolution operation result as the high-frequency component detection result, wherein the convolution operation includes a plurality of regions including at least the region of interest. A pixel group composed of pixels is set as a calculation target, and the convolution pattern data includes coefficients obtained by switching positive and negative for each successive pixel with respect to each pixel in the pixel group as the calculation target. Image processing method .

Images