JP6929185B2 - Image processing equipment and image processing methods, programs, storage media - Google Patents

Description

The present invention relates to a technique for performing noise correction of an image signal.

In recent years, the number of pixels of an image sensor has been increasing, and the pixels tend to be miniaturized. Due to the miniaturization of the pixels, the noise of the image signal obtained from the image sensor increases. Therefore, as a method of suppressing this noise by signal processing, a method of using multi-rate signal processing is known.

For example, there is known a method of dividing an image signal into a plurality of frequencies, suppressing noise in each divided image, and then resynthesizing the image signal (see Patent Document 1). In Patent Document 1, the image signal is divided for each frequency band, and noise suppression processing is performed on each divided image. There are various methods for dividing the frequency component. For example, in order to generate a low frequency image, the original image is prefiltered and then the size is thinned out in half. Further, in order to generate a high-frequency image, it is obtained by enlarging the low-frequency image so that it has the same size as the original image, and then subtracting the enlarged image from the original image.

By dividing into frequency components in this way, it is possible to set an application amount of noise suppression suitable for the frequency components for each divided image. Further, by reducing the size of each image, it is possible to reduce noise with a small filter length. It should be noted that the high-frequency noise is not visually noticeable because the graininess of the noise is fine. On the other hand, low-frequency noise has a large graininess and is easily visually noticeable. Therefore, it is desirable to remove low-frequency noise more strongly than high-frequency noise.

Further, in the process of synthesizing the divided image divided into frequency components, it is possible to perform a simple calculation such as enlarging the low frequency image and adding it to the high frequency image. However, as is clear from the calculation method of the compositing process, when the divided image is blurred by the noise suppression process, the blurred image of the corresponding frequency band remains as it is in the image after the compositing process. Therefore, in the method of suppressing noise by dividing it into frequency bands, it is possible to suppress noise differently for each frequency band, but it is necessary to suppress the noise suppression effect to the extent that blurring is acceptable in each frequency band. There is a problem that there is.

Therefore, as another noise suppression method, first, a plurality of reduced images are generated from the image signal, noise suppression processing is performed on the plurality of reduced images, and an edge signal is extracted and each image is based on the edge signal. Find the composition ratio. Then, a method of synthesizing a plurality of image signals based on the composition ratio is known (see Patent Document 2).

For example, an image in which the sizes in the horizontal and vertical directions are halved and an image in which the sizes are halved are generated. Then, noise suppression processing is performed on each image, and an edge signal is extracted from the reduced image. When compositing images, the edge signal is enlarged to the original size, and the compositing ratio is determined based on the magnified edge signal. That is, the portion where the edge signal is strong is configured to use a larger size image, and the portion where the edge signal is weak is configured to use a reduced image enlarged to the original size.

With this configuration, the original size image is often used for the edge portion of the composited image, and conversely, the reduced image is not often used. Therefore, even if the noise suppression process is applied to the reduced image and the edge portion is blurred, the final image is not affected. Therefore, it is possible to apply a strong noise suppression process to the reduced image, and it is possible to obtain an image after composition in which the noise suppression effect of the low frequency component is enhanced without blurring the edges.

Japanese Unexamined Patent Publication No. 2008-15741 JP-A-2009-199104

However, the method disclosed in Patent Document 2 is premised on synthesizing an input image and a reduced image. Therefore, as compared with the noise suppression process in which a plurality of images are not combined, the noise suppression effect is high, but there is a problem that the edge resolution tends to decrease.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to obtain a noise suppression image having a higher resolution when the noise suppression processing is performed in layers and combined to obtain a noise suppression image. It is to provide an image processing apparatus which can be obtained.

The image processing apparatus according to the present invention includes a first noise suppression processing means that performs noise suppression processing on an input image without performing layer division, and layer division of the input image using a reduced image. The output image of the first noise suppression processing means and the output image of the second noise suppression processing means according to the edge signal of the second noise suppression processing means that performs the noise suppression processing while performing the above. The output image of the first noise suppression processing means and the second noise based on the first calculation processing means for calculating the first synthesis ratio for synthesizing and the first synthesis ratio. The second noise suppression processing means includes a first synthesis processing means for synthesizing the output image of the suppression processing means, and the second noise suppression processing means performs noise performed by the first noise suppression processing means without reducing the input image. Outputs of a third noise suppression processing means that performs noise suppression processing different from the suppression processing, a fourth noise suppression processing means that performs noise suppression processing by reducing the input image, and the third noise suppression processing means. It is characterized by comprising a second synthesis processing means for synthesizing an image and an output image obtained by performing enlargement processing on the output image of the fourth noise suppression processing means.

According to the present invention, it is possible to obtain a noise suppression image having a higher resolution when the noise suppression processing is performed in layers and combined to obtain a noise suppression image.

The block diagram which shows the schematic structure of the digital camera in one Embodiment of this invention. The block diagram which shows the structure of the noise suppression processing part included in an image processing circuit. The flowchart which shows the processing procedure by a noise suppression processing part. The figure which shows the pixel of interest and the reference area. The figure which shows the relationship between the edge amount and the hierarchical composition ratio. The figure which shows the pattern of the direction determination of an edge. The figure for demonstrating the method of obtaining the similarity between a pixel of interest and a reference pixel using a pixel group. The figure which shows the relationship between the edge amount and the composition ratio.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a schematic configuration of a digital camera 100 according to an embodiment of the present invention. The present invention is not limited to digital cameras, but can be realized by digital video cameras, personal computers, and the like as long as they are devices capable of performing image processing on image signals.

In FIG. 1, the optical system 101 includes a lens group composed of a zoom lens and a focus lens, an aperture device, and a shutter device. The optical system 101 adjusts the magnification, focus position, or amount of light of the subject image reaching the image sensor 102. The image pickup device 102 is a photoelectric conversion element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor, and converts a subject image into an electric signal to generate an image signal.

The preprocessing circuit 103 includes a CDS (Correllated Double Sampling) circuit and an amplifier circuit. The CDS circuit suppresses the dark current contained in the image signal generated by the image sensor 102, and the amplifier circuit amplifies the image signal output from the CDS circuit. The A / D converter 104 converts the image signal output from the preprocessing circuit 103 into a digital image signal.

The image processing circuit 105 performs white balance processing, noise suppression processing, gradation conversion processing, edge enhancement correction processing, and the like on the image signal, and outputs the image signal as the luminance signal Y and the color difference signals U and V. The image processing circuit 105 also calculates the brightness value of the subject and the focusing value indicating the focus state of the subject from the image signal. The image processing circuit 105 can perform the same image processing not only on the image signal output from the A / D converter 104 but also on the image signal read from the recording medium 109.

The control circuit 106 controls each circuit constituting the digital camera 100 of the present embodiment to control the operation of the digital camera 100. The control circuit 106 also performs drive control of the optical system 101 and the image sensor 102 based on the brightness value obtained from the image signal processed by the image processing circuit 105 and the instruction input from the operation member 110.

The display memory 107 is a memory that temporarily stores an image signal that is a source of an image to be displayed on the display device 108. The display device 108 is composed of a liquid crystal display or an organic EL (Electro Luminescence) display, and displays an image signal generated by the image sensor 102 and an image related to the image signal read from the recording medium 109. By updating and displaying the continuous image signal read from the image sensor 102 at any time, it is possible to function as an electronic viewfinder. Further, the display device 108 displays the status of the digital camera 100, character information such as shutter speed, aperture value, and sensitivity information determined by the user side or the digital camera 100 side, a graph showing the brightness distribution measured by the image processing circuit 105, and the like. Can also be displayed.

The recording medium 109 may be configured to be removable from the digital camera 100, or may be built into the digital camera 100. The operation member 110 is a member operated by the user to input an instruction to the digital camera 100. The bus 111 is used for exchanging image signals, instruction signals, and the like between the image processing circuit 105, the control circuit 106, the display memory 107, and the recording medium 109.

Next, the operation of the digital camera 100 at the time of shooting in the present embodiment will be described. When the operation member 110 is operated by the user and an instruction to start shooting preparation is input, the control circuit 106 starts controlling the operation of each circuit.

First, the image sensor 102 photoelectrically converts the subject image transmitted through the optical system 101 to generate an analog image signal, and the A / D converter 104 digitally converts the analog image signal processed by the preprocessing circuit 103. To become. The image processing circuit 105 performs white balance processing, noise suppression processing, gradation conversion processing, contour correction processing, and the like on the image signal output from the A / D converter 104.

The image signal processed by the image processing circuit 105 is displayed as an image on the display device 108 via the display memory 107. As described above, the image sensor 102 continuously generates an image signal, updates the read continuous image signal in real time and displays it on the display device 108, so that the display device 108 functions as an electronic viewfinder. do.

Then, these processes are repeated until the user operates the shutter button included in the operation member 110. When the user operates the shutter button, the control circuit 106 readjusts the operation of the optical system 101 based on the brightness value and the in-focus value obtained by the image processing circuit 105 to take a still image. The image processing circuit 105 performs various image processing including noise suppression processing on the image signal of the still image. Then, the image signal output from the image processing circuit 105 is recorded on the recording medium 109.

Here, the noise suppression processing in the image processing circuit 105 will be described in detail. FIG. 2 is a block diagram showing a detailed configuration of the noise suppression processing circuit 230 included in the image processing circuit 105.

The noise suppression processing circuit 230 of the present embodiment includes a band-specific noise suppression processing unit 200, a single-layer noise suppression processing unit 221, a composition ratio calculation processing unit 222, and an image composition processing unit 223. Further, the band-specific noise suppression processing unit 200 includes a reduction processing unit 201, a noise suppression processing unit 202, an edge signal detection processing unit 203, a noise suppression processing unit 204, an edge signal detection processing unit 205, an enlargement processing unit 206, and an enlargement processing unit. 207 is provided. Further, the band-specific noise suppression processing unit 200 includes a reduction processing unit 208, a noise suppression processing unit 209, an edge signal detection processing unit 210, an enlargement processing unit 211, and an enlargement processing unit 212. Further, the band-specific noise suppression processing unit 200 includes a reduction processing unit 213, a noise suppression processing unit 214, an enlargement processing unit 215, a composition ratio calculation processing unit 216, and an image composition processing unit 217. The single-layer noise suppression processing unit 221 is a processing unit that performs noise suppression processing on the input image without performing layer division, and the band-specific noise suppression processing unit 200 uses a reduced image for the input image. This is a processing unit that performs noise suppression processing while dividing the layers.

FIG. 3 is a flowchart showing a processing procedure by the noise suppression processing circuit 230. When the image processing circuit 105 receives the image signal output from the A / D converter 104, the image processing circuit 105 performs color interpolation processing of the pixels, and then performs the processing shown in FIG. 3 for each color in the noise suppression processing circuit 230. It should be noted that known techniques may be applied to the white balance processing, the color conversion processing, the gradation conversion processing, and the like, and since they are not related to the gist of the present embodiment, the description thereof will be omitted.

First, in step S301, the reduction processing unit 201 applies a pre-filter to the input image signal and then thins out the pixels to 1/2 the size in the horizontal and vertical directions to generate a reduced image. As the pre-filter, a coefficient of [1 2 1] is applied both horizontally and vertically. This is a coefficient for averaging the weighting on the pixel of interest and the weighting on each pixel adjacent to both sides of the pixel of interest in a ratio of 2: 1. Similarly, the reduction processing unit 208 and the reduction processing unit 213 generate reduced images having a size of 1/4 and 1/8 of the input image, respectively.

Next, in step S302, the noise suppression processing unit 202 performs noise suppression processing on the input image. In the noise suppression processing, as shown in FIG. 4, for example, a reference area of 7 × 7 pixels is set around the pixel of interest, and the difference between the value of each pixel in the reference area and the value of the pixel of interest is calculated. Then, the average value is calculated with the value of only the pixels whose difference is within the threshold value, and the average value is output as the value of the pixel of interest. By performing the above processing for each pixel, it is possible to obtain an output image in which noise is suppressed for the entire image. It is desirable to change the threshold value for each image, and further, the threshold value may be changed according to the signal level in consideration of the optical shot noise of the sensor.

Next, in step S303, the edge signal detection processing unit 203 detects the edge signal with respect to the input signal. To detect the edge signal, the Laplacian filter represented by the equation (1) is applied to the input image to detect the edge signal. Here, an example of setting a reference area of 3 × 3 pixels will be given, but the size of the reference area may be 5 × 5 pixels or 7 × 7 pixels.

Next, in step S304, the noise suppression processing unit 204, the noise suppression processing unit 209, and the noise suppression processing unit 214 perform noise suppression processing on the input images reduced to their respective sizes. The method of noise suppression processing is the same as the processing content of the noise suppression processing unit 202.

Next, in step S305, the edge signal detection processing unit 205 and the edge signal detection processing unit 210 detect the edge signal for the input image reduced to the respective size. The method of detecting the edge signal is the same as the processing content of the edge signal detection processing unit 203.

Next, in step S306, the enlargement processing unit 206, the enlargement processing unit 211, and the enlargement processing unit 215 perform enlargement processing on the reduced image in which noise is suppressed by bilinear interpolation so as to have the same size as the input image. ..

Next, in step S307, the enlargement processing unit 207 and the enlargement processing unit 212 perform enlargement processing on the calculated edge signal of the reduced image by bilinear interpolation so as to have the same size as the input image.

Next, in step S308, the synthesis ratio calculation processing unit 216 enlarges the output of the edge signal detection processing unit 203 that detects the edge signal from the input image and the edge signal detected from the reduced image. From the output signals of each of the unit 207 and the enlargement processing unit 212, the calculation processing of the composition ratio for image composition used in the image composition processing unit 217 is performed. As shown in FIG. 5A, the synthesis ratio calculation processing unit 216 synthesizes the output signal of the enlargement processing unit 211 rather than the output signal of the enlargement processing unit 215 as the edge signal obtained by the enlargement processing unit 212 is larger. Set a large ratio. Specifically, between the output signal of the enlargement processing unit 215 and the output signal of the enlargement processing unit 211, the edge signal obtained by the enlargement processing unit 212 is compared with the threshold values TH1 and TH2. Then, if the edge signal is smaller than the threshold value TH1, the combined ratio of the output signals of the enlargement processing unit 211 is set to 0, and if it is larger than the threshold value TH2, it is set to 1. The synthesis ratio is linearly calculated between the thresholds TH1 and TH2. That is, the larger the value of the edge signal, the higher the synthesis ratio is given priority to the output signal of the enlargement processing unit 211 including a large number of high frequency bands. Similarly, the composite ratio calculation processing unit 216 sets the composite ratio of the enlargement processing unit 206 to be larger than the output signal of the enlargement processing unit 211 as the edge signal obtained by the enlargement processing unit 207 is larger.

Further, the synthesis ratio calculation processing unit 216 sets the ratio of the output signal of the enlargement processing unit 211 to the output signal of the noise suppression processing unit 202 according to the magnitude of the edge signal obtained by the edge signal detection processing unit 203. .. However, here, as shown in FIG. 5B, if the edge signal obtained by the edge signal detection processing unit 203 is smaller than the threshold value TH3, the combined ratio of the output signals of the noise suppression processing unit 202 is set to 0. If the edge signal obtained by the edge signal detection processing unit 203 is equal to or more than the threshold value TH3 and less than the threshold value TH4, the larger the edge signal, the larger the composition ratio of the output signal of the noise suppression processing unit 202 is set. If the edge signal obtained by the edge signal detection processing unit 203 is equal to or higher than the threshold value TH4, the larger the edge signal, the smaller the composition ratio of the output signal of the noise suppression processing unit 202 is set. Alternatively, the synthesis ratio calculation processing unit 216 adds the edge signals obtained by the enlargement processing unit 207 and the edge signal detection processing unit 203, respectively, and compares the added edge signals with the threshold values TH3 and TH4. If the edge signal in the input image is large, a certain amount of edge signal remains even in the image reduced after applying the prefilter. Therefore, if the edge signal obtained by the addition is smaller than the threshold value TH4, the larger the edge signal, the larger the combined ratio of the output signals of the noise suppression processing unit 202, and if the edge signal is TH4 or more, the larger the edge signal. The synthesis ratio of the output signal of the noise suppression processing unit 202 is reduced.

Then, each composition ratio set based on the edge signal obtained by each of the enlargement processing unit 212, the enlargement processing unit 207, and the edge signal detection processing unit 203 is integrated, and the final image of each image is finalized for each pixel. Set the composition ratio.

Here, when the edge signal is the threshold value TH4 or more, the reason why the synthesis ratio of the output signal of the noise suppression processing unit 202 is reduced as the edge signal becomes larger will be described.

In order to detect the edge signal, as described in step S303, the values of the peripheral pixels of the target pixel for which the edge signal is obtained are also referred to. Therefore, if the peripheral pixels include edges, the value obtained as the edge signal of the target pixels may become large even if the target pixels do not include the edges. Therefore, in the pixels in the vicinity of the edges, the composition ratio of the high-resolution image may be high as in the case where the edges are present even though the edges are not present. Therefore, depending on whether or not it is near the edge, there is a difference in the sense of noise even though the flat portion is the same.

Therefore, the band-specific noise suppression processing unit 200 of the present embodiment intentionally lowers the composition ratio of high-resolution images in pixels and regions where the edge signal is larger than a predetermined threshold value, so that there is a lot of noise in the vicinity of the edge. It suppresses the remaining. However, since the edge is also blurred by this alone, the image composition processing unit 223 uses the image generated by the band-specific noise suppression processing unit 200 and the image generated by the single-layer noise suppression processing unit 221 as described later. Synthesize.

Next, in step S309, the image composition processing unit 217 converts the output signals of the noise suppression processing unit 202, the enlargement processing unit 206, the enlargement processing unit 211, and the enlargement processing unit 215 into the output signals of the composition ratio calculation processing unit 216. The output signal of the band-specific noise suppression processing unit 200 is generated. In the image composition processing, the output signals of the noise suppression processing unit 202, the enlargement processing unit 206, the enlargement processing unit 211, and the enlargement processing unit 215 are pixelated according to the composition ratio which is the output signal of the composition ratio calculation processing unit 216. An output image is generated by weighting and adding each time. With the above composition ratio, it is possible to acquire a composite image having a high noise suppression effect because the usage ratio of the output signal of the enlargement processing unit 206 is high in the place where the edge amount is small.

Next, in step S310, the single-layer noise suppression processing unit 221 performs noise suppression processing on the input image. The input image of the single-layer noise suppression processing unit 221 and the input image of the band-specific noise suppression processing unit 200 are the same image, and their resolution and image size are the same. The single-layer noise suppression processing unit 221 detects the direction of the edge and the intensity of the edge from the input image, and outputs the result image of the noise suppression and the edge intensity. The noise suppression processing performed by the single-layer noise suppression processing unit 221 can maintain a higher edge resolution than the noise suppression processing performed by the noise suppression processing unit 202 of the band-specific noise suppression processing unit 200.

Edge direction determination and edge strength detection are performed, for example, by pattern matching using the pattern (7 taps) shown in FIG. 6 for each pixel of interest. For each reference region shown in FIGS. 6 (a) to 6 (h), the sum of the absolute values of the differences between the pixel of interest and each pixel included in the search region is calculated. Next, the pattern in which the calculated sum of the absolute values of the eight differences is minimized is set as the edge direction of the pixel of interest, and the sum of the absolute values of the differences of the patterns orthogonal to the pattern in the edge direction is output as an edge signal. For example, when the sum of the absolute values of the differences in the pattern of FIG. 6A is the minimum, it is determined that the edge extends in the horizontal direction, the horizontal direction is set as the edge direction, and the vertical direction orthogonal to this (FIG. 6 (A). b) The sum of the absolute values of the differences using the pattern of)) is used as the edge signal of the pixel of interest. Noise suppression processing is performed according to the detected edge direction. In the noise suppression process, the difference between each pixel in the detected reference region in the edge direction and the pixel of interest is taken. Then, the average value is calculated only for the pixels whose difference is within the threshold value, and the average value is output as the value of the pixel of interest. By performing the above processing for each pixel, it is possible to obtain a noise-suppressed output image having a high edge resolution with respect to the entire image. It is desirable to change the threshold value for each pixel, and further, the threshold value may be changed according to the signal level in consideration of the optical shot noise of the sensor.

Alternatively, the single-layer noise suppression processing unit 221 may perform the noise suppression processing described below. FIG. 7 is a diagram for explaining another noise suppression process by the single layer noise suppression process unit 221. FIG. 7A shows an input image 701. In the input image 701, the upper left pixel is represented as (0,0), and the pixel value of each pixel is represented as I (x, y). Here, assuming that the pixel 702 is the pixel of interest, the pixel value of the pixel of interest is I (4,4). For the pixel of interest, the value of the pixel after the noise reduction processing is determined by the weighted average of the reference pixel and the weight corresponding to the reference pixel. If the number of reference pixels used for the weighted average is NS, the pixel value of the reference pixel is Ij (j = 1 to NS), and the weight corresponding to the reference pixel is wj (j = 1 to NS), the noise reduction processing is performed. The pixel value Inew of the pixel of interest later is given by the following equation.

The weight corresponding to the reference pixel is calculated by the similarity of the reference pixel with respect to the pixel of interest. The degree of similarity between the pixel of interest and the reference pixel is determined using the region including the pixel of interest and the region including the reference pixel. The area used to determine the degree of similarity is defined as the matching area. The input image 701 shown in FIG. 7A will be described as an example. It is assumed that the rectangular range 704 of 5 pixels × 5 pixels including the pixel of interest 702 is set as a reference pixel group, and each of the 25 pixels included in the reference pixel group is set as a reference pixel. Further, it is assumed that a 3 × 3 area is set in the matching area. The similarity between the pixel of interest and the reference pixel is calculated from the set matching area. Assuming that the pixel 706 is a reference pixel, the similarity between the pixel of interest 702 and the reference pixel 706 is as follows: the pixel group 703 including the pixel of interest 702 (hereinafter, the region of interest 703) and the pixel group 705 including the reference pixel 706 (hereinafter, It is determined by comparing with the reference area 705). Specifically, the similarity is set for each pixel included in the region of interest 703 and each pixel included in the reference region 705 based on the sum of the squares of the differences in the pixel values of the corresponding pixels. As shown in FIG. 7B, assuming that the pixels in the region of interest 703 are bs (p, q) and the pixels in the reference region 705 are bj (p, q) (j = 1-NS), the similarity Cj is an equation. It becomes (3).

The smaller the value of the similarity Cj, the more similar the pixel of interest and the reference pixel are. Here, the sum of the squared differences of the values between the corresponding pixels in the region of interest and the reference region is calculated as the degree of similarity Cj. However, in addition to the sum of squared differences, the sum of the absolute values of the differences and the evaluation function may be used.

Next, the weight for the reference pixel is derived according to the similarity. It may be determined that the smaller the similarity Cj is, the larger the weight is, and the larger the similarity Cj is, the smaller the weight is determined. For example, it is determined by the equation (4).

Wj = exp (-Cj / h ² ) (4)
Here, h is a variable for controlling the magnitude of the weight. Increasing the variable h enhances the noise reduction effect, but blurs the edges of the image data. As described above, all the pixels included in the reference pixel group 704 are set as reference pixels, and the weight for each reference pixel is determined. The pixel of interest itself is also set as a reference pixel.

Next, in step S311, the composition ratio calculation processing unit 222 calculates the composition ratio for image composition used by the image composition processing unit 223 from the edge signal detected by the single-layer noise suppression processing unit 221. .. The composition ratio calculation processing unit 222 calculates the composition ratio from the input edge intensity signal according to the relationship shown in FIG. That is, the edge signal is compared with the threshold values TH5 and TH6. Then, if the edge signal is smaller than the threshold value TH5, the composition ratio of the output signal of the single-layer noise suppression processing unit 221 is set to 0 among the two images to be combined, and if it is larger than the threshold value TH6, it is set to 1. The synthesis ratio is linearly calculated between the thresholds TH5 and TH6. The threshold values TH5 and TH6 in FIG. 8 may be the same values as the threshold values TH1 and TH2 in FIG. 5, or may be different values, respectively. That is, as the value of the edge signal is larger, the output signal of the single-layer noise suppression processing unit 221 including many high-frequency bands is prioritized and the synthesis ratio is increased.

Next, in step S312, the image composition processing unit 223 sets the output signal of the single-layer noise suppression processing unit 221 and the output signal of the band-specific noise suppression processing unit 200 according to the output signal of the composition ratio calculation processing unit 222. And synthesize, and output the final noise suppression result image. In the image composition processing, the output signal of the single-layer noise suppression processing unit 221 and the output signal of the band-specific noise suppression processing unit 200 are weighted for each pixel according to the composition ratio which is the output signal of the composition ratio calculation processing unit 222. An output image is generated by adding. Due to the above synthesis ratio, the resolution is maintained because the usage ratio of the output signal of the single-layer noise suppression processing unit 221 is high where the edge amount is large, and the output signal of the band-specific noise suppression processing unit 200 where the edge amount is small. Since the usage ratio of is high, it is possible to acquire a composite image having a high noise suppression effect.

As described above, according to the present embodiment, the output result of the noise suppression processing for each band and the output result of the single layer noise suppression processing are combined according to the edge strength, so that the noise has a higher resolution. It becomes possible to realize the reduction processing.
(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Digital camera, 101: Optical system, 102: Image sensor, 103: Pre-processing circuit, 104: A / D converter, 105: Image processing circuit, 106: Control circuit, 107: Display memory, 108: Display device , 109: Recording medium, 110: Operating member, 111: Bus

Claims (11)

A first noise suppression processing means for performing noise suppression processing on an input image without performing layering, and
A second noise suppression processing means that performs noise suppression processing while classifying the input image into layers using a reduced image.
A first to calculate a first synthesis ratio for synthesizing the output image of the first noise suppression processing means and the output image of the second noise suppression processing means according to the edge signal of the input image. Calculation processing means and
A first synthesis processing means for synthesizing the output image of the first noise suppression processing means and the output image of the second noise suppression processing means based on the first synthesis ratio.
With
The second noise suppression processing means includes a third noise suppression processing means that performs noise suppression processing different from the noise suppression processing performed by the first noise suppression processing means without reducing the input image, and the input. The fourth noise suppression processing means that reduces the image to perform noise suppression processing, the output image of the third noise suppression processing means, and the output image of the fourth noise suppression processing means are enlarged. An image processing apparatus comprising: a second compositing processing means for compositing the obtained output image. When the value of the edge signal of the input image is the second value, the first calculation processing means is the first value in which the value of the edge signal of the input image is smaller than the second value. The image processing apparatus according to claim 1, wherein the composition ratio of the output images of the first noise suppression processing means is made larger than that of the case. The second noise suppression processing means enlarges the output image of the third noise suppression processing means and the output image of the fourth noise suppression processing means at least according to the edge signal of the input image. The present invention according to claim 1 or 2, wherein the second calculation processing means for calculating the second synthesis ratio for synthesizing the output image obtained by performing the above-mentioned second synthesis processing means is provided. Image processing equipment. When the edge signal of the input image is a fourth value, the second noise suppression processing means has a third value in which the value of the edge signal of the input image is smaller than the fourth value. The second composite ratio is calculated so that the ratio of the output image of the third noise suppression processing means is larger than that of the third noise suppression processing means, and the edge signal of the input image is a sixth value larger than the fourth value. In some cases, the value of the edge signal of the input image is larger than the fourth value and smaller than the sixth value, which is smaller than that of the output image of the third noise suppression processing means. The image processing apparatus according to claim 3, wherein the second composite ratio is calculated so that the ratio becomes small. When the added edge signal obtained by adding the edge signal of the input image and the edge signal of the reduced image obtained by reducing the input image is the fourth value, the second noise suppressing processing means has a fourth value. The second composition is such that the ratio of the output image of the third noise suppression processing means is larger than that when the value of the edge signal of the input image is a third value smaller than the fourth value. When the ratio is calculated and the edge signal after the addition is a sixth value larger than the fourth value, the value of the edge signal of the input image is larger than the fourth value and is larger than the sixth value. 3. The third aspect of claim 3, wherein the second composite ratio is calculated so that the ratio of the output image of the third noise suppression processing means is smaller than when the fifth value is also small. Image processing device. The image processing apparatus according to any one of claims 1 to 5, wherein the output image of the first noise suppression processing means has a higher resolution than the output image of the second noise suppression processing means. .. The first noise suppression processing means detects the direction of the edge with reference to the peripheral pixels of the pixel to be corrected, performs noise suppression processing along the detected edge direction, and suppresses the second noise. The processing means is characterized in that noise suppression processing is performed using the values of the peripheral pixels of the pixels to be corrected and the values of the pixels whose difference from the values of the pixels to be corrected is within the threshold value. The image processing apparatus according to claim 6. The first noise suppression processing means obtains a weight corresponding to the degree of similarity between the peripheral pixels of the pixel to be corrected and the pixel group including the pixel to be corrected, and sets the weight. Noise suppression processing is performed using the values of peripheral pixels, and the second noise suppression processing means has a threshold value of the difference between the peripheral pixels of the pixels to be corrected and the values of the pixels to be corrected. The image processing apparatus according to claim 7, wherein noise suppression processing is performed using the values of pixels within. The first noise suppression processing step of performing noise suppression processing on the input image without performing layering, and
A second noise suppression processing step of performing noise suppression processing while dividing the input image into layers using a reduced image, and
A first combination ratio for synthesizing the output image of the first noise suppression processing step and the output image of the second noise suppression processing step according to the edge signal of the input image is calculated. Calculation process and
A first synthesis processing step of synthesizing the output image of the first noise suppression processing step and the output image of the second noise suppression processing step based on the first synthesis ratio.
Have,
The second noise suppression processing step is
A third noise suppression processing step that performs noise suppression processing different from the noise suppression processing performed in the first noise suppression processing step without reducing the input image, and a noise suppression processing that reduces the input image to perform noise suppression processing. The fourth noise suppression processing step, the output image of the third noise suppression processing step, and the output image obtained by enlarging the output image of the fourth noise suppression processing step are combined. An image processing method comprising two synthetic processing steps. A program for causing a computer to execute each step of the image processing method according to claim 9. A computer-readable storage medium that stores a program for causing a computer to execute each step of the image processing method according to claim 9.

Images