JP4886663B2 - Moving image noise reduction apparatus and moving image noise reduction program - Google Patents

Description

  The present invention relates to a moving image noise reduction device and a moving image noise reduction program for reducing moving image noise.

Conventionally, there is a general moving image noise reduction technique that focuses on differences in pixel values between successive frames to reduce moving image noise. For example, a digital filter is used to reduce the moving image noise. There is a method for removing image noise (for example, random noise) (see Non-Patent Document 1).
In this method, the characteristics of the digital filter can be changed by changing the program, and it is possible to cope with various types of noise.

In recent years, a TV (Total Variation) method (see, for example, Non-Patent Document 2) has been disclosed as a method for reducing noise in still images. This TV method is intended to reduce noise by repeatedly calculating a recurrence formula by numerical approximation so that the total variation of the luminance value of the image (Total Variation) is minimized. Incidentally, when reducing the noise of a still image, the number of repetitions of the recurrence formula calculation is about 100 times. It is known that when a TV method is used for a still image, a clear image can be obtained as compared with a conventional noise reduction method.
Yagi, Inoue, Hayashi, Okui, Koshi "Practical digital video processing learned in C language" Ohmsha 2001 Leoid I.I. Rudin, Stanley Osher and Emad Fatemi "Nonlinear total variation based noise removal algorithms" Physica D60 (1992) 259-268.

However, if the TV method is to be used to reduce the noise of moving images, the TV method was originally applied to a still image, and therefore, as the number of images (number of frames) increased There is a problem that the amount increases dramatically.
Further, in a general moving image noise reduction technique, for example, when pixel values between frames are averaged, there is a problem that blur occurs in a moving portion.

  Furthermore, in the TV method, there is a problem that a clear image may not be obtained due to a difference in parameters of the recurrence formula that is repeatedly calculated, and it is often difficult to set parameters in the recurrence formula. is there.

  Therefore, the present invention solves the above-described problem, facilitates parameter setting of equations used for calculation, suppresses increase in the amount of calculation, and reduces noise in moving images without causing blur in the moving portion. It is an object of the present invention to provide a moving image noise reduction apparatus and a moving image noise reduction program.

In order to solve the above-described problem, the moving image noise reduction device according to claim 1 is a moving image noise reduction device that outputs a noise-reduced moving image in which noise of an input moving image that has been input is reduced. Determining means, change amount determining means, and total variation method applying means, wherein the simplified total variation method is such that u is a pixel value, k is the number of times of application, i and j are x coordinates of pixels in the frame, and y coordinate, h is 1 / √ (horizontal resolution × vertical resolution), Δ ^x ₊ is the backward difference of the pixel value in the horizontal direction, Δ ^x ₋ is the forward difference of the pixel value in the horizontal direction, and Δ ^y ₊ is the vertical direction The pixel value is obtained by the following expression when the backward difference of the pixel value at Δ, Δ ^y ₋ is the forward difference of the pixel value in the vertical direction, and m is the minmod function.

According to such a configuration, the moving image noise reduction device detects the change in the luminance value of the pixel in the continuous frame of the input moving image by the region determination means, and the total value obtained by summing the change in the luminance value is the minimum. Is determined as a region for extracting the value of the noise dispersion parameter σ ² . The noise reduction device for moving image uses a change amount determining unit to calculate a pixel value before applying a simplified total variation method from a pixel value that has been applied n times in advance to the region determined by the region determining unit. When the subtracted variance value is equal to or greater than the value obtained by multiplying the parameter σ ² by a predetermined coefficient (greater than 0 and less than 1), and the pixel value before application is subtracted from the pixel value applied n-1 times. Is set to be smaller than a value obtained by multiplying the parameter σ ² by the predetermined coefficient, a change amount parameter Δt that defines the noise reduction width in the simplified total variation method is determined. Then, the moving image noise reduction apparatus uses the total variation method applying unit to calculate the simplified total variation method using the parameter Δt determined by the change amount determining unit, and to extract each region from which the region is extracted by the region determining unit. Applying it to the pixel value, a noise-reduced moving image in which noise is reduced from the initial noise is output.

The moving image noise reduction device according to claim 2 is a moving image noise reduction device that outputs a noise-reduced moving image in which noise of an input moving image that has been input is reduced, and includes an input unit and a frame difference acquisition unit. An edge extraction unit, a luminance average minimum block extraction unit, a change amount determination unit, and a total variation method application unit, wherein the simplified total variation method uses u as the pixel value, k as the number of applications, i And j are the x- and y-coordinates of the pixels in the frame, h is 1 / √ (horizontal resolution × vertical resolution), Δ ^x ₊ is the backward difference of the pixel value in the horizontal direction, and Δ ^x ₋ is the pixel value in the horizontal direction. the forward difference, directional difference after the pixel value delta ^y ₊ in the vertical direction, delta ^y _- forward difference of pixel values in the vertical direction, the pixel values by following expressions a upon the minmod function m And wherein the Mel.

According to this configuration, the moving image noise reduction device inputs the frame numbers of consecutive frames of the input moving image by the input unit. Then, the moving image noise reduction device acquires the difference between the luminance values of the pixels in the two frames corresponding to the frame number according to the frame number input by the input unit by the frame difference acquisition unit. Then, the moving image noise reduction device extracts edges for each of the two frames corresponding to the frame number by the edge extraction unit according to the frame number input by the input unit. Then, the moving image noise reduction device adds the difference acquired by the frame difference acquisition unit and the edge extracted by the edge extraction unit by the luminance average minimum block extraction unit, so that the average luminance value is minimized. Is a region in which the value of the noise variance parameter σ ² is extracted. Then, the moving image noise reduction device uses a simplified total variation method applied to a region determined by the luminance average minimum block extraction unit by the change amount determination unit, from a pixel value obtained by applying n times in advance. The variance value when the pixel value before application is subtracted is equal to or larger than the value obtained by multiplying the parameter σ ² by a predetermined coefficient, and the pixel value before application is subtracted from the pixel value applied n-1 times. A change amount parameter Δt that defines a reduction width of the noise in the simplified variation method is determined so that a variance value becomes smaller than a value obtained by multiplying the parameter σ ² by the predetermined coefficient. Then, the moving image noise reduction apparatus uses the parameter Δt determined by the change amount determining means by the total variation method applying means, and uses the simplified total variation method for each frame from which the area is extracted by the area determining means. Apply to pixel values and output a noise-reduced moving image.

  A moving image noise reducing device according to a third aspect is the moving image noise reducing device according to the first or second aspect, wherein the change amount determining means includes an adjusting means.

According to such a configuration, the moving image noise reduction device applies the i times when the simplified total variation method is applied to the region i times less than n times by the adjusting means of the change amount determining means. When the value obtained by subtracting the pixel value before application from the applied pixel value is equal to or greater than the value obtained by multiplying the parameter σ ² by a predetermined coefficient, the change amount used in the simplified total variation method for the i-th time is calculated. The value obtained by subtracting the pixel value before application from the pixel value applied n times is smaller than the value obtained by multiplying the parameter σ ² by a predetermined coefficient, while changing the parameter Δt to the amount of change smaller than the parameter Δt. In this case, the change amount parameter Δt is larger than the change amount parameter Δt used in the n-th simplified total variation method.

  The moving image noise reducing device according to claim 4 is the moving image noise reducing device according to any one of claims 1 to 3, wherein the total variation method applying means includes a plurality of arithmetic means. This computing means is connected in series.

  According to this configuration, the moving image noise reduction device performs the calculation by the simplified total variation method by the calculation means of the total variation method application means.

6. The moving image noise reduction program according to claim 5, wherein a computer is used to output a noise-reduced moving image in which the noise of the input moving image is reduced, the region determining means, the change amount determining means, and the total variation method. In the simplified total variation method, u is a pixel value, k is the number of times of application, i and j are x and y coordinates of pixels in the frame, and h is 1 / √ (horizontal resolution × vertical) Resolution), Δ ^x ₊ is the backward difference of pixel values in the horizontal direction, Δ ^x ₋ is the forward difference of pixel values in the horizontal direction, Δ ^y ₊ is the backward difference of pixel values in the vertical direction, and Δ ^y ₋ is vertical. It is characterized in that the pixel value is obtained by the following equation when the forward difference of the pixel value in the direction, m is a minmod function.

According to this configuration, the moving image noise reduction program detects the change in the luminance value of the pixel in the continuous frame of the input moving image by the region determination unit, and the total value obtained by summing the change in the luminance value is the minimum. A region consisting of a predetermined number of pixels is determined as a region from which the value of the noise dispersion parameter σ ² is extracted, and the total variation is simplified by the variation determination means with respect to the region determined by the region determination means. A pixel obtained by subtracting a pixel value before being applied from a pixel value to which the method is applied n times in advance is equal to or greater than a value obtained by multiplying the parameter σ ² by a predetermined coefficient, and a pixel having been applied n-1 times as the dispersion values at the time of subtracting the pixel value before applying the value is smaller than the value obtained by multiplying the predetermined coefficient to the parameter sigma ^2, the simplified variation method Determining the parameters Δt of variation defining a reduced width of definitive the noise. Then, the moving image noise reduction program uses the total variation method applying unit to calculate a simplified total variation method using the parameter Δt determined by the change amount determining unit, and to extract each region from which the region is extracted by the region determining unit. Applying it to the pixel value, a noise-reduced moving image in which noise is reduced from the initial noise is output.

  According to the first and fifth aspects of the present invention, by applying the simplified total variation method, the parameter setting can be easily performed, and the image value of the initial frame is not repeatedly used. It is possible to reduce the noise of the moving image while suppressing the increase of.

According to the second aspect of the present invention, it is possible to determine a region for extracting the value of the parameter σ ² of the noise variance by designating only the frame number and not designating the coordinates of the pixels in the frame. In addition, by applying the simplified total variation method, parameter setting can be easily performed, and the image value of the initial frame is not used repeatedly. Noise can be reduced.

  According to the invention described in claim 3, by appropriately changing the parameter Δt of the amount of change according to the number of times of applying the simplified total variation method, the appropriately simplified total variation method can be used. A moving image with reduced noise can be output.

  According to the fourth aspect of the invention, by performing the calculation by the simplified total variation method a plurality of times, it is possible to output a moving image with reduced noise while suppressing an increase in calculation amount.

Next, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
(Configuration of noise reduction device for moving images)
FIG. 1 is a block diagram of a moving image noise reduction apparatus. As shown in FIG. 1, the moving image noise reduction device 1 outputs an output moving image (noise reduced moving image) with reduced noise by inputting an input moving image. , A buffer 5, a change amount determination unit 7, a TV method application unit 9, and a control unit 11. In addition, the moving image noise reduction device 1 is used by the user of the device 1 to input a frame number, coordinates within the frame, the number of times of application of the simplified total variation method (hereinafter referred to as simplified TV method), and the like. A user input means (input means) 2 is provided as an interface.

  The moving image noise reduction apparatus 1 applies a simplified TV method to each frame of a moving image to reduce noise of the moving image while suppressing an increase in the amount of calculation. When the noise is reduced, the simplified TV method is applied while appropriately changing a change amount parameter Δt, which will be described later, so that the moving image is not blurred and the noise can be surely reduced. .

  When the input moving image of the moving image noise reduction apparatus 1 is a color moving image including the luminance component y and the color difference components u and v, it is sufficient to apply the simplified TV method only to the luminance component y. Noise reduction effect can be obtained. Of course, even if the simplified TV method is applied to the color difference components u and v, the noise reduction effect can be obtained in the moving image noise reduction device 1.

  Here, if the difference between the luminance value (that is, the pixel value of the luminance component) and the pixel value is clarified, in the case of a black and white moving image, the pixel value is composed only of the luminance component. Value is synonymous (same value). In the case of a color moving image, since the pixel value is composed of a luminance component and a color difference component, the pixel value means a pixel value of the luminance component or a pixel value of the color difference component, and strictly speaking, the luminance value is This is the pixel value of the luminance component.

The area determining means 3 determines an area composed of a predetermined number of pixels in the frame of the input moving image. The region composed of a predetermined number of pixels is a region composed of a minimum of four pixels. This region is used to extract the value of the noise variance parameter σ ² of the input moving image, and the variance of the pixel values in this region becomes the noise variance parameter σ ² . In general, the value of the parameter σ ² of noise dispersion added to the image is unknown. A configuration example of the area determination unit 3 will be described with reference to FIG.

  As shown in FIG. 2A, the region determining unit 3 includes a region extracting unit 31. This area extraction means 31 extracts the area s of the frame of the input moving image according to the frame number and the in-frame coordinates (represented by x-coordinate and y-coordinate, at least one coordinate) input from the user input means 2 It is.

  That is, the region extracting means 31 is for manually specifying the region s when both the frame number and the in-frame coordinates are input. That is, the region s is a flat region selected by the user as a result of actually viewing the frame of the input moving image (a region in which the luminance value can be estimated to be almost unchanged in a state where no noise is added). ) Or an area to be flattened (an area in which noise is reduced and a change in luminance value is reduced as much as possible).

  The area extraction unit 31 specifies the frame of the input moving image by the frame number, and when one in-frame coordinate of the specified frame is specified, the number of pixels is centered on the specified in-frame coordinate. Extracts a region s in a predetermined number of ranges. When a plurality of in-frame coordinates are designated, the region s is extracted so as to include the designated in-frame coordinates. When three or more in-frame coordinates are designated, a range surrounded by the in-frame coordinates is extracted as the region s.

  FIG. 2B is a diagram showing another embodiment of the region determining means 3. As shown in FIG. 2B, the region determination unit 3 includes a frame difference acquisition unit 32, an edge extraction unit 33, an addition unit 34, and a luminance average minimum block extraction unit 35.

  As shown in FIG. 2B, the configuration example of the area determining unit 3 can determine the area s automatically when only the frame number is input from the user input unit 2. Is. That is, the region s is a flat region as a result of automatic extraction of a region with little change in a continuous frame at least one frame before or after and almost no change in the frame.

  The frame difference acquisition unit 32 acquires a change in luminance value of pixels (difference in luminance value) between consecutive frames of the input moving image. The frame difference acquisition unit 32 acquires a difference in luminance value between frames for all pixels in the frame for a frame (for example, frame number 5 and frame number 6) that is continuous with the frame having the designated number. Note that the method of acquiring the luminance value difference by the frame difference acquisition unit 32 may be an existing method. Further, the luminance value difference is, for example, an absolute value difference. Then, the frame difference acquisition unit 32 outputs the luminance value difference to the addition unit 34.

  The edge extracting means 33 extracts an edge of a frame having a designated number in the input moving image. Note that an existing method may be used as an edge extraction method. For example, an edge may be extracted using a Laplacian filter. Then, the edge extraction unit 33 outputs the extracted edge (the coordinates of the edge portion, the pixel value) to the addition unit 34.

  The adding unit 34 adds the luminance value difference acquired by the frame difference acquiring unit 32 and the edge extracted by the edge extracting unit 33. The adding means 34 is composed of a general adder. The adding unit 34 outputs the addition result to the luminance average minimum block extracting unit 35.

  The luminance average minimum block extraction unit 35 extracts a region s (block region) in which the average of the luminance values of the pixels is minimum from the region for one frame that is the result of addition by the addition unit 34. . That is, this block region can be said to be a portion that is a flat region estimated to be the flattest in the frame. Returning to FIG.

  In this embodiment, it is assumed that at least the user of the apparatus 1 inputs a frame number in the area determination unit 3, but a predetermined frame (for example, the first frame or scene) of the input moving image is always used. A region may be determined from a frame in which a change is detected.

  The buffer 5 temporarily stocks each frame of the input moving image, and associates the region s in each frame determined by the region determining unit 3 by stocking the corresponding frame together, thereby applying the TV method applying unit. 9 is output.

The change amount determining means 7 satisfies the conditional expression using the noise dispersion parameter σ ² when the simplified TV method is applied to the area s determined by the area determining means 3 n times in advance. A parameter Δt of such a change amount is determined.

It can be said that the change parameter Δt defines the amount of noise reduction in the region s. When the change amount parameter Δt is increased, the input moving image is likely to be flat (the change in luminance value is small, so that noise appears to be small). On the other hand, the image is blurred at the same time. It has the property of occurring. Therefore, the change amount determining means 7 determines a change amount parameter Δt that is as small as possible and satisfies the conditional expression using the noise dispersion parameter σ ² . The change amount parameter Δt is obtained so that the moving image before removing the noise and the moving image after removing the noise can maintain the noise dispersion parameter σ ² within a predetermined range.

  The simplified TV method applies the total variation method to the (n + 1) th time using the pixel value of the initial frame, which is the initial frame of the input moving image as the normal total variation method, and the pixel value to which the n times total variation method is applied. This is a method of calculating the pixel value to which the total variation method is applied n + 1 times by using the pixel value to which the total variation method is applied n times instead of the pixel value of the initial frame.

  The preset number of times n is the number of times that the simplified TV method is applied to the pixel values of the pixels included in the region s, that is, the number of iterations. In this embodiment, n is set to 30 or less. From the experimental results and the like, the noise of the input moving image can be sufficiently reduced in about 10 times in practice.

  The mathematical formula used in the simplified TV method will be described using the following mathematical formula (1) and mathematical formula (2).

Formula (1) is a recurrence formula used in the normal TV method. In Equation (1), u _ij ^{n + 1} indicates the result of applying the TV method n + 1 times for the pixel value of the coordinates (i, j), and if u _ij ^n, it indicates that the TV method has been applied n times. , N + 1 is 0, that is, the pixel value of the initial frame to which the TV method is not applied (pixel value of the original image with noise).

  In Equation (1), h is a parameter indicating the resolution of a frame to which the TV method is applied. In Formula (1), when horizontal resolution (number of horizontal pixels) = vertical resolution (number of vertical pixels) = N, h = 1 / N. Many moving image frames have different horizontal and vertical resolutions. As a result, it is assumed that h = 1 / √ (horizontal resolution × vertical resolution). For example, in the case of SD resolution, h = 1 / √ (720 × 480) ≈0.0017.

Further, in Equation (1), Δ ^x ₊ is a backward difference in pixel values in the horizontal direction, Δ ^x ₋ is a forward difference in pixel values in the horizontal direction, Δ ^y ₊ is a backward difference in pixel values in the vertical direction, Δ ^y ₋ indicates the forward difference of the pixel value in the vertical direction.

Furthermore, in Equation (1), m is a limiting function minmod. In this limiting function minmod, sgnx is an sgn function with an argument x, and sgny is an sgn function with an argument y. In Equation (1), the term including λ represents a constraint condition that the variance of noise added to the image is the parameter σ ² . Then, instead of removing the term including λ, the change amount parameter Δt is obtained as described above.

Equation (2) is a recurrence formula used in the simplified TV method. In Equation (2), k (number of times the simplified TV method is applied) is used instead of n, and the pixel value obtained by applying the simplified TV method k + 1 times for the pixel value at the coordinates (i, j). When u _ij ^{k + 1} is calculated, the noise dispersion parameter σ ² is not necessary.

  Furthermore, calculation of a term including λ becomes unnecessary, and the amount of calculation can be reduced as compared with the calculation of the recurrence formula of the conventional TV method (calculation can be simplified). This is obtained from a result of an experiment for reducing noise on an input moving image to which noise is actually added. That is, the influence of the change amount parameter Δt is larger than the influence of the λ term, and the simplified TV method can sufficiently reduce noise even if the λ term in the conventional TV method is ignored (omitted). Can be planned.

However, even if the simplified TV method is used, the noise dispersion parameter σ ² cannot be completely ignored, and the change amount determining means 7 applies this simplified TV method to the region s. The change amount parameter Δt is determined so as to satisfy a constraint condition (conditional expression) in which the variance obtained by subtracting the pixel value u ⁰ of the initial frame from the pixel value u ⁿ applied n times satisfies σ ² .

In the change amount determination means 7, the change amount parameter Δt in the region s is obtained by the following calculation using a preset number of iterations n. In this embodiment, n = 15.
The conditional expression is expressed by the following expression (3) when the change amount parameter Δt = Δt ₀ is set.

  In Equation (3), α is a constant value of 0 <α <1 for determining a threshold value for determining whether the change amount parameter Δt is larger or smaller than the current value. 0.9. The value of α is set to 0.9 so that the following equation (4) is satisfied when the image becomes flat (a state in which the added noise is reduced and there is almost no change in luminance value). (In other words, α = 1) is ideal, but in reality, the image cannot be expected to converge to a flat state. If the value of α is decreased, an image with less blur (output moving image) tends to be obtained, but at the same time, noise may be insufficiently reduced.

  In the change amount determining means 7, h in the above-described equation (1) may be adopted as the initial value of the change amount parameter Δt. The initial value can be given an arbitrary value.

Then, the change amount determining means 7 is the change amount parameter Δt = Δt ₀ when the conditional expression shown in Equation (3) is satisfied. When i <n and the following formula (5) is satisfied, the change amount parameter Δt is set to a value smaller than the current value. For example, Δt = a * Δt (0 <a <1).

  Then, the change amount determining means 7 sets the change amount parameter Δt to a value larger than the current value when the following formula (6) is obtained. For example, Δt = b * Δt (b> 1, b * a ≠ 1).

  The TV method applying means 9 uses the change amount parameter Δt determined by the change amount determining means 7 to perform the simplified TV method using a frame and a series of frames extracted from the area s determined by the area determining means 3. This is applied to pixel values and outputs an output moving image (noise-reduced moving image) in which noise is reduced from the beginning.

Here, with reference to FIG. 3, the detailed structure of the variation | change_quantity determination means 7 and the TV method application means 9 is demonstrated.
As shown in FIGS. 3A and 3B, the change amount determining means 7 includes a Δt adjusting means 71, and the TV method applying means 9 includes simplified TV method executing means 9 ₁ , 9 ₂ , 9 _3,..., has a _{9 n.}

3A shows the operation when determining the change amount parameter Δt in the change amount determining means 7 together with the detailed configuration. FIG. 3B shows the operation in the TV method applying means 9. The operation when applying the simplified TV method is shown together with a detailed configuration. That is, the change parameter Δt in FIG. 3A is used when the TV method applying means 9 is tried, whereas the change parameter Δt ₀ in FIG. 3B is obtained as a result of the trial. Therefore, the TV method applying means 9 is used when calculating the output moving image μ _n from the input moving image μ ₀ .

In FIG. 3A, the simplified TV method is repeatedly applied to the region s by the simplified TV method executing means 9 _{1 to} 9 _n while performing the fine adjustment of the change amount parameter Δt by the Δt adjusting means 71 (trial). ) To obtain the parameter Δt ₀ (see FIG. 3B) of the amount of change when the simplified TV method is applied to the input moving image μ ₀ (see FIG. 3B) for which noise is actually desired to be reduced. It shows the flow at the time.

That is, the change amount parameter Δt ₀ is set to another change amount parameter Δt after all the results of applying the simplified TV method by the simplified TV method execution units 9 _{1 to} 9 _n are input to the Δt adjustment unit 71. And is obtained by this repetition.

  The Δt adjusting unit 71 stores the conditional expressions from the above formulas (3) to (6), and the pixel value obtained by applying the simplified TV method by the simplified TV method executing unit 9 to the region s. The change amount parameter Δt is output.

That is, in the Δt adjustment means 71, simplified TV method execution unit _{9 1,} 9 _2, 9 3, ..., corresponding to the pixel value output from the _{9 n,} and determine not meet or satisfy the condition in the area s result, by adjusting the amount of change of the parameter Delta] t, the adjusted Delta] t, simplified TV method execution unit _{9 1,} 9 _2, 9 3, ..., and outputs the respective _{9 n.}

The simplified TV method execution means 9 ₁ , 9 ₂ , 9 ₃ ,..., 9 _n are configured by pipelines connected in series, and the simplified TV method execution unit 9 ₁ , 9 ₂ , 9 ₃ ,. One dose is applied. The pipeline stage number n is configured to be equal to the number of iterations n. Therefore, the n-th stage output of the pipeline is obtained by applying the simplified TV method n times. Returning to FIG.

  The control unit 11 controls the moving image noise reduction device 1 and outputs a control signal for controlling the units 3 to 9 to the units 3 to 9 and the units 3 to 9. The output signal representing the processing state in is received.

According to the moving image noise reduction device 1, TV method simplified TV method execution unit ₉ first application means _9, 9 _2, 9 3,..., The _{9 n,} by applying the simplified TV method, Since the image value of the initial frame is not used repeatedly, it is possible to suppress an increase in the amount of calculation and reduce the noise of the moving image.

Further, the moving image noise reduction device 1 extracts the value of the noise dispersion parameter σ ² by designating only the frame number by the region determination means 3 and not designating the coordinates of the pixels in the frame. An area can be determined.

  Furthermore, the moving image noise reduction apparatus 1 uses the simplified TV method appropriately by changing the change amount parameter Δt according to the number of times the simplified TV method is applied by the change amount determining means 7. It is possible to output a moving image with reduced noise.

(Operation of noise reduction device for moving images)
Next, the operation of the moving image noise reduction apparatus 1 will be described with reference to the flowcharts shown in FIGS. 4 and 5 (see FIG. 1 as appropriate). FIG. 4 is a flowchart showing the overall operation of the moving image noise reduction apparatus 1, and FIG. 5 is a flowchart showing the operation when determining the change amount parameter Δt.

As shown in FIG. 4, first, the moving image noise reduction apparatus 1 determines the region s in the frame of the input moving image by the region determining means 3 (step S1). Subsequently, the moving image noise reduction device 1 calculates a noise dispersion parameter (dispersion of pixel values in the area s) σ ^{2 in the} determined area s (step S2).

  Then, the moving image noise reduction device 1 determines the change amount parameter Δt for the region s by the change amount determination means 7 while applying the simplified TV method (step S3). Then, as shown in FIG. 3B, the moving image noise reduction apparatus 1 applies the simplified TV method to each frame by the TV method applying means 9, and outputs an output moving image (step). S4).

  Note that the noise of the input moving image is assumed to have the same property in a series of frames, for example, until a scene change is detected. Therefore, in the series of frames, the parameter Δt having the same value of change may be used. The change parameter Δt is derived from, for example, the first frame.

  As shown in FIG. 5, the moving image noise reduction apparatus 1 uses the change amount determining means 7 shown in FIG. 3A to change the simplified TV method with the TV method applying means 9 using the change amount parameter Δt. When the number of times is applied, k that satisfies the conditional expression (substitute n = k in Expression (3)) is obtained (step S11). This conditional expression is expressed by the following mathematical expression (7).

  Then, the moving image noise reduction apparatus 1 determines whether or not the number of times k is larger than the preset number of times n (step S12). If it is determined that the number is larger (Yes in step S12), the change amount parameter Δt is set to a value larger than the current value (step S13), and the process returns to step S11.

  In addition, when it is not determined in step S12 that the number of times k is greater than the preset number n (No in step S12), the moving image noise reduction device 1 has the number of times k greater than the preset number n. It is determined whether or not the value is small (step S14). If it is determined that the value is small (Yes in step S14), the change amount parameter Δt is set to a value smaller than the current value (step S15), and the process returns to step S11.

  If it is not determined in step S14 that the number of times k is smaller than the preset number n (No in step S14), k == n (k is equal to n). The change amount parameter Δt is determined (step S16).

  In the operation of the moving image noise reduction apparatus 1, the determination that the number k is actually larger than the preset number n (Yes in step S <b> 12) is satisfied by satisfying the condition of the above-described equation (6). The determination that the number of times k is smaller than the number of times n set in advance (Yes in step S14) is made by satisfying the condition of Equation (5) described above.

(Application example of noise reduction device for moving images)
An application example of the moving image noise reduction device 1 of the present invention will be described.
For example, the moving image noise reduction device 1 is suitable for use as a pre-filter of a video encoder. This is because it has been conventionally known that encoding efficiency is improved by reducing noise as preprocessing for video encoding.

  Further, the moving image applied noise reduction device 1 of the present invention is suitable for use in moving images taken with a high sensitivity camera. The reason for this is that for moving images (high-sensitivity camera images) shot with a high-sensitivity camera, thermal noise, which is Gaussian noise, and the number of photons captured by the photoreceptor in the high-sensitivity camera when shooting with the high-sensitivity camera This is because it includes random noise having different properties such as shot noise generated by fluctuations in the frequency. And it has been confirmed by experiments that the moving image applied noise reduction device 1 of the present invention can reduce thermal noise and shot noise.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. For example, although the present embodiment has been described as the moving image noise reduction device 1, it may be configured as a moving image noise reduction program in which the operation of each unit is described to be executable.

It is a block diagram of the noise reduction apparatus for moving images which concerns on embodiment of this invention. It is the figure which showed the structural example of the area | region determination means. It is the figure which showed the structural example of the variation | change_quantity determination means and the TV method application means. 3 is a flowchart for explaining an overall operation of the moving image noise reduction apparatus shown in FIG. 1. It is a flowchart for demonstrating the operation | movement at the time of determining parameter Δt of variation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Moving image noise reduction apparatus 3 Area | region determination means 5 Buffer 7 Change amount determination means 9 TV method application means 11 Control means

Claims (5)

A noise reduction device for a moving image that outputs a noise-reduced moving image in which noise of an input moving image is reduced,
A difference between the luminance values of pixels in two consecutive frames of the input moving image is obtained, and an area composed of a predetermined number of pixels having a minimum total value of the luminance value differences is defined as the variance of the noise. An area determining means for determining the value of the parameter σ ² as an area to be extracted;
A variance value obtained by subtracting a pixel value before application of a simplified total variation method, which has been applied to the region determined by the region determination unit, from a preset pixel value n times is the parameter σ ^2. And a variance value obtained by subtracting the pixel value before application from the pixel value applied n-1 times is smaller than the value obtained by multiplying the parameter σ ² by the predetermined coefficient. As described above, a change amount determining means for determining a change amount parameter Δt that defines a reduction range of the noise in the simplified variation method,
Using the parameter Δt determined by the change amount determining means, the simplified total variation method is applied to the pixel value of each frame from which the area is extracted by the area determining means, and the total noise output video is output. Variation method application means,
In the simplified total variation method, u is a pixel value, k is the number of application times, i and j are x and y coordinates of a pixel in the frame, h is 1 / √ (horizontal resolution × vertical resolution), and Δ ^x ₊ is The backward difference of pixel values in the horizontal direction, Δ ^x ₋ is the forward difference of pixel values in the horizontal direction, Δ ^y ₊ is the backward difference of the pixel values in the vertical direction, and Δ ^y ₋ is the forward direction of the pixel value in the vertical direction. A moving image noise reduction apparatus characterized in that a pixel value is obtained by an expression shown below when a difference, m is a minmod function.

A noise reduction device for a moving image that outputs a noise-reduced moving image in which noise of an input moving image is reduced,
Input means for inputting frame numbers of consecutive frames of the input moving image;
In accordance with the frame number input by the input means, frame difference acquisition means for acquiring a difference between the luminance values of the pixels in the two frames corresponding to the frame number;
Edge extraction means for extracting an edge for each of two frames corresponding to the frame number in accordance with the frame number input by the input means;
By adding the difference acquired by the frame difference acquisition means and the edge extracted by the edge extraction means, the block having the smallest average luminance value is determined as the value of the noise dispersion parameter σ ² . Luminance average minimum block extraction means as an area to be extracted;
A variance value obtained by subtracting the pixel value before applying the simplified total variation method, which has been simplified, for the region determined by the luminance average minimum block extracting unit from the preset pixel value n times the parameter sigma ² with greater than or equal to a value obtained by multiplying a predetermined coefficient, the value variance at the time of subtracting the pixel value before applying the n-1 times the applied pixel value is multiplied by the predetermined coefficient to the parameter sigma ² Change amount determining means for determining a change amount parameter Δt that defines a reduction width of the noise in the simplified variation method,
Using the parameter Δt determined by the change amount determining means, the simplified total variation method is applied to the pixel value of each frame from which the area is extracted by the area determining means, and the total noise output video is output. Variation method application means,
In the simplified total variation method, u is a pixel value, k is the number of application times, i and j are x and y coordinates of a pixel in the frame, h is 1 / √ (horizontal resolution × vertical resolution), and Δ ^x ₊ is The backward difference of pixel values in the horizontal direction, Δ ^x ₋ is the forward difference of pixel values in the horizontal direction, Δ ^y ₊ is the backward difference of the pixel values in the vertical direction, and Δ ^y ₋ is the forward direction of the pixel value in the vertical direction. A moving image noise reduction apparatus characterized in that a pixel value is obtained by an expression shown below when a difference, m is a minmod function.

The change amount determining means is a value obtained by subtracting a pixel value before application from the pixel value applied i times when the simplified total variation method is applied less than n times i times to the region. Is equal to or greater than a value obtained by multiplying the parameter σ ² by a predetermined coefficient, the change amount parameter Δt is smaller than the change amount parameter Δt used in the i th simplified total variation method. In addition, when the value obtained by subtracting the pixel value before application from the pixel value applied n times is smaller than the value obtained by multiplying the parameter σ ² by a predetermined coefficient, the simplified total variation of the nth time 3. The moving image for moving image according to claim 1, further comprising adjusting means for changing the parameter Δt of the amount of change larger than the parameter of change Δt used in the method. It is reducing device.   4. The total variation method applying means includes a plurality of calculation means for performing calculations according to the simplified total variation method, and the calculation means are connected in series. The noise reduction apparatus for moving images described in 1. In order to output a noise-reduced moving image with reduced noise in the input moving image,
A difference between the luminance values of pixels in two consecutive frames of the input moving image is obtained, and an area composed of a predetermined number of pixels having a minimum total value of the luminance value differences is defined as the variance of the noise. Area determination means for determining the value of parameter σ ² as an area to be extracted;
A variance value obtained by subtracting a pixel value before application of a simplified total variation method, which has been applied to the region determined by the region determination unit, from a preset pixel value n times is the parameter σ ^2. And a variance value obtained by subtracting the pixel value before application from the pixel value applied n-1 times is smaller than the value obtained by multiplying the parameter σ ² by the predetermined coefficient. A change amount determining means for determining a change amount parameter Δt that defines the noise reduction width in the simplified variation method,
Using the parameter Δt determined by the change amount determining means, the simplified total variation method is applied to the pixel value of each frame from which the area is extracted by the area determining means, and the total noise output video is output. Function as a variation method application means,
In the simplified total variation method, u is a pixel value, k is the number of application times, i and j are x and y coordinates of a pixel in the frame, h is 1 / √ (horizontal resolution × vertical resolution), and Δ ^x ₊ is The backward difference of pixel values in the horizontal direction, Δ ^x ₋ is the forward difference of pixel values in the horizontal direction, Δ ^y ₊ is the backward difference of the pixel values in the vertical direction, and Δ ^y ₋ is the forward direction of the pixel value in the vertical direction. A moving image noise reduction program characterized in that a pixel value is obtained by the following equation when the difference, m is a minmod function.

Images