JP3070041B2 - Video / still image converter - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a television camera, a VT
The present invention relates to a moving image / still image conversion device for outputting a moving image input from R or the like as a still image to a CRT or a printer.

[0002]

2. Description of the Related Art Conventionally, image signals applied to a printer or the like have been mainly input from a still image input device such as an image scanner.
A wide variety of video equipment such as video cameras and VTRs have been used. In addition, with the increase in definition of an image of a high-definition television or the like, there is a demand for printing a high-definition television image by a printer or a printing device.

[0003]

However, since the input image of the television camera is a moving image, while the output image of the printer is a still image, a device for converting the moving image into a still image is required. . At the time of moving image / still image conversion, there is a problem in that noise components are more visually conspicuous in a still image than in a moving image. That is, in a moving image, noise in one frame is difficult to see due to the integration effect in the time axis direction of the visual system, whereas in a still image from which one frame is extracted, there is no visual system integration effect, and image quality degradation due to noise is conspicuous. .

[0004] Noise removal methods include inter-frame smoothing and intra-frame smoothing, but they have advantages and disadvantages. That is, in the inter-frame smoothing, if there is a moving portion in the image, the image position between the frames shifts. Therefore, not only a portion from which noise is not removed remains but also an unclear portion such as blur or color blur occurs. In the intra-frame smoothing, high-frequency components are removed, so that an edge portion of an image becomes unclear. Various filtering methods for preserving edges and performing intra-frame smoothing have been proposed. However, since only information within the frame is used, noise removal is insufficient.

Accordingly, an object of the present invention is to provide a moving picture / still picture conversion apparatus which solves such a problem.

[0006]

According to the present invention, there is provided a moving image / still image conversion apparatus according to the present invention, wherein an image memory capable of storing a predetermined number of images is freely movable, and at least two images stored in the image memory are moved for each attention area. Detecting means for detecting the amount and the edge amount; and noise reducing means for performing noise reduction processing on the image stored in the image memory at a rate of two-dimensional spatial processing and processing in the time axis direction according to the detection result of the detecting means. Wherein the noise reduction means comprises a specified value holding means for freely changing a specified value for defining the degree of the time axis direction processing and the two-dimensional spatial processing for the motion amount and the edge amount.

[0007]

According to the above means, the ratio of the processing in the time axis direction and the two-dimensional spatial processing in accordance with the amount of motion and the amount of edge of the image,
Since noise removal is performed, it is possible to preserve or enhance the edges and to obtain a still image in which image quality deterioration such as blurring or color blur is not visually noticeable. Further, since the degree of the processing in the time axis direction and the degree of the two-dimensional spatial processing can be changed, the noise reduction processing according to the target image can be easily performed.

[0008]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention and its peripheral devices. Reference numeral 10 denotes a moving image / still image conversion device according to an embodiment of the present invention, 12 denotes a moving image input device such as a television camera, a video camera, or a VTR, and 14 denotes a still image output device such as a color printer or a color monitor. It is.

In the moving picture / still picture conversion apparatus 10 of the present embodiment, 16 is an interface with the moving picture input apparatus 12, 18 is an interface with the still picture output apparatus 14, 20 is an image data bus, and 22 is a control signal bus. , 2
4 is a control circuit for controlling the whole; 26 is an image memory capable of storing several frames of moving images in frame units; 28 is a motion amount detecting circuit for detecting the amount of motion of images between frames;
Is an edge amount detection circuit for detecting an edge amount in a frame. Reference numeral 32 denotes a noise removal circuit, which includes an intra-frame processing circuit 34 for performing noise reduction processing within a frame and an inter-frame processing circuit 36 for performing noise reduction processing between frames. Reference numeral 38 denotes an operation device including a key switch for inputting various operation instructions to the moving image / still image conversion device 10 (specifically, the control circuit 24).

FIG. 2 is a detailed circuit block diagram of a portion including the motion amount detecting circuit 28, the edge amount detecting circuit 30, and the noise removing circuit 32. Hereinafter, an example will be described in which one screen is divided into blocks each including 3 × 3 pixels and noise removal processing is performed in units of the blocks. In the edge amount detection circuit 30, reference numeral 40 denotes an image Ai of a block of interest.
A minimum value detector for detecting the minimum value Amin in j, a maximum value detector for detecting the maximum value Amax, and a difference device 44 for calculating a difference As between the maximum value Amax and the minimum value Amin. In the motion amount detection circuit 28, reference numeral 46 denotes a frame difference image (Aij-) between the image Aij of the target block of the first frame and the image Bij at a position corresponding to the image Aij in the second frame (for example, the frame following the first frame). B
ij), and 48 is a frame difference image (Ai)
j-Bij), a minimum value detector for detecting the minimum value Smin, 50 a maximum value detector for detecting the maximum value Smax, 5
Reference numeral 2 denotes a differentiator for calculating a difference Ss between the maximum value Smax and the minimum value Smin.

In the intra-frame processing circuit 34 of the noise removal circuit 32, reference numeral 54 denotes an output As of the edge amount detection circuit 30.
, A multiplier 56 for multiplying the output Sij of the inter-frame processing circuit 36 by the coefficient Kij, and an adder 58 for adding the output of the multiplier 56. In the inter-frame processing circuit 36, reference numeral 60 denotes a coefficient generator for generating a coefficient T indicating the degree of inter-frame processing based on the output Ss of the motion amount detection circuit 28, and 62 denotes an output (Aij-Bij) of the differentiator 46. Is multiplied by a coefficient T, and 64 is a differentiator for subtracting the output of the multiplier 62 from the image Aij.

The noise removal circuit 32 also has an edge amount A
A default memory 66 for storing a predetermined relation table of s and coefficient Kij and a motion amount Ss and coefficient T, and a modification memory 68 for storing a relation table obtained by modifying the relation table.
Is provided. The coefficient T indicates the degree of inter-frame processing, and the coefficient Kij
Indicates the degree of intra-frame processing. Default memory 6
The user can select which of the modification memory 6 and the modification memory 68 to use via the control circuit 24 by the operation device 38, and the user can arbitrarily change the storage contents of the modification memory 68. .

The operation of this embodiment will be described. When the user inputs a moving image input request using the operation device 38, the control circuit 24
And activates the image memory 26 to store the image data
An access right to the bus 20 is given. Interface 1
6 fetches a moving image signal from the moving image input device 12 in synchronization with a synchronizing signal such as a frame synchronizing signal, and outputs it on the image data bus 20. Of course, if the output signal of the moving image input device 12 is an analog signal, the interface 16 converts the output signal into a digital signal and converts the signal into a digital signal.
Output to the top. The image memory 26 stores the image data bus 2
The image data on 0 is sequentially captured and stored for several frames (at least two frames) in frame units. A predetermined number of frames of image data are stored in the image memory 26.
The control circuit 24 makes the interface 16 and the image memory 26 non-active via the control bus 22. Thus, the input of the moving image is completed.

Next, when the user inputs a request for conversion into a still image from the operation device 38, the moving image / still image conversion device 10
Forms a still image from which noise has been removed from a plurality of frame images stored in the image memory 26, and outputs the still image to the still image output device 14. That is, the control circuit 24 controls the control bus 22
To activate the image memory 26, the motion amount detection circuit 28, the edge amount detection circuit 30, and the noise removal circuit 32 to give the right to access the image data bus 20.
The image Ai of the target block of the first frame in the image memory 26
j is supplied to the motion amount detection circuit 28 and the edge amount detection circuit 30, and the image Bij of the second frame is supplied to the motion amount detection circuit 28.

In the edge amount detection circuit 30, the minimum value detector 40 detects the minimum value Amin in the block of interest, the maximum value detector 42 detects the maximum value Amax, and the differentiator 44 detects the maximum value Amax and the minimum value. A difference As from Amin is output. The output of the differentiator 44, that is, the output As of the edge amount detection circuit 30, indicates the degree of dispersion of the image Aij of the block of interest. In the motion amount detection circuit 28, the differentiator 46
The difference image between the frame and the second frame (Aij-Bi
j), the minimum value detector 48 detects the minimum value Smin of the difference image in the block of interest, the maximum value detector 50 detects the maximum value Smax of the difference image, and the differentiator 52 determines the maximum value Smax and the minimum value. The difference Ss from the value Smin is output. The output of the differentiator 52, that is, the output Ss of the motion amount detection circuit 28
Represents the degree of dispersion of the difference image (Aij-Bij).

The edge amount As, the motion amount Ss, and the difference image (A
ij-Bij) and the image Aij are applied to the noise removal circuit 32. Although not shown, the input of the noise elimination circuit 32 is temporarily held by an appropriate latch circuit or a buffer memory.

In the noise elimination circuit 32, first, the edge amount As and the coefficient Kij are read from the parameter memory 70.
Is read out and set in the coefficient generator 54, and a relational table between the motion amount s and the coefficient T is read out and set in the coefficient generator 60. At this time, the control circuit 24 determines whether to use the relation table stored in the default memory 66 or the relation table stored in the modification memory in accordance with an operation instruction from the operation device 38.

The coefficient generator 60 includes a parameter memory 70
Is applied to the relation table read and set from the difference table 52, the corresponding coefficient T (coefficient indicating the degree of inter-frame processing) is generated, and the multiplier 62 outputs the difference image (Aij-Bij). ) Multiplied by a coefficient T, T (Aij
-Bij). The differentiator 64 converts the image Aij to T
Sij ｛= Aij−T obtained by subtracting (Aij−Bij)
(Aij-Bij)} is output. The Sij thus obtained is an image that has been subjected to inter-frame processing. Sij
Is applied to the intra-frame processing circuit 34, where the intra-frame processing corresponding to the edge amount As is performed. That is, the coefficient generator 54 applies the edge amount As from the differentiator 44 to the relation table read and set from the parameter memory 70, and outputs the corresponding coefficient Kij (coefficient indicating the degree of intra-frame processing). , A multiplier 56 multiplies the output Sij of the inter-frame processing circuit 36 by a coefficient Kij for each pixel.
The adder 58 integrates the product Kij · Sij for each pixel in the block of interest and outputs P ₂₂ (= ΣKij · Sij).
P ₂₂ corresponds to the center pixel value that has been processed in a frame in the block of interest.

Coefficient Ki generated by coefficient generators 54 and 60
Examples of j and T are shown in FIGS. 3 and 4, respectively.

[0021] In FIG. 3, the coefficient K ₂₂ and Ka for the center pixel in the block of interest among the coefficients Kij, (in this example 8 pixels) other peripheral pixels in the block of interest and the coefficients for the coefficient Kb of the same value . The sum of the coefficients in the block of interest is set to 1, that is, Ka + Kb × 8 = 1.
It is standardized so that As the edge amount As increases, the coefficient Ka increases, the coefficient Kb decreases, and the degree of intra-frame processing decreases. In particular,
Coefficient Ka is between the edge amount As is 0 predetermined value As _1,
Kb is 1/9, and the output P of the noise removal circuit 32 is
₂₂ becomes ΣSij / 9, which corresponds to intra-frame averaging. If the edge amount As is larger than the predetermined value As ₂ , Ka becomes 1, Kb becomes 0, and P ₂₂ becomes S ₂₂ , which corresponds to no intra-frame processing. Between the edge amount As from As ₁ of As _2, coefficient Ka
Takes a continuous value between 1/9 and 1, and the coefficient Kb is 1
/ 9 take continuous values between 0, P ₂₂ is ShigumaSij /
Continuously varies between S ₂₂ from 9, which is equivalent to the degree of the frame processing is continuously changed.

In FIG. 4, as the amount of movement Ss increases, the coefficient T
Becomes smaller, and the degree of inter-frame processing becomes smaller. Specifically, during the motion amount Ss is 0 predetermined value Ss _1,
The coefficient T = 1/2, so Sij is (Aij + Bi
j) / 2, which corresponds to inter-frame averaging. If the motion amount Ss is larger than the predetermined value Ss _2, Sij = Aij becomes a factor T = 0, which corresponds to not being processed between frames. Between motion amount Ss from Ss ₁ of Ss _2, coefficient T takes a continuous value between 1/2 of 0, Sij from (Aij + Bij) / 2 Ai
It changes continuously in the range of j. This means that the degree of inter-frame processing changes continuously.

The noise elimination circuit 3 thus obtained
Second output P ₂₂ is stored in the output frame memory area of the image memory 26 via the image data bus 20. Thereafter, by shifting the block of interest one pixel at a time and performing the above processing on the entire image area of the frame, the noise-reduced frame image is stored in the output frame storage area of the image memory 26. However, strictly speaking, the outermost one pixel of the frame is a singular case.

When the noise reduction processing is completed, the control circuit 2
Reference numeral 4 denotes an output stored in the output frame storage area of the image memory 26 by making the motion amount detection circuit 28, the edge amount detection circuit 30, and the noise removal circuit 32 inactive via the control bus 22 and making the interface 18 active. Are sequentially read to the image data bus 20.
The interface 18 outputs the image data on the image data bus 20 to the still image output device 14 in synchronization with the still image output device 14. Thus, the still image output device 14 forms and outputs a visible still image. When the signal input format of the still image output device 14 is an analog format, the interface 18 outputs an analog image signal by an internal D / A converter.

Next, a method of modifying the degree of the intra-frame processing and the degree of the inter-frame processing from the default setting value will be briefly described. As a simple method, by changing As ₁ , As ₂ and Ss ₁ , Ss ₂ shown in FIGS. 3 and 4, the relationship between the edge amount As and the coefficient Kij, and the motion amount s and the coefficient T, The relationship can be modified. The user operates the operating device 3
In step 8, a request to change the ratio between the intra-frame processing and the inter-frame processing is input. The control circuit 24 responds to the modification request,
Characteristic values As ₁ from the default memory 66, As _2, Ss
₁ and Ss ₂ are read out and displayed on the display screen of the operation device 38. The user looks at the displayed values and inputs desired values for each. If the characteristic values As ₁ and As ₂ are increased, the area to be smoothed in the frame increases, and if the characteristic values As ₁ and As ₂ are decreased, the area decreases. Also, the area to be smoothed between frames increases when the characteristic values Ss ₁ and Ss ₂ are increased, and decreases when the characteristic values Ss ₁ and Ss ₂ are reduced. Further, the interval between the characteristic values As ₁ and As ₂ and Ss ₁ and Ss ₂
By changing the interval, the inclination of the characteristic curves shown in FIGS. 3 and 4 can be changed.

The changed characteristic values As ₁ , As ₂ , Ss ₁ ,
Ss ₂ is transferred to the control circuit 24. The control circuit 24 checks the validity of these characteristic values (e.g., whether the relationship is large or small, and whether it is within an appropriate range), and if appropriate, changes the characteristic values As ₁ , As ₂ , S
Each characteristic curve is newly calculated based on s ₁ and Ss ₂ . The calculated characteristic curve is written in the modifying memory 68 as the above-mentioned relation table in the form of a table sampled at predetermined intervals. If inappropriate, re-input is prompted on the display screen of the operation device 38.

It goes without saying that the function of each circuit block may be realized by individual hardware, or may be realized by software processing by a dedicated or general-purpose digital processing unit. Processes at appropriate locations may be simultaneously executed by pipeline processing. In this embodiment, the number of images to be processed is two, but of course, noise reduction processing may be performed sequentially using images of three or more frames. A field unit may be used for interlaced scanning moving images.

Further, in the above embodiment, the difference between the maximum value and the minimum value of the image of the region of interest is used as the edge amount. However, the present invention is not limited to this, and statistical processing (for example, correlation value)
Another edge amount may be detected from structural analysis such as processing at a spatial frequency or the like. As the motion amount, a motion amount obtained by various motion vector detection methods may be used.

As a noise elimination method, a weighted average process between frames and a weighted average process within a frame are used in combination, and the weighted average coefficient is set to a value corresponding to the amount of motion and the amount of edge. Although the rate of intra-frame processing is controlled, the present invention is not limited to this. That is, different types of inter-frame processing and intra-frame processing such as various types of filtering processing may be employed. The parameters controlled by the movement amount and the edge amount may be not only the weighted average coefficient but also other parameters or a parameter in which the edge amount and the movement amount are complexly related. The order of the inter-frame processing and the intra-frame processing is not limited to the above example, and may be the reverse of the above example or may be partially related to each other. The block of interest, which is a processing unit, is 3 × 3 pixels, but the size and shape of the block of interest may be determined according to the inter-frame processing and intra-frame processing to be used. For example, a configuration in which the processing unit itself is controlled by the amount of motion and the amount of edge may be employed.

It goes without saying that the function of one or more circuit blocks described in the present embodiment may be realized by software processing.

[0031]

As can be easily understood from the above description, according to the present invention, edges are preserved or emphasized, and
It is possible to obtain a clear still image in which image quality deterioration such as blur or color blur is not conspicuous. In particular, since the ratio between the two-dimensional spatial processing and the processing in the time axis direction can be continuously adjusted according to the amount of motion and the amount of edge, image distortion generated due to processing discontinuity can be suppressed.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 is a configuration block diagram of a main part of FIG. 1;

FIG. 3 is a characteristic diagram of coefficients Kij for intra-frame processing.

FIG. 4 is a characteristic diagram of a coefficient T of inter-frame processing.

[Explanation of symbols]

10: moving image / still image conversion device 12: moving image input device
14: Still image output devices 16, 18: Interface 20: Image data bus 22: Control signal bus 24:
Control circuit 26: Image memory 28: Motion amount detection circuit
30: Edge amount detection circuit 32: Noise removal circuit 3
4: In-frame processing circuit 36: Inter-frame processing circuit
38: Operating device 40, 48: Minimum value detector 42, 5
0: maximum value detector 44, 46, 52, 64: differentiator
54, 60: coefficient generator 56, 62: multiplier 58:
Adder 66: Default memory 68: Modification memory 70: Parameter memory

Claims (1)

(57) [Claims] An image memory capable of storing a predetermined number of images, detection means for detecting a motion amount and an edge amount for each attention area from at least two images stored in the image memory, and detection of the detection means A noise reduction unit that performs a noise reduction process on the image stored in the image memory at a rate of the two-dimensional spatial processing and the processing in the time axis direction according to the result. A moving image / still image conversion device, comprising: a specified value holding means for variably holding a specified value for defining the degree of axial processing and two-dimensional spatial processing.