JPH07118817B2 - Image signal processing method and apparatus for implementing the same - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hard copy of a screen arbitrarily selected from a continuous image such as a television signal recorded during broadcasting or recorded on a video tape or the like, and further for printing. In order to obtain the high-resolution images necessary to create the film and printing plates of the above, we are going to submit a new method to correct the blur (moving part) and noise that occur on the still screen. .

Conventional technology In recent years, due to the practical use of satellite broadcasting, the progress of television technology centered on high-definition television, the development and popularization of video technology, and changes in publishing trends associated with the diversification of printed materials, television broadcasting, video tapes, etc. If a predetermined screen obtained from continuous video information using the above as a medium can be used for a photo or a manuscript for printing, various merits can be expected.

For example, in the case of the conventional method in which a cameraman shoots a film using a still camera, he has to go to the scene and actually shoot the film.
Since various types of electronic means are used for sending photographs, they lack the ability to promptly send information. In particular, if a manuscript for printing can be taken immediately from the TV broadcast screen of a news program or a live broadcast of sports, etc., it is easy and quick to obtain a news photo of a newspaper, etc. You can expect speedup.

In addition, when shooting with a still camera, there are restrictions on shooting opportunities and other shooting conditions, but it is also possible to freely select any screen when recording a TV broadcast on a video tape, etc. It is also possible to obtain.

In addition to news media, hard copy such as photographs and manuscripts for printing can be created directly from television broadcast images, television images recorded on videotapes, etc., or images obtained from movie films through telecine equipment. If it can be done, various needs can be expected to develop.

However, when a still image is extracted from a continuous video image projected on a television, there are the following problems.

That is, the television broadcasting currently generally performed in Japan is called the NTSC system, and one screen is formed with 525 scanning lines regardless of the size of the television image. A television image on one screen is composed of images of two fields, and 525 television signals are
It is divided into two and is interlaced. Therefore 1
In order to obtain a hard copy or the like from the screen image information, the scanning line information of the first field (odd field) and the second line
It is necessary to sequentially combine scanning line information of fields (even fields). However, between the image information of the first field and the image information of the second field,
Since there is a time lag, if a moving part exists on the screen, blurring (movement) will occur in the still image.

A simulated representation of this blurring situation is shown in FIG.

FIG. 1 shows a partial image of one screen of a television, which is composed of a moving part (Y) and a stationary part (X). Then, the state of the screen when the image is captured as the scan line information of the odd field is represented by a, the scan line situation of the odd field is represented by c, and the screen when the scan line information of the even field is captured is also shown. The scan line information of the b and even fields is indicated by d.

Since there is a slight time lag (t) when the information is read in the odd field and the even field, the scan line information of each of the odd field and the even field is simply combined as shown by e. Originally, for example, an image shown by f is necessary, but if a still image is obtained from a moving screen, the screen becomes blurred. In the case where the entire screen moves uniformly, for example, blurring can be prevented by moving only the scanning line information of the odd field by the time shift and combining it with the scanning line information of the even field. In normal TV images, there is almost no case where the entire screen moves uniformly,
With irregular movements in different parts of the screen,
It is not possible to correct a blurred image only by correcting the above-mentioned time lag.

Conventionally, as a method of correcting blur, one screen of a television is used as one field (only odd or even fields).
It has been performed to form one screen by using only the scanning line information of the above and inserting it between the scanning lines. However, in this case, the screen is distorted or blurred,
The intended clear image could not be obtained.

As another method, instead of simply inserting the information, one of the scan line information of the odd or even fields is used, and the information corresponding to the other fields is entirely interpolated based on the information of one field. It was also newly formed by and made into one screen. But in this case,
Compared with the above-mentioned method, there was less distortion and blurring, but a clear image could not be obtained.

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention The present invention can obtain an image without blurring or an improved image without deteriorating the image quality even when an image in which an arbitrary part of one screen moves is used. It is intended to provide a method and an apparatus therefor.

Means for Solving the Problems In order to achieve the above object, a method of processing an image signal according to the present invention is a method of digitizing an image signal of an image configured by an interlace method, and using the digitized image as an original image. The first step of loading the image signal level of each pixel into the frame memory and the image signal of either the odd field or the even field loaded into the frame memory as the basic image is fixed. However, the image signal level of each pixel of interest belonging to another field different from the basic image, and the image signal of the pixel of the basic image at the position corresponding to the pixel of interest in the row before and after the row to which the pixel of interest belongs The second step of finding the product of the respective differences with the levels, and For each pixel of the basic image at the corresponding position, the difference in the image signal level of the pixel at the position corresponding to the pixel of interest in the row adjacent to the outside of the row to which each pixel of the basic image at the corresponding position belongs The third step of obtaining the product of these differences, and the pixel in which the product obtained in the second step is positive and the product obtained in the third step is negative, are taken as the blur portion. Using the fourth step of detecting as equivalent and the image signal level of the peripheral pixels of each pixel of the blur portion in the original image, a new image signal level is obtained, and each pixel of the blur portion is obtained. And a fifth step of replacing with the image signal level of.

Now, the image signal processing method according to the present invention will be described in more detail.

First, a TV signal from a television, a VTR signal from a VTR, a VDR signal from a laser disk or the like can be used as an image signal of an original image (original image) to which the image signal processing method according to the present invention is applied. . Further, a telecine signal obtained by using a film or the like through a telecine device can also be used.
(Hereinafter, these signals are collectively referred to as TV signals.) Since these TV signals are analog signals, A /
It is quantized by the D converter and stored in the frame memory as a signal having an address. On the other hand, a previously digitized signal, for example, a digital signal from an optical disc or computer graphics (CG) can also be used as an input signal and is stored in the frame memory without passing through the A / D converter. If the TV signal is a color image signal, it is separated into R, G, B signals by the RGB decoder, and then converted into digital signals by the A / D converters, and R, G, B
Each of the signals will be stored in each frame memory.

As described above, in the case of an input image signal such as a TV signal, each frame memory (for example, 512 × 512) address (corresponding to a pixel) has a density value of, for example, 8 bits (hence 256 levels). Remembered.

The digital image processing method according to the present invention will now be described in more detail with reference to the simulated image of a moving object image shown in FIG.

FIG. 2a is a diagram showing the image signals stored in the frame memory, from the j-th row (where j is an odd number) to the (j-th) scanning line.
The state of the part up to +10) line is shown. In this figure, each square in the square represents one pixel, and each pixel in the shaded area belongs to a moving object and has a non-zero (eg, positive) image signal level. In this figure, pixels with diagonal lines have an image signal level of 5, and pixels with cross diagonal lines have an image signal level of 10.
Each pixel in the white portion belongs to the background and has an image signal level of 0.

Here, a method of extracting a moving part of the object will be described. First, each row (j, j + 2, j + 4, ... j + 10) corresponding to the scanning line of the odd field (first field) stored in the frame memory shown in FIG. 2a (hereinafter simply referred to as an odd-numbered row) The image signal of each pixel is assumed to be an image (hereinafter referred to as a basic image) signal of a non-moving (still) portion of the object. Then, with respect to the portion represented by each pixel in this odd-numbered row, each of the rows (hereinafter, simply referred to as even-numbered rows) corresponding to the scanning lines of the even field (second field) shown in FIG. 2a It is checked whether the image signal of the pixel indicates a blur (movement) portion. (Note that the original image in the even field shown in FIG. 2a includes pixels in a part that does not move relative to the basic image and pixels in a part that does move.) Therefore, The image signal levels of the corresponding pixels of two rows (for example) before and after the row to which the pixel of interest belongs are compared with the image signal level of the pixel of interest centering on the pixel of interest of each row of the even field. First, the difference between the level of the image signal of each pixel in the even-numbered row and the level of the image signal of the pixel in the adjacent odd-numbered row before and after it is examined.
Specifically, first, the i-th row in the (j + 1) -th row shown in FIG.
Regarding the image signal level d (i, j + 1) of the th target pixel, the image signal levels d (i, j) and d (i, j + 2) of the corresponding i-th pixel in the j-th row and the (j + 2) th row
Compare with. That is, S (j + 1, j) = d (i, j + 1) -d (i, j) ... (1) S (j + 1, j + 2) = d (i, j + 1) -d (i, j + 2) ...
(2) is obtained, and further the product of the obtained image signal level differences,
That is, it is checked whether or not S (j + 1, j) × S (j + 1, j + 2) ... (3) is positive. When this product is negative or 0, it can be regarded as a slope portion or an edge portion.

In this way, the image signal level is similarly compared with the pixels of the corresponding odd-numbered rows of each pixel of each row of the even field shown in FIG. 2A. As a result of such comparison of the image signal levels, each pixel in the even field in which the product of the image signal level differences is positive is indicated by a circle.
Figure b. The image signal level of the pixel whose product of the image signal level difference is negative or corresponds to 0 is used as it is.

Next, the pixel of interest (i, j
For the pixels (i, j) and (i, j + 2) in the rows before and after +1), the pixels in the rows outside thereof, that is, S (i, j-1) and S (i, j-1)
Examine the relationship with (i, j + 3). That is, (i, j
The image signal level of the pixel of (-1) is d (i, j-1), and the image signal level of the pixel of (i, j + 3) is d (i, j +).
3), S (j + 1, j) = d (i, j + 1) -d (i, j) ... (1) S (j, j-1) = d (i, j) -d (i, j -1) ... (4) S (j + 1, j) * S (j, j-1) <0 ... (5) or S (j + 1, j + 2) = d (i, j + 1) -d (i, j + 2) …
(2) S (j + 2, j + 3) = d (i, j + 2) -d (i, j + 3) ...
(6) When S (j + 1, j + 2) × S (j + 2, j + 3) <0 ... (7) is satisfied, it can be considered that the target pixel d (i, j + 1) corresponds to the blur portion. When the product is positive or 0, it can be regarded as a slobe portion or an edge portion.

The original signal level is used as it is as the image signal level of the pixel corresponding to the slope portion or the edge portion.

For example, the portion indicated by a triangle in FIG. 2c can be regarded as a blurred portion (moving portion).

As described above, by fixing either one of the image signal information of the even field and the image signal information of the odd field,
The image signal level is compared with the corresponding pixels in each of the two rows before and after the row to which the pixel of interest belongs, and the second pixel
The portion satisfying that the product in the step of is positive and the product in the third step is negative can be regarded as a blurred portion (moving portion).

Next, the image signal level of the pixel at the position indicated by the triangle in FIG. 2c is once removed to 0, and the interpolation processing of the image signal level is performed based on the value of the image signal of the peripheral pixel for each of these pixels. By doing so, the image signal level of a moving portion can be corrected, and thereby a target image can be obtained.

FIG. 2d is a conventionally known bi-linear method in which the image signal level of a pixel included in a portion corresponding to a blur is determined by using image signal levels of pixels in rows before and after the row to which the pixel belongs. 3 shows an image obtained by interpolating by. In this figure, the numbers shown in the squares showing the pixels indicate the new image signal levels obtained by interpolation, and the squares marked with-indicate that the image signal level obtained as a result of the interpolation is 0. The pixel is shown.

In the above description, the scanning line information in the odd field is fixed, but it may be fixed in the even field.

As an interpolation method for newly forming an image signal in a pixel having no signal, in addition to the bilinear method, a conventionally known Nearest neighbor method is also used.
method), the cubic convolution method (Cubi
c convolution method) etc. can be used. In addition, Japanese Patent Application Nos. 61-286351 and 61
It is also possible to use the linear extrapolation averaging method disclosed in Japanese Patent Application No. 286352 and Japanese Patent Application No. 62-217419. Particularly in the case of the present invention, since the pixels of one field arranged every other row are interpolated, it is desirable to use the linear extrapolation averaging method capable of signal enhancement.

Hereinafter, the image signal processing apparatus as a preferred embodiment of the present invention will be described more specifically.

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 3 is a schematic block diagram showing an image signal processing apparatus for carrying out the image signal processing method according to the present invention.

As the input image signal, a TV signal (1) from a television, a VTR signal (2) from a VTR, and a VDR signal (3) from a laser disk or the like can be used. In addition, a telecine signal (4) obtained from the film through a telecine device.
Is also available. Below, including these signals, generically
It is referred to as a TV signal. Since these TV signals are analog signals, they are quantized by the A / D converter (8) and stored in the frame memory (9) as signals having addresses.
On the other hand, previously digitized signals, such as optical disc (5) or computer graphics (CG) (6)
Digital signals from etc. can also be used as input signals, A / D
It is stored in the frame memory (9) without going through the converter (8). The digitized input image signal is stored in one of the frame memories 9-1 and 9-2 which is not used in the subsequent processing, for example, the frame memory 9-1 in FIG. The control unit (10) selects the frame memory and the input image signal. D /
The A converter (11) and the monitor display (12) are devices for displaying the signal of the frame memory (9) on the display as needed.

Next, the input image signal stored in the frame memory (9-2) is supplied to the image quality improvement processing section (13). At this time, the other frame memory (9-1) stores the next input image signal according to an instruction from the control unit (10). In this way, the frame memories (9-1) and (9-2) are used by being switched alternately by the above-mentioned method. When it is necessary to store many image signals in parallel, a required number of frame memories can be prepared.

If the TV signal is a color image signal, it is separated into R, G, B signals by the RGB decoder (7) and then A / D respectively.
It is converted into a digital signal by the converter (8) and each of the R, G and B signals is stored in each frame memory.

As described above, in the case of an input image signal such as a TV signal, each address (corresponding to a pixel) of this frame memory (corresponding to 512 × 512) is, for example, a density value of 8 bits (hence 256 levels). Remembered.

The input image signal stored in the frame memory (9) is then subjected to necessary image quality improvement processing in the image quality improvement processing section (13). Further, the image quality improvement processing unit (13) implements the image signal processing method according to the present invention.

In the image quality improvement processing unit (13), the image signal read from the frame memory (9) is subjected to processing such as aspect ratio correction and gradation correction as necessary in addition to the above processing, and after processing The image signal of is stored in the RAM (15) via the CPU (14). At this time, RAM (15)
The image signal stored in can be stored in an image file (16) such as a floppy disk or hard disk, if necessary, and can be displayed and confirmed on the CRT display (17) if necessary. It can be done.

In this way RAM (15) or image file (1
The image signal read from 6) and stored in the RAM (15) can be interpolated to obtain a high-resolution image if necessary, and a new pixel image obtained by interpolation calculation, etc. The signals are sequentially stored in the image memory (18).

The image data stored in the image memory (18) is output by the output device (20) via the interface (19) to, for example, photographic film, photographic paper, or the like.
Although not shown in FIG. 4, the output device means a laser beam plotter or the like, and the output part of the scanner can be used. If the output device is equipped with a buffer memory, etc., it is also possible to temporarily store the image data in the output and then output the image data. It is also possible to output after applying.

FIGS. 4 (A) and 4 (B) are flowcharts exemplifying the flow of procedural operations when the image signal processing method according to the present invention is carried out using the apparatus shown in FIG.

First, the image size (number of pixels) of the original digital image (original image) is initialized (step S1). That is, the data of the number of pixels in the horizontal and vertical directions of the original image are respectively set as variables x and y
Set to. Then, the image signal level data of the original image stored in the frame memory is read into the array d (step S2). For the elements of this array d, for example, the image signal level data of the i-th pixel in the j-th row is
It is expressed as d (i, j). Furthermore, as the value of each element of the array d ′ that stores the corrected image data, the corresponding d
Set the value of the image signal level of the element of (step S
3).

Next, the values of the control variables (counters) i and j are set to 1 (step S4).

Here, first, the i-th pixel of the j-th row of the original image, the i-th pixel of the corresponding preceding and following (j−1) -th row and the (j + 1) -th row, and the corresponding (j-th) outer pixel -2) and the data of the image signal level of the i-th pixel in the (j + 2) -th row are calculated as in the following equations (8) to (11) (step S5).

S (j, j + 1) = d (i, j) -d (i, j + 1) ... (8) S (j, j-1) = d (i, j) -d (i, j-1) ... ( 9) S (j-1, j-2) = d (i, j-1) -d (i, j-2) ...
(10) S (j + 1, j + 2) = d (i, j + 1) -d (i, j + 2) ...
(11) Next, the value of S (j, j + 1) * S (j, j-1) is positive and S (j, j-1) * S (j-1, j)
-2) or the value of S (j, j + 1) * S (j + 1, j + 2) is negative (step S6). If the condition of step S6 is not satisfied, the image signal d (i, j) of the pixel (i, j) of the original image is directly d ′ (i,
j) is set to the pixel (i, j) of the array d '. (Step 7).

If the condition of step S6 is satisfied, it corresponds to a blur portion, and in step S8, the image signal level of the pixel of interest (i, j) is set to 0, and the following equation d ′ (ij) = {d (i, j− 1) + d (i, j + 1))} / 2
The value obtained by (12) is used as the new image signal level d '(i,
As j), the value of the image signal level of the corresponding pixel of d'which stores the corrected image signal level is set. Then, the i-th data d (i, j) of the (j + 1) th line
+1) is set as it is as the image signal level of the corresponding pixel in the array d '. (Step S9) Next, the value of the control variable i is increased by 1 (Step S1
0) Further, it is checked whether the value of the control variable i is larger than the number of pixels in the horizontal direction of the original image of the variable x (step S1).
1). Then, when the value of the control variable i is smaller than the number of pixels in the horizontal direction of the original image, the above steps S5 to S11 are further performed on the pixel next to the i-th row of the original image and the j-th row of the original image. The same process is repeated until all the pixels are processed. If the value of the control variable i is larger than the number of pixels in the horizontal direction of the original image, the process proceeds to step S12, the control variable j is increased by 2, and steps S5 to S11 are performed for each pixel in the next row of the original image. Perform the same processing as. Next, in step S13, it is checked whether or not the value of the control variable j is larger than the value of (y), and if not larger, the process returns to step S4 and the above step S5 is performed for the (j + 2) th row.
The process of step S13 is performed. And in step S13,
The processes of steps S4 to S13 are repeated until the value of the control variable j becomes larger than the value of (y), that is, for all the rows of the original image.

In this embodiment, the conventionally known bilinear method is used as a method of correcting the pixel corresponding to the blur portion, but other conventionally known interpolation methods can be used as described above.

When j takes a value of 1 or a value of (y-1), that is, in the peripheral portion of the image, for example, d (i, j-1), d (i, j
-2) or there is no signal corresponding to d (i, j + 2), but in this case, the signal level is 0,
It can be dealt with by calculating. Even if the peripheral part is ignored, there is no problem in image quality.

As described above, according to the present invention, the image signal level of the pixel of interest and the image signal level of the pixel of the basic image at the position corresponding to the pixel of interest in the row adjacent to the row to which the image of interest belongs And the difference between the image signal level of the pixel of the basic image and the image signal level of the pixel at the position corresponding to the pixel of interest in the row adjacent to the outside of the basic image are then obtained. , A product of this difference and a difference related to the pixel of the basic image which is obtained first, and a pixel in which the product obtained first is positive and the product obtained next is negative is defined as a blur portion. It is configured so that it can be detected as a corresponding one and this blurring portion can be replaced with a new image signal level. Therefore, from an image which is arbitrarily selected from a continuous image such as a television signal which is being broadcast or recorded on a video tape, etc., and which has a blurred portion (moving portion) or noise therein, An image without blurring can be obtained without deteriorating the image quality.

In particular, the present invention does not perform the detection of the blurring portion based on the image signal levels of the pixels in the rows before and after the pixel of interest as in the prior art, but the image signal levels of the pixels in the rows before and after the pixel of interest, and This is performed based on the image signal level of the pixels on the outer rows of the front and rear rows, and the detection accuracy of the blurring portion can be significantly improved compared to the conventional technique, and noise and the like can be accurately removed. . Furthermore, in the image processing of the prior art, an image of an edge portion having a firm contour is smoothed and edge blurring occurs, but this point is also prevented in the present invention.

[Brief description of drawings]

FIGS. 1 (a) to (f) are diagrams showing an example of an image composed of a moving part and a stationary part of an object on one screen of the television, and FIGS. 2 (a) to (f) FIG. 3D is an image simulation diagram for explaining the image signal processing method according to the present invention, and FIG. 3 is a schematic block diagram showing an apparatus for performing the image signal processing method according to the present invention. FIG. 4 (A) and FIG. 4 (B) are flowcharts showing the flow of procedural operations for carrying out the image signal processing method according to the present invention. In the drawings, 7 ... RGB decoder, 8 ... A / D converter, 9 ... Frame memory, 10 ... Control unit, 11 ... D / A converter, 12 ... Monitor display, 13 ... Image quality improvement processing unit, 14 ... CPU, 15 ... RAM, 16 ...
Image file, 17 ... CRT, 18 ... Image memory, 19 ... Interface, 20 ... Output device.

Front page continuation (72) Kenji Okamori Kenji Okamori 2-6-2 Higashi-Tenma, Kita-ku, Osaka City, Osaka Prefecture Sakata Inx Co., Ltd. in the Central Building of Minamimorimachi (72) Masatoshi Fukumoto 2 Higashi-Tenma, Kita-ku, Osaka City, Osaka Prefecture Sakata Inx Co., Ltd. in Minamimorimachi Chuo Bldg. 6-62 Minamimorimachi Central Building (72) Inventor Noriya Kato 2-6-2 Higashitenma, Kita-ku, Osaka City, Osaka Sakata Inx Co. Ltd. In-house (56) References JP 61-196684 (JP, A) JP 63-266982 (JP, A)

Claims (2)

[Claims] 1. A first step of digitizing an image signal of an image formed by an interlace system, and taking a pixel signal level of each pixel of the digitized image as an original image into a frame memory, the frame memory The image signal of one of the odd field and the even field taken in is fixed as a signal representing the basic image, and the image signal level of each pixel of interest belonging to another field different from the basic image. , A second step of obtaining a product of respective differences between the image signal level of the pixel of the basic image at a position corresponding to the pixel of interest in a row adjacent to the row of the image of interest, and in the row adjacent to each other. The image signal level of the pixel of the basic image at the position corresponding to the pixel of interest, and the image signal level of the row to which the pixel of the basic image belongs The difference between the image signal levels of the pixels at the positions corresponding to the pixel of interest in the row adjacent to the side is obtained, and the product of this difference and the difference related to the pixel of the basic image obtained in the second step is obtained. A third step, and a fourth step in which a pixel in which the product obtained in the second step is positive and the product obtained in the third step is negative is detected as corresponding to a blur portion. And a fifth step of obtaining a new image signal level by using image signal levels of peripheral pixels of each pixel of the blur portion in the original image and replacing the image signal level of each pixel of the blur portion with the fifth step. A method of processing an image signal, comprising: 2. A first storage section for storing an image signal of the digital image after selecting an image signal of a desired digital image as an original image, reading it from a medium, performing A / D conversion if necessary. And an image signal stored in the first storage unit, and an image signal of one of an odd field and an even field stored in the first storage unit, which represents a basic image. The signal level is fixed, and the image signal level of each pixel of interest belonging to another field different from the basic image and the basic position of the position corresponding to the pixel of interest in the row adjacent to the row to which the image of interest belongs. First means for obtaining a product of respective differences with image signal levels of pixels of the image; image signal levels of pixels of the basic image at positions corresponding to the pixel of interest in the adjacent rows, The difference between the image signal levels of the pixels at the positions corresponding to the pixel of interest in the row further outside the row to which the pixel of the basic image belongs is found, and this difference and the pixel of the basic image found by the first means. And a pixel for which the product obtained by the first means is positive and the product obtained by the second means is negative, as the blur portion. A new image signal level is obtained using the third means for detecting as equivalent and the image signal levels of the peripheral pixels of each pixel of the blur portion in the original image, and the new image signal level of each pixel of the blur portion is obtained. An image improvement processing unit including a fourth storage unit that replaces the image signal level with the image signal, and an image improvement processing unit including a second storage unit that stores the image signal subjected to the image quality improvement processing by these units, and is stored in the second storage unit. Output unit for outputting the image signal An image signal processing device comprising: