JP2552742B2 - Adaptive contour correction circuit - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial application] The present invention adaptively corrects, for each pixel, deterioration of a spatial frequency component that occurs when registration correction of a camera using a solid-state image sensor is electrically performed. The present invention relates to a contour correction circuit for

[Conventional technology]

As is conventionally known, misregistration of registration in a solid-state image sensor occurs due to mounting error and lens aberration. Here, the attachment is performed by using an extremely precise jig, but it is difficult to completely eliminate the error.

Further, in the case of a solid-state image pickup device, the registration deviation is determined by its mounting accuracy, so that it is impossible to dynamically correct the chromatic aberration of magnification of the lens.

[Problems to be Solved by the Invention]

However, in reality, when a zoom lens is used, the registration shift occurs due to the chromatic aberration of the lens in the telephoto state and the wide-angle state, and the image quality is significantly impaired in the composite output of the camera.

Therefore, the applicant of the present invention has provided a registration correction circuit capable of not only statically correcting a sticking error of a solid-state image sensor by electrical image processing but also dynamically changing an error such as chromatic aberration of magnification in real time. Another application at the same time as the applicant (Japanese Patent Application No. 1-29)
No. 0098; JP-A-3-15178).

That is, in the above registration correction circuit, in a color television camera using a plurality of sets of solid-state image pickup devices, the relationship between the image signals obtained from the respective solid-state image pickup devices and the light-receiving units facing each other is different. In many cases, the combinations of the respective sets do not match due to structural problems. In this relationship, one combination is used as a reference, and the remaining combinations are matched with the combinations of the reference. Is used for coordinate conversion and pixel interpolation of the image signal.

In other words, the registration correction circuit can shift the input color image by the address conversion of the random access memory and the interpolation calculation. However, when the correction information is stored in one pixel unit to perform this operation, It requires a huge amount of memory. Therefore, in the registration correction circuit, a sawtooth wave (SA) that is used to control the deflection system of a conventional analog camera tube camera is used.
W) / Parabola (PARABOLA) signal is generated and corrected Various signals are generated digitally as time-series data in the same manner as in the past, and the mixing ratios have already been determined and stored so that they can be individually set. It can be controlled by the value.

Further, in the registration correction circuit, in the coordinate conversion for moving in units of one pixel and the pixel interpolation processing for moving in a small range not exceeding one pixel, DC preset as a correction coefficient, SAWTOO
Digital data such as H and PARABOLA will be used.

Further, in the above registration correction circuit, in the pixel interpolation method, in the case of interlaced pixels, inter-field interpolation and intra-frame interpolation are included, and switching between them detects the motion of the image, and the pixel with motion is detected. It is preferable to use intra-field interpolation and intra-frame interpolation for pixels having no motion.

Moreover, in the above-mentioned registration correction circuit, in these coordinate conversions and pixel interpolations, in order to deal with correction of lateral chromatic aberration due to the zoom ratio of the zoom lens and conversion of the lens, it is possible to selectively select among a plurality of coefficient data prepared in advance. Is proposed to be used for.

However, in the case of obtaining the image information which is displaced by a certain image processing, the displacement amount of one pixel unit can be adjusted by physically moving the address by a randomly accessible memory or the like. In this case, interpolation calculation is not used. However, there is no deterioration in the frequency characteristics, but in order to express the amount of deviation within one pixel unit, an interpolation calculation is performed using adjacent pixels. In this case, the interpolation coefficient is different in each pixel. For example, in the case of an image composed of a single high frequency component, there arises a problem that the amplitude characteristic is not uniform on the screen.

For example, as shown in FIG. 10, consider a fine vertical stripe image in which black and white are repeated on the screen at regular intervals t. For this image, m pixel pitch (0 ≦ m
≦ 1) When shifted to the right, the frequency characteristic deteriorates only in that part. This deterioration of the frequency characteristic shows a roll-off characteristic that shows the maximum attenuation at 1 / (2τ) (τ = 1 pixel pitch) as shown in FIG.
The maximum is 1/2 pixel. In the case of the image in FIG. 10, t
The frequency deterioration due to the coefficient m is maximum when = τ.

That is, if different coefficients are applied to different parts of the image having a uniform single frequency on the screen as described above, shading-like color unevenness will occur.

Furthermore, this interpolation operation is also performed in the vertical direction, and in a system that performs interlaced scanning, in particular, since different pixels are used depending on the motion, this phenomenon becomes three-dimensional, and up to now, this correction has not been performed. Was not done.

Therefore, in view of the above points, a first object of the present invention is to eliminate blur caused by interpolation in a system in which adjacent pixels are used to perform interpolation calculation of different coefficients for each pixel to match a reference image with correlation. An object of the present invention is to provide a contour correction circuit that reduces the number of pixels and significantly improves the image quality.

A second object of the present invention is to provide a contour correction circuit capable of adaptively correcting each pixel in real time according to the interpolation coefficient of each pixel.

[Means for solving the problem]

The adaptive contour correction circuit according to the present invention uses the pixel data of a specific color output from a specific solid-state imaging device among a plurality of solid-state imaging devices as reference pixel data to move pixel data of another color by An adaptive contour correction circuit that corrects, in real time, a deterioration in spatial frequency caused by the registration correction when performing registration correction for aligning reference pixel data,
Coefficient output means for outputting interpolation coefficient data representing a shift amount in a range smaller than one pixel required for performing the registration correction, coefficient conversion means for generating complementary coefficients of the interpolation coefficient data, and the reference A processing means for forming inverse frequency characteristic data by multiplying the pixel data by the complementary coefficient, and an adding means for adding the inverse frequency characteristic data to pixel data other than the reference pixel data. Is.

Further, another adaptive contour correction circuit uses the pixel data of a specific color output from a specific solid-state imaging device among a plurality of solid-state imaging devices as reference pixel data to move pixel data of another color An adaptive contour correction circuit that corrects, in real time, the deterioration of the spatial frequency caused by the registration correction when performing registration correction for aligning the reference pixel data. Pixel data storage means for storing a color video signal in the form of pixel data, and a registration for compensating a registration deviation between each pixel data by capturing a predetermined test image using the solid-state image sensor. Registration correction information storage means for storing correction information in advance; Waveform generating means for generating a plurality of predetermined waveform data in synchronization with pixel data, and a plurality of multiplying means for multiplying each of the waveform data by a predetermined correction coefficient based on the registration correction information. , First addition means for accumulating the respective data obtained from the plurality of multiplication means and outputting the correction waveform data, and necessary for compensating the registration deviation based on the correction waveform data And a pixel data stored in the pixel data storage unit or pixel data read from the pixel data storage unit. On the other hand, by multiplying the interpolation coefficient data, the pixel data output from the solid-state image sensor is equivalently transferred within a range not exceeding one pixel. Interpolation means, coefficient conversion means for generating complementary coefficients of the interpolation coefficient data, processing means for forming inverse frequency characteristic data by multiplying the reference pixel data by the complementary coefficients, and the reference Second addition means for adding the inverse frequency characteristic data to the pixel data other than the pixel data is provided.

[Action]

Since the interpolation coefficient used in the adaptive contour correction circuit according to the present invention is given by some coefficient generator in the feedforward method, which part of the pixel is subjected to what kind of interpolation is preset. This means that it is decided which pixel has what kind of deterioration in frequency characteristics.

Further, in the present invention, it is premised that the pixels are interpolated to be matched with the reference video signal, and therefore, the video signal after the interpolation has the same signal position with respect to the same part of the subject as the reference video signal. Conceivable.

That is, by converting to a coefficient that obtains the inverse characteristic of the frequency characteristic that deteriorates due to the coefficient used for interpolation in the interpolated image signal and multiplying that value by the reference image signal,
Extract the frequency component that seems to be lost by interpolation,
By using it as a correction signal for the frequency deterioration of the interpolated signal, it is possible to obtain a signal having the same frequency component as the original signal before interpolation.

Furthermore, in a system that performs interlaced scanning, even when motion detection is performed and interpolation calculation is performed by dividing it into in-frame and in-field, the motion detection signal is applied to both the circuit system that generates the correction signal and the circuit system that is corrected. By giving it, processing can be performed under exactly the same conditions.

〔Example〕

Before describing a specific embodiment of the present invention, the principle of contour correction according to this embodiment will be described.

Now, an impulse as shown in FIG. 4 (A) having a level of "1" on a part of the screen is moved to the right by m pixel pitch (0
≦ m ≦ 0.5) Considering the case of shifting using linear interpolation,
By using the coefficient of FIG. 4 (B), interpolation calculation is performed on two adjacent pixels for each pixel, and FIG. 4 (C) can be obtained.

In this case, the spatial frequency characteristic of the image signal converted as shown in FIG. 4 (C) is as shown in FIG. 5 (A).
The roll-off characteristic shows the maximum attenuation at 1 / 2τ (τ = 1 pixel width).

On the other hand, in order to correct the frequency characteristic, the characteristic shown in FIG. 5 (B) may be inversely corrected, and the coefficient at that time is
It looks like Figure 6.

Since the interpolated signal is considered to be in position with the reference signal by shifting the m pixel pitch at this point, the reference signal (for example, Gch signal) is multiplied by the coefficient shown in FIG. Extract the frequency component with the characteristics of B),
By adding to the interpolated signal, blurring due to the interpolation can be eliminated, and particularly the luminance signal can be ideally improved.

 Next, consider the case where this is expanded into two dimensions.

For example, assuming that the horizontal shift is m and the vertical shift is n, and the interpolation coefficient is shown in FIG. 7A, the coefficient of the reference signal for correcting the blur due to interpolation in this case is the eighth coefficient.
It becomes like FIG. (A) -FIG. 8 (D).

The correction method is the same as that described in the first dimension, and the coefficient is applied to the adjacent four pixels.

Next, the case of a system that performs interlaced scanning will be described.

In an interlaced scanning system, as shown in FIG. 3, pixels that are vertically adjacent have different fields. Therefore, there is a method of switching between in-frame and in-field pixels according to the motion.

Even in this case, the interpolation coefficient of FIG.
Both coefficients and processing (in-frame / in-field) are switched according to the motion signal detected from the reference signal using the correction coefficient in FIG.

By performing the above-described operation by adapting the coefficient to each part of the screen, it is possible to perform appropriate frequency interpolation even if the interpolation coefficient is different.

As a matter of course, if the interpolation is not performed, the video signal is not affected at all.

FIG. 1 is a block diagram showing an entire embodiment of the present invention. In this embodiment, the registration deviation of the three-primary-color interlaced solid-state image pickup device camera is electrically matched with the G channel signal by using the G channel signal as a reference. .

In FIG. 1, 1,2,15,16 are 1-field delay circuits, 3,3 ', 17,17' are 1-line delay circuits, 4,5,6 are random access memories, 18,19,20 are n Line delay circuit (n =
RAM4,5,6 delay), 7,21 are multiplexers, 8,9,22,2
3 is a 1-pixel delay element, 10, 11, 12, 13, 25, 26, 27, 28 are coefficient units, 14, 29, 30, 30 'are adders, 24 are coefficient converters, 31 is a motion detector, 32 is a coefficient generator, 33 is a synchronizing signal generator, 34 is a microcomputer, and 35 is a deviation amount control system.

In the present embodiment, in order to match the registration shift, in the case of the R and B channels with respect to the reference G channel, if there is a shift for each pixel unit, coordinate conversion using a random access memory is performed and one pixel The shift within the unit is performed by the interpolation calculation using the adjacent four pixels as described above.

The input R / B channel signal is processed by the two 1-field delay circuits 1 and 2 and the 1-line delay circuit 3.
A signal delayed by 1 frame + 1 line from the input is used as a timing reference, and a signal one field later and a signal one line later are obtained. These signals are “static” and “moving” depending on the motion signal generated by the motion detection circuit 31.
In this case, it is switched by the multiplexer 7 as shown in the figure.

The RAMs 4, 5 and 6 perform coordinate conversion by controlling the address with integer information of the coefficient given from the coefficient generator 32, and correct the deviation in units of one pixel.

Next, the deviation of less than one pixel is corrected by the interpolation calculation using the adjacent four pixels as described above by each of the circuits 8 to 14. At this time, the coefficient units 10 to 13 use the decimal information given from the coefficient generator 32.

Reference numerals 15 to 29 in the figure are amplitude characteristic deterioration correction systems peculiar to this embodiment, which will be described later in detail.

FIG. 2 shows the detailed structure of the coefficient generator 32 shown in FIG. The coefficient generator 32 is composed of a horizontal displacement amount control system and a vertical displacement amount control system, each of which is composed of a plurality of waveform generators of correction signals expected in advance and a multiplier for controlling the amount thereof.

The waveforms are generated digitally, and the lower m bits are the fractional part and the upper n bits are the integer part, and the ratios of the respective waveforms are controlled by the microcomputer and then added.

The integer part of the data added in the horizontal and vertical directions is output to the address control part of the RAM that performs coordinate conversion, and the decimal part is output as four coefficients for the four pixels to be interpolated.

Further, the intra-field interpolation coefficient and pixel and the intra-frame interpolation coefficient and pixel can be switched by the motion signal.

The data representing the mixing ratio of the respective waveforms output from the microcomputer 34 is the data obtained by actually photographing a test chart and adjusting it by a person in advance so that the registrations match.

 The waveform generator generates the following waveforms, for example.

・ DC y = a (a is a constant) ・ SAW y = ax + b (on axis) ・ PARA y = a (x−b) ² + c (on axis) ・ SAW x = ay + b (on axis) ・ PARA x = a (Y−b) ² + c (off-axis) In this way, the registration shifts are matched, and the R and B channels are used with respect to the reference G channel, and the random access memory is used for the shift of one pixel unit or more. The coordinate conversion is performed, and the shift within 1 pixel unit is performed by the interpolation calculation using the adjacent 4 pixels described above. Deviations of one pixel unit, that is, those that can be adjusted by coordinate conversion, have no effect on the frequency characteristics of the video signal, but those that include deviation components within one pixel unit such as 0.5 pixels and 1.3 pixels Since the values are adjusted by the interpolation calculation, the frequency characteristic is deteriorated according to the coefficient.

The reference G channel signal system is basically the same as the R / B channel circuit configuration. The input G channel signal is generated by the two 1 field delay circuits 15 and 16 and the 1 line delay circuit 17. , 1 frame +1 compared to input
With the line-delayed signal as a reference, a signal after one field and a signal after one line are obtained. Among them, the signal after one field and the signal after one line are converted by the multiplexer 21 into the former signal in the case of "static" and the latter signal in the case of "moving" by the motion signal generated by the motion detection circuit 31. Can be switched.

The signal obtained here and the signal delayed by 1 frame + 1 line (see FIG. 3) are respectively the 1 pixel delay circuits 22 and 23.
By passing through, four adjacent pixels are obtained.

Since the main line signal is delayed by n lines in order to represent the vertical deviation in the positive direction, Rch and Bch are phase-matched by the n line delay circuits 18 to 20.

The adjacent 4 pixels thus obtained are then subjected to the coefficient multipliers 25 to 28.
It is added after passing through. These coefficient units use the fractional part of the coefficient generated by the coefficient generator 32, that is, information on the deviation within one pixel unit, and the coefficient converter 24 uses the inverse characteristic of the interpolation coefficient as described above. What is converted into a coefficient indicating is given in real time.

The relationship between the coefficients (a, b, c, d) used for interpolation and the coefficients (A, B, C, D) for correcting the amplitude characteristic is described above with reference to FIGS. 6 to 9. This corresponds to four adjacent pixels having the same coordinates.

The correction signal added by the adder 29 is added by the adder 30 to the R channel or B channel signal, and the correction is completed.

The multiplexers 7 and 21 can be replaced with a known adaptive addition circuit. The adaptive addition circuit mixes in-field and in-frame calculation results at a predetermined ratio. This makes it possible to smooth the transition from the moving image to the still image or from the still image to the moving image.

〔The invention's effect〕

By implementing the present invention, the following special effects are obtained.

In a system that interpolates different coefficients depending on the part of the screen, blurring of pixels that may vary depending on the coefficient can be appropriately corrected according to the coefficient at the part of the screen, so particularly in the frequency domain of the luminance signal component,
It is possible to configure an interpolation circuit that can obtain a resolution almost equal to that of the image signal before interpolation.

It has no effect on the parts that are not interpolated.

Since real-time operation is possible, for example, dynamic correction according to the state of the zoom lens is also possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing an entire embodiment of the present invention, FIG. 2 is a block diagram showing a detailed configuration of the coefficient generator 32 shown in FIG. 1, and FIG. 3 is an interface in this embodiment. Adjacent 4 during race scan
FIG. 4 is a diagram showing a pixel, FIGS. 4 to 9 are diagrams showing the principle of contour correction in the present embodiment, and FIGS. 10 and 11 are diagrams explaining the technology which is the premise of the present invention. 1,2,15,16 …… 1 field delay circuit, 3,3 ′, 17,17 ′…
... 1 line delay circuit, 4,5,6 ... Random access memory, 18,19,20 ... n line delay circuit (n = delay of RAM4,5,6), 7,21 ... Multiplexer, 8, 9,22,23 …… 1 pixel delay element, 10,11,12,13,25,26,27,28 …… Coefficient unit, 14,
29,30,30 '... Adder, 24 ... Coefficient converter, 31 ... Motion detector, 32 ... Coefficient generator, 33 ... Synchronous signal generator, 34
…… Microcomputer, 35 …… Displacement control system.

Claims (2)

(57) [Claims] 1. Using a pixel data of a specific color output from a specific solid-state imaging device among a plurality of solid-state imaging devices as reference pixel data, and moving pixel data of another color to align with the reference pixel data. An adaptive contour correction circuit that corrects the spatial frequency deterioration caused by the registration correction in real time when performing the registration correction, the range being smaller than 1 pixel required for performing the registration correction. Coefficient output means for outputting interpolation coefficient data representing the amount of shift of the interpolation coefficient, coefficient conversion means for generating a complementary coefficient of the interpolation coefficient data, and multiplying the reference pixel data by the complementary coefficient to obtain an inverse frequency Processing means for forming characteristic data; and for the pixel data other than the reference pixel data, the inverse frequency And an adding means for adding characteristic data. 2. The pixel data of a specific color output from a specific solid-state imaging device among a plurality of solid-state imaging devices is used as reference pixel data, and the pixel data of another color is moved to align with the reference pixel data. An adaptive contour correction circuit for correcting in real time the deterioration of spatial frequency caused by the registration correction when performing the registration correction, wherein a plurality of color video signals output from the plurality of solid-state image pickup devices are Pixel data storage means for storing the data in the form of data and registration correction information for compensating the registration deviation between the respective pixel data are stored in advance by capturing a predetermined test image using the solid-state image sensor. The registration correction information storage means and the reference pixel data are synchronized with each other. Waveform generating means for generating a plurality of predetermined waveform data; a plurality of multiplying means for multiplying each of the waveform data by a predetermined correction coefficient based on the registration correction information; Accumulate each obtained data,
First adding means for outputting the correction waveform data, and interpolation coefficient data representing a shift amount in a range smaller than one pixel required for compensating the registration shift based on the correction waveform data. Output from the solid-state image pickup device by multiplying coefficient data output means and pixel data stored in the pixel data storage means or pixel data read from the pixel data storage means by the interpolation coefficient data Interpolation means for equivalently moving the generated pixel data within a range not exceeding one pixel, coefficient conversion means for generating complementary coefficients of the interpolation coefficient data, and multiplying the reference pixel data by the complementary coefficients. Processing means for forming the inverse frequency characteristic data, and the inverse frequency characteristic data for the pixel data other than the reference pixel data. Adaptive contour correction circuit, characterized by comprising a second adding means for adding data.