JP4131049B2 - Signal processing apparatus and signal processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a prediction coefficient generation method and a signal processing apparatus, and more particularly to a prediction coefficient generation method and a generated prediction coefficient for generating a prediction coefficient used when classifying and performing interpolation processing or the like for each class. The present invention relates to a signal processing apparatus that performs processing by using the signal processing apparatus.
[0002]
[Prior art]
In recent years, various techniques for improving the resolution or sampling frequency of video and audio signals have been proposed. For example, in the case of up-conversion from a standard television signal corresponding to standard or low resolution to a so-called HDTV signal of high resolution, or in the case of performing sub-sample interpolation, for example, Japanese Patent Laid-Open It is known that a better result can be obtained by performing the conversion process with classification as disclosed in JP-A-6-178277 and JP-A-7-79418.
[0003]
The conversion process accompanied by the class classification is, for example, when converting a standard television signal (SD signal) corresponding to a standard or low resolution into a high resolution signal (HD signal), and classifying the input SD signal into a class classification. Then, a class code is generated, and a prediction coefficient preset for each class is selected using this class code, and the input SD signal is subjected to a predictive calculation process to obtain an HD signal. For the prediction calculation process, for example, a linear linear expression is used.
[0004]
In addition to the up-conversion from the SD signal to the HD signal as described above, the conversion process with the above classification is performed when performing interlace / progressive conversion for the scanning method of the video signal, converting the number of scanning lines, Various signal conversions such as converting a blurred image into a clear image, electronic zoom processing, frame number conversion processing (for example, conversion between 24 and 30 frames per second), format conversion processing, etc. Favorable results have been obtained when applied to processing.
[0005]
[Problems to be solved by the invention]
By the way, in the case of improving the resolution of the original signal by the above-described conversion process with classification, for example, the degree of improvement in the portion with large blur may be lower than that in the portion with small blur. In order to increase the degree of improvement in the portion where the blur is large, it is conceivable that pixels are thinned out in advance during classification, or that the interval between prediction taps during prediction calculation is thinned out.
[0006]
However, when the above thinning is performed, there is a drawback that the resolution cannot be sufficiently improved for a portion with a small blur.
[0007]
The present invention has been made in view of the above-described circumstances. For an input signal such as a video signal having a small blur portion and a large blur portion, an optimal resolution improvement for each portion is performed. It is an object of the present invention to provide a signal processing apparatus and a signal processing method that can perform conversion processing.
[0008]
[Means for Solving the Problems]
In order to solve the above-described problem, the present invention uses the data of the target position to be processed of the input signal and a plurality of data around it as a class tap, obtains class classification information from the class tap, and classifies the input signal as a class. Class classification means for classifying, prediction coefficient output means for outputting a prediction coefficient preset for each class according to the class classified by the class classification means, and data on the target position to be processed by the input signal And a plurality of data around it as a prediction tap, a prediction calculation means for predicting and calculating the prediction tap using a prediction coefficient from the prediction coefficient output means, and outputting a prediction value of the data of the target output signal; and in response to an input signal and a decimation control means for controlling each sampling interval of the input data to the input data and the prediction calculation means to said classification means The thinning-out control means includes an arithmetic means for obtaining, as an activity, an absolute value sum of errors between a value obtained by interpolating the thinned data and a true value at a plurality of preset thinning intervals, The activity obtained by the means is compared with a predetermined threshold value in order from the smallest thinning interval, and the thinning rate determining unit determines the thinning interval as the optimum thinning rate when the predetermined threshold value is exceeded for the first time .
[0009]
In the thinning control, it is preferable to obtain an activity representing the degree of change for a plurality of preset thinning intervals, and to determine the optimum thinning interval based on the obtained activity.
[0010]
In the thinning-out control, it is preferable that the activities are obtained in order starting from the smallest thinning-out interval, and the thinning-out interval when the activity for each thinning-out interval exceeds a predetermined threshold for the first time is determined as the optimum thinning-out rate.
[0011]
Specifically, class taps are generated for the class classification, prediction taps are generated for the prediction calculation, and the thinning rate for these class taps and prediction taps is controlled.
[0012]
Optimal thinning of the class tap and prediction tap intervals at the target position where the input signal is to be processed, and processing is performed with a short tap interval when the blur of the original input signal is small and with a long tap interval when the blur is large .
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments according to the present invention will be described below with reference to the drawings.
First, as a signal processing apparatus according to the present invention, a specific example of a signal conversion apparatus that performs the conversion process with the above-described class classification will be described with reference to FIG. In this specific example, an image information conversion device that converts the resolution of an image SD signal into an HD signal is assumed.
[0014]
By the way, for the class classification in the conversion process with the class classification described above, the data of the current position to be processed and a plurality of data around it are used, and these data correspond to the taps of the filter. Therefore, it is called a class tap. The prediction calculation process is also referred to as a prediction tap because it uses the data of the position of interest and a plurality of data around it, and these data are taps of the prediction filter.
[0015]
In FIG. 1, a standard resolution SD signal is supplied to the input terminal 11 and sent to the prediction tap generation circuit 12 and the class tap generation circuit 15 via the thinning control unit 20. The prediction tap output from the prediction tap generation circuit 12 is sent to the prediction calculation circuit 13 for prediction calculation. The class tap from the class tap generation circuit 15 is supplied to the class classification circuit 16 for class classification, and the prediction coefficient ROM 17 is accessed by using the determined class index as an address, and the prediction coefficient is read out. It is sent to the prediction calculation circuit 13.
[0016]
The prediction calculation circuit 13 is also referred to as an estimation calculation circuit or an interpolation value generation circuit. The prediction calculation circuit 13 uses the prediction coefficient adaptively selected for each class and supplied from the prediction coefficient ROM 17 as a prediction tap from the prediction tap generation circuit 12. Based on the prediction calculation, HD data for the input SD data is calculated. The HD signal obtained by the prediction calculation in the prediction calculation circuit 13 is taken out from the output terminal 14.
[0017]
The class classification circuit 16 outputs class information by subjecting the class tap from the class tap generation circuit 15 to, for example, ADRC (adaptive dynamic range coding). The adaptive dynamic range coding in the class classification circuit 16 is originally developed for high-efficiency compression coding of video signals, as disclosed in Japanese Patent Application Laid-Open No. 61-144899. Since the level distribution pattern of the video signal can be efficiently expressed with a short word length, information for class classification is obtained based on this ADRC encoded data. That is, the dynamic range DR in the block that is a coding unit of the video signal is determined by the maximum value MAX and the minimum value MIN in the block,
DR = MAX-MIN + 1
The requantization code Q when the level L of each pixel data in the block is requantized to n bits is
Q = [(L−MIN + 0.5) · 2 ⁿ / DR]
However, [] means truncation processing. Information for classifying the video signal of the block is obtained using the requantization code Q in the block.
[0018]
This class information is sent to the prediction coefficient ROM 17 as a class code or a class index, and a prediction coefficient corresponding to the class is read out and sent to the prediction calculation circuit 13.
[0019]
Next, the thinning control unit 20 will be described.
The thinning control unit 20 is adapted to change the thinning rate of the class tap or the prediction tap according to the degree of change of the input signal, for example, whether the change is gentle or the degree of blur in the case of the image signal. Control. Specifically, in the thinning control unit 20 in FIG. 1, the SD signal from the input terminal 11 is sent to the activity calculation circuit 21 to calculate the activity, and the thinning rate determination circuit 22 determines the thinning rate according to this activity. To do. The tap thinning rate of the prediction tap generation circuit 12 and the class tap generation circuit 15 is controlled by the determined thinning rate. The delay circuit 23 of the thinning control unit 20 delays the input signal by the processing time in the activity calculation circuit 21 and the thinning rate determination circuit 22.
[0020]
The control of the thinning rate based on the activity of the input signal will be described with reference to FIGS. Here, for the sake of simplicity, it is assumed that the resolution conversion processing of an image is performed using a one-dimensional class tap and a prediction tap, but it goes without saying that it can be easily extended to two dimensions and three dimensions.
[0021]
As shown in FIG. 2A, it is assumed that the input pixel is sampled in a one-dimensional direction and takes five class taps and 13 prediction taps around the target pixel to be processed. And If there is a high frequency component in the vicinity of the target pixel, resolution improvement processing with a narrow tap interval will improve resolution, but there will be little high frequency component in the vicinity of the target pixel and it will change smoothly. If so, it is known that there is little improvement. In that case, the resolution is improved by taking a tap so as to cover a wide area with a thinning interval as shown in FIG. In this case, paying attention to the optimum relationship among the activity of the input image signal, the thinning rate, and the improvement level, in the embodiment of the present invention, the activity of the input image signal is obtained by calculation. Based on the above, the thinning rate of the class tap and the prediction tap is determined.
[0022]
2A and 2B, the hatched pixels indicate class taps and prediction taps that are extracted from the input pixels and sent to the next stage, and FIG. 2A is when there are many high-frequency components. , (B) respectively show a case where the high frequency component is small.
[0023]
Here, a specific example of the activity is based on a dense (without thinning) one-dimensional class tap as shown in FIG. 2A, and one or more pixels as shown in FIG. 2B are thinned out. The sum of absolute values of interpolation errors in class taps is regarded as activity. The optimum thinning-out interval is determined by comparing the activity thus obtained with a predetermined threshold th.
[0024]
FIG. 3A shows a one-dimensional input signal with the vertical axis as the level direction, and shows an example in which class taps are taken by one decimation. When the pixel values of the five class taps obtained after the thinning are X _{1 to} X ₅ and the pixel values thinned between these taps are Y ₁ to Y ₄ , the center X ₃ is the target pixel. is there. The activity at this time (that is, the average of the sum of absolute values of interpolation errors in this embodiment) A ₁ is
A ₁ = {Σ | Y _i − (X _i −X _{i + 1} ) / 2 |} / 4
However, 1 ≦ i ≦ 4
It is expressed by the following formula.
[0025]
FIG. 3B shows an example in the case of two thinning out similarly. In this case, for the five class taps (pixel values X _{1 to} X ₅ ), two pixels are thinned out between the taps, and these are sequentially assigned to Y ₁ , Y ₂ , Y ₃ , Y _4. , Y ₅ , Y ₆ , Y ₇ , Y ₈ . The activity (average of the sum of absolute values of interpolation errors) A ₂ at (B) in FIG.
A ₂ = {Σ | Y _2i-1 − (2X _i + X _{i + 1} ) / 3 | +
| Y _2i − (X _i + 2X _{i + 1} ) / 3 |} / 8
However, 1 ≦ i ≦ 4
It is expressed by the following formula.
[0026]
In the same manner, the average value (A ₃ , A ₄ ,...) Of the sum of absolute values of interpolation errors when the thinning interval is increased is obtained. When there is no decimation, the activity A ₀ is
A ₀ = α {Σ | X _i −X _{i + 1} |} / 4
However, α <1, 1 ≦ i ≦ 4
The average value of the sum of absolute values of adjacent pixel differences expressed by the following equation is used.
[0027]
The activities obtained as described above are sequentially compared with the predetermined threshold th from A ₀ , and the thinning rate indicated by the activity when it becomes larger than the threshold for the first time is determined as the optimum thinning rate.
[0028]
If this is expressed as a logical expression,
if (A ₀ > th) No decimation
else if (A ₁ > th) 1 decimation
else if (A ₂ > th) 2 decimation
else if (A ₃ > th) 3 decimation
else if (A ₄ > th) 4 thinning
else if (A ₅ > th) 5 decimation
It becomes.
[0029]
FIG. 4 is a flowchart for explaining the procedure for determining the thinning rate based on the activity in the thinning rate determination circuit 22 in FIG. 1, and shows a specific example in the case of performing the thinning rate up to eight.
[0030]
In FIG. 4, in the first step S41, it is determined whether or not the activity A ₀ obtained by the average value of the absolute value sum of the adjacent pixel differences is larger than a predetermined threshold th. If YES, the process proceeds to step S42, where the thinning rate determination circuit 22 generates a code without thinning and sends it to the prediction tap generation circuit 12 and the class tap generation circuit 15 in FIG. When NO is determined in the step S41, the process proceeds to a step S43, and it is determined whether or not the activity A ₁ obtained as an average value of the sum of absolute values of the interpolation errors at the time of the one thinning is larger than a predetermined threshold th. Judging. If it is determined as YES in step S43, the process proceeds to step S44 to generate one thinning code, and if it is determined NO, the process proceeds to the next step S45. In step S45, it is determined whether or not the activity A _{2 at} the time of the two thinning is greater than a predetermined threshold th. If YES, the process proceeds to step S46 to generate a two thinning code and determine NO. When it is done, proceed to the next step. In the same manner, the above processing is sequentially performed for 3 to 6 thinning out. When the activity A _{6 at} the time of 6 thinning is equal to or less than the predetermined threshold th, the process proceeds to step S47, and the activity A _{7 at} the time of the 7 thinning is determined as the predetermined. It is determined whether or not it is larger than the threshold th. If it is determined as YES in step S47, the process proceeds to step S48 to generate a 7 thinning code, and if it is determined to be NO, the process proceeds to the next step S49 to generate an 8 thinning code. Of course, the 1 to 8 decimation codes generated by the decimation rate determination circuit 22 are sent to the prediction tap generation circuit 12 and the class tap generation circuit 15 of FIG.
[0031]
Next, a learning apparatus for obtaining the prediction coefficient for each class of the conversion process with class classification shown in FIG. 1 by the least square method will be described with reference to FIG. In this specific example, as the signal processing apparatus shown in FIG. 1, a standard television signal (SD signal) corresponding to a standard or low resolution video signal is converted into a so-called HDTV signal (HD signal) which is a high resolution video signal. FIG. 5 shows a learning apparatus for performing learning for obtaining a prediction coefficient used in a conversion apparatus that converts this SD signal into an HD signal, assuming a conversion apparatus.
[0032]
The input terminal 51 in FIG. 5 is supplied with a high-resolution HD signal as a teacher signal. The HD signal is sent to the down-converter 52 and subjected to low-pass filtering processing, down-sampling processing, etc. A corresponding pseudo standard definition SD signal is obtained. These HD signal and SD signal are sent to the normal equation generation circuit 53. The SD signal from the down converter 52 is sent to the prediction tap generation circuit 54 and the class tap generation circuit 55 via the thinning control unit 60.
[0033]
The thinning control unit 60 has the same configuration as the thinning control unit 20 in FIG. 1, and the downconverted SD signal is sent to the activity calculation circuit 61 to calculate an activity. Based on this activity, the thinning rate is reduced. The determination circuit 62 determines the thinning rate of the prediction tap and the class tap. The delay circuit 63 delays the input signal by the processing time in the activity calculation circuit 61 and the thinning rate determination circuit 62. The activity calculation circuit 61, the thinning rate determination circuit 62, and the delay circuit 63 correspond to the activity calculation circuit 21, the thinning rate determination circuit 22, and the delay circuit 23 of FIG. 1, respectively. Since it is the same as each part, description is abbreviate | omitted.
[0034]
The prediction tap generation circuit 54 and the class tap generation circuit 55 to which the SD signal is supplied via the delay circuit 63 of the thinning control unit 60 have a thinning control signal or a thinning code from the thinning rate determination circuit 62 of the thinning control unit 60. Is input, and the thinning interval of each tap is controlled. The prediction tap from the prediction tap generation circuit 54 is sent to the normal equation generation circuit 53, the class tap from the class tap generation circuit 55 is classified by the class classification circuit 56, and class information is sent to the normal equation generation circuit 53. It is done. The operations of the prediction tap generation circuit 54, the class tap generation circuit 55, and the class classification circuit 56 are the same as those of the prediction tap generation circuit 12, the class tap generation circuit 15, and the class classification circuit 16 shown in FIG. Is omitted.
[0035]
The normal equation generation circuit 53 is a circuit that generates a normal equation for each class. The normal equation generation circuit 53 sends a normal equation for each class obtained after the learning material is input to the simultaneous equation solving circuit 57, and for each class. A simultaneous equation is solved from the normal equation to obtain a prediction coefficient for each class, and the prediction coefficient for each class is stored in the prediction coefficient memory 58. The stored contents of the prediction coefficient memory 58 are written and used in the prediction coefficient ROM 17 of FIG.
[0036]
According to the embodiment of the present invention as described above, the interval between the class tap and the prediction tap at the target position to be processed of the input signal is optimally thinned, and a short tap interval at a place where the blur of the original input signal is small. Thus, by performing the processing at a long tap interval in a place where the blur is large, there is no decrease in the degree of improvement in a portion where the blur is large or a portion where the blur is large, and the resolution can be further improved.
[0037]
The present invention is not limited to the above-described embodiment. For example, the activity calculation circuit 21, the thinning rate determination circuit 22, and the delay circuit 23 are represented by hardware, but can be realized by software. Of course. In addition, the present invention is not limited to the case of converting a standard resolution video signal (SD signal) to a high resolution video signal (HD signal). Of course, the present invention can also be applied to the case where an audio signal having a sampling frequency or the number of sample bits is converted into an audio signal having another sampling frequency or the number of sample bits.
[0038]
【The invention's effect】
According to the present invention, when the input signal is classified, and the prediction calculation is performed using the prediction coefficient prediction coefficient corresponding to the classified class and the prediction value is output, the classification and the classification are performed according to the input signal. By controlling each thinning interval of the data to be subjected to the prediction calculation, signal processing is performed at an optimum thinning rate corresponding to each part of the input signal, and the quality can be further improved.
[0039]
In addition, as the thinning control, an activity indicating the degree of change in a plurality of preset thinning intervals is obtained, the activities are obtained in order from the smallest thinning intervals, and the activity for each thinning interval has a predetermined threshold value. By determining the thinning interval when it exceeds the first time as the optimum thinning rate, the optimal thinning interval can be reasonably obtained.
[0040]
Further, class taps are generated for the class classification, and prediction taps are generated for the prediction calculation. The decimation rate for these class taps and prediction taps is controlled, and the input signal is processed. By optimizing the interval between the class tap and the prediction tap at the target position, processing is performed with a short tap interval in a portion where the blur of the original input signal is small, and a long tap interval in a portion where the blur is large. However, even in the part where the blur is large, the improvement degree does not decrease and the quality can be further improved.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an example of an image information conversion apparatus that performs conversion processing with class classification as an embodiment of the present invention.
FIG. 2 is a diagram showing when a high frequency component is large and when a class tap and a prediction tap extracted from an input signal are large.
FIGS. 3A and 3B are diagrams showing a signal change of a class tap taken out from an input signal, when thinning out one time and when thinning out two times, respectively;
FIG. 4 is a flowchart for explaining a procedure for determining a thinning rate based on an activity of an input signal.
FIG. 5 is a block diagram showing an example of a learning apparatus for obtaining a prediction coefficient of conversion processing with class classification for explaining an embodiment of the present invention.
[Explanation of symbols]
12 prediction tap generation circuit, 13 prediction calculation circuit, 15 class tap generation circuit, 16 class classification circuit, 17 prediction coefficient ROM, 20 decimation control unit, 21 activity calculation circuit, 22 decimation rate determination circuit

Claims (7)

Class classification means for classifying the input signal by obtaining class classification information from the data of the target position to be processed of the input signal and a plurality of data around it as class tap ,
Prediction coefficient output means for outputting a prediction coefficient preset for each class according to the class classified by the class classification means;
The target position data to be processed of the input signal and a plurality of data around it are set as prediction taps, and the prediction tap is predicted using the prediction coefficient from the prediction coefficient output means, and the target output signal is output. A prediction calculation means for outputting a predicted value of the data ;
In response to the input signal have a and decimation control means for controlling each sampling interval of the input data to the input data and the prediction calculation means to said classification means,
The decimation control means is
An arithmetic means for obtaining, as an activity, an absolute value sum of errors between a value obtained by interpolating the thinned data and a true value for each of a plurality of preset thinning intervals;
A decimation rate determination unit that compares the activity obtained by the calculation unit with a predetermined threshold value in order from the smallest decimation interval and determines the decimation interval when the predetermined threshold value is exceeded for the first time as an optimum decimation rate. Signal processing device. The signal processing apparatus according to claim 1, wherein the class classification information is obtained from encoded data when the class tap is compression encoded . The signal processing device according to claim 1, wherein the number of data of the prediction tap is larger than the number of data of the class tap . A class tap generation means is provided in the preceding stage of the class classification means, and a prediction tap generation means is provided in the previous stage of the prediction calculation means, and the thinning-out control means includes each tap in the class tap generation means and the prediction tap generation means. the signal processing apparatus according to claim 1, wherein that controls the thinning ratio for obtaining. The input signal is a video signal, each tap in the class-tap generating means and the prediction tap generating means, Ru 1D tap or two-dimensional tap der claim 4 signal processing apparatus according. 6. The signal processing apparatus according to claim 5, wherein the target output signal is a video signal having a higher resolution than the input signal. A class classification step for classifying the input signal by obtaining class classification information from the data of the target position to be processed of the input signal and a plurality of data around the class tap, and class classification information from the class tap ;
A prediction coefficient output step for outputting a prediction coefficient preset for each class according to the class classified by the class classification step ;
The target position data to be processed of the input signal and a plurality of data around it are set as prediction taps, and the prediction tap is predicted using the prediction coefficient from the prediction coefficient output step, and the target output signal A prediction calculation step for outputting a predicted value of the data;
A decimation control step for controlling each decimation interval of the input data to the class classification step and the input data to the prediction calculation step according to the input signal ,
The decimation control step is
A calculation step for obtaining, as an activity, an absolute value sum of errors between a value obtained by interpolating the thinned data and a true value for each of a plurality of preset thinning intervals,
A decimation rate determination step of determining the decimation interval as the optimum decimation rate when the activity obtained by this calculation step is compared with a predetermined threshold in order from the one with the smallest decimation interval and the predetermined threshold is exceeded for the first time Signal processing method.

Images