JPH0411045B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Technical field to which the invention relates] The present invention relates to a signal processing device that decomposes and/or combines signals. That is, in the signal processing device of the present invention, the highest frequency to be processed is
f real time of the frequency spectrum of the information component (with one or more dimensions) of a given temporal signal not higher than ₀ (with an unavoidable inherent time delay between input and output of the device) ) decomposition and/or real-time (but with unavoidable inherent time delays between input and output of the device) synthesis of such time signals from their decomposed frequency spectra; It is used. In addition, in this specification, as mentioned above, when there is an unavoidable time delay inherent to the device between the input of a signal to the device, such as a pipeline structure, and the output thereof, Here, real-time processing (processing such as decomposition and composition) will be simply referred to as real-time processing (decomposition and composition). Although the invention is sometimes suitable for two-dimensional spatial frequency real-time image processing of television images defined by temporal video signals, it is not limited thereto.

[Description of prior art]

Many studies have been conducted on creating models of the operation of the human visual system. It has been found that it appears to calculate the primitive spatial frequency decomposition of luminescent images. Each band has a width of approximately one octave, and the center frequency of each band differs from its neighbors by approximately twice as much. Research has shown that the spatial frequency range of the human visual system has approximately seven bands or "channels" ranging from 0.5 to 60 cycles/degree. The significance of these findings is that the human visual system independently processes spatial frequency information that is more than twice as far away from other spatial frequency information.

It is also known that the processing of spatial frequencies that occurs in the human visual system is spatially localized. The signal in each spatial frequency channel is therefore calculated over small subregions of the image, which subregions overlap each other and have a width of approximately two cycles at a particular frequency.

Using a sinusoidal grating image as a test pattern, it can be seen that the threshold contrast versus sensitivity function for this image rolls off quickly as the spatial frequency of the image increases. That is, higher spatial frequencies require higher contrast (approximately 20% at 30 cycles/degree), whereas lower spatial frequencies require relatively lower contrast (approximately 0.2% at 3 cycles/degree).

The ability of the human visual system to detect changes in the contrast of sinusoidal grating images above a threshold has also been found to be better at low spatial frequencies than at high spatial frequencies.
Specifically, the average human needs 3 to correctly discriminate a contrast that changes 75% of the time.
A sinusoidal grating at cycles/degree requires a contrast change of approximately 12%, and a grating at 30 cycles/degree requires a contrast change of 30%.

Dr. Peter J. Burt, who is aware of the above-mentioned properties of the human visual system, uses a computer to decompose the two-dimensional spatial frequency of an image into multiple separate spatial frequency bands. We have developed an algorithm (hereinafter referred to as Bart's Pyramid).The width of each spatial frequency band (except for the lowest spatial frequency band) is preferably one octave.Therefore, the highest spatial frequency of interest for image processing is If not higher than f ₀ , its highest frequency band spans the octave from f ₀ /2 to f ₀ (with a center frequency of 3f ₀ /4), and the next band after this highest frequency band spans from f _{0 /4 to f 0} /4. It spans an octave up to f ₀ /2 (with a center frequency of 3f ₀ /8).
The same applies below.

We cite the following list of documents written or co-authored by Dr. Burt that detail various aspects of Burt's Pyramid.

IEEE Transactions on Systems Man and Cybernetics,
Man, and Cybernetics) December 1981 issue
SMC-11, line 12, pages 802-809, Bart et al., “Segmentation and Estimation of Image Region Characteristics by Joint Rank Calculation”
Region Properties Through Cooperative
Hierarchical Computation).

I.E.E. Transactions on Communications (IEEE
Transactions on Communications) April 1983
COM-31, No. 4, pp. 532-540, "The Laplacian Pyramid as a compact image code" by Bart et al.
a Compact Image Code).

Computer Vision Graphics and Image Processing
Vision, Graphics, and Image Processing)
21, 1983, pp. 368-382, Burt, “Fast Algorithms for Estimating Local Image Properties.”
Algorithms for Estimating Local Image
Properties)”.

Computer Graphics and Image Processing
Image Processing) 1980 No. 14 No. 271-280,
"Tree and Pyramid Structure for Encoding Hexagonally Extracted Binary Images" by Bart.
Pyramid Structures for Coding Hexagonally
Sampled Binary Images).

SPIE No. 360
Pyramid-based Extraction of Local Image Features Applicable to Motion and Tissue Analysis by Bart, pp. 114-124.
Features with Applications to Motion and
Texture Analysis).

Computer Graphics and Image Processing
Image Processing) 1981 No. 16, pp. 20-51,
"Fast Filter Conversion for Image Processing" by Bart.
Filter Transforms for Image Processing).

Rensselaer Institute of Technology, Department of Electrical and Computer Systems Engineering, Image Processing Laboratory, published June 1983,
“A Multiresolution Spline with Application to Image Mosaic” by Burt et al.
Applications to Image Mosaics).

Rensselaer Polytechnic Institute, Department of Electrical and Computer Systems Engineering, Image Processing Laboratory, published July 1982, "The Pyramid as a Structure for High-Efficiency Computing" by Burt.
as a Structure for Efficient
Computation)”.

Burt's pyramid algorithm uses a special sampling method to convert a relatively high-resolution original image into N orders (N is a plurality of integers) of component images, each consisting of an octave with a different spatial frequency of the original image. The term "pyramid" refers to the spatial frequency and sample density of each order of each component image. This is related to a continuous decrease from the highest octave component image to the lowest octave component image.

The first advantage of Burt's pyramid algorithm is that
The original high-resolution image can be synthesized from the component image and the residual image without introducing pseudo spatial frequencies due to aliasing.The second advantage is that the one-octave spatial frequency bandwidth of each order of the component image is The goal is to match the characteristics of people's visual systems. This allows each spatial frequency of each order of the component images to be selectively processed in a different and independent manner (i.e. without any signal processing of one component image to the detriment of all other component images). or can be modified to produce and enhance some other desired effects in the composite image derived from the processed component images. One example of this desired effect is the polyresolution spline technique detailed in the article "Polyresolution Splines Applied to Image Mosaic" cited above.

Until now, Burt's pyramid algorithm has been implemented in non-real time on general purpose digital computers. The level of each pixel sample of the original image is multi-bit (e.g. 8 bits) stored in each address location of the computer.
expressed in numbers. For example, each of the two dimensions is 2 ⁹
A relatively high-resolution two-dimensional original image composed of = 512 pixel samples has 2 ¹⁸ = 262144 storage locations for storing each multi-bit number representing the level of each pixel sample that makes up the original image. Requires large storage device.

The original image stored in the storage device can be processed by a digital computer according to Burt's pyramid algorithm. This process consists of iteratively performing steps such as convolving the pixel samples with a predetermined kernel weight function, subtracting the samples, expanding the samples by interpolation, and subtracting the samples. The size of the kernel function (one or more dimensions) is small (in terms of the number of pixels) compared to the size of each dimension of the entire image. Pixel subregions or windows (the size is equal to the kernel function and are arranged symmetrically around each pixel) are multiplied by the kernel weight function and summed by a convolution operation.

The kernel weighting functions are chosen to act as low-pass filters of the multidimensional spatial frequencies of the image being convolved. The nominal "cutoff frequency" (also known in filter technology as the "corner frequency" or "break frequency") of the low-pass filter characteristic given to each dimension by the kernel function is the frequency of that dimension of the signal being convolved. It is chosen to be effectively 1/2 of the highest frequency in question. This low-pass filter characteristic need not have a "brick wall" rolloff at a given cutoff frequency, but may have a relatively gradual rolloff, in which case the nominal cutoff frequency is Some pre-selected attenuation value (e.g.
3dB) is defined as the frequency at which this occurs. Because Burt's pyramid inherently compensates for the aliasing introduction of spurious frequencies caused by progressive roll-off low-pass filter characteristics, filters with more gradual roll-off characteristics can be used. The convolved image is decimated by effectively sampling every other convolved pixel in each considered dimension of the image sequentially;
This reduces the number of pixels in the convolved image in each dimension by half. Since the image is usually a two-dimensional image, the convolved and thinned image consists of only 1/4 of the number of pixels contained in the image before thinning. This convolved and thinned out image (this is called a Gaussian image) is stored in the second storage device.

Starting from the stored pixel samples of the original image, we perform the convolution-reduction process described above.
decimation) procedure is repeated N times (N is a plural integer, i.e. 2 or any integer greater than 2), the original high-resolution image and the reduced resolution N
(N+1) images consisting of a rank pyramid of Gaussian additional images are generated. Here, the number of pixel samples in each dimension (sample density) of each additional image is only 1/2 of the number of pixel samples in each dimension of the immediately preceding image. The original high resolution is represented by G ₀ representing the stored image, and the ranks of the stored N additional images being represented by G ₁ to G _N , respectively, and the pixel samples of these N additional images are sequentially decreased. The number is stored separately in N storage devices, respectively. Therefore, adding the stored original images, there is a total of (N+1) storage devices.

According to the non-real-time implementation of Burt's Pyramid Algorithm, the following operational steps generate an additional sample of the interpolated value between each memory G ₁ pixel sample pair in each dimension, thereby reducing the memory image The sample density of G ₁ is expanded to the sample density of the original stored image G ₀ . The digital value of each pixel sample of this enlarged image G ₁ is then subtracted from the stored digital value of the corresponding image sample of the original image G ₀ to generate a difference image (known as a Laplace image). This Laplace image (denoted by L ₀ ), which has the same sample density as the original image G ₀ , combines the spatial frequencies contained in the original image within the octave from f ₀ /2 to f ₀ and often the reduced G ₁ image. from the derivation of the sample density and a small low spatial frequency error compensation component corresponding to the information loss caused respectively by the reduction step used to introduce the interpolated samples that occurs when expanding the sample density to that of the original image _G0 . Become. This Laplace image L ₀ is then (N+1)
The image is stored in the first of the pyramid storage devices in place of the original image.

By repeating this procedure in the same way, additional (N-1) Laplace images L ₁ to
A rank consisting of L _N-1 is derived and the Gaussian image G ₁
or an additional (N-1) in which G _N-1 is stored
(so that the Gaussian images G ₁ to 1 in the memory
G _N-1 is replaced). A Gaussian image G _N (with the lowest sample density) is not replaced with a Laplace image in its corresponding storage, and is a Gaussian image consisting of the lowest spatial frequencies (i.e. less than L _N-1 octaves) contained in the original image. remains in the storage device as an afterimage.

According to Burt's pyramid algorithm, the stored afterimage G _N is expanded to the sample density of the image L _N-1 , this is added to the stored Laplace image L _N-1 to create a sum image, and then this sum image is The process of enlarging the image _L can be recovered. Additionally, after decomposing one or more original images into N Laplace images and Gaussian afterimages, any necessary special image processing or modification steps (e.g. splining) can be performed before synthesizing the complete high-resolution image from this. It can also be introduced.

The non-real-time implementation of Burt's Pyramid Algorithm by computer processing is effective for processing fixed image information;
cannot be applied to the decomposition of Decomposition of such time-varying sequential images requires real-time implementation of Burt's pyramid algorithm as provided by the present invention.

[Summary of the invention]

More specifically, the present invention relates to a signal processing device that uses a pipeline configuration to decompose the frequency spectrum of the information component of a given temporal signal in delayed real time. The highest frequency to be processed (of interest) in the frequency spectrum is f ₀ (i.e., there may be frequencies higher than f ₀ in the spectrum that are of no interest and therefore not directly targeted for processing). ), and the information component of the given time signal corresponds to information with a given number of dimensions. The apparatus includes means for relaying a set of N separate sampled signals arranged in ordinal order (N being a plural integer;
2 or any integer greater than 2), each relay means having first and second input terminals and first and second output terminals. A first input terminal of the first relay means of this group of means is coupled to receive a given time signal, and a first input of each of the second to Nth relay means is connected to the respective immediately preceding relay means. coupled to each first output terminal of
Each of the second to Nth relay means is adapted to send its signal to each relay means immediately following it.
and a second input terminal of each relay means is coupled to receive a respective separate sampling clock signal;
With this arrangement, each relay means produces a signal at its first and second output terminals at a frequency equal to the sampling frequency of the clock signal applied thereto.

Further, each relay means connects its first input terminal to the first input terminal.
exhibits a low-pass transfer function between its output terminals for the information component of the signal applied to its first input terminal. The low-pass transfer function of each relay means is a direct function of the sampling frequency of the clock signal applied to the second input of that relay means (as the value of the independent variable increases (or decreases), the value of the dependent variable also increases (or decreases)). has a nominal cut-off frequency that is an increasing (or decreasing) function). The clock signal applied to the second input terminal of the first relay means of the relay means is (a) twice f ₀ and (b) has a low pass of the first relay means in the above information component. have a sampling frequency that gives a nominal cut-off frequency for the transfer function that is less than f ₀ . Also, each second relay means of the second to Nth relay means of the means group
the clock signal applied to the input terminal of the respective relay means is (a) lower than the clock frequency applied to the second input terminal of the relay means immediately preceding the respective relay means, and (b) the clock signal applied to its first input terminal (c) providing its low-pass transfer function with a nominal cut-off frequency that is at least equal to twice the highest frequency of the information component transmitted; and (c) lower than the nominal cut-off frequency of its immediately preceding relay means.

The signal drawn at the second output terminal of each relay means corresponds to the difference between the information component applied to its first input terminal and a direct function of the information component drawn to its first output terminal. .

The information components of a given temporal signal processed by the signal processing device of the invention correspond to respective two-dimensional spatial frequency components of successive frames of a television image linearly scanned, for example, in each of the two dimensions. There are some, but it is not limited to this.

In general, the invention is useful for decomposing the frequency spectrum of a signal derived from a spatial frequency or non-spatial frequency signal source in one or more dimensions regardless of the characteristics of the source, and thus for example On top of a two-dimensional visual image source such as
It is useful for decomposing one-, two-, three-, or more dimensional complex signals derived from signal sources such as audio, radar, seismic recorders, robots, etc.

Moreover, this invention uses a pipeline configuration,
The present invention also relates to a signal processing device that synthesizes a composite signal in delayed real time according to a group of decomposed signals.

[Description of recommended examples]

In FIG. 1, a group of extracted signals (sampled signals) relay means 10 arranged in ordinal order
0-1 to 100-N (where N is a plural integer, i.e. 2 or any integer greater than 2) each has two input terminals and two output terminals, and is used for a given temporal signal defining information. G ₀ is applied as an input to the first of the two input terminals of the first relay means 100-1. Time signal G ₀
may be a continuous analog signal (such as an audio or video signal) or a sampled analog signal, but in the latter case it may also be expressed in terms of each sample level and directly in amplitude level (i.e. the time signal G ₀ is It may also be represented indirectly in digital numbers (by passing the amplitude level of each sample through an analog-to-digital converter, not shown in FIG. 1) before being applied to the first input terminal of 100-1. The frequency spectrum of G ₀ is 0 (i.e. DC)
to a certain frequency f ₀ (ie, a range including all frequencies to be processed corresponding to information of a given number of dimensions). In other words, it may be a pre-filtered signal that does not contain frequencies higher than G ₀ and f ₀ . In this case, the clock frequency 2f ₀ of the relay means 100-1 satisfies the Nyquist criterion for all frequency components of f ₀ . However, G ₀ may also include some untargeted frequency components higher than f ₀ .
In this latter case, the Nyquist criterion is not satisfied;
Some aliasing occurs, but from a practical point of view, such aliasing is not desirable, but is often acceptable (as long as it is not too large).

In FIG. 1, each of the other relay means 100-2...
The first input terminal of 100-N is coupled to the first of the two output terminals of each relay means immediately preceding it. That is, the first signal relay means 100-1
The output terminal of the relay means 100-2 is coupled to the first input terminal of the relay means 100-2, the first output terminal of the relay means 100-2 is coupled to the first input terminal of the relay means 100-3 (not shown), and... Relay means 100 without
-(N-1)'s first output terminal is the relay means 100
-N's first input terminal. In this manner, the signal processing apparatus shown in FIG. 1 uses a pipeline configuration for mutually coupling the relay means of the relay means group.

A different sampling frequency clock is applied to the second of the two input terminals of each relay means 100-1, . . . 100-N. Specifically, the relay means 100-1 receives the sampling frequency clock _CL1 as its second input, and the relay means 100-1 receives the sampling frequency clock CL1 as its second input.
0-2 has the sampling frequency clock CL ₂ applied as its second input, and...relay means 100-
N has a sampling frequency clock CL _N applied as its second input. The mutual values CL ₁ , CL ₂ . . . CL _N of each clock frequency are constrained as shown in FIG. 1, and the meaning of this constraint will be explained in detail below.

Further, in FIG. 1, the relay means 100-1 draws out the second output terminal _L0 to its second output terminal, and similarly, the other relay means 100-2...100-N each have their second output terminal L0. the second output signal L ₁ ,
…Pull out L _N-1 .

Each of the relay means 100-1, ... 100-N is connected between the first input terminal and the first output terminal of the input signal applied to the first input terminal, regardless of its special internal structure. It can be viewed as a black box that exhibits a low-pass transfer function for the frequency spectrum of its components. Furthermore, the low-pass transfer function of each relay means 100-1, 100-2, ... 100-N has a roll-off with a nominal cut-off frequency that is a direct function of the sampling frequency applied to its second input terminal. have As mentioned above,
In the case of Bart's Pyramid, the Rohrhof is not a "brick wall" but a gradual one.

To be more specific, the above-mentioned input signal G ₀ is applied to the first input terminal of the relay means 100-1,
The highest frequency to be processed in the frequency spectrum of G ₀ is not higher than f ₀ . Also, relay means 1
The frequency CL ₁ of the sampling clock applied to the second input terminal of G 0 is equal to 2f ₀ (i.e., the frequency that satisfies the Nyquist criterion for all frequencies to be processed in the frequency spectrum of G ₀ ). Under this condition, the low-pass transfer function between the first input terminal and the first output terminal of the relay means 100-1 is not greater than f ₁ in the frequency spectrum of G ₀ (provided that f ₁ < f ₀ ) Frequency is the only relay method 1
00-1, and thereby the first output terminal of the relay means 100-1.
An output signal G ₁ is drawn at the output terminal of , which has a frequency spectrum (determined by the particular characteristics of its low-pass transfer function) that essentially consists of the lower part of the frequency spectrum of G ₀ . This signal G ₁ is then applied as an input to the first input terminal of the relay means 100-2.

As shown in FIG. 1, the sampling frequency (applied to the second input terminal of the relay means 100-2) is lower than 2f ₀ (sampling frequency of clock CL ₁ ), but at least 2f ₁ (sampling frequency of clock CL ₁ ). (twice the highest frequency f ₁ of the frequency spectrum). Therefore the clock
The sampling frequency of CL ₂ is the relay means 100-1.
is not high enough to satisfy the Nyquist criterion for the highest frequency of interest f ₀ that may be present in the frequency spectrum of G ₀ applied to the first input terminal of 1
It is still sufficient to satisfy the Nyquist criterion for the frequency spectrum of G ₁ applied to the first input terminal of 00-2. The form of this relationship (the higher the normal position of a relay means, the higher the
(the sampling frequency of the clock applied to the input terminal of the clock is lower) is generally applied. Specifically, the sampling frequency of the clock applied to the second input terminal of each of the relay means 100-2, . . . 100-N is as follows: (b) at least equal to twice the highest frequency of the information component of the signal applied to its first input terminal; and (c) its nominal cutoff frequency for the low-pass transfer relationship. The value is reduced to a value lower than that of the immediately preceding relay means. Therefore, the highest frequency f ₂ of the signal G ₂ occurring at the second output terminal of the relay means 100-2 is lower than f ₁ ...Finally, the frequency of the signal G _N (occurring at the output terminal of the relay means 100-N) The highest frequency f _N of the spectrum is the signal (appearing at the first output terminal of a relay means (not shown) immediately before the relay means 100-N and applied to the first input terminal of the relay means 100-N).
The highest frequency f of the frequency spectrum of G _N-1 is lower than _N-1 .

If we consider each of the relay means 100-1,...100-N as a black box again, each relay means 1
Each output signal L ₀ ,...L _N-1 drawn out to the second output terminal of 00-1,...100-N is the information component of the signal applied to the first input terminal of the relay means and its relay. It corresponds to the difference between the information component of the signal drawn at the first output terminal of the means and the forward function. Therefore, the first
As shown, L ₀ is equal to (or at least corresponds to) the difference G ₀ −g(G ₁ ). However, g (G ₁ ) is
Either G ₁ itself or some particular forward function of G ₁ . Similarly, if L ₁ is equal to (or at least corresponds to) G ₁ −g(G ₂ ), ...L _N-1 is G _N-1 − g(G _N )
is equal to (or at least corresponds to).

In general, each sampling clock frequency f ₀ ,...
The only restriction on the relative value of f _N-1 is as shown in FIG. 1, but typically each relay means 100-1,...
The value of the sampling clock frequency applied to the second input terminal of 100-N is expressed as each ratio CL ₂ /CL ₁ ,
It is convenient to specify CL ₃ /CL ₂ ...CL _N /CL _N-1 to be equal to 1/2 (or to an integer power of 1/2 according to the number of dimensions of the information component of the signal to be decomposed). be. As a result, the decomposition output of the frequency spectrum of the original signal G ₀ becomes the Laplace component signal L ₀ ,...
L each divided into _N-1 separate parallel frequency passbands,
Each bandwidth is one octave for each dimension of the information component (ignoring all sampling errors due to loss of signal information or addition of spurious aliasing frequency components caused by less than or equal to the sampling density), and It contains only frequencies that are within the frequency spectrum of the original signal G ₀ that falls within. In this case, the frequency of the frequency spectrum of the original signal G ₀ lower than the lowest octave Laplace component signal L _N-1 is the residual Gaussian signal G _N of the decomposition output.
include.

In general, N is a plurality of integers having an arbitrary predetermined value of 2 or more, but even if the predetermined value of N is relatively small, it is sufficiently high that all frequencies of interest in each dimension of the frequency spectrum of the original signal _G0 can be set sufficiently high. There is information in a format that can be resolved sufficiently with resolution. For example, in the case of visible images, it is often found that a value of N of 7 is sufficient, in which case the frequency of each dimension of the residual signal G _N is the highest frequency f ₀ to be processed in the frequency spectrum G ₀ of the original signal. lower than 1/128 (1/2 ⁷ ).

FIG. 1a shows in generalized form a first type of digital embodiment of each extracted signal relay means 100-1, . . . 100-N of the pipeline group of FIG. In the figure, any one of the first type embodiments among the relay means 100-1,...100-(N-1) is represented by 100 a -K, and the first type embodiment immediately after that relay means is represented by 100 _a -K.
00 _a - (K-1).

The relay means _100a -K includes an m-tap digital convolutional filter 102 (m is a plural integer of 3 or more, preferably an odd number), a decimator 104, and an expander 106.
and n-tap digital interpolation filter 108 (n is 3
or more, preferably odd integers), a delay device 109, and a subtractor 110. Each of these elements 102, 104, 106, 1
A sampling frequency clock CL _K (that is, the clock applied to the second input terminal of each relay means of the relay means group 100 _a -K in FIG. 1) is applied as each control input of 08, 109, and 110.

The signal G _K-1 applied to the first input terminal of the relay means group 100 _a -K is applied as an input to the convolution filter 102 and is also applied as an input to the subtracter 110 via the delay device 109 . Ru. 1st
_The sample density shown in FIG _.
is arranged in the time domain by the sampling frequency of the clock CL _K. Therefore, each sample making up G _K-1 is processed by filter 102.
The purpose of convolutional filter 102 (as described above with respect to FIG. 1) is to reduce the highest frequency of its output signal G _K with respect to the highest frequency of its input signal G _K-1 , whereas FIG. The sample density of the output of filter 102 is still at the sample frequency of _CLK , as shown in FIG.

The output of the filter 102 is a decimeter 1
04 input. The reducer 104 extracts only some (but not all) of the successive samples of each dimension applied to its input from the filter 102.
to its output. Therefore, the sample density of each dimension at the output of the reducer 104 is reduced than the sample density of that dimension at its input. In particular, as shown in FIG. 1a, the sample density CL _K+1 of each dimension of the output of the reducer 104 is applied to the second input terminal of the relay means 100 _a -(K+1) immediately thereafter. can be placed in the time domain at a low frequency defined by a clock CL _K+1 with a reduced sampling frequency.

Also, a reducer 104 placed within this time domain
A reduced sampling density of each dimension of the signal G _K of the output of is produced in phase with the sampling frequency clock CL K+1 applied to the second input terminal of the relay means 100 _a -( _K+1 ) immediately thereafter. In FIG. 1a, the G _K output signal of the reducer 104 (relay means 10
0 _a -K) is applied to the first input terminal of the immediately succeeding relay means 100 _a -(K+1). Therefore, the relay means 100 _a −(K+
1) The isochronous relationship between the reduced sampling density of the samples of G _K at the first input and the reduced sampling frequency clock CL _K+1 at its second input terminal is:
This is similar to the isochronous relationship between the high sampling density of the samples at the first input of the relay means 100 _a -K (described above) and the high sampling frequency clock CL _K at the second input.

The preferred embodiment of the reducer 104 is to reduce the sample density of its input in that dimension by 1/1 for each dimension of signal information.
The function is to reduce the number by 2, but it is not limited to this. In this case, the reducer 104 serves to send every other sample of its input to the output in each dimension. Therefore, for one-dimensional signal information, the sample density CL _K+1 is (1/2) ¹ or 1/2 of the sample density CL _K , and for two-dimensional signal information, the sample density for each of the two dimensions is
CL _K+1 is 1/2, giving a two-dimensional sample density of (1/2) ² , or 1/4.

The baseband frequency spectrum of G _K is the same at the input and output of reducer 104, but
As the sample density of the signal at the output of G _K becomes lower, some of the phase information present in the higher sample density signal applied to its input is lost.

The output of the reducer 104 is applied to the first input terminal of the relay means immediately following it, and is also applied to the input of the expander 106. The enlarger 106 is the reducer 104
It serves to insert a null (a digital number representing a zero level) as an additional sample at each sample position of the clock CL _K where there is no sample from the output of the clock. This restores the sample density of the output of expander 106 to the sample density of the input of reducer 104. In the recommended case where the sample density in each dimension is reduced by 1/2, expander 106 inserts a null in each dimension between each pair of adjacent samples in that dimension of the output of reducer 104.

Expander 106 increases the sample density of the output relative to its input, but does not change the G _K signal information of the output relative to the input. However, the introduction of the null has the effect of adding an image or copy of the baseband G _K signal information that occurs as harmonics of the sideband frequency spectrum CL.

The signal G _K output from the expander 106 is passed through the interpolation filter 1
Pass through 08. This interpolation filter 108 passes the baseband G _K signal, but the sideband frequency spectrum CL
It is a low pass filter that blocks harmonics. Filter 108 therefore serves to replace each null-valued null sample with the value of a sample having information surrounding it. The effect of this interpolated value sample is to define the envelope of the informative samples with higher resolution. The interpolating filter 108 thus substantially removes high frequency components above the baseband of the signal G _K output from the expander 106, but does not include the G _K interpolated signal at the output of the expander 106. Low sample density G _K does not and cannot add any information that is not already present in the signal. In other words, expander 106 convolves the reduced sample density in each dimension of the G _K signal into the output of filter 102.
G Functions to return to the sample density in each dimension of the _K signal.

The subtracter 110 supplies the G _K signal generated at the output of the interpolation filter 108 to the first input terminal of the relay means 100 _a -K and applies it as an input to the convolution filter 102 and also passes it through the delay device 109 . G _K-1 signal applied to the subtractor 110. Delay device 109 is convolution filter 1
02, provides a delay equal to the total delay provided by compressor 104, expander 106, and interpolating filter 108. Therefore, since the two signals applied to the input of the subtractor 110 have the same sample density _CLK in each dimension and their delays are also equal, the subtracter 110 calculates the digital number of each sample of its _GK signal input. The level represented by is subtracted from the level represented by the digital number of the corresponding sample of the G _K-1 input. In this way, the output of the subtracter 110 constitutes the Laplace signal L _{K-1 which} is taken out to the second output terminal of the relay means 100 _a -K.

Only the signal component G _K-1, which is not also present in the signal G _K applied to the subtracter 110, will be present in the Laplace signal L _K-1 output from the subtracter 110, but the first of these components consists of the high frequency part above the passband of the convolutional filter 102 of the frequency component of the G _K-1 signal, and therefore, for example, the relay means 100 _a
-K corresponds to the _relay means _{100-1 in FIG .} Contains frequencies within f ₀ . However, the Laplace output L _K-1 of the subtractor 110 also adds to this component substantially due to the phase information present in the densely sampled G _K signal at the output of the convolutional filter 102 but lost in the reduction process (described above). It includes a second component for error compensation consisting of frequencies within the passband of the corresponding convolutional filter 102. Therefore, the relay means 100 _a −(K+
The lost phase information of the low sample density (reduced) G _K signal sent to the first input terminal of 1) is added to the Laplace signal L taken out to the second output terminal of the relay means 100 _a -K. It is essentially held in _K-1 .

Each relay means 100-1,...100-N is the 1a
It has the configuration of the relay means _100a -K shown in the figure, and in this case, the last relay means 100-N of this relay means group.
The sampling density of each dimension of the residual signal G _K of the decomposed output taken out at the first output terminal of is smaller than the sampling density of each dimension of the G _N-1 signal applied to its first input (1/ 2 is preferred). However, since there is no relay means after the relay means 100-N by definition, in most applications (except for compressed data transmission) the sample density of the residual signal G _N is lower than the relay means 100-N.
It is important that the sample density be smaller than the sample density of the G _N-1 signal applied to the 0-N first input terminal. Therefore, in this case, the last relay means 100-N is
Rather than including the entire structure of the relay means _100a -K,
The configuration shown in FIG. 1c (each relay means 100-1,...100-(N-1) of the first type relay means group)
may also have relay means _100a -K). In FIG. 1c, the G _N signal output of the convolutional filter 102 (of which the sample density in each dimension is the same as the G _N-1 signal applied to the input of the convolutional filter 102) does not pass through the reducer, but , is directly supplied as the residual G _N output signal of the last relay means 100 _a -N of the first type relay means group. In this case, since there is no reduction, there is no need for expansion or interpolation, and therefore the G _N signal of the output of the convolutional filter 102 is directly applied as the G _N input of the subtractor 110. In other words, the configuration of the relay means 100 _a -N of FIG. 1c is the same as that of the relay means 100 _a of FIG. 1a.
-K, there is no reducer 104, expander 106, or interpolation filter 108. In this case, delayer 109 provides only a delay equal to the delay introduced by convolutional filter 102.

Figure 1a (or alternatively Figures 1a and 1c)
The first type shown in Figure 1 executes Burt's pyramid algorithm in real time. It goes without saying that in its most useful form, the bandwidth of each Laplace component of the decomposition output derived by Burt's pyramid algorithm is one octave in each dimension. The most useful form of Burt's pyramid algorithm is obtained by making each dimension's sampling frequency clock CL _K+1 one-half the sampling frequency clock CL _K for that dimension in the real-time implementation of FIG. 1a.

Next, I will cite other forms of hierarchical pyramids as substitutes for Barth's pyramids. This substitute pyramid is called a "filtered subtraction reduction (FSD)" pyramid, which lacks some of the desired properties of Burt's pyramid, but has desirable properties not found in Burt's pyramid. For example, a desirable property of Burt's pyramid (not found in FSD pyramid) is that the pseudo-aliasing frequencies present in each of the Laplace component and the residual component of the decomposed output are compensated for in the synthesis of the reproduced original signal, but depending on the application, FSD pyramids require less hardware and are therefore cheaper to run.

The signal processing apparatus of the present invention using pipeline techniques is also useful for implementing FSD pyramids in real time. This FSD pyramid (as an alternative to the above-mentioned relay means _100a -K-like stages used in Burt's pyramid)
Each sample signal relay means 100 of _FIG .
-a, . . . 100-N type 2 configuration is included.

The relay means 110 _b -K in FIG _. 1b is different from the relay means 100 - 1 , . . . ₁₀₀ - (N-1) in FIG.
The second type of digital embodiment described above uses relay means such as +1). Further, the relay means 100 _b -(K+1) in FIG. 1b represents one of the relay means 100-1, . . . 100N immediately after the relay means 100 _b -K.

As shown in FIG. 1b, the relay means 100 _b -K
consists only of an m-tap digital convolution filter 102, a reducer 104, a delay unit 109, and a subtracter 110. The configuration of the second type relay means 100 _b -K shown in FIG. 1b is the first type relay means 1.
00 _a -K (Fig. 1a) and (sample density
The G _K-1 signal of CL _K is applied as an input to the filter 102 and is passed through the delay device 109 to the subtractor 11.
0 and the output signal G _K (also of sample density CL _K ) is passed through a reducer 104 to reduce its sample density to CL _K+1 for each dimension, and this reduction in sample density The G _K signal thus obtained is applied to the first input terminal of the relay means 100 _b -(K+1) immediately after that.

The difference between the second type relay means 100 _b -K and the first type relay means 100 _a -K is that the subtracter 110 has G _K
The sample density (in each dimension) applied to the input from the output of the filter 102 to the input of the reducer 104 is CL _K
The G _K signal of is directly applied. That is, this is different from the first type relay means _100a -K which uses the _GK signal in which the sample density (in each dimension) of the output of the reducer 104 is reduced to CLK ₊₁ . The first type thus uses an expander 106 and an interpolating filter to restore its sample density (in each dimension) to C _K before the G _K signal is applied to the G _K input of subtractor 110. It takes 108. Second type relay means 10
Since the G _K input of the 0 _b -K subtractor 110 is not taken from a reduced sample density source, the expander 106 and interpolation filter 108 are not required in the configuration of the relay means 100 _b -K. Thus, in FIG. 1b, delay 109 provides only a delay equal to the delay introduced by convolutional filter 102. Also, subtractor 1
The output L _K-1 of 10 is also a convolutional filter 102
It consists only of relatively high-frequency components of the frequency spectrum of the G _K-1 signal that are not present in the output G _K signal.

According to the second type of configuration, the last relay means 1
00-N may have the configuration of the relay means _100b -K, but may also have the configuration of FIG. 1c.

The first and second embodiments shown in FIGS. 1a and 1b are digital embodiments in which an analog-to-digital converter is first used to convert an analog signal into a digital signal, usually represented as a multi-bit binary number. It does not matter whether the first type or the second type of this invention is implemented in digital form. Charge-coupled device (hereinafter referred to as CCD)
Means for relaying extracted signals using the same method are well known to those skilled in the art. For example, a CCD like a split gate filter.
Horizontal filters can be designed as convolutional filters or interpolation filters. Since the CCD signal is composed of a series of individual samples, each sample having an analog amplitude level, the invention can be implemented in either digital or analog format.

The filtering characteristics of a tapped filter depend on factors such as the number of taps, the effective delay between taps, and the designated level and polarity of each individually applied weighting factor for each tap. For explanation, convolutional filter 1
02 is assumed to be a one-dimensional 5-tap filter. FIG. 2 shows an example of weighting factors all of the same polarity (positive in FIG. 2) and specified magnitude, each with five individual taps. This also represents the effective delay time between each adjacent tap. Specifically, as shown in Figure 2, the effective delay time between each adjacent tap is
seed or second type relay means 100-1,...100
-N (FIGS. 1a, 1b, 1c) is 1/CL _K determined by the sampling frequency clock CL _K applied individually to each of the convolutional filters 102 (FIGS. 1a, 1b, and 1c). Therefore, each relay means 100
The absolute value of the delay CL _K of the convolutional filter 102 of -2, . . . 100-N is longer than that of the relay means immediately preceding it.

In FIG. 2, the weighting coefficients belonging to the five taps are all positive in polarity and have specified value levels symmetrically distributed with respect to the third tap. That is, in the example of FIG. 2, the weighting coefficient for the third tap has a specified value of 6, the weighting coefficients for the second and fourth taps have the same lower specified value of 4, and the weighting coefficients for the second and fourth taps have the same lower specified value of 4. That of the tap has the same weighting factor of 1, which is even lower. The envelope 202 of each weighting factor defines the kernel function of the convolutional filter 102 of each relay means 100-1, . . . 100-N (and thus the shape of the filtering characteristic in that frequency domain). That is, all samples 200 are (1) of the same polarity (positive in Figure 2) and (2)
The convolutional filter 102 is arranged symmetrically with respect to the center (third) sample, and (3) the level becomes smaller as the sample deviates from the center. Indicates filtering characteristics. In Figure 2, all the weighting coefficients are of the same polarity (positive), but this is not important in a low-pass filter; some of the weighting coefficients are of opposite polarity (negative), unless their algebraic sum is 0. is also possible. The kernel function waveform (for example, that of envelope 202 in FIG. 2) is the same for all convolutional filters 102 of each relay in the relay group, so that the relative low-pass frequency characteristics (in that frequency range ) is (not important) for the total filter 10
It is also possible that the same is true for 2. However, the absolute value of the low-pass nominal cut-off frequency of the filter of each relay means has a scaling that depends on the sampling frequency period 1/CL _K for that filter, and the level of the weighting factor (the specific value in Figure 2 1, 4,
It is not necessary to have 6) by appropriately selecting
A low pass of substantially 1/2 of the highest frequency of the G _K-1 signal input of the convolutional filter 102 (or in the case of G ₀ , the highest frequency f ₀ that may be included) A nominal cutoff frequency is obtained for the output signal G _K of convolutional filter 102 (with sample density in each dimension C _K ). In this case, the reducer 104 reduces the samples of that dimension to 1 in each dimension.
By sampling every third time, we reduce the one-dimensional sample density of the G _K signal to CL _K/2 , but the G _K signal (defined by the sample envelope 202) is
04 (although some phase information is lost due to the low sampling density of the output of 04).

We now describe some recommended embodiments of the real-time implementation of Bart's Pyramid, which forms the first species of the species of FIG. 1 (FIG. 1a).

FIG. 3 is a block diagram of a spectral decomposer, spectral conversion circuit, and signal synthesizer operating on electrical signals representing one-dimensional information signals (eg, time-varying information signals of any type).

FIG. 3 shows that the original electrical signal to be spectrally resolved is applied in analog form to an analog-to-digital (AD) converter 305 to be digitized. Let the extracted (sampled) digital response from the AD converter 305 be G ₀ . The high-pass spectrum _L0 , which is the high frequency response of _G0 , is 0
The next decomposition stage 310 extracts G ₁ , which is the low-pass filtered response of G ₀ . The band spectrum L ₁ , which is the high frequency part of this G _1, is extracted in the primary decomposition stage 315.
Leave G ₂ for the low-pass filtered response of G ₁ . The band spectrum L ₂ lower than the band spectrum L ₁ which is the high frequency part of G ₂ is extracted in the secondary decomposition stage 320 and converted into G ₂
leaving a low-pass filtered response of _G3 . A band spectrum L ₃ lower than the band spectra L ₁ and L ₂ which is the high frequency part of G ₃ is extracted in the third decomposition stage 325 and converted into G ₃
leaving G ₄ with a low-pass filtered response. Band spectrum that is the high frequency part of G ₄ Band spectrum lower than L ₃
L ₄ is extracted in a fourth order decomposition stage 330 leaving G ₅ the low pass filtered response of G ₄ . The high frequency portion of G ₅ , the lower band spectrum than the other band spectra, is extracted in a fifth decomposition stage 335 leaving G ₆ as the residual low pass filter response of G ₅ . This response G ₆ is actually a six times low pass filtered response of the original signal G ₀ .

Decomposition stage 310, 315, 320, 325, 33
0, 335 are initial low-pass filter stages 311, 316, 321, 326, 33 whose passbands are sequentially narrower, respectively.
1,336, these filters 311,31
The low pass response of 6,321,326,331,336 is sufficiently narrower than its input signal that it may be resampled at a lower frequency before being sent to the next decomposition stage. The sample reduction is regular, i.e. filters 311, 316, 321, 326,
Reduction (thinning) circuits 312, 317, 322, 327, 332, 3 following 331, 336, respectively
This is done by thinning out at 37. Particularly useful is octave spectral decomposition, in which every other sample is eliminated by a decimation process.

The high frequency portion of the input signal applied to each decomposition stage is extracted by removing the low frequency portion from the input signal. The decimated low frequency portion of the input signal is in a sampling matrix with a lower resolution than the input signal and has the disadvantage of being delayed relative to the input signal. The first one in this problem is the expansion circuit 313, 318, 323,
328, 333, and 338 by introducing nulls at the missing sample points in the low-pass filter response sample matrix and eliminating the incidentally introduced pseudo-harmonic spectra by low-pass filtering; The problem is that the input signal of the decomposition stage is
The solution is to delay before subtracting from the low-pass filter response expanded by 333, 338.

Delayed subtraction processing is carried out at decomposition stages 310, 315, and 32.
0, 325, 330, 335 circuits 314, 3
19,324,329,334,339. (As discussed below, each element may be conveniently allocated between the initial low-pass filter and the delay subtraction circuit of each decomposition stage if appropriate).

The above spectral decomposition has the property of a pipeline, L ₁ sample, L ₂ sample, L ₃ sample,
The L ₄ sample has a time lag that becomes longer with respect to the L ₅ sample and the L ₀ sample. The "time lag" used here means, for example, the decomposed output signals L ₀ , L ₁ , L ₂ , L ₃ , L ₁ ,
It refers to a time delay difference of a predetermined known amount that occurs between corresponding samples of parallel signals that are informationally related, such as between corresponding samples of L ₅ and G ₆ . Decomposition of the signal by the spectral techniques described below requires a backward time shift for each group of samples, as shown in Figure 3 (typically, for example, in a shift register or read-after serial storage device). (including other types of storage devices that perform equivalent functions such as delay lines 340,3)
41, 342, 343, 344 make the circuit 34
5,346,347,348,349, it can be given before modification. Spectral modification can also be done before applying the delay, or the delay can be divided before and after modification and done in various ways.
For example, spectral modifications can also be performed in parallel in time. In some cases, the correction circuit 34
It is contemplated that means could be provided within 5,346,347,348,349 itself to provide a delay difference as part of the overall delay difference condition.

The spectra of L ₅ and L ₆ are corrected by circuits 350 and 35.
1 will be corrected. For some types of signal processing, the correction circuits 345 to 351 may not be necessary, and in this case they may be replaced by direct connection. The spectral decomposition procedure described above can be extended by adding more decomposition stages or reduced by fewer decomposition stages. In such a case, the residual low-pass spectrum G〓 does not become G ₆ at the end of spectral decomposition.

When recombining the sometimes modified spectral decomposition components to synthesize a signal, it is necessary to cancel the thinning of the sampling matrix between each decomposition stage, so the adders 353, 355, 357,
359, 361, 363 can be used to sum the spectral samples. This is delay circuit 3
This is added to the time skew correction action in steps 40 to 344. This thinning essentially consists of expansion circuits 338, 333, 328, 323, 318,
313 and the same expansion circuits 352, 354,
356, 358, 360, 362. In fact, multiplexing allows one circuit to perform two functions. The residual low-pass spectrum G〓 is displaced forward in time with respect to the neighboring low-pass spectrum L〓 _-1 , and its expansion causes its sample to fit that of L ₍ 〓 _-1) . In Figure 3, G〓 is G ₆ , and
L〓 _-1 (the _third
In the figure, the new G〓 _-1 added to L ₅ ) and synthesized
(G ₅ ' in Figure 3). The output of the adder 353 is expanded by an expansion circuit 354, the adder 355 combines the delay-corrected L ₄ and the new G ₄ ' added, and the output of the adder 355 is expanded by an expansion circuit 356.
The adder 357 adds it to the delay-corrected L ₃ to synthesize a new G ₃ ', and the output of the adder 357 is expanded in an expansion circuit 358 and adds it to the delay-corrected L ₂ in an adder 359 to synthesize a new G 3 '. G ₂ ' and adder 3
The output of 59 is expanded by an expansion circuit 360 and added to the delay-corrected L ₁ by an adder 361 to create a new
_Finally , the output of the adder 361 is expanded in an expansion circuit 362, and added with delay-corrected L ₀ in an adder 363 to synthesize a new G ₀ '. The new G ₀ ′, G ₁ ′, G ₂ ′, G ₃ ′, G ₄ ′, G ₅ ′, and G ₆ ′ are also represented with dashes (′) in the signal synthesis circuit of FIG. The new G ₀ ' can also be converted to analog form by a digital-to-analog converter (not shown) if desired.

Circuits 352, 354, 356, 358, 36
The expansion process at 0.362 removes the upper band at each stage of the synthesis process, but when the bandpass spectrum is not wider than an octave, correction circuits 345-351 occur, introducing pseudo-aliasing frequencies to synthesize the signal. This suppresses all harmonics that could interfere with the

Figure 4 shows 310, 315, 320, 325, 33 used for spectrum decomposition by octave.
The structure of the spectral decomposition stage for one-dimensional information such as 0,335 is shown in more detail. This stage is a K-order spectral decomposition stage where K is 0 or a positive integer. In the case of a zero-order spectral decomposition stage, its clock frequency is the original input signal G ₀ on which the spectral decomposition is performed.
is the sampling frequency R, but if K is a positive integer, it is reduced to 1/ ^2K .

The input signal G _K of the spectral decomposition stage in Fig. 4 is
M clocked at clock frequency R/ ₂ K
It is applied as an input to the stage's shift register 470. The (M+1) samples representing sequentially longer delays provided by the input and each output of shift register 470 act as a multi-tap delay line of a low-pass delay line filter, with each sample being weighted and summed in circuit 471. linear phase low-pass filter response G _(K+1)
Generate a sample of In all decomposition stages except for the first one where K exceeds 0, the clock frequency 1/2 (relative to the clock frequency of the previous stage) used in the first shift register 470 and the adder in the weighted summation circuit 471 are G Reduce G _(K+1) with respect to _K. The response G _(K+1) is applied as one input to multiplexer 472, which
The G _K+1 input signal and the null input signal are alternately selected at a frequency R/2K to produce the signal G _(K+1) ^* .

The signal G _(K+1) ^* is twice the spectrum of G _(K+1) and G _(K+1)
has a baseband frequency spectrum mixed with a first double sideband carrier-suppressed harmonic spectrum of peak amplitude of . Here, the next spectral decomposition stage has correctly timed G _{(K+1) instead of G(K+1)} as input.
It turns out that G _(K+1) ^* can be used. The signal G _(K+1) ^* is applied as an input signal to another shift register 473 having multiple stages (M stages or otherwise) and clocked at a frequency R/2K.

The (M+1) samples supplied from each stage of the shift register 473 by its input and output signals are applied to another weighted summation circuit 474 similar to the circuit 471. This circuit 474 is G _(K+1) ^*
The first harmonic spectrum of G is suppressed and an expanded version of G _(K+1) is supplied to a sample matrix having many samples similar to the sample matrix of G _K.

In the subtraction circuit 475, the G _K delayed by the shift register 470 and the delay circuit 476 is expanded.
G _K+1 is subtracted. in shift register 470
The M cycle delay of G _K is compensated for by the center sample M/2 cycle delay to the weighted summation circuit 471 for the G _K input of the spectral decomposition stage of FIG.
A similar M/2 cycle delay between G _(K+1) ^* and the center sample to weighted summation circuit 474 is compensated. The delay circuit 476 is a weighted summation circuit 471, 47
A delay is introduced to compensate for the delay due to the addition in 4, which can be easily formed by extending the shift register 470 by the required number of stages. The output signal L _K of the subtraction circuit 475 is one of the possible spectral decomposition components, the lower frequency limit of which is set by the low-pass filtering performed in the Kth spectral decomposition stage of FIG. 4, and the upper frequency limit, if any. for example, by low-pass filtering in the next spectral decomposition stage.

FIG. 5 illustrates a method of reducing the number of shift register stages used in a spectral decomposition device constructed in accordance with the present invention. Each sample defining G _(K+1 ) ^* to be weighted summed to perform the low-pass filtering associated with the interpolation from _{G (K+1)} is transferred to the next spectral decomposition stage without shift register 473. It results from a tapped delay line structure used to support the first low-pass filtering of G _(K+1) .

FIG. 5 shows, by way of example, how this is done between the zero-order decomposition stage used to generate L ₀ and the next decomposition stage. Elements 570-0, 571-0, 5
75-0, 576-0 are the elements 470, 471, 475, 476 of the K-th spectral decomposition stage in FIG.
Each element of the zero-order spectral decomposition stage corresponds to . Elements 570-1, 5 of the primary spectral decomposition stage
71-1 is each element 570-0, 57 of the 0th order spectrum decomposition stage except that the clock frequency is 1/2.
Same as 1-0. Shift register 570-1
The four samples extracted from the input and the first three outputs are fed in parallel at clock frequency R/2, interleaved with nulls, and the results are weighted in two patterns by a 7-filter weighting pattern ABCDCBA. Sequentially, a pair of sample groups are formed, and the subtracter 5
A clock pulse R is subtracted from the delayed G ₀ at 75-0.

The earlier of each pair of consecutive samples subtracted from delayed G ₀ is applied to the input of shift register 570-1 and the output of the first three weighting circuits 580, 581, 58.
The filter weights A, C, C, A are multiplied by 2,583, respectively, and each weighted sample is summed by the circuit 5.
It is obtained by summing up 87. For this positioning of the _G1 vs. filter weighting pattern, the inserted nulls come at points that are weighted by B, D, B. The later sample of each pair of samples subtracted from _G0 is sent to the input of shift register 570-1 by weighting circuit 58.
4,585,586 filter weights B, D,
B and summing each weighted sample in a summing circuit 588. This G ₁
For positioning of the filter weighting pattern, the nulls to be inserted come at points weighted by A, C, C, A.
Multiplexer 58 operating at clock frequency R
9 alternately selected each sample of the output of the summation circuits 587 and 588 and delayed it in the subtracter 575-0.
G forms a sample stream that is subtracted from ₀ .

FIG. 6 shows one stage of the signal combiner of FIG. 3 in more detail. G _K ′ (i.e. delay corrected G〓)
The samples are alternated with nulls in a multiplexer 692, and the resulting expanded signal is clocked at the expanded sampling frequency by an M-stage (or other multiple stages) shift register 6.
93 as an input. The input of the shift register 693 and the output of each stage thereof are supplied to a weighted summation circuit 694, and from this weighted summation circuit 694 to an adder 695, the G _K (or G〓) Spectrum is supplied and added to this re-extracted filtered
G _K ′ (or G〓) is combined with a modified L _(K-1) ′ delayed in time to match the sample. A multiplexer 692, a shift register 693 and a weighted sum circuit 694 are connected to the element 47 in the spectral decomposition process.
2,473,474.

In this respect, it is useful to consider the low-pass filtering characteristics used in the low-pass filter stage of the spectral decomposition procedure and in the expansion stage of the spectral decomposition and signal synthesis procedure. Since the low pass filter is linear phase, the filter weighting pattern is symmetric about the center sample. The sum of the filter weights is 1 in order to suppress as many low frequencies as possible in the high-pass spectrum L ₀ and the band-pass spectra L ₁ , L ₂ , L ₃ . If the spectral decomposition is performed by octaves and the subbands removed by the low-pass filter of each spectral decomposition stage should be reduced by a factor of 2 in the re-encoding, then 2/2 of the octave center frequency is added to the low-pass filtrate. It is desirable to remove frequencies below 3. The graded frequency response of the filter (the so-called "brick wall" response) introduces an overshoot in the filtered signal, resulting in the G _(K+1) function extracted by the spectral decomposition stage and the _G generated by subtracting _(K+1)
Increase dynamic range with L _(K+1) function. This is an example of the Gibbs phenomenon, which can be relaxed by using a less abrupt truncation of the Fourier series.
A number of truncated windows are known that provide a filter response with reduced Gibbs effects, such as those by Hanning, Hamming, Blackman and Kaiser. For example, in 1975 Prentice
Published by Prentice-Hall Inc., pages 239-251 of ``Digital Signal Processing'' by AVOppenhemi and RWSchafer,
Chapter 5.5 “Design of FIR Filters Using Windows”
Quote.

In practice, the number of samples in low-pass filtering is usually limited to a very small number. For filters with odd samples, the filter response contains a dc component and a series of cosine harmonics; for filters with even samples, the filter response contains a dc component and a series of sine harmonics. The required response curve is approximated using a computer through trial and error calculation of weights to obtain the smoothest fit.

Although non-octave wide equal-Q spectra can also be generated by the present invention, such methods are believed to have limited utility. If you reduce the low-pass response by choosing every second sample and filtering out frequencies below 1/2 of the bandpass spectrum center frequency to generate that low-pass response, e.g. yields a group of bandpass spectra that are sequentially narrower by 1/3.

The sample modification circuits 345-351 of FIG. 3 can take a variety of forms, some of which may be replaced by direct feedthroughs.
For example, in order to remove low-level background noise from various spectra, each of the correction circuits 345 to 351 may be configured with the baseline clipper 700 shown in FIG. 7. The clipper 700 may simply clip the lower bits of the signal.

FIG. 8 shows a circuit in which each modification circuit 345-351 can be used to form a spectral equalizer. The rotary switch 897 is adapted to generate a binary code for each of the plurality of axial displacements, and the code is supplied via a latch 898 to a two-quadrant multiplier to multiply the input spectral sample and synthesize it. and generates an output spectral sample that generates G ₀ '. The latch 898 is connected to the multiplier 889 while the setting of the rotary switch 897 is changed.
Retains sign input. Each octave spectrum is subdivided using a digital filter using the same sampling frequency or half the sampling frequency used to generate the octave spectrum, and the gain of each subdivision of the spectrum is adjusted separately. It can also be done. By subdividing the octave into 1/12, individual scale and chromatic adjustment of the signal encoding music, for example, is obtained.

The modification circuit may also be a read only memory (ROM) that stores the nonlinear transfer function. For example, the ROM 990 that stores the logarithmic response to the input signal shown in FIG. 9 is used in each sample correction circuit 345 to 351 of the transmitter, and the ROM 1091 that stores the exponential response to the input signal shown in FIG. 10 is used for each corresponding sample correction circuit of the receiver. This enables pre-emphasis of the signal before transmission and de-emphasis after reception. Other complementary pre-emphasis and de-emphasis characteristics can also be alternately stored in the ROM modification circuits of the transmitter and receiver spectrally resolved signal synthesizers.

FIG. 11 is a variation of the spectral decomposition signal synthesis scheme of FIG. 3, in which the delay between decomposition and synthesis is partitioned to provide spectral samples without time lag for processing. For example, spectral decomposition is used to separate the signal into its spectrum before stretching, so that the stretched spectrum can be filtered to suppress distortions that occur during rapid signal compression or expansion. Good consistency is desirable. The amplitude of the original signal applied to the AD converter 305 in FIG.
111, 1112, 1113, 1114, 111
5,1116, which results in the rapid onset and slow decay of the expanding and contracting signal. The stretchers 1110-1116 essentially have a control signal CC generated from an AD converter cascaded after conventional analog circuitry that senses the signal to be stretched or compressed and generates an analog stretch control signal in response to this detection. Ru2
It can also be configured with quadrant digital multipliers.

Expanding device 1110, 1111, 1112, 111
3, 1114, 1115, 1116 are spectra
L ₀ , L ₁ , L ₂ , L ₃ , L ₄ , L ₅ , L ₆ as delay circuit 110
0,1101,1102,1103,1104,
1105 and 1106 to coincide with each sample in time and then operate on them. Next, delay circuits 1120, 1121, 1
Signals L ₀ ′, L ₁ ′, L 2 ′, L _{3 ′} , L ₄ ′, L ₅ ′ and
G ₆ ' is shifted to suit the signal synthesis process using elements 352-363 of FIG.

The delay of delay circuits 1106 and 1125 is essentially M/2 cycles of a clock frequency of R/ _2K (K=5) or 16M cycles of a fundamental clock frequency R, and the delay of the final spectral decomposition stage 335 is equal to Occurs when assembling samples for This 16M cycle delay is due to the expansion circuit 33
Delay time D ₁ to adapt to the addition time in 8,352, delay subtraction circuit 334 and adder 353
Delay time D to accommodate the addition time in ₂
only. It is assumed that all addition processes are performed at the fundamental clock frequency R, and D ₁ and D ₂ are expressed as the number of clock cycles.

The delay of the delay circuit 1104 is equal to the clock frequency R.
It is longer than 16M+D ₁ +D ₂ cycles by the difference between the time to generate L ₅ from G ₃ and the time to generate L ₄ from G ₅ . The time to generate L ₆ from G ₅ is M cycles of clock cycles R/2 ⁵ to collect the samples twice for weighting and summing, or 32 M cycles of the fundamental clock frequency, for summing two sets of samples. 2D ₁ and for sample subtraction
D ₂ is added. The time to generate L ₄ from G ₅ is M/2 cycles of frequency R/2 ⁴ to collect samples for weighted summation, or 8M cycles of basic clock frequency, D ₁ for sample summation, and This is the addition of D ₂ for subtraction. 24M of extra delay of the base clock frequency to align the L ₄ samples in time with the L ₅ samples.
+D Requires ₁ cycle. Therefore, the delay circuit 104
The total delay is 40M of the fundamental clock frequency R + 2D ₁ +
D is ₂ cycles. By similar calculation, each cycle of the basic clock frequency R at which each sample is delayed in delay circuits 103, 102, 101, and 100 is 52M+3D ₁ +D ₂ and 58M, respectively.
+4D ₁ +D ₂ , 61M+5D ₁ +D ₂ and (621/2)M
It is determined that +6D ₁ +D ₂ .

The delay required by delay circuit 1124 beyond that provided by delay circuit 1125 is the time required for expansion in circuit 354 and the delay D ₂ associated with the addition in adder 55. The former delay consists of M/2 cycles of the clock frequency R/2 ⁴ for collecting samples for weighting and summing, plus 8M cycles of the fundamental clock frequency R and D ₁ for the summation of the weighted summing process. The delay circuit 112
The total delay for 4 is 24M+D ₁ +D ₂ . By similar calculation, delay circuits 1123, 1122, 112
The total delay of 1,1120, counting in cycles of the fundamental frequency R, is 28M + 3D ₁ + 3D ₂ and 30M + respectively.
4D ₁ +4D ₂ , 31M+5D ₁ +5D ₂ , (311/2)M+6D ₁
+6D ₂ .

The total delay of delay circuits 340 to 344 in FIG.
Assuming that the delays of the correction circuits 345-351 are all equal, it can be determined by a similar calculation. Delay circuits 340, 341, 342, 343,
The delays of 344 and 345, respectively expressed in cycles of the basic clock frequency R, are 77M + 12D ₁ +
7D ₂ , 76M + 10D ₁ + 6D ₂ , 72M + 8D ₁ + 5D ₂ , 64M
+6D ₁ +4D ₂ , 48M+4D ₁ +3D ₂ .

The digital filtering used in spectral decomposers is commonly referred to as hierarchical filtering, where low-pass and bandpass filtering over a large number of samples is achieved with a relatively small number of samples that are always weighted together.

Although this invention can be applied to the use of the spectrum of a signal representing one-dimensional information, Burt's pyramid was originally developed to resolve the spatial frequencies of two-dimensional image information. The invention enables real-time spectral decomposition of the spatial frequencies of changing image information, such as occurs in successive video frames of a television display.

As known from television technology, successive video frames (NTSC) occur sequentially at a frame frequency of 30 frames per second. Each frame is
Consisting of 525 interlaced horizontal scan lines, each odd-numbered horizontal scan line is transmitted sequentially during the first field period, and each even-numbered horizontal scan line is transmitted during the second field period following the first field period. They are transmitted sequentially, followed by the first field of the next frame. Although the length of each field period is 1/60 second, it is necessary to store at least the number of pixels within the field period so that the total spatial frequency of the image can be determined in delayed real time.

A technique known as progressive scanning is known in television technology for sequentially extracting complete frames of 525 lines from an NTSC video signal at a rate of 60 frames per second. This technique delays each successive NTSC field by a field period of 1/60 second. Thus, during the currently occurring odd field of each successive field, each successive scan line of that odd field is completed in alternating combination with each successive scan line of the immediately preceding even field delayed one field time. form a frame of pixels. Similarly, during the currently occurring even field of each frame, each scan line of that even field is combined alternately with each scan line of the immediately preceding odd field delayed one field time to form a complete frame of pixels. do.

The progressive scanning method described above is particularly useful for producing high resolution image displays, known as high definition television (HDTV), which are currently under development in the television industry. The invention is also useful for improving image display in this HDTV.

FIG. 12 illustrates a spectral decomposition apparatus employing the principles of the present invention for operation on signals representing two-dimensional information, such as spatial frequency image information contained in sequentially scanned television video frames.
However, such two-dimensional information can also be obtained from a non-interlaced television camera or a line-interlaced television camera with suitable buffer storage.

Although FIG. 12 depicts monochromatic processing of the luminance signal for ease of explanation, the technique described may be applied individually to the primaries of a color television signal or to signals derived therefrom by algebraic mixing. I can do it. The original video signal is supplied to the AD converter 1205 in raster scan format, where it is sampled if not already sampled, resampled if already sampled, and finally digitized. This video sample as a digitized signal is denoted G ₀ and contains the complete two-dimensional spatial frequency spectrum of the original signal and the harmonic spectrum due to the sampling process. This harmonic spectrum is symmetric with respect to the sampling frequency and each of its harmonics. The general fact of its existence is noted since the harmonic spectrum must be considered in the design of the two-dimensional low-pass spatial frequency filter used in the spectral decomposition device of FIG. This is because the harmonic spectrum generates aliasing frequencies during spectral decomposition and during signal synthesis from the decomposed spectrum.

The zero-order spectral decomposition stage 1210 separates the high-pass spectrum L ₀ from G ₀ . This high-pass operation low-pass filters G ₀ from its timing coming from AD converter 1205 to the same extent that _{the lower frequency portion of G 0 is} delayed in the low-pass filter response. and this delayed G ₀
This is essentially done by subtracting the low-pass filter response from . If the spectral decomposition is performed by octaves, the two-dimensional low-pass spatial frequency filter 12
The cutoff frequency of 11 is chosen to be the highest frequency of the bandpass spectrum L ₁ of the next octave bandwidth to be decomposed, i.e. 4/3 times its center frequency. In the reducer 1212, every other row and column of the sample is deleted in order to extract the low-pass filtered G ₀ at a spatial frequency of R/2, and the signal with the reduced sample frequency is further sent to the stage 1210 for spectral decomposition. is provided as the low frequency output response of Here, this low-pass filtered G ₀ with a low sample frequency is
Proceedings of June 1973
The I.E.E. (Proceedings of
61, No. 6, pages 692-702 of the IEEE)
Rabiner's paper "A Digital Signal Processing Approach to Interpolation"
Interpolation is performed according to the method outlined in ``Interpolation''. In the enlargement circuit 1213, the reducer 121
The samples canceled in 2 are null-replaced to produce the input signal for another two-dimensional low-pass spatial frequency filter 1214. This filter may use the same sample weighting factors as the original low-pass filter, but always has substantially the same cut-off frequency as the original low-pass filter. The signal obtained thereby has a sampling matrix of the same size as G ₀ delayed by the delay circuit 1215, and the signal obtained by the subtracter 1216
High-pass output response subtracted from G ₀ delayed by
produces L ₀ . L ₀ is not only the high frequency part of G ₀ , but also
Also, during resynthesis of the video signal from the spectral decomposition as described above, a low frequency phase error correction term is used to compensate for errors introduced by resampling G ₀ at a lower sampling frequency in the downscaler 12. Contains.

This separation of the signal into low and high frequency parts, which are resampled at 1/2 frequency, is repeated at each spectral decomposition stage. Each successive spectrally decomposing stage receives as its input signal the resampled low-pass output response of the previous spectrally decomposing stage, and the sampling frequency is one-half that of the previous spectrally decomposing stage for each spectrally decomposing stage. First stage 1210
Each subsequent spectral decomposition stage 1220, 1230,
The high frequency output response of 1240, 1250, and 1260 has an upper limit given by the low frequency response characteristics of its predecessor, so this "high frequency" output response is in fact an equal-Q bandpass spectrum of decreasing spatial frequency. be. The response decimation rate of the first low-pass filter in each stage is 1/2 and the cut-off frequency of the low-pass filter in each stage is 2/3 of the center frequency of the spectrum in which it occurs. This is the factor that makes the spectrum a two-dimensional spatial frequency of gradually decreasing octaves.

The decimated low frequency output response G ₁ of spectrally decomposing stage 1210 is provided from its reducer 1212 as an input signal to the next spectrally decomposing stage 1220 . The spectral decomposition stage 1220 includes each element 1211, 1212, 121 of the spectral decomposition stage 1210.
Elements 1221, 1222, 1223, which are similar to 3, 1214, 1215, and 1216, respectively, but have a difference in operation because the sampling frequency of stage 1220 is 1/2 that of stage 1210 in two dimensions.
1224, 1225, 1226. Low-pass filters 1221 and 1224 are low-pass filter 1, respectively.
211, 1214, but with the same weighting factor as stage 1
When the sampling frequency of stage 220 is halved for stage 1210, the cutoff frequency of filters 1221 and 1224 is halved for filters 1211 and 1214. As far as delay circuit 1215 is concerned, the delay before subtraction in delay circuit 1225 is twice as long, and if this delay is a clocked delay such as a shift register, then this delay structure is equal to delay circuit 1.
This is similar to the ratio 2/1 of the delay given by the ratio 1/2 of each delay clocking frequency of 225 and 1215. Spectral decomposition stage 1220 high frequency output response L ₁
is the bandpass spectrum just below the spectrum L ₀ .

The decimated low frequency output response G ₂ of spectrally decomposing stage 1220 is provided from its reducer 1222 as an input signal to the next spectrally decomposing stage 1230 .
Bandpass spectrum L ₂ one octave lower than L ₁
is the spectral decomposition stage 1 for its input signal G ₂
230 high-pass output response. The difference is that the sampling frequency of the spectral decomposition stage 1230 is 1/2, but each element 1221, 1222, 1223, 1224,
123 corresponding to 1225 and 1226 respectively
1,1232,1233,1234,1235,
It has 1236.

The decimated low frequency output response G ₃ of spectrally decomposing stage 1230 is provided from its reducer 1232 as an input signal to the next spectrally decomposing stage 1240 . The bandpass spectrum _L3, which is one octave lower than _L2 , is the highpass output response of the spectral decomposition stage 1240 to its input signal _G3 . The difference is that the spectral decomposition stage 1240 has a sampling frequency of 1/2, but each element 1231, 1232, 1233, 1234,
Elements 12 corresponding to 1235 and 1236, respectively
41, 1242, 1243, 1244, 124
5,1246.

The decimated low frequency output response G ₄ of spectrally decomposing stage 1240 is provided from its reducer 1242 as an input signal to the next spectrally decomposing stage 1250 . The bandpass spectrum _L4, which is one octave lower than _L3 , is the highpass output response of the spectral decomposition stage 1250 to its input signal _G4 . The sampling frequency of the spectral decomposition stage 1250 is 1/2
The difference is that each element 1241, 1242, 1243, 124 of the spectral decomposition stage 1240
Elements 1251, 1252, 1253, 1254, 12 corresponding to 4, 1245, 1246, respectively
55,1256.

The decimated low frequency output response G ₅ of spectrally decomposing stage 1250 is provided from its reducer 1252 as an input signal to the next spectrally decomposing stage 1260 . The bandpass spectrum L ₅ one octave below L ₄ is the highpass output response of the spectral decomposition stage 1260 to its input signal G ₅ . The difference is that the spectral decomposition stage 1260 has a sampling frequency of 1/2, but each element 1251, 1252, 1253, 1254,
Elements 12 corresponding to 1255 and 1256, respectively
61, 1262, 1263, 1264, 126
It has 5,1266.

The decimated low-pass output response G 〓 provided by the reducer of the last spectral decomposition stage 1260 is here G ₆ provided by the reducer 1262 of the spectral decomposition stage 1260, which is the residual low-pass spectral response. It is. This serves to recombine the signal by summing the interpolated band spectral response of the subsequent spectral decomposition stage and the capstone high band spectral response of the first spectral decomposition stage. L ₀ , L ₁ , L ₂ ,
L ₃ , L ₄ , and L ₅ are in a time-shifted relationship and are supplied while increasing the amount of delay sequentially. Residual low frequency spectrum G〓
(G ₆ here) is earlier in time than the last band spectrum G〓 _-1 (G ₅ here) and is oblique in the opposite direction.

As will be explained later, the iterative method of synthesizing signals from spectral components also requires that the spectral components L ₀ , L ₁ , L ₂ , L ₃ , L ₄ , and L ₅ be time-shifted in opposite directions. . Before describing the processing of the spectrally resolved components and the synthesis of signals from the processed spectrally resolved components, the configuration of the spectrally resolved stage will now be described in further detail. First, consider the first two-dimensional low-pass filter structure.

As is well known in the art of filter design, there are two types of two-dimensional filter structures: non-separable types and separable types. Separate filtering in the first and second dimensions involves first filtering in the first direction using a first one-dimensional filter, then filtering in the first direction using a second one-dimensional filter. This is achieved by filtering in a second orthogonal direction.
Therefore, since the low-pass characteristics of the two cascaded individual one-dimensional filters constituting the decomposed two-dimensional low-pass filter are completely independent of each other, the core of each of the two low-pass filters is Functions and structures are secondary
It may be similar to that described above with respect to Figures a, 2b, and 3 to 11.

For television images consisting of a raster of horizontal scan lines, the two orthogonal directions of the separate filter are preferably horizontal and vertical. When separate two-dimensional low-pass filtering is used in the practice of this invention, certain benefits are obtained by performing horizontal low-pass filtering before vertical low-pass filtering, and performing vertical low-pass filtering before horizontal low-pass filtering. This provides other benefits. For example, performing horizontal filtering and decimation first reduces by 1/2 the number of pixel samples per horizontal scan line that must be acted upon by the vertical kernel function during subsequent vertical filtering, but doing vertical filtering first reduces
In addition to providing a relatively long delay, the spectral decomposition stages 1210, 1220 of FIG.
1230, 1240, 1250, 1260 subtracters 1216, 1226, 1236, 1246,
Signals G ₀ and 1256 and 1266 are connected to the positive terminals, respectively.
Each compensation delay 1 to send G ₁ , G ₂ , G ₃ , G ₄ , G ₅
215, 1225, 1235, 1245, 125
The same delay structure required to provide 5,1265 is now available.

The integrated filtering response of a separable two-dimensional spatial frequency filter may be square or rectangular in cross-section parallel to the spatial frequency plane, whereas the filtering response of a non-separable filter may have other cross-sectional shapes. Circular or elliptical cross-sections are particularly important for filtering raster-scan television signals because filters with such cross-section responses can be used to reduce excess diagonal resolution in television signals. Uniformity of image resolution in all directions is also important, for example in television systems where images need to be rotated between a camera and a display device.

2-D low-pass filters 1211, 122 in FIG.
1, 1231, 1241, 1251, 1261 and 2-D low pass filter 1214, 1224, 12
A matrix of weights for a filter with a pattern exhibiting quadrant symmetry and linear phase response, which are particularly suitable filtering characteristics as 34, 1244, 1254, 1264, is shown below.

ABCBA DEFED GHJHG DEFED ABCBA A kernel function matrix having this pattern of weighting coefficients acts on each successive image sample in turn, and each pixel sample corresponds to a weighting coefficient J whose position is located at the center of the matrix when acted upon. In a low pass filter, the weighting factor J has the highest relative intensity level, and each of the other weighting factors has an intensity level that decreases away from the center position. Therefore, the strength level of the weighting coefficient A at the four corners is the lowest.

In the case of a non-separable two-dimensional filter, A, B,
Although the specific chosen values for each intensity level of C, D, E, F, G, H, and J are completely independent of each other,
In the case of a two-dimensional separable filter, the intensity level of the weighting coefficient is obtained from the cross product of each value of the horizontal and vertical one-dimensional kernel weighting coefficients, so A, B, C, D, E,
The values of F, G, H, and J are not completely unrelated to each other.

An apparatus for synthesizing electrical signals from component spectra that can take the general form shown in FIG. 13 is important to the invention. Spectral components G ₆ ′, L ₅ ′, L ₄ ′,
L ₃ ', L ₂ ', L ₁ ', and L ₀ ' respond without their darts (') provided by the spectral separator of FIG. Spectral components L ₀ , L ₁ , L ₂ , L ₃ ,
L ₄ , L ₆ , and L ₅ are sequentially supplied with a delay in time by the spectral decomposition device shown in FIG.
G ₀ ′, L ₅ ′, L ₄ ′, L ₃ ′, L ₂ ′,
In order to sequentially supply L ₁ ′ and L ₀ ′ with a time delay, it is necessary to delay them differentially.

FIG. 13 shows a plurality of consecutive signal combining stages 136.
0,1365,1370,1375,1380,
1385 shows a signal synthesizer including a 1385. Each stage uses interpolation to expand the sample matrix of spectral components so that it is the same length as the next highest spectral component in spatial frequency and can be added to that spectral component. This expansion of the sample matrix alternates the nulls with each sample point within the matrix and low pass filters the result to remove harmonic structures. Preferably, this low-pass filter has the same filtering characteristics as the low-pass filter associated with the corresponding interpolation process in the spectral decomposer of FIG.

The low-pass filtering associated with interpolation in the signal synthesizer can occur in a modification circuit (as described above with respect to FIG. 3) that may be inserted between the spectral decomposer of FIG. 12 and the synthesizer of FIG. It suppresses the harmonics associated with the G〓 or L〓 signal, which is modified by nonlinear processing. This nonlinear processing may produce visible aliasing phenomena in the composite composite image without the low pass filtering associated with the interpolation process used in the signal synthesizer.

In the synthesizer shown in Fig. 13, the low-frequency spectrum
Each sample of G ₆ ' is interleaved with nulls in the expansion circuit 1361 and passes through a similar two-dimensional low spatial frequency filter 1362 of the filter 1265 of the spectral decomposer of FIG. The samples of the filter 1362 response are summed with the samples of L ₅ ' in summer 1363 to produce G ₅ ', which is similar to or equivalent to a hypothetical delayed replica of G ₅ . This G ₅ ' sample is interleaved with nulls in an expansion circuit 1366, passed through a low pass filter 1367 similar to low pass filter ₁₂₅₄ in _FIG . Generates G ₄ ′ similar or equivalent to delayed replication. This sample G ₄ ' is alternated with nulls in an expansion circuit 1371 and low pass filtered in a filter 1372 similar to filter 1244 in FIG. This filter 13
The response of 72 is added to L ₃ ' by adder 1373.
Generates G ₃ ′ which is similar or equivalent to delayed replication of G ₃ . The G ₃ ' samples are interleaved with nulls in an enlarger circuit 1376 and low pass filtered in a filter 1377 similar to filter 1234 in FIG. Filter 1
The response of 377 is added to L ₂ ' in adder 1378 to produce G ₂ ', which is similar to or equivalent to a delayed replica of G ₂ . A null is inserted between the G ₂ ' samples in an expansion circuit 1381, and low-pass filtered in a filter 1382. The response of filter 1382 is added by adder 1383.
Similar or equivalent to G ₁ added with L ₁ ′ and delayed
G ₁ ′ is generated. This G ₁ ' sample is fed to an expansion circuit 1386 for interpolation and to a filter 1387 in FIG.
8 with L ₀ ' to produce a composite signal G ₀ ' represented by the same image as represented by G ₀ with possible modifications.

Although the two-dimensional embodiment of the present invention is particularly suited for use in processing the spatial frequencies of images in real time, the two-dimensional information of interest to the present invention is not limited to the spatial frequency spectrum of two-dimensional images. For example, one of the two dimensions may correspond to spatial frequency information and the other may correspond to time frequency information.

Further, the present invention is useful for decomposing real-time frequency spectra of information determined in three or more dimensions. For example, in the case of three-dimensional information, all three dimensions may correspond to spatial information, or two dimensions may correspond to spatial information and the other one dimension may correspond to temporal information. Of interest here are image processing devices that respond to motion in displayed television images. In this case, the portion of the spatial frequency spectrum of the displayed image that corresponds to a stationary subject remains unchanged between each video frame of video information, whereas the portion that corresponds to a moving subject changes between frames. A spectral decomposition device using the principles of the present invention can also be used in such an image processing device that utilizes a three-dimensional low-pass filter. The two three-dimensional low-pass filters are spaces, and the two three-dimensional low-pass filters attached to each stage of the two-dimensional spectral decomposer shown in FIG.
The third dimension is time and is responsive to the fine structure characteristics of the three-dimensional spectrum due to the changes caused by the interlocking objects in the intensity levels of the corresponding pixels of the displayed image from frame to frame.

In the above description of the embodiment of the present invention, it was assumed that the time signal G ₀ was a baseband signal having a frequency spectrum that determined information of one dimension or more.
As is known, such baseband information is
It is often communicated in a frequency multiplexed format such that it consists of sidebands of a carrier frequency modulated by baseband information components. 1st
By using appropriate modulators and demodulators for each relay means 100-1,...100N shown in the figure, any of G ₀ and/or G ₁ ,...G _N and/or L ₀ ,
... _LN can be made into a frequency multiplexed signal.

The term "shift register" is to be interpreted in the claims as including means performing the same function, such as read-then-write serial storage devices.

[Brief explanation of drawings]

FIG. 1 is a functional block diagram showing the invention implemented in its most general and general form; FIG. 1a is a first type digital embodiment of any one of the group of sample signal relay means of FIG. 1; FIG. 1b is a diagram showing a second type digital embodiment of any one of the sample signal relay means group in FIG. 1, and FIG.
A diagram illustrating a last alternative digital embodiment of the sample signal relay means of the first or second type shown in the figure;
FIG. 2 is a diagram showing an example of a kernel weighting function that can be used in carrying out the present invention, and FIG. 3 is a one-dimensional block diagram of a spectral decomposition device, a spectral modification circuit, and a signal synthesis device embodying the principles of the present invention.
FIG. 4 is a block diagram of one of the decomposition stages used for the iterative calculation of the spectral decomposition process of FIG. 3 in which the present invention is implemented, and FIG. 5 is a continuation of the decomposition stages of FIG. 4 in another embodiment of the present invention. 6 is a block diagram of one of the synthesis stages used in the iterative process of FIG. 3 for synthesizing signals from spectral components; FIGS. 7, 8, 9 and 9. 10 is a block diagram of the representative spectral modification circuit of FIG. 3 used in the present invention, and FIG. 11 is a block diagram of the representative spectral modification circuit of FIG. Figure 12 is a block diagram of a two-dimensional spatial frequency spectrum decomposition device using a pipeline structure to perform spectrum decomposition in real time, and Figure 13 is a block diagram of a modification of the method shown in Figure 12. 1 is a block diagram of an apparatus for synthesizing a signal representative of a decomposed sample field from its output spectrum; FIG. 1a, 1b...pipeline, 100-1,...
100-N...sample signal relay means, 102,
104...first means, 102...m-tap convolution filter, 109, 110...second means,
110...Sample subtraction means, 106, 108,
109... Delay means (third means), 104...
Reducer (thinner), 106... Enlarger, 108...
...Interpolation means, 353-363...Sample signal combination means.

Claims (1)

[Scope of Claims] 1. A signal processing device for decomposing the frequency spectrum of information components of a given time signal into (N+1) separate frequency bands; wherein the components have a given number of dimensions. , N is a plurality of integers, the highest frequency of interest in the frequency spectrum is f ₀ , and in order to decompose the frequency spectrum in real time, the frequency spectrum is arranged in order to form a group. and a pipeline consisting of N extracted signal relay means, each of which has a first and second input terminal and a first and second input terminal.
an output terminal of a first relay means in said group of relay means, said first input terminal of said first relay means in said group of relay means being coupled to receive said given time signal; The first input terminal of each of the Nth relay means is coupled to the first output terminal of the relay means immediately preceding each of the relay means in the group of relay means, The relay means sends a signal to the relay means immediately following it, and the second input terminal of each relay means in the group of relay means is coupled to receive a clock signal of a different sampling frequency. A signal extracted at a frequency equal to the sampling frequency of the applied clock signal is extracted to the first and second output terminals of each of the relay means, respectively, and the relay means group each of said relay means in said group of relay means exhibits a low pass transfer function for said information component between its first input terminal and first output terminal; a pass transfer function having a nominal cut-off frequency that is a direct function of the sampling frequency of the clock signal applied to the second input terminal of each of said relay means; The clock signal applied to the input terminal of 2 is
a sampling frequency that is (a) twice f ₀ and (b) provides a nominal cut-off frequency lower than f ₀ for said low-pass transfer function of said first relay means in said group of relay means for said information component; and each of the second to Nth relay means in the group of relay means
(a) is lower in frequency than the clock signal applied to the second input of the relay means immediately preceding each relay means in the group of relay means; and (b) the frequency of the clock signal applied to the input terminal of at least equal to twice the highest frequency of the information component of the signal applied to the input terminal of
(c) said second output of each said relay means in said group of relay means having a sampling frequency which gives its low pass transfer function a lower nominal cut-off frequency than that of said relay means immediately preceding said relay means in said group of relay means; The information component of the signal drawn to the terminal is the difference between the information component of the signal applied to the first input terminal and the information component of the signal drawn to the first output terminal. Correspondingly, each of the (N+1) separate frequency bands corresponds to each of the N signals occurring at the second output terminal of the N relay means and the first frequency band of the N relay means. A signal processing device comprising: a signal generated at an output terminal. 2. Each of said relay means in said group of relay means has a first and second input terminal and a first output terminal coupled to said respective relay means to provide said low pass transfer function of said respective relay means. 1 means, the first means and a second input terminal and a second output terminal of each of the relay means;
and a second means for drawing the difference signal to an output terminal, and the first means is configured to generate information about a signal applied to the first input terminal of the relay means, when m is a given plural integer. an m-tap convolution filter for convolving the component by a predetermined kernel function at a sampling frequency corresponding to the frequency of a clock signal applied to a second input terminal of the relay means, said predetermined kernel function and said relay means; the sampling frequency of the convolutional filter of the means respectively defines the shape and nominal cut-off frequency of the low-pass transfer function of the relay means in each dimension of the information component; third means including said delay means for coupling sample subtraction means to said first means via delay means;
Said sample subtraction means inputs each successively occurring sample level of the convolved sample of said relay means to a first input terminal of said relay means in a time-aligned relationship at the sampling frequency of said convolved sample of said relay means. The output of the sample subtraction means is subtracted from each successively occurring level of the information component of the applied signal before being convolved by the predetermined kernel function of the convolution filter of the relay means. , each difference sample level occurring sequentially at the sampling frequency of the convolved sample of the relay means, each difference sample level constituting an information component of a signal drawn at a second output terminal of the relay means. The signal processing device according to claim 1, characterized in that: 3 at least one of said first means of said relay means in said group of relay means has the given form;
First means of the given type are coupled in series between said convolutional filter, said filter, an output of said filter and a first output terminal of said relay means in said group of relay means. said convolutional filter of said first means of said type given, said convolutional filter having said convolutional filter of said first means of said type given, said information corresponding to the sampling frequency of the clock signal applied to said second input terminal of said relay means at its output; Deriving a particular sample density for each dimension of the component, the reducer of the first means of the given form convolves the first means of the given form in each dimension of the information component. Only some, but not all, of the convolved samples occurring at the output of the filter are sent to the first output terminal of the relay means, thereby reducing the amount of the information component at the first output terminal of the relay means. The reduced sample density of said convolved samples in each dimension is reduced to said specific sample density of said information component of said convolution filter output of said relay means in a corresponding dimension. The signal processing device according to claim 2, characterized in that: 4 In said at least one of said relay means, said third means is coupled between said convolution filter output and said sample subtraction means to convolution filter said convolved information component. 4. The signal processing apparatus according to claim 3, further comprising fourth means for directly applying the signal from the sample subtracting means to the sample subtracting means. 5 Said third means are further coupled between said reducer and said sample subtractor for reducing said convolved samples in each dimension of said information component at said first output terminal of said relay means. fifth means for extending the sample density back to the particular sample density of the convolved sample in that dimension of the sample subtraction means;
a sample expander for inserting an additional sample of a respective zero value level whose occurrence corresponds to each convolutional sample of the output of the convolutional filter that is not at the reduced sample density; 4. A signal processing apparatus according to claim 3, further comprising interpolation means for replacing the zero value level of each inserted additional sample with an interpolated value sample level.