JPH0792824B2 - Real-time video processor - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a real-time signal processor that realizes digital signal processing such as digital filtering and high-efficiency coding with software for moving picture signals such as television signals.

(Prior art and its problems) The advantage of real-time digital signal processing is that it is possible to realize a filter and modulator / demodulator with high accuracy or high stability, which cannot be realized by analog technology. The time-varying adaptive filter can be easily realized. Furthermore, by incorporating the results of digital LSI technology, which has been rapidly developing in recent years, it is possible to reduce the size and power consumption of real-time digital signal processing circuits, and gradually replace analog circuits and apply them to higher functionality. Is progressing to. For more detailed advantages of digital signal processing, see pages 1280 to 1284 of the December 1982 issue of the Institute of Electronics and Communication Engineers.

Digital signal processing, which has many advantages in this way, also has the drawback of requiring a huge amount of calculation. In order to perform real-time signal processing, digital signal processing must be performed within a sampling period per sampled input signal sample, for example, telephone speech (8KHz sampling).
When a 4th-order cyclic digital filter process is applied to
8 multiplications and 8 additions are required within 125 microseconds. Therefore, the frequency bandwidth is more than 1000 times wider than that of telephone voice, and therefore the signal processing circuit for telephone voice is at least 1000 times faster than the signal processing circuit for telephone voice in order to perform signal processing on video signals with a sampling period of 1/1000 or less. A circuit is needed.

For this reason, high-level digital signal processing can be performed only for signals in the audio region at present, and moving image signals are currently limited to very simple processing.

Furthermore, regarding digital signal processing for signals in the voice domain, we want to perform advanced digital signal processing,
Often, various parameters are changed or part of the signal processing algorithm is changed. Therefore, there is a strong demand for a signal processing device whose algorithm and parameters can be changed by software. As hardware for performing digital signal processing by conventional software, there are IEEE Journal of Solid State Circuits No. SC-16.
Volume 4 (August 1981), pages 372 to 376, includes signal processors, and typical applications of this signal processor include Broodings of International, published in 1982.
Conference on Acoustics Speech Signal Processing (Proceedings of Acoustics, S
There is 32k bps ADPCM published from page 960 to page 963 of peach signal processing), which is also targeted for telephone voice processing.

With such a conventional processor format, it is not possible to easily expect a speed of 1000 times or more, no matter how fast the arithmetic circuit is made,
It was not suitable for a software-controlled processor to perform high-speed digital signal processing for moving images.

(Object of the Invention) An object of the present invention is to provide a software-controlled circuit capable of performing advanced digital signal processing on a moving image signal such as a television signal.

(Structure of the Invention) The present invention comprises: (a) a control unit for generating a predetermined input partial screen position signal and output partial screen position signal from a synchronization signal for notifying the beginning of one screen of a moving image signal such as a television signal, A capturing unit that captures a partial screen signal specified by the input partial screen position signal of a moving image signal that is input with an input partial screen position signal from the control unit, and the capturing unit that is connected to the capturing unit A processing unit for performing signal processing on the moving image signal captured by the device until the start of capturing of the next screen, and a processing unit connected to the output side of the processing unit for storing the processing result of the processing unit and separately controlling the control. A plurality of unit processors each including an output unit that outputs the stored processing result to a partial screen position designated by the output partial screen position signal input from the unit; and (b) the plurality of unit processors. An input bus for supplying the synchronizing signal and the moving image signal to each of the input / output terminals, and (c) means for obtaining the entire image signal by adding all outputs of the plurality of unit processors. The feature is that it eliminates the exchange and realizes signal processing with a delay of one screen.

(Principle of the Invention) According to the principle of the present invention, one screen (frame) is divided into a plurality of partial screens, and one unit signal processor is allocated to each partial screen, whereby a moving image is displayed by a plurality of unit signal processors. Is to be processed.

First, if it is treated as a one-dimensional signal suitable for transmitting a moving image signal, it is necessary to sample at about 10 MH as described above. In this case, it was necessary to perform processing per sample within a period of about 100 nsec. When the moving image signal is treated as a two-dimensional signal called a screen, for example, a television signal sends only 30 screens per second. In other words, if one screen can be processed in 33 milliseconds, a delay of one screen will occur, but the real-time property is maintained.

A plurality of unit signal processors are prepared to process the sampled signal for one screen, an area to be processed is preset between the unit signal processors, and each unit signal processor is assigned an assigned process. The moving image signal required for the partial screen area is selectively fetched.
In this case, the capture partial screen is generally larger than the processing partial screen. For example, the two-dimensional sampling signal of coordinates (i, j) is x (i, j)
And the two-dimensional signal is impulse response {h
Consider filtering through (i, j)}. At this time, filter operation is performed for each of the partial areas O and P. And the output signal y (i, j) in the partial area O is
It is realized by calculating from the impulse response h (i, j) and the input signal x (i, j) defined by

Therefore, the input signal {x (i,
The section Q of j)} becomes Q = {(i, j) :-( M + N) i (M + N),-(M + N) j (M + N)} (3) from the expressions (1) and (2). FIG. 2 shows the relationship between the data capture screen Q and the processing screen O, and shows a square capture image section Q of one side 2 (M + N) and a square processed image section O of one side 2N.

The expression (2) is called a convolution operation, but other than this, the correlation operation can be expressed almost in the same way as the expression (2), and the relationship between the captured image and the processed image can be expressed as shown in FIG.

An example in which the captured screen and the processed screen match is digital signal processing using the Fourier transform. There is FFT (Fast Fourier Transform) as a method for performing the Fourier transform at high speed by digital signal processing. This FFT is to transfer a time domain signal to a frequency domain signal, and after performing signal manipulation in the frequency domain, it is returned to the time domain by an inverse FFT (IFFT).

In order to perform the FFT of the M × M region, the following formula is usually used.

Here, (k, l) is obtained by multiplying the input signal x (k, l) by a window function w (k, l) such as a Hamming window (k, l) = w (k, l) x ( k, l) (5) This window function has a value that decreases toward the boundary of the defined M × M region, so normally It is general to perform FFT by overlapping each area in four directions, perform signal processing in the frequency domain, perform inverse FFT, return to the time domain, and add the overlapped comrades. In other words, the processing of overlapping and taking in and adding the output results of the overlapping portions is often used.

When these digital signal processings are applied to partial screens, convolution and correlation calculation ensure a wider captured screen than the processing screens, that is, overlap between unit signal processors that process between partial screens. By capturing the moving picture signal, the subsequent digital signal processing can be processed only by the data inside the unit signal processor.

In addition, even in correlation calculation applying FFT, moving image signals are captured by overlapping between unit signal processors, and then each unit signal processor independently processes digital signals. The processing result of the processor and the captured screen are overlapped by the overlapping amount, and the normal processing result can be obtained by adding all the processing results of this overlapping amount.

Therefore, a circuit that can input data to the unit signal processor in collaboration with other unit processors, and the output of each unit signal processor is always zero except for the area corresponding to the processing screen, and the processing result is output to the part corresponding to the processing screen. However, if the outputs of all the unit signal processors are added, if there is overlap output between the unit signal processors, the overlapped signals are added, and if there is no overlap output, the single signal processor Since the output is obtained as a true output, the above-described basic operations of digital signal processing can be independently processed by the unit signal processors provided in parallel. That is,
It is sufficient for each unit signal processor to process the allocated partial screen within the above-mentioned 33 ms which is one frame sampling period. By operating many unit signal processors in parallel, real-time video processing can be performed. It will be possible.

(Example) Next, the Example of this invention is described, referring drawings. FIG. 1 shows a sync signal input terminal 1, a moving picture signal input terminal 2, unit signal processors 3, 4, 5, 6 and a sync signal output terminal 7, which is an embodiment of the present invention when four unit signal processors are used. It consists of a video signal output terminal 8 and adders 91, 92 and 93, and the unit signal processors 3, 4, 5 and 6 are each capturing units 10,
The processing unit 11, the reading unit 12, and the control unit 13 are included. Capture part
The reading unit 12 and the reading unit 12 are storage circuits, and the details of the processing unit 11 and the control unit 13 will be described later.

The synchronization signal input from the terminal 1 is input to the control unit 13 of each of the unit signal processors 3, 4, 5 and 6. Control unit 13
Then, the time point at which the signal belonging to the pre-assigned capturing partial screen area is input to the terminal 2 is identified from the input synchronizing signal, and the capturing unit 10 is notified as a capturing signal.

The capturing unit 10 captures and stores the moving image signal input to the terminal 2 according to the capturing signal transmitted from the control unit 13.

The control unit 13 also transmits an execution signal to the processing unit 11 when the signal of the predetermined capture partial screen area is input based on the synchronization signal input from the terminal 1, and the processing unit 11 is input from the control unit. Predetermined digital signal processing by the execution signal, for example, the convolution calculation of the formula (2) or the FFT calculation of the formula (4) is performed on the capture screen signal stored in the capture unit 10, and the calculation is performed. The result is written in the reading unit 12.

The control unit 13 further detects a predetermined processing partial screen area output time point from the synchronization signal input from the terminal 1, and when the processing partial moving image area is reached, transmits an output command signal to the output unit 12, and the output unit 12 controls the control unit. Based on the output command signal from 13, the processed data written and processed by the processing unit 11 is sequentially read when the output command signal is on, and zero is output when the output command signal is off.

The output signals from the output sections of the unit signal processors 3, 4, 5 and 6 are added by adders 91, 92 and 93 and transmitted to the output terminal 8.

FIG. 3 shows the case where the capture signal, the execution signal, and the output command signal used in the unit signal processors 3 and 4 in the moving image processor of FIG. Is shown. In order to simplify the explanation, it is considered that the moving image signal used in FIG. 3 has undergone the scanning line conversion in which the scan signal over the entire screen is rearranged for each partial screen. The sync signal (a) applied to the terminal 1 signals the start of one screen, and in the unit signal processor 3 that processes the first first divided screen, the capture signal (b) generated by the control unit 13 is the sync signal. At the same time, it starts up and continues to instruct the acquisition until the acquisition area ends. Further, the control unit 13 transmits the execution signal (c) to the processing unit 11 after the end of the capture. As a result, the processing unit 11 performs signal processing from the rising edge of the execution signal (c) to the next rising edge of the fetch signal (b). The control unit 13 also sends the output command signal (d) to the output unit 12. This output command signal (d) is the same as the acquisition signal (a) in applications such as FFT, but in applications such as convolution, the acquisition area is the processing area as shown in FIG. Since it is wider, the duration may be shorter than the capture signal (a).

Signals (b '), (c'), (d ') are a take-in signal, an execution signal, and an output command signal of the unit processor 4 which processes the second divided screen. The processing time allowed for the processing unit 11 of the unit processor 4 is from the rising of the execution signal (c ') to the rising of the output command signal, and this length is the processing unit 11 of the unit processor 3.
It is the same as the time allowed for.

Although the control signals of only the unit processors 3 and 4 have been described with reference to FIG. 3, the unit processors 5 and 6 are similarly operated. The time when each unit processor outputs is only when each output command signal is ON, so the time when the outputs other than zero of the two unit signal processors are added by any of the adders 91, 92 and 93 is (e ). The portions shown as A, B, C in the signal (e) are adders 91, 92, 93, respectively.
Indicates the time when the addition operation is performed with a date other than zero. This addition is as detailed in the Principles of the Invention section.
It is necessary for FFT processing. As a result, the moving image signal shown in (f) is obtained at the output terminal 8. Here, V ⁱ is the output of the i-th unit signal processor, and V ^{i + j} is the addition result of the outputs of the i-th and j-th unit signal processors. Therefore, a moving image signal can be continuously obtained at the output terminal 8, and real-time signal processing can be achieved.

FIG. 4 shows an example of the control unit 13 used in the unit signal processors 3, 4, 5 and 6, which includes a synchronization signal input terminal 20, a clock signal input terminal 21, a capture signal output terminal 22, and an execution signal output terminal. 23, output command signal output terminal 24, column counter 25, row counter 26, read-only memories 27, 28, gate circuits 29, 30,
It consists of 31.

The read-only memory 27 is a 3-bit output. The first bit is programmed to output 1 when the value of the input address matches the line number of the capture screen, and the other is programmed to output zero. Is programmed to output 1 when the value of the input address is the line number on the screen at the time when you want to output the execution command, and 0 for the other, and the third bit is the processing screen Are programmed to output 1's for those that match the line number and zero for others.

Similarly, the read-only memory 28 has a 3-bit output,
Bits are programmed to output 1 if the input address value matches the column number on the capture screen, and 0 for the others, and the 2nd bit indicates the time when the input address value outputs the execution command. 1 for the column number on the screen,
The others are programmed to output zero, and the third bit is programmed to output 1 when the value of the input address matches the column number of the processing screen and the other outputs zero.

When the synchronizing signal is input from the terminal 20, the column counter 25 and the row counter 26 are reset and both output zero.
Assuming that the control unit of the unit processor 3 for processing the first section in FIG. 1 is considered, the read-only memory 28 indicates the first bit indicating the capture screen and the output screen by the value 0 of the column counter. "1" is output to the third bit and the second bit is "0". Further, when the value of the row counter is 0, the read-only memory 27 outputs "1" at the first bit indicating the capture screen and the third bit indicating the output screen, and the second bit is "0". Therefore, the gates 29, 30, and 31 output "1" to the fetch signal output terminal 22, "0" to the execution signal output terminal 23, and "1" to the output command output terminal 24, respectively.

Every time a sampled moving image signal is applied to the terminal 2 in FIG. 1, a signal is applied to the clock terminal 21 in FIG. 4 to advance the column counter 25, and when the column counter 25 completes one column of the entire screen, the row counter is counted. Step 26 forward and the row counter 25 will return to zero. Therefore, the first bit of the read-only memories 28 and 27 outputs "1" as long as the column counter 25 and the row counter 26 indicate the columns and rows belonging to the capture screen, and the gate 29 thus captures them. "1" is output to the terminal 22 for the sample position belonging to the screen.

Similarly, the read-only memories 28, 2 only when the column counter 25 and the row counter 26 indicate the values of the column and the row to instruct the start of processing.
7 outputs "1", and at this time, the gate 30 outputs "1" to the terminal 23 as an execution signal.

Similarly, when the column counter 25 and the row counter 26 indicate the columns and rows corresponding to the output screen, the read-only memories 28 and 27 each output "1", and as a result, the gate 31 outputs "1" to the terminal 24 as an output command signal. Output 1 ".

FIG. 5 shows an embodiment of the processing units in the unit signal processors 3, 4, 5 and 6 of FIG.
Register 41, gate 42, input terminal 43 from the capturing section, address output terminal 44 to the capturing section, output terminal 45 to the output section, address output terminal 46 to the output section, write signal output to the output section It is composed of a terminal 47, an execution signal input terminal 48, and an acquisition section output prohibition signal output terminal 49. It is assumed that the signal processor 40 uses the NEC µPD7720 described in the second document of the present invention. The μPD7720 is a signal processing processor that has a unique bus configuration and has a multiplier and an adder inside, but the details are given in the second document. The μPD7720 is designed so that interrupt processing can operate when a signal arrives at the interrupt input terminal (INT), and the programmable output bit P
I have 1, P2. Input / output is performed via a bidirectional parallel bus (D), and when a signal is input to the write terminal (W), it is used as an input direction bus, and when a signal is not input to the write terminal (W), it is output direction. Used as a bus.

Now, the execution signal from the control unit 13 in FIG. 1 is the terminal in FIG.
Upon joining 48, the signal processor 40 starts digital signal processing as an interrupt processing. For this reason, the capture part of FIG.
Input data from 10 is required. First, prepare the required address in port D and set bit output port P1 to "1".
Is output. At this time, the gate 42 outputs "0", the data of the port D can be output from the signal processor 40 to the outside, and the address is stored in the register 41. Next, when P1 is set to "0", the content of the register 41 is transmitted to the capturing unit 10 via the terminal 44, and the corresponding data is input from the terminal 43 to the port "D".

Similarly, in order to transfer the data processed by the signal processor 40 to the output unit 12, the address of the output unit 12 is specified. Therefore, the required address is prepared in the port D and the bit output port P1 outputs “1”. Is output and the address is written in the register 41. This address is output to the output unit 12 via the output terminal 46.
Transmitted to. Next, the processed data is prepared in the port D and "1" is output from the bit output port P2. At this time, the gate 42 outputs "0", and the port D outputs the signal processor 40.
Further, the data is output to the outside, and the output section 49 is notified to the capturing section that the output is prohibited. Therefore, the data on the D port is transmitted to the output section through the terminal 45. In addition, "1" of the bit output port P2 is transmitted to the output section through the terminal 47, and commands the writing of the data transmitted from the terminal 45 to the output section.

FIG. 6 shows an example of the configuration of the output section 12 of the unit signal processor shown in FIG. 1, which has a write signal input terminal 51 and a data input terminal 5.
2, address input terminal 53, output command input terminal 54, output terminal 55, clock input terminal 56, zero input terminal 57, random access memory (RAM) 60, selectors 61, 62, counter 63, flip-flop 64, gate It consists of 65.

When the output command signal output from the terminal 24 of the control unit in FIG. 4 is off, this signal is input to the terminal 54 in FIG. 6, controls the selector 61, and is input to the output terminal 55 from the terminal 57. Output zero. At the same time, when the output command signal is off, the processing unit shown in FIG.
It is controlled so that it can be transmitted to RAM60. At this time, the signal at the terminal 47 shown in FIG. 5 is sent to the write input terminal 51 shown in FIG. 6, and the data output signal at the terminal 45 shown in FIG. 5 is sent to the data input terminal 52 shown in FIG.
Since the signal at the address output terminal 46 in FIG. 5 is connected to the terminal 53 in FIG. 6, the data processed by the processing unit shown in FIG. 5 is transferred to the address determined by the signal at the terminal 53 in FIG. , Is written via the terminal 52 by the write signal of the terminal 51.

When the output command signal input to the terminal 54 changes from off to on, the selectors 61 and 62 are respectively in the RAM 60 and the counter 6
At the same time when 3 is selected, the flip-flop 64 that stores the OFF status one hour ago and the current ON status gate 65
, The gate 65 detects the change from the previous time and outputs "1". The output of this gate 65 causes the counter 63
Is reset to "0".

Therefore, the value of “0” of the counter 63 is passed through the selector 62.
It is connected to the address of RAM 60 and the value of RAM 60 at address 0 is output to terminal 55 via selector 61.

When the output command signal input to the terminal 54 continues to be on, both the value of the flip-flop 64 and the value of the terminal 54 are on (“1”), so the counter 63 is not reset after that and each sample is not reset. The counter 63 is incremented by 1 at each time, and the value of the counter 63 is accessed by incrementing the address of the RAM 60 by 1 via the selector 62 and the data of the corresponding address is selected.
It will be output to the terminal 55 via 60.

The present invention can be implemented as described above.

(Effects of the Invention) As has been seen above, according to the present invention, moving image signals are not communicated with each other by a plurality of unit signal processors, and there is no effect on digital signal processing at the boundary between unit signal processors. Digital signal processing can be realized without giving. Therefore, real-time digital signal processing can be applied to a moving image signal by using many unit signal processors. Also, the unit signal processors placed in parallel differ only in the designation of the capture screen and the processing screen, and the processing section of each unit signal processor should be processed by the same digital / signal processing program, so program development is simple. It is sufficient to perform only one unit signal processor, and the programs of the other unit signal processors may be copies of the developed program, so that the program work becomes easy.

Further, since only the areas of the capture screen and the processing screen are different between the unit signal processors, many unit signal processors are provided in parallel to prohibit the output of the unit signal processor in which a failure has occurred, and Since the failure can be recovered by changing only the definitions of the capture screen and the processing screen, it can be used as a highly reliable signal processor.

(Modification of the Present Invention) In the embodiment of the present invention, the read-only memory is used for the control unit. However, by replacing the read-only memory with a random access memory or the like, it is possible to move the predetermined positions of the captured partial image and the processed partial image. It is also within the scope of the present invention to make such changes.

Further, in the present invention, the control unit for designating the positions of the captured partial image and the processed partial image is dispersedly provided in each unit signal processor, but these are centralized and only the control signal is supplied to each unit signal processor. The method of dispensing is also within the invention.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention, FIG. 2 is a diagram for explaining the principle of the present invention, and FIG. 3 is an operation timing of the embodiment of the present invention shown in FIG. FIG. 4, FIG. 4 is a diagram showing a configuration example of a control unit, FIG. 5 is a diagram showing a configuration example of a processing unit, and FIG. 6 is a diagram showing a configuration example of an output unit. In FIG. 1, 1 ... Sync signal input terminal, 2 ... Video signal input terminal, 3,
4,5,6 …… Unit signal processor, 91,92,93 …… Adder, 8 …… Video output terminal, 10 …… Capture section, 11 …… Processing section, 12 …… Reading section, 13 …… The control unit.

Claims (1)

[Claims] 1. A control unit for generating a predetermined input partial screen position signal and output partial screen position signal from a sync signal for notifying the beginning of one screen of a moving picture signal such as a television signal. A capturing unit that receives an input partial screen position signal and captures a partial screen signal of the separately input moving image signal specified by the input partial screen position signal; and a moving image signal that is connected to the capturing unit and that is captured by the capturing unit. On the other hand, a processing unit that performs signal processing until the next screen capture starts, and an output unit that is connected to the output side of the processing unit and stores the processing result of the processing unit and that is separately input from the control unit. A plurality of unit processors each including an output unit that outputs the stored processing result to a partial screen position designated by a screen position signal; and (b) the plurality of unit processors. An input bus for supplying the synchronization signal and the moving image signal to each of the processor units; and (c) means for adding all outputs of the plurality of unit processors to obtain an entire image signal. The frame image is divided into partial images that allow overlap, each partial image is processed by the unit processor, and the entire processing image is obtained by adding the processing screens to each other. A real-time video processor characterized by finishing by the arrival time of the frame.