JPH0452669B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a video and audio relative time difference compensation device,
In particular, it is designed to automatically compensate for the relative time difference between a video signal and an audio signal that occurs when a television signal is transmitted via a frame synchronizer.

A digital converter (frame synchronizer, hereinafter abbreviated as FS) is used to match the synchronization phase of television signals with different current synchronization to that of the local station, and to perform image processing. . This FS stores video in memory in synchronization with an input video signal (hereinafter referred to as "video"), and reads out this stored video with reference synchronization given by the user.The output video is synchronized with an audio signal (hereinafter referred to as "video"). (abbreviated as "audio"), development occurs later than that of audio. As a result, the audio advances relative to the video, and one
Even if it is below the detection limit when using FS, if multiple units are connected in cascade, the progress time will exceed the allowable limit. In the future, it is possible that multiple FSs will be connected to the line system, and it will be necessary to consider countermeasures for lead times.

As a countermeasure to this problem, a method of delaying the audio using a fixed delay line may also be considered. However, since the video delay time is a function of time and changes discontinuously due to the FS overtaking and overtaking phenomena, this fixed delay line method cannot be used.

Here I will explain the phenomenon of overtaking and being overtaken. As mentioned earlier, FS writes in synchronization with the input video and reads in reference synchronization, but if the frequencies of both synchronizations do not match, the output will be determined based on the reference synchronization. The video read memory address may overtake the memory address where the input video is written (if the frequency of the reference period is higher than the frequency of the input video)
On the other hand, development occurs that is overtaken (when the frequency of the reference synchronization is lower than the frequency of the input video), and as a result, the output video is divided into two frames by one frame.
You may have to read it twice or omit one frame. This is called the "overtaking/overtaking phenomenon", and as a result, the video/audio relative time difference changes discontinuously.

An object of the present invention is to eliminate the above-mentioned drawbacks, and to continuously change the audio delay amount to follow the continuous change in the video delay amount, and to respond to the discontinuous change in the video delay amount. The aim is to correct and absorb the relative time difference between video and audio by immediately following the amount of audio delay in a manner that does not affect the auditory sense.

To achieve this objective, the present invention uses an audio memory having a predetermined storage capacity, adjusts the amount of delay in this audio memory based on the phase difference between the input video and output video of the FS, and adjusts the amount of delay between the video and audio. Synchronize.

That is, the present invention provides phase difference detection means for detecting a phase difference between an input video signal and an output video signal of a frame synchronizer, a phase difference zero prediction means for predicting the time when the phase difference becomes zero, and memory control means for generating a predetermined first address signal sequence during a predetermined period related to the time when the phase difference becomes zero, and generating a second address signal sequence according to the phase difference during periods other than the predetermined period; and an audio signal memory for writing and reading audio signals inputted along with the video based on each of the first and second address signal strings, and controlling the writing and reading of the memory by the second address signal strings. The apparatus is characterized in that the relative time difference is compensated for by the first address signal sequence, and the audio signal is delayed by a predetermined time until the time difference becomes zero by controlling the writing and reading of the memory by the first address signal string. It is.

Hereinafter, the present invention will be explained in detail with reference to the drawings.

FIG. 1 shows an example of a block configuration of a compensation device according to the present invention, where 1 is the aforementioned FS, 2 is its input video, 3 is an output video, and 4 is a reference synchronization signal. The input image 2 and the output image 3 are supplied to a phase difference detection means 5. This phase difference detection means 5 detects the phase difference between the two and generates phase difference information 6. The phase difference is, for example, the time difference between synchronization signals, or
It can be determined from the difference between the write and read addresses of FS1. This information 6 is supplied to the audio frame memory control means 7. As described later, the memory control means 7 reads and writes a write signal 8 (hereinafter abbreviated as R/W control signal) based on this information 6.
An address signal 9 and a fade in/out signal 10 are generated. The signals 8 and 9 are supplied to the audio frame memory 11, and the signal 10 is supplied to the amplifier 12.
supply to. A PCM encoder 13 digitizes the analog input audio 14 by pulse code modulation. The output signal 15 is supplied to the audio frame memory 11 and sequentially written to predetermined addresses under the control of the R/W control signal 8 and address signal 9 described above. Reference numeral 16 denotes a PCM decoder, which converts the pulse signal 17 sequentially read out from the memory 11 under the control of the R/W control signal 8 and address signal 9 into an analog audio signal 18 by pulse code demodulation. This signal 1
8 to an amplifier 12 to obtain an output pitch 19.

In the device shown in the first figure, the audio frame memory control means 7 controls the writing and reading of the audio frame memory 11 based on the phase difference information 6 supplied from the phase difference detection means 5, and outputs the video 3 and the output video 3. The phase and time difference occurring between the input audio 14 and the input audio 14 is always automatically corrected. Regarding the automatic correction, for example, the following two forms can be considered.

The first method is to control the write/read clock frequencies of the audio frame memory 11 to eliminate the relative time difference between video and audio. That is, by fixing one of the write clock frequency and the read clock frequency and changing the other, or by changing both, the amount of audio delay in the audio frame memory 11 is changed, and the audio delay is increased. quantity
Match the amount of video delay in FS1. In addition,
Audio frame memory 1 by changing the clock frequency
Since the pitch changes while the delay time in 1 is changed, it is preferable that the amount of change in the clock frequency, that is, the amount of change in the amount of delay per unit time, be within a range that does not affect the auditory sense.

This "range that does not affect the auditory sense" will be considered as follows.

According to the paper, the human frequency discrimination limit is about 0.1% around 1KHz, where discrimination ability is highest. Use this value below, 1KHz is 1Hz, 30K
Assume that it is possible to discriminate a difference of 30 Hz in Hz. Now, if the write clock is 30KHz, the audio memory will require one video frame time (that is, 1/30 second), and its capacity will be 30,000 x 1/30 = 1,000 words.
When the relative time difference between video and audio is maximum (ie, 1/30 second), approximately 1000/30≒30 seconds are required to correct this.

An example of the configuration of the audio frame memory control means 7 in the case of adopting this first form is shown in FIG. In this example, the write clock frequency is fixed and the read clock frequency is varied. That is, 21 is a read clock frequency control means, and this control means 2
1 calculates the current amount of delay in the audio frame memory 11 by calculating, for example, the difference data between the two addresses from the write address information 23 from the write address generation means 22 and the read address information 25 from the read address generation means 24. Since this delay amount corresponds to the difference between the delay amount and the video delay amount in FS1 represented by the phase difference information 6, the readout clock frequency is increased or decreased in the readout address generation means 24. A control signal 26 is supplied to enable the control. As a result, the amount of audio delay in the audio frame memory 11 changes, and the amount of delay in the FS1 and the amount of delay in the audio frame memory 11 match.

In the second form, the write/read clock frequency is fixed and the audio delay amount is adjusted by operating one or both of the write address and the read address.In this case, the audio frame memory control means 7 An example of the configuration is shown in FIG. In this example, only the read address is manipulated. Here, reference numeral 31 is a read address operating means, which controls the read address generating means 24 to skip and read twice so as to generate a read address signal string corresponding to the amount of video delay. A signal 32 is provided. Based on this control signal 32, the read address generating means 24 skips or reads the address twice at a constant control period _T1 (every n times of reading), as shown in FIG. Let's do it. As can be understood from FIG. 4, the amount of delay in the audio frame memory 11 decreases by skipping addresses, and increases by reading twice. It is preferable that the frequency of this skipping and double reading operations is set to a level that does not affect the auditory sense, as in the case of the first embodiment.

Next, there are three methods for processing when input video VS (video synchronization signal) switching occurs, that is, when the relative time difference between video and audio changes instantaneously.

The first processing method is to perform processing in the first form or the second form when video VS switching occurs, and if a silent period is detected by 34 silence detection means during this phase adjustment period, 7 audio frames are detected. The control means forcibly controls the read address and adjusts the amount of delay. This is the process normally performed.

The next method is that even if input video VS switching occurs,
This is used when it is desired to maintain audio continuity, such as when broadcasting a marathon. No forced read address control,
Processing is performed within a certain period in the first form or the second form. If you do this, the audio will be continuous, but it will take up to 30 seconds to adjust the delay amount.

The third method is to gradually reduce the amplification degree of the amplifier 12 using the fade-in/out signal 10 and fade out the audio when the video VS switching is detected. Then, after the read address is forcibly controlled by the audio frame control means 7 and the delay amount is instantaneously adjusted, the fade-in/out signal 1
0 gradually increases the amplification degree of the amplifier 12 and fades in the audio.

Which of the above three processing methods is appropriate differs depending on the situation, and it would be difficult to determine this with a machine, so a changeover switch should be used to set it.

Next, countermeasures against discontinuous changes in the video/audio relative time difference due to overtaking and overtaking phenomena will be explained.

As mentioned above, the phenomenon of overtaking and being overtaken is
This is caused by a mismatch between the input video synchronization and the reference synchronization, that is, a slight difference in the oscillation frequencies of the respective original oscillators that form both synchronization signals, and the time interval T ₂ is at least 1 in terms of the accuracy of the crystal oscillator. It takes about 50 minutes. In this case, the normal processing method, that is, the first form or the second form, may be used, but in this case, if the frequency of the input video synchronization is lower than the reference synchronization, the development of audio progress will be difficult. This will be explained below, including countermeasures.

In other words, as shown in Figure 5, when an overtaking/overtaking phenomenon occurs, at this point tx, the FS
The amount of delay at 1 suddenly changes from "0" to "1/30 seconds". Assuming that the delay amount in the audio frame memory 11 at this moment follows the video delay amount and becomes "0", at this time tx, the audio has instantaneously advanced by 1/30 seconds relative to the video. Therefore, during the period T ₃ (approximately 30 seconds) until the relative time difference becomes zero due to correction, the state in which the audio advances relative to the video continues. By the way, intuitively,
It feels less unnatural when the audio lags behind the video rather than ahead of it. Therefore, the time tx at which this overtaking/overtaking phenomenon occurs is predicted in advance, and the amount of delay in the audio frame memory 11 is calculated in the FS at that time during a predetermined period _T4 before this time tx (see Part 6).
Gradually increasing the amount of video delay in step 1, the overtaking and overtaking phenomenon occurs,
Time tx when the delay amount in FS1 suddenly changes to 1/30 seconds
In this step, the amount of delay in the audio frame memory 11 is predictively controlled so that the amount of delay is 1/30 second.

That is, in the first embodiment, a zero phase difference prediction means is provided in the read clock control means 21, and in the second embodiment, the read address operation means 31 is provided with a zero phase difference prediction means to predict the time tx at which the overtaking/overtaking phenomenon occurs. This time tx can be easily determined from the current phase difference provided by the phase difference information 6 and the rate of change of the phase difference per unit time, and the prediction means for this can be configured by, for example, a microprocessor. . Overtaking When the time tx of occurrence of the overtaking phenomenon approaches, as described above, the amount of delay in the audio frame memory 11 is gradually increased during the predetermined period _T4 before that time. This increase in the amount of delay is achieved by controlling the read address generating means 24 using the clock frequency control signal 26 in the first embodiment and the interlaced twice read control signal 32 in the second embodiment. The period _T4 for increasing the amount of delay is about 30 seconds in order to control the delay amount so as not to affect the auditory sense, and it is preferable that the amount of delay is increased at a constant rate of increase. Note that even if the audio delay amount at time tx is not 1/30 seconds, the influence of the overtaking phenomenon can be reduced to some extent. Therefore, the period T ₄ for predictive control may be set to extend before and after the time tx, and the amount of audio delay at the end of the period T ₄ may be appropriately reduced from 1/30 seconds.

As a result, the address generating means 24 reads
In either form, two types of address signal sequences are used, namely, the address in period _T4 for controlling the readout so that the delay amount is 1/30 second in advance in preparation for the phenomenon of overtaking. signal train,
Address signal strings for other periods are supplied to the audio frame memory 11 for controlling the readout so that the delay amount matches the delay amount in FS1.

Finally, in the case of input video loss, the delay amount at the time of input video loss is retained, and the audio delay amount is adjusted using the first or second mode when the input video is restored. do.

Note that the device of the present invention may be configured integrally with the FS, but if it is used as a separate device and added to the FS as necessary, the number of devices required can be reduced. Furthermore, it goes without saying that the format of the FS used with the device of the present invention is arbitrary.

As explained above, according to the present invention, even when multiple FSs are connected in series in a grid,
By using one device of the present invention for each FS, it becomes possible to automatically compensate for the relative time difference between video and audio throughout the system, contributing to high-quality broadcasting.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing a configuration example of the apparatus of the present invention, FIG. 2 is a block diagram showing a first configuration example of the audio frame memory control means shown in FIG. 1, and FIG. 3 is a block diagram showing a second configuration example. FIG. 4 is a diagram showing the state of double reading and interlaced reading of the audio frame memory shown in FIG. FIG. 6 is a diagram showing the correspondence relationship between delay amounts, and FIG. 6 is a diagram showing the correspondence relationship when predictive control is similarly performed. 1...FS, 2...Input video signal, 3...Output video signal, 4...Reference synchronization signal, 5...Phase difference detection means, 6...Phase difference information, 7...Audio frame memory control means, 8...R/W control signal, 9...Address signal, 10...Fade in/out signal, 11...Audio frame memory,
12...Amplifier, 13...PCM encoder, 1
4...Input audio signal, 15...PCM encoder output signal, 16...-PCM decoder, 17...Pulse signal, 18...PCM decoder output signal, 1
9...Output audio signal, 21...Read clock frequency control means, 22...Write address generation means, 23...Write address information, 24...Read address generation means, 25...Read address information, 26...Reading Clock frequency control signal, 3
DESCRIPTION OF SYMBOLS 1...Reading address operation means, 32... Skipping, twice reading control signal, 33... Address setting signal when switching video VS, 34... Silence detection means, T ₁
... Control cycle, T ₂ ... Time interval between overtaking and overtaking and phenomenon occurrence, T ₃ ... Required time for relative time difference compensation after overtaking and overtaking phenomenon occurs, T ₄ ... Predetermined period for predictive control.

Claims (1)

[Scope of Claims] 1. Phase difference detection means for detecting a phase difference between an input video signal and an output video signal of a frame synchronizer, and zero phase difference prediction means for predicting a time when the phase difference becomes zero. memory control means that generates a predetermined first address signal sequence during a predetermined period related to the time when the phase difference becomes zero, and generates a second address signal sequence corresponding to the phase difference during periods other than the predetermined period; , an audio signal memory for writing and reading audio signals inputted along with video based on each of the first and second address signal strings, and writing and reading the audio signals in and out of the memory using the second address signal strings. The relative time difference is compensated by control, and the audio signal is delayed by a predetermined time by the time when the time difference becomes zero by controlling the writing and reading of the memory by the first address signal string. Video and audio relative time difference compensation device. 2. The video/audio relative time difference compensating device according to claim 1, wherein the predetermined period is 30 seconds or more, and the rate of increase in the delay time of the audio signal during this period is approximately constant. 3. When the input video signal is interrupted, the relative time difference compensation operation is interrupted, the delay amount at the relevant time point in the audio frame memory is held, and when the video signal is restored, the relative time difference compensation operation is stopped. The video/audio relative time difference compensating device according to claim 1 or 2, characterized in that the device restarts the operation. 4. Claim 4, characterized in that in response to a sudden change in the phase difference, the audio signal is delayed immediately or within a certain period of time by controlling the second address signal train. The video/audio relative time difference compensating device according to item 1, item 2, or item 3.