JP3063682B2 - Video / audio decoder having audio playback time control function - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video / audio decoder for simultaneously reproducing video and audio.

[0002]

2. Description of the Related Art FIG. 8 shows a conventional video / audio decoder reproduction time control circuit. An image / audio decoder playback time control circuit is used for adjusting the playback time of an image and an audio in a system for simultaneously playing an image and an audio, such as an MPEG (Moving Picture Expert Group) decoder. The reproduction time of the image and the audio are shifted due to the difference in the decoding time between the data and the decoded data, the error in the encoded data, and the like, and it is necessary to correct it.

First, each part will be described with reference to FIG.
When the encoded audio data 8 is input, the audio decoder 7 decodes the encoded audio data 8 and outputs it as output audio data 9 to the outside. When the pause signal 4 and the skip signal 5 are input, a pause operation and a skip operation for sound reproduction are performed. The pause operation stops the decoding of the audio for a time corresponding to the difference between the audio playback time and the image playback time when the audio playback time is ahead of the image playback time,
Thereafter, the decoding of the audio is restarted. The skip operation skips encoded audio data for a time corresponding to the difference between the audio playback time and the image playback time when the audio playback time is later than the image playback time, and then skips the audio playback time. This is an operation to restart decoding. The image playback time 1 and the audio playback time 2 are signals indicating the current image and audio playback time. The control circuit 3 compares the input image reproduction time 1 and the audio reproduction time 2 and, when the difference between them becomes equal to or more than a certain value, the pause signal 4 or the skip signal 5 and the control time 6 are converted to the audio decoder 7. To enter. If the audio playback time is advanced, the pause signal 4 is input to the audio decoder 7, and the audio playback is stopped for the input control time 6 to delay the playback time.
If the audio playback time is later, the skip signal 5 is input to the audio decoder 7, and the audio playback is skipped by the input control time 6 to advance the playback time. In this way, the reproduction of the audio is stopped or skipped, the reproduction time of the audio is adjusted to the reproduction time of the image, and the reproduction of the image and the audio is synchronized.

The control circuit 3 is configured as shown in FIG. The audio playback time 30 is subtracted from the input image playback time 29 to obtain a playback time difference 31. The absolute value calculator 34 calculates the absolute value of the reproduction time difference 31. This value is determined value t
_The comparing unit 35 determines whether or not the _{threshold value} is larger than _{th. If the threshold value} is larger than the _{threshold value th} , the synchronization processing enable signal 36 is output to the pause / skip signal output unit 37. The positive / negative determination unit 32 determines whether the reproduction time difference 31 is positive or negative, and outputs a positive / negative determination signal 33 to the pause / skip signal output unit 37. The pause / skip signal output unit 37 outputs a pause signal 38 or a skip signal 39 when the synchronization processing enable signal 36 is input. At this time, if the positive / negative determination signal 33 is positive, the skip signal 39 is output because the reproduction of the sound is delayed, and if negative, the pause signal 38 is output because the reproduction of the sound is advanced. Further, the difference between the image reproduction time and the audio reproduction time is output as the control time 40.

FIG. 10 shows the pause and skip operations. If the pause operation of the control time t _p is performed at time t ₁ as shown in the upper diagram of Fig. 10a, discontinuity of the output sound is generated between the start and end time pause as shown in the figure below.
Also, in the case of performing the skip only control time t _s at time t ₂ as shown in the upper diagram of FIG. 10b, the discontinuity of the output sound at skipping occurs. As described above, in the conventional video / audio decoder playback time control circuit, pause and skip control is performed at an arbitrary time, so that the discontinuity of the voice occurs even in the case where the amplitude of the output voice is large, which is unpleasant to human hearing. felt.

It is conceivable that the synchronous control is not pause and skip, but continuously changes the audio reproduction speed.
Also in this case, if the operation is performed at a place where the amplitude of the output sound is large, it is unnatural in audibility, and the quality of the reproduced signal is impaired.

[0007]

The above-described conventional video / audio decoder reproduction time control circuit controls skip and pause at an arbitrary time. Therefore, when output sound is a sound, skip and pause are controlled. If you have a human hearing,
The reproduced sound is unnatural and unpleasant. Therefore, an object of the present invention is to solve the above-mentioned disadvantages of the conventional example, and to provide an image / audio decoder which does not give any unpleasantness to human hearing even if discontinuity exists in audio data at the time of synchronous processing. To provide.

[0008]

SUMMARY OF THE INVENTION An image / audio decoder according to the present invention, when decoding and reproducing coded image / audio data simultaneously, generates an audio signal when a time difference occurs between the image reproduction time and the audio reproduction time. A synchronization process is performed to synchronize the audio playback time with the image playback time by the decoder. This synchronization process analyzes the decoded audio data using the silence detection circuit and detects whether the amplitude of the audio data is smaller than a predetermined silence threshold. When the amplitude of the audio data is smaller than the silence threshold and the time difference between the two reproduction times exceeds a predetermined value, the control circuit causes the audio decoder to perform synchronization processing.

The above-described synchronization processing includes pause processing for interrupting decoding of audio data and delaying a reproduction time, and skip processing for skipping encoded audio data and skipping the reproduction time. This is performed so as to compensate only for the difference between the playback time and the playback time of the audio data.

In the synchronous processing, the reproduction speed of the audio data can be controlled to increase or decrease the reproduction speed. The reproduction speed of the audio data depends on the reproduction time of the image data and the audio data at that time. It is determined according to the time difference from the data reproduction time.

[0011]

FIG. 1 is a diagram showing a first embodiment of the present invention.

First, each part will be described with reference to FIG.
When the coded audio data 17 is input, the audio decoder 16 decodes the coded audio data 17 and outputs it as output audio data 18 to the outside. When the pause signal 13 and the skip signal 14 are input, a pause operation and a skip operation for sound reproduction are performed. The silence detection circuit 19 outputs the output audio data 1
It is determined whether or not 8 is silent, and if no, a silence detection signal 20 is output. The image reproduction time 10 and the audio reproduction time 11 are signals indicating the current reproduction time of the image and the audio. The control circuit 12 receives the image reproduction time 10
And the sound reproduction time 11 are compared, and when the difference between them is equal to or more than a certain value and the silence detection signal 20 is input from the silence detection circuit 19, the pause signal 13 or the skip signal 14 and the control time 15 is input to the audio decoder. If the audio playback time is advanced, the pause signal 13 is input to the audio decoder 16, and the audio playback is stopped by the input control time 15 to delay the audio playback time.
If the audio playback time is later, the skip signal 14 is input to the audio decoder 16, and the audio playback is skipped by the input control time 15 to advance the playback time. As described above, the reproduction of the audio is stopped or skipped only while the output audio data 18 is silent, the reproduction time of the audio is adjusted to the reproduction time of the image, and the reproduction of the image and the audio is synchronized.

The control circuit 12 can be configured as shown in FIG.
The audio playback time 42 is subtracted from the input image playback time 41 to obtain a playback time difference 44. The absolute value calculator 47 calculates the absolute value of the reproduction time difference 44. The comparing unit 48 determines whether or not this value is larger than a predetermined value t _{th. If} it is larger, the synchronization processing enable signal 49 is output to the pause / skip signal output unit 50. The positive / negative determination unit 45 determines whether the reproduction time difference 44 is positive or negative, and outputs a positive / negative determination signal 46 to the pause / skip signal output unit 50. Pause·
The skip signal output unit 50 outputs the synchronization processing enable signal 4
When 9 is input and the silence detection signal 43 is input, a pause signal 51 or a skip signal 52 is output. At this time, when the positive / negative determination signal 46 is positive, the skip signal 52 is output because the reproduction of the audio is delayed, and when the positive / negative determination signal 46 is negative, the pause signal 51 is output because the reproduction of the audio is advanced. Also, the control section 53 outputs how much the audio playback time is advanced or delayed.

The silence detection circuit 19 can be configured as shown in FIG. The input output audio data 21 is converted to an absolute value calculation unit 22.
And the absolute value 23 of the output audio data is calculated. A value obtained by multiplying this by a constant determined by the multiplier 24 and a value obtained by multiplying the value stored in the register 25 by the constant determined by the multiplier 26 are added to the register 21.
To be stored. The data stored in the register 21 is
It is considered that the absolute value 23 of the audio output data is like an average value from the past. The value stored in the register 21 is compared with a predetermined silence threshold value by the comparison circuit 27. If the value is equal to or less than this value, the silence detection signal 2
8 is output.

The pause and skip operations will be described with reference to FIG. In the circuit of the present embodiment, the silence detection circuit 1
9 controls pause and skip when the amplitude of the output audio data 18 is smaller than the silence threshold.
Figure 4a from time t ₁ at which the amplitude of the sound becomes smaller than the silence threshold, only the control time t _p, is a case of performing a pause operation. In this case, the sound is discontinuous at the start and end of the pause, but since it is a silent part where the amplitude of the sound is small, it has little effect on human perception. Also, FIG.
It is when the amplitude of the audio sound from the time t ₂ becomes smaller than the silence threshold was only skip operation control time t _s. Also in this case, since the sound is discontinuous in a silent part having a small amplitude of the sound, the reproduced sound is not perceived as unnatural by human perception.

It is conceivable that silence is not detected for a long time and synchronization processing cannot be performed. However, the silence threshold of the silence detection circuit 19 is increased in proportion to the time during which silence is not detected. This can be solved by making the synchronization process more likely to occur.

In this embodiment, the silence detecting circuit 19
Determines whether or not there is silence based on the magnitude of the amplitude of the output voice data 18. However, by using the characteristics of human hearing, a voice whose output voice data 18 has a large amplitude but is difficult to hear by human hearing is used. It is easy to configure the audio decoder playback time control circuit of the present invention even if a silence detection circuit that determines that the audio signal is silent is used.

FIG. 5 is a view showing a second embodiment of the present invention.

First, each part will be described with reference to FIG.
When the encoded audio data 60 is input, the audio decoder 59 decodes the encoded audio data 60 and outputs it as output audio data 61 to the outside. When the reproduction speed control signal 57 is input, the audio reproduction speed is controlled in accordance with the separately input control speed 58. That is, when the control speed 58 is positive, the reproduction speed is increased, and when the control speed 58 is negative, the reproduction speed is reduced. The silence detection circuit 62 determines whether or not the output audio data 61 is silence, and outputs a silence detection signal 63 if the output speech data 61 is determined to be silence. The image playback time 54 and the audio playback time 55 are signals indicating the current image and audio playback times, respectively. The control circuit 56 compares the input image reproduction time 54 and the audio reproduction time 55, and if the difference between them is equal to or more than a certain value and the silence detection signal 63 is input from the silence detection circuit 62, the reproduction is performed. The speed control signal 57 and the control speed 58 are input to the audio decoder 59. If the playback time of the audio is advanced, a control speed 58 for reducing the playback speed of the audio decoder 59 is input to the audio decoder 59, and the playback speed of the audio decoder 59 is reduced to delay the playback time of the audio. . If the audio playback time is later, the audio decoder 59
The control speed 58 is input to the audio decoder 59 so that the reproduction speed of the audio is increased, and the reproduction speed of the audio decoder 59 is increased to advance the audio reproduction time. As described above, only while the output audio data 61 is silent, the audio reproduction speed is increased or decreased to match the audio reproduction time with the image reproduction time, and synchronize the image and audio reproduction. .

The control circuit 56 can be configured as shown in FIG.
The audio playback time 65 is subtracted from the input image playback time 64 to obtain a playback time difference 67. The absolute value calculator 68 calculates the absolute value of the reproduction time difference 67. The comparing unit 69 determines whether this value is larger than a predetermined value t _th , and if it is larger, the synchronization processing enable signal 70 is output to the reproduction speed control signal output unit 71. The reproduction speed control signal output unit 71 outputs a reproduction speed control signal 72 when the synchronization processing enable signal 70 is input and the silence detection signal 66 is input. Further, a value obtained by multiplying the reproduction time difference 67 by a constant determined by the multiplier 73 is output as a control speed 74 indicating how much the audio decoder increases or decreases the reproduction speed.

The operation of the reproduction speed control will be described with reference to FIG. In the circuit of the second embodiment, the silence detection circuit 6
2, the playback speed is controlled when the amplitude of the output audio data 61 is smaller than the silence threshold. FIG. 7 shows that the control of the reproduction speed is started at time t ₁ when the amplitude of the audio becomes smaller than the silence threshold, and at time t ₂ , the time difference between the image reproduction time and the audio reproduction time becomes equal to or less than a certain value, This shows a case where the control of the reproduction speed has been completed. As an example, a case where the reproduction time of the image is later than the reproduction time of the audio at time t ₁ and the reproduction speed of the audio decoder is delayed from t ₁ to t ₂ is shown. In this case, at the start and end of the control of the playback speed, the playback speed of the audio is discontinuous, but since the audio is a silent part with a small amplitude of the audio, the reproduced audio is unnatural due to human perception. I can't feel there.

[0022]

As described above, according to the present invention, when a time difference of a predetermined value or more occurs between the image reproduction time and the audio reproduction time by using the silence detection circuit and the control circuit, the output audio data By performing the synchronization processing of the audio reproduction time so as to eliminate the time difference only when there is no sound, there is an effect that it is possible to reduce the influence of the discontinuity of the audio generated at the time of the synchronization processing of the image / audio decoder on human perception.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of a control circuit according to the first embodiment.

FIG. 3 is a diagram illustrating a configuration of a silence detection circuit according to the first and second embodiments.

FIG. 4 is a diagram illustrating pause and skip operations of audio reproduction according to the first embodiment.

FIG. 5 is a diagram illustrating a configuration of a second embodiment.

FIG. 6 is a diagram illustrating a configuration of a control circuit according to a second embodiment.

FIG. 7 is a diagram illustrating a synchronization process based on a change in a reproduction speed of audio reproduction in a second embodiment of the present invention.

FIG. 8 is a diagram showing a configuration of a conventional example.

FIG. 9 is a diagram showing a configuration of a control circuit in a conventional example.

FIG. 10 is a diagram showing pause and skip operations of audio reproduction in a conventional example.

[Explanation of symbols]

 1 Image reproduction time 2 Audio reproduction time 3 Control circuit 4 Pause signal 5 Skip signal 6 Control time 7 Audio decoder 8 Encoded audio data 9 Output audio data 10 Image reproduction time 11 Audio reproduction time 12 Control circuit 13 Pause signal 14 Skip Signal 15 Control time 16 Audio decoder 17 Encoded audio data 18 Output audio data 19 Silence detection circuit 20 Silence detection signal 21 Output audio data 22 Absolute value calculation unit 23 Absolute value of output audio data 24 Multiplier 1 25 Register 26 Multiplication Unit 2 27 Comparison circuit 28 Silence detection signal 29 Image reproduction time 30 Audio reproduction time 31 Reproduction time difference 32 Positive / negative judgment unit 33 Positive / negative judgment signal 34 Absolute value calculation unit 35 Comparison circuit 36 Synchronization processing enable signal 37 Pause / skip signal output unit 38 Pause signal 39 Skip Signal 40 Control time 41 Image reproduction time 42 Audio reproduction time 43 Silence detection signal 44 Reproduction time difference 45 Positive / negative judgment unit 46 Positive / negative judgment signal 47 Absolute value calculation unit 48 Judgment unit 49 Synchronization processing enable signal 50 Pause / skip signal output unit 51 Pause Signal 52 Skip signal 53 Control time 54 Image playback time 55 Audio playback time 56 Control circuit 57 Playback speed control signal 58 Control speed 59 Silence detection signal 60 Encoded audio data 61 Output audio data 62 Silence detection circuit 63 Silence detection signal 64 Image reproduction time 65 Audio reproduction time 66 Silence detection signal 67 Reproduction time difference 68 Absolute value calculation unit 69 Comparison unit 70 Synchronization enable signal 71 Reproduction speed control signal output unit 72 Reproduction speed control signal 73 Multiplier 3 74 Control speed

Claims (5)

(57) [Claims] 1. A method for decoding and reproducing encoded image data and audio data simultaneously, and synchronizing the audio reproduction time by an audio decoder with the image reproduction time when a time difference occurs between the image reproduction time and the audio reproduction time. A video / audio decoder that performs a synchronization process for converting the decoded voice data into an audio data, and detects whether or not the amplitude of the audio data is smaller than a predetermined silence threshold. the amplitude of the data is detected Ikoto smaller than the silence threshold value, and, when the time difference exceeds a predetermined value, and having a control circuit which subjected the synchronization process to the audio decoder And an image / audio decoder having a function of controlling the audio reproduction time. 2. The synchronizing process comprises a pause process for interrupting decoding of the audio data to delay a playback time, and a skip process for skipping the encoded audio data and leading a playback time. 2. The video / audio decoder according to 1. 3. The pause processing is performed when the reproduction time of the audio data is earlier than the reproduction time of the image data, and the skip processing is performed when the reproduction time of the audio data is later than the reproduction time of the image data. 3. The video / audio decoder according to claim 2, wherein the video / audio decoder is compensated by a difference between the reproduction time and the reproduction time of the audio data. 4. The video and audio decoder according to claim 3, wherein said synchronization process controls a reproduction speed of increasing or decreasing a reproduction speed of said audio data. 5. The video and audio decoder according to claim 4, wherein the reproduction speed of the audio data in the synchronization processing is determined according to a time difference between a reproduction time of the image data and a reproduction time of the audio data at that time.