JPS6242280B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a method for converting an audio signal into an arbitrary band by compressing or expanding the frequency spectrum while maintaining the time length of the audio signal.

Compressing and expanding the frequency spectrum of speech without changing its duration has long been considered as one of the challenges in speech information processing. For example, when audio that has been recorded on a tape is played back by changing the tape speed, the spectrum is expanded during low-speed playback, and the audio is recorded in the original band while still being heard as a document. This technology is necessary to compress the spectrum to make it easier to understand and understand its contents in a short time. It is also a necessary technology for narrowband transmission systems that combine compression and expansion, and for improving the intelligibility of helium speech.

By the way, methods of thinning out the audio waveform or repeatedly reproducing the same waveform are used to compress and expand the frequency spectrum of audio. For this reason, tape recorders with rotating playback heads and devices that implement the same principle using semiconductor memory have been manufactured. However, according to this method,
Since the waveform signals are cut out and connected regardless of the fundamental period of the voice, there is a lot of noise and distortion, making it impossible to obtain high-quality voice. It has also been considered to reduce noise and distortion by detecting the period of a waveform, using one period of the waveform as a unit, and expanding and contracting the waveform. This method has not become widespread because the input and output waveforms may be out of phase or the system may become unstable if the input audio signal-to-noise ratio is low.

On the other hand, analysis and synthesis systems such as LPC vocoders and PARCOR can compress and expand the frequency spectrum, but the sound quality of analysis and synthesis systems is poor.
This method is far from practical use because the deterioration is significant when the S/N of the input audio is low.

As a method to overcome the above problems, a frequency spectrum compression/expansion method using a short-time autocorrelation function was invented.
May 28, 1975; filing date: July 8, 1975);
It is named SPAC (Masashi Suzuki, SPAC for audio processing using autocorrelation function, Institute of Electronics and Communication Engineers, technical research report EA75-25, published in July 1975). SPAC can process even audio with a low S/N ratio of 0 dB, and has relatively little distortion and is highly practical. However, when comparing the audio processed with SPAC and the original audio, there is a slight deterioration in quality.
I also feel that there is a lack of high frequency sounds. These causes include a decrease in the unspeech level due to the use of the autocorrelation function, spectral distortion that emphasizes the strength and weakness of the frequency spectrum, and the fact that the amount of calculation for the correlation function increases when the frequency band is widened. This is because it is difficult to obtain a wide band. One way to eliminate the drawbacks of SPAC is to divide the frequency band into two channels, high and low, apply SPAC to the lower frequency channel, and apply asynchronous waveform expansion and contraction to the higher frequency channel, which is unrelated to the audio period. (Name of invention: Audio information conversion method, Patent number: 1063230, Registration date: September 22, 1980, Application date: March 1978)
12th). However, even with this method, distortion due to waveform discontinuity remains, resulting in unsatisfactory quality.

The present invention uses SPAC (one period is determined from the short-time autocorrelation function of the waveform signal in the low frequency band of the audio signal, and the time origin of the audio waveform for which the correlation function is calculated is set at a time interval proportional to the one period). The waveforms of one period of the correlation function are connected sequentially to produce the output signal.), and the waveform of one period of the correlation function is It is characterized by sequentially cutting out and connecting them to obtain an output waveform signal by combining both, and its purpose is to compress and expand the frequency spectrum of the audio signal without distortion.

The present invention will be explained below with reference to the drawings. FIG. 1 is a block diagram of an embodiment of the present invention, in which 20 is an input terminal;
1 and 31 are BPF (bandwidth filter) - A and B, 22
and 32 are sampling circuits A and B, 23 and 33 are memories A and B, 24 is a maximum amplitude position detection circuit, and 25
and 35 are window circuits - A and B, 26 and 36 are DA converters - A and B, 27 and 37 are IPF (low pass filters) - A and B, 34 are pitch cycle determining circuits, 38
is a correlator, 39 is a memory-C, 40 is a control circuit,
49 is a sum circuit, and 50 is an output terminal.

Figure 2 is a diagram showing the waveforms of each part in Figure 1.
The number of each waveform is shown in FIG. 1 is the input speech waveform, 2 is the sampled output waveform of BPF-B, 3 is the waveform of the short-time autocorrelation function S ₁ regarding time t ₁ , and 4 is the short-time autocorrelation function S ₂ regarding time t ₂ .
5 is the waveform edited from 3 and 4, 6 is the sampled output waveform of BPF-A, 7 is the waveform cut out and edited from 6, and 8 is the output waveform of the sum of 5 and 7.

The input audio waveform 1 applied to the input end 20 is
BPF-A21 and BPF-B31 divide the signal into two frequency bands, high and low. 21 is 31
This is a higher frequency band BPF, and 21 and 31 cover one required frequency band.
The output signal of 31 is a waveform of low frequency components, and this becomes a sampling signal of sampling period I in sampling circuit-B32. This is shown by 2 in FIG. 2
is temporarily stored in memory-B33. A sample (referred to as ai) with the signal t ₁ as the origin is read out from 33 and sent to a correlator 38, where a short-time autocorrelation function S is calculated using equation (1).

Here, NI corresponds to product-sum (integration) time and LI corresponds to delay time. Let S at t ₁ be S ₁ , which is shown by 3 in FIG. 3 is stored in memory C39. The pitch cycle determination circuit 34 determines the cycle T ₁ from 3 read from the pitch cycle 39 . It is a known technique to determine the pitch period (fundamental period) of speech from the short-time autocorrelation function in this way. T ₁ and 3 are applied to a window circuit-B 35, which cuts out a sample corresponding to T ₁ from 3 according to a certain logic. The DA converter-B36 converts this sample into an analog quantity at a sampling period kI, and a waveform (R ₁ ) of 5 is obtained as an output signal. Note that k>0. The waveform extraction logic (window function) in 35 may be of any type as long as it does not cause distortion or noise, but considering that random noise included in the input signal gathers around j = 0 in S, It is sufficient to avoid including nearby waveforms. In the case of FIG. 2 and 3, the waveform is cut out using a rectangular window based on the first point in time when the amplitude of S changes from negative to positive.

Next, based on the information of T ₁ determined in 34, 33
A sample is read out from the contents of , with time t ₂ (t ₂ =t ₁ +kT ₁ ) as the origin, and S is calculated in step 38 using equation (1).
This S is stored in 39, which is shown in FIG. From 4 read from 39, period T ₂ at 34
Determine. T ₂ and 4 are added to 35, and a sample corresponding to T ₂ is cut out from 4 according to a certain logic. 36 converts this sample into an analog quantity at a sampling period kI, and the waveform (R ₂ ) shown in FIG. 25 is obtained as the output signal. Note that R ₂ is output continuously to R ₁ .

Next, S ₃ is calculated for time t ₃ (t ₃ =t ₂ +kT ₂ ), and the same process is repeated. As a result, waveforms R ₁ , cut out from S ₁ , S ₂ , S ₃ . . .
R ₂ , R ₃ ... are obtained.

On the other hand, the output waveform of 21 is sampled at a sampling period J in a sampling circuit-A22. This is shown in FIG. 16. 6 is temporarily stored in memory-A23.
From 23 to a certain time window U with time t ₁ as the origin,
The corresponding sample is read out and sent to the maximum value detection circuit 24. In step 24, the maximum value of the amplitude is detected and the time u ₁ at that time is determined. For window circuit-A25, 23
Regarding 6 read from , cut out a sample corresponding to T ₁ according to a certain logic based on u ₁ ,
Send to DA converter-A26. Here, T ₁ is the period determined in 34. Note that any window function may be used for cutting out the waveform in 25, but in FIG. 2, the waveform is cut out using a rectangular window from the point in time when the amplitude is 0, which is before u ₁ in 6. At 26, this is converted into an analog quantity at a sampling period kJ, and the waveform _W1 shown in FIG. 27 is obtained as the output signal at 26.

Next, from 23, a sample corresponding to a certain time window U is sent to 24 with time t ₂ as the origin, and time u ₂ at which the amplitude is the maximum value is determined. In 25, for 6 read from 23, a sample corresponding to T ₂ is cut out, converted to an analog quantity in 26 with a sampling period of kJ, and the second
Waveform W ₂ in FIG. 7 is obtained as 26 output signals. Note that W ₂ is connected to W ₁ and output. Next, obtain u ₃ with respect to time t ₃ and repeat the same process. As a result, output 7 of 26 contains
A continuous waveform of W ₁ , W ₂ , W ₃ . . . is obtained during the time kT ₁ , kT ₂ , kT ₃ .

Waveform signals 7 and 5 are LPF-A27 and LPF-B3
After being smoothed in step 7, the signals are added in summation circuit 49 to obtain output waveform 8. As a result, 8 is a waveform signal obtained by converting the frequency spectrum of the input audio signal to 1/k.

In the method of the present invention, for signals in high frequency bands, only processing such as waveform storage and cutting/editing is performed, and no special calculations are performed. Therefore, it is easy to widen the frequency band, and since the original waveform is used as it is, high quality can be obtained.

I and J may be arbitrary as long as they satisfy the sampling theorem, but if I is made an integral multiple of J, the relationship between R and W can always be kept constant. In addition, in the block diagram shown in Figure 1, two sets of DA converters and LPF are used, but 35 output signals are interpolated to convert them into a signal sequence with a sampling period of J, and 25 output signals are used. It is also easy to obtain 8 by digitally adding the signal, performing DA conversion, and smoothing.

The time window U is set to an arbitrary time longer than the maximum possible pitch period, or is set to the period T determined in step 34. In the latter case, the difference in phase between the signals of the two channels is smaller.

By the way, when k=2, 3, 4..., 2
4 can be omitted, and the principle of this system will be explained using the waveforms in FIG. 51 is the same waveform as 6, 52 is the converted waveform corresponding to 7,
Here, the case where k=2 is shown. In the present invention, 51 waveforms of one period T ₁ are expanded by k times with t ₁ as a reference, resulting in 26 output waveforms 52, W ₁ . t ₂ = t ₁ +
kT ₁ , and the phase positional relationship of t ₁ and t ₂ with respect to the waveform 51 is equal. Therefore, W ₂ expanded based on t ₂ is smoothly connected to W ₁ , and there is no need to determine the peak of the waveform from 51 and cut out and edit the waveform based on that time u.

Next, when 1/k=m=2, 3, 4,...
A third invention in which 24 can be omitted will be explained with reference to the waveforms in FIG. Here, 53 is the same waveform as 2, and 53 is the same waveform as 2.
4 is the correlated waveform corresponding to 3, 55 is the converted waveform corresponding to 5, 56 is the same waveform as 51 and 6, and 57 is the converted waveform corresponding to 7, these are shown as m=2. ing.

In the present invention, the period T ₁ is detected from S ₁ , 54 calculated for t ₁ , and one period of the waveform is cut out and repeated m times at the sampling period kI to produce an output signal 55 (this is referred to as mR ₁₎ . ). Next, for time t ₂ = t ₁ + T ₁ , calculate S ₂ and convert mR ₂ to mR ₁
Output to connect to. The next process is t ₃ = t ₂ +
It is performed for T ₂ , and in 55, mR ₁ , mR ₂ , mR ₃ ,
...is obtained. On the other hand, regarding the waveform 55 read from 23, one period of the waveform is cut out based on time _t1 , and converted at the sampling period kJ to form the output waveform 57 of 26, which is repeated m times. this
Let mW be ₁ . Next, one cycle of the waveform is cut out based on _t2 , connected to _mW1 , and output repeatedly m times. Next, similar processing is performed for _t3 . As a result, the 26 output waveform signals 57 are mW ₁ , mW ₂ ,
This is a waveform in which the same waveform as mW ₃ is repeated m times, and each waveform is connected at the same position in terms of phase. There is no need to determine the peak of the waveform from 56 and perform cutting and editing of the waveform based on that time u. Note that in this method as well, the frequency spectrum is converted to 1/k.

When a periodic signal is converted into an autocorrelation function, the amplitude of each frequency component has the property of being squared. SPAC uses an equalizer for preprocessing that compresses the amplitude of the frequency spectrum to reduce the quality loss caused by this property. In each method of the present invention, the bandwidth processed by SPAC is narrow, so distortion is small, but
The sampling circuit-B32 can be made into an AD converter with square root characteristics, or can be made into an equalizer by replacing the sampled amplitude value with its square root.

In addition, in the present invention, the short-time autocorrelation function is calculated using equation (1), but the signals of the zero crossing waves of ai and ai _are
When, the short-time cross-correlation function of ai and bi is used instead of the short-time autocorrelation function (title of invention: waveform signal conversion method, patent number: 1059232, registration date: August 25, 1982, Inventor: Makoto Suzuki). In this case, correlation calculation becomes simple and an equalizer is not required.

The operation of this method has been described above for quasi-periodic signals. No significant periodicity or peaks are observed from the S or waveform of an aperiodic signal such as unvoiced sound. However, it is sufficient to obtain the local maximum value of S or the waveform, determine T in the same way as for periodic signals, or perform processing to cut out the waveform.

Next, examples of the present invention will be shown. In the method shown in Figure 1,
21 from 1000 to 3400Hz, 31 from 100 to 1000Hz, I=
Sampling and processing were performed in J=100 microseconds.
Note that in equation (1), N=200 and L=170. The peak positions of the 24 waveforms were searched between t ₁ −T ₁ /2 and t ₁ +T ₁ /2. When processed with k=1, a sound that was almost the same as the original sound was obtained. Also, play the recorded audio at 1/3 to 3 times the speed, and
1, 31, I and J are changed proportionally, k
When processed at 1/3 to 3, it was possible to obtain audio whose duration was increased by 1/3 to 3 times while keeping the audio frequency band unchanged. The quality of the voice was more natural and less noisy than the voice converted using any conventional method. Note that the input audio S/
Even when N was set to about 0 dB, stable operation was observed, and no deterioration in quality was noticed.

As described above, according to the present invention, it is possible to compress and expand the frequency spectrum of a quasi-periodic signal stably without signal discontinuity or distortion, even in the presence of noise, and to reproduce the recording/reproducing method. In combination with changing the speed, it is possible to shorten or extend the signal time length, so it can be used for recording and fast listening of lectures and conversations, information services, adjusting the length of audio broadcast programs, generating special sound effects, and producing helium audio. It can be widely applied to voice information processing in general, such as improving intelligibility, assisting people with hearing loss in communication, and narrowband transmission systems.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of an embodiment of the present invention, and FIGS. 2 to 4 are waveforms illustrating the operation of the system of the present invention. 20...Input end, 50...Output end, T...Waveform period, I and J...Waveform sampling period, k...Constant.

Claims (1)

[Claims] 1. Divide the audio waveform signal into frequency bands of two high and low channels, sample the signal in the low frequency band at a sampling period, and obtain a short-time autocorrelation function S ₁ at time t ₁ . Determine the period T ₁ from S ₁ , cut out the samples corresponding to T ₁ , obtain the signal R ₁ arranged at the sampling period kI, and then calculate the short-time autocorrelation at time t ₂ (t ₂ = t ₁ + kT ₁₎ . Find the function S ₂ , determine the period T ₂ of S ₂ , cut out the sample corresponding to T ₂ , obtain the signal R ₂ arranged at the sampling period kI, connect R ₂ next to R ₁ , and then Similar processing is performed for time t ₃ (T ₃ = t ₂ + kT ₂ ) to obtain R ₃ , and the process of connecting to the next one after R ₂ is repeated, resulting in a continuous waveform of R ₁ , R ₂ , R ₃ , etc. The signal (A) is obtained, and on the other hand, the signal in the high frequency band is sampled at the sampling period J, and the signal is sampled in the vicinity of the basic synchronization range of the audio (hereinafter referred to as "the vicinity of t ₁ ") based on time t ₁ . Detect the maximum value of the waveform and find the time u ₁ ,
Samples corresponding to T ₁ are cut out based on u ₁ to obtain a signal W ₁ arranged at a sampling period of kJ, and then the maximum value of the waveform is detected near time t ₂ to obtain that time u ₂ . Samples corresponding to T ₂ are cut out based on u ₂ and arranged at a sampling period of kJ to obtain a signal W ₂ , and W ₂ is converted into W ₁
Then, perform the same process for time t ₃ to obtain W ₃ , repeat the process of connecting after W ₂ ,
By obtaining a continuous waveform signal (B) of W ₁ , W ₂ , W ₃ , ..., and obtaining a waveform signal that combines A and B, we can expand the frequency spectrum of the voice when 0<k<1. An audio signal conversion method that is characterized by being distortion-free. 2 Divide the audio waveform signal into frequency bands of two high and low channels, sample the signal in the low frequency band at sampling period I, find the short-time autocorrelation function S ₁ at time t ₁ , and calculate the period from S ₁ . Determine T ₁ , cut out the samples corresponding to T ₁ , obtain the signal R ₁ arranged at the sampling period kI, and then calculate the short-time autocorrelation function S ₂ at time t ₂ (t ₂ = t ₁ + kT ₁ ). , determine the period T ₂ of S ₂ , cut out the samples corresponding to T ₂ , obtain the signal R ₂ arranged at the sampling period kI, connect R ₂ next to R ₁ , and then select the signal R 2 at the next time t ₃ ( t ₃ = t ₂ + kT ₂ ), the same process is performed to obtain R ₃ , and the process of connecting next to R ₂ is repeated to obtain a continuous waveform signal (R ₁ , R ₂ , R ₃ , etc.) ), and on the other hand, the signal in the high frequency band is sampled at the sampling period J, the maximum value of the waveform is detected near time t ₁ , the time u ₁ is obtained, and T ₁ is obtained based on u ₁ .
Cut out samples corresponding to , and obtain the signal W ₁ arranged at a sampling period of kJ. Next, detect the maximum value of the waveform near time t ₂ to find the time u ₂ , and use u ₂ as a reference.
A sample corresponding to T ₂ is cut out to obtain a signal W ₂ arranged at a sampling period kJ, W ₂ is connected next to W ₁ , and the same process is performed for time t ₃ to obtain W ₃ , W ₂ Repeat the process of connecting next, W ₁ , W ₂ , W ₃ ,...
An audio signal conversion method is characterized in that when k>1, the frequency spectrum of audio is compressed without distortion by obtaining a continuous waveform signal (B) and a waveform signal that is the sum of A and B. 3 Divide the audio waveform signal into two frequency bands, high and low, sample the signal in the low frequency band at sampling period I, find the short-time autocorrelation function S ₁ at time t ₁ , and calculate the period from S ₁ . Determine T ₁ , cut out a sample corresponding to T ₁ , and cut out the sampling period kI (k=
2, 3, 4 _... ) is obtained, then the short-time autocorrelation function S ₂ is obtained for time t ₂ (t ₂ = t ₁ + kT ₁ ), and the period T ₂ of S ₂ is determined. A signal obtained by cutting out samples corresponding to T ₂ and arranging them at a sampling period kI.
Obtain R ₂ , connect R ₂ next to R ₁ , and at the next time t ₃ (t ₃ =
Perform the same process for t ₂ + kT ₂ ) to obtain R ₃ ,
Repeat the process of connecting R ₂ next, R ₁ , R ₂ , R ₃ ,
A continuous waveform signal (A) is obtained, and on the other hand, the signal in the high frequency band is sampled at the sampling period J,
Using time t ₁ as a reference, samples corresponding to T ₁ are cut out to obtain a signal W ₁ arranged at a sampling period kJ, and then samples corresponding to T 2 are cut out using time t ₂ as a reference, and a sampling period kJ is obtained. We get the signal W ₂ arranged by , W ₂ is
Connect after W ₁ , perform the same process for time t ₃ to obtain W ₃ , repeat the process of connecting after W ₂ , and create a continuous waveform of W ₁ , W ₂ , W ₃ , etc. An audio signal conversion method characterized by compressing the audio frequency spectrum to 1/k without distortion by obtaining a signal (B) and obtaining a waveform signal that is a combination of A and B. 4 Divide the audio waveform signal into two frequency bands, high and low, sample the signal in the low frequency band at sampling period I, find the short-time autocorrelation function S ₁ at time t ₁ , and calculate the period from S ₁ . Determine T ₁ , cut out a sample corresponding to T ₁ , and cut out the sampling period kI (k=
1/m, m = 2, 3, 4, ...), this is repeated m times to obtain a signal mR ₁ , and then at time t ₂ (t ₂
= t ₁ + T ₁ ), find the short-time autocorrelation function S ₂ , cut out samples corresponding to the period T ₂ of S ₂ and arrange them at the sampling period kI, and repeat this process m times.
Get mR ₂ , connect mR ₂ next to mR ₁ , next time t ₃
Perform the same process for (t ₃ = t ₂ + T ₂ ) to obtain mR ₃ , repeat the process of connecting to the next mR ₂ , and mR ₁ ,
A continuous waveform signal (A) of mR ₂ , mR ₃ , ... is obtained, and on the other hand, the signal in the high frequency band is sampled at a sampling period J, and a sample corresponding to T _{1 is cut out with time t 1 as} the reference. This is repeated m times to obtain a signal mW _1. Next, samples corresponding to T ₂ are cut out based on time t ₂ and arranged with a sampling period kJ, and this is m signal repeated times
Obtain mW ₂ , connect mW ₂ next to mW ₁ , perform the same process for time t ₃ to obtain mW ₃ , repeat the process of connecting next to mW _{2, mW 1 ,} mW ₂ , mW ₃ ...
An audio signal conversion method characterized by obtaining a continuous waveform signal (B) and obtaining a signal that is the sum of A and B, thereby expanding the audio frequency spectrum by m times without distortion.