JP3866171B2 - Phoneme determination method, apparatus and program thereof - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to, for example, a phoneme determination method, an apparatus, and a program for automatically obtaining a phoneme or a phoneme boundary in a speech signal, which is used for obtaining a speech unit when speech synthesis is performed by connecting speech units.
[0002]
[Prior art]
Conventionally, the labeling (labeling) indicating the phoneme or the phoneme boundary for extracting the speech segment has been determined manually by looking at the sonogram of the speech signal. Although manual labeling can be done fairly accurately, it is laborious. In particular, there are a large number of speech segments required for synthesizing speech with various human voice qualities and various utterance styles, and it is extremely difficult to obtain this by manual labeling.
[0003]
Conventionally, it has been proposed to automatically perform labeling. One of them is phoneme labeling by speech recognition using a phoneme model based on mel cepstrum. The literature is Brugnara, F., Falavigna, D., and Omologo, M., “Automatic segmentation and labeling of speech based on Hidden Markov Models, "Speech Communication, No. 12, pp. 357-370, 1993. Also, phoneme labeling using spectrogram reading knowledge has been published in the literature by Koichiro Hatasaki, Shinichi Tamura, Go Kawabata, Kiyohiro Shikano, “Trial of phoneme segmentation using spectrogram reading knowledge”, Proc. 21-22, Mar., 1991.
[0004]
However, since conventional automatic labeling only specifies phonemes by speech recognition, the accuracy of specifying each phoneme segment was relatively poor. For this reason, the synthesized speech obtained by connecting the speech units obtained based on the automatic labeling has poor quality.
[0005]
[Problems to be solved by the invention]
An object of the present invention is to provide a phoneme determination method, an apparatus thereof, and a program capable of accurately distinguishing phonemes and accurately specifying phoneme sections, that is, phoneme boundaries.
[0006]
[Means for Solving the Problems]
According to the method of the present invention, the input audio signal is subjected to mel frequency division into a plurality of bands for each frame, the power of the input audio signal for each of the divided bands is calculated as a mel filter bank coefficient, and the input audio signal is calculated for each frame. Similar to the phoneme or phonological boundary of the acoustic feature vector of the input speech signal to the acoustic model that calculates the energy of the signal and is associated with the phonological or phonological boundary for each acoustic feature vector including the mel filter bank coefficient and energy The degree is obtained, and a label indicating the phoneme or the phoneme boundary having the highest degree of similarity is given for each frame.
[0007]
According to the apparatus of the present invention, a band dividing unit that divides an input audio signal into a plurality of bands for each frame, and a bank coefficient that calculates the power of the divided input audio signal in each band as a mel filter bank coefficient A calculation unit; an energy calculation unit for obtaining energy of the input speech signal for each frame; a model storage unit storing an acoustic model of each phoneme or phoneme boundary; an acoustic feature sequence including the Mel filter bank coefficient and the energy And a similarity calculation unit for obtaining a similarity between the acoustic model and a label providing unit for providing a label indicating a phoneme or a phoneme boundary corresponding to the acoustic model having the highest similarity.
[0008]
According to this configuration, the acoustic feature vector including the mel filter bank coefficient and energy is analyzed, and the label indicating the phoneme or the phoneme boundary is specified based on the analysis, and the characteristic power fluctuation for each band at the phoneme boundary is specified. Although it occurs, since the acoustic model reflecting the property is used, it becomes easy to specify the phoneme boundary accurately.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
In the present invention, an acoustic model is used for labeling. The creation of the acoustic model will be briefly described first. For a speech signal that a speaker has read a meaningless sentence, the frame length is set to, for example, 20 milliseconds, and the input speech signal is captured and divided into Mel frequencies while the frame is shifted by, for example, 2 milliseconds. The number of divisions is, for example, 10 or more, preferably 10. In the mel frequency division, a frequency scale is converted into a mel frequency scale by the following equation, and the speech signal is equally divided with respect to the mel frequency scale.
Mel (f) = 2595log ₁₀ (1+ (f / 700))
[0010]
The logarithmic power of the audio signal for each band is obtained as a mel filter bank coefficient, and the energy of the audio signal for each frame is obtained to obtain an acoustic feature quantity vector including the mel filter bank coefficient and energy. For example, a hidden Markov model (HMM) is used as an acoustic model, and the HMM has an appearance probability associated with a phoneme or a phoneme boundary for each acoustic feature vector sequence between one or more finite frames for each state. Using a speech waveform or a corresponding acoustic feature vector sequence whose correspondence with a phoneme or a phonetic boundary is known in advance, The frequency of appearance at each phoneme boundary is counted, and for example, an HMM is created by initializing with a segmental k-means method, which is a known method, and learning with a Baum-Welch method.
The phoneme boundary acoustic model is created based on a series of acoustic feature vectors of several frames before and after one phoneme and its adjacent phoneme boundary.
[0011]
Next, an embodiment of the method of the present invention will be described with reference to FIG.
Step S1: The input audio signal is subjected to Mel frequency division into a plurality of bands for each frame. The frame length is set to 20 milliseconds, for example, and the input audio signal is captured while the frame is shifted by 2 milliseconds, for example, and the Mel frequency is divided. The number of divisions is, for example, 10 or more, preferably 10.
[0012]
Step S2: The power of the divided audio signal in each band is calculated as a Mel filter bank coefficient, and the energy of the audio signal in that frame, that is, the power is obtained. An acoustic feature vector including mel filter bank coefficients and energy is obtained. The inter-frame difference and Δ component of each power (Mel filter bank coefficient) and the inter-frame difference ΔΔ component (secondary difference component) are also obtained. Similarly, the inter-frame difference Δ component of energy and the inter-frame difference ΔΔ component are also calculated. It is preferable to obtain both the Δ component and the ΔΔ component as an acoustic feature vector. Only the Δ component of power, or only the Δ component of energy, or only the Δ component and ΔΔ component of power, or only the Δ component and ΔΔ component of energy may be included. Further, it is easier to process the power (energy) obtained as a logarithmic power.
[0013]
Step S3: The similarity between the obtained acoustic feature vector sequence and the acoustic model is calculated. The input speech signal is a sentence (text) read out, and therefore the input speech signal knows in advance which phoneme or phoneme boundary sequence is made, so it is similar to each acoustic model of the corresponding phoneme or phoneme boundary. Find the degree. When the acoustic model is, for example, an HMM, the appearance probability and transition probability in the HMM are obtained using the acoustic feature vector obtained for each frame, for example, the branch metric value is obtained by the Viterbi algorithm, and the path metric value is further obtained. The likelihood for the entire input speech signal is calculated. In this case, since the target HMM is known in advance, the calculation amount is small, and the likelihood is obtained as the similarity.
[0014]
Step S4: In the case of similarity or HMM, a label indicating the phoneme or phoneme boundary of each acoustic model in the sequence of acoustic models having the highest likelihood is attached to the corresponding portion of the input speech signal. Normally, the acoustic model is created using a normalized audio signal, and the input audio signal is also normalized, and then the processes in step S1 and subsequent steps are performed.
FIG. 2 shows an embodiment of the apparatus of the present invention. An audio signal input from the input terminal 10 (normally, for example, a digital signal sampled at a sampling frequency of 22 kHz) is normalized by the normalization unit 11. That is, for example, the maximum value of samples in the input audio signal is set to the maximum value that can be taken by the bit width of the data to be processed, and this conversion coefficient is multiplied by all other sample values, or in the input audio signal This is done by dividing all sample values by the maximum sample value.
[0015]
The audio signal thus normalized is divided into frames by the frame dividing unit 12. In the above example, an audio signal of 20 milliseconds is captured every 2 milliseconds.
For each frame, the audio signal is frequency-melled linearly divided into a plurality of, for example, 10 bands by the band dividing unit 13. This division is performed by, for example, the above formula.
In the bank coefficient calculation unit 14, the divided audio signal power of each band is calculated as a mel filter bank coefficient, and the mel filter bank coefficient for each band is stored in the storage unit 15. Further, the energy of the audio signal is calculated by the energy calculation unit 16 for each frame, for example, by taking the square sum of the sample values of the frame, and stored in the storage unit 15. If the mel filter bank coefficient and energy are obtained as logarithmic values, the processing becomes easy. In this example, the inter-frame difference Δ component of the mel filter bank coefficient is calculated by the primary difference calculation unit 17, and the inter-frame difference ΔΔ component of the Δ component is calculated by the secondary difference calculation unit 18, and these Δ component and ΔΔ component are calculated. Is stored in the storage unit 15. Similarly, in this example, the inter-frame difference Δ component of the energy is calculated by the primary difference calculation unit 21, and the inter-frame difference ΔΔ component of the Δ component of the energy is calculated by the secondary difference calculation unit 22. The ΔΔ component is also stored in the storage unit 15. Any storage in the storage unit 15 is performed in association with the number of each frame.
[0016]
Thereafter, the similarity calculation unit 23 obtains a sequence of acoustic feature vectors (in this example, mel filter bank coefficients, energy, and Δ components and ΔΔ components thereof) extracted from the storage unit 15 in the order of frame numbers, and a model storage unit 24. The similarity with the sequence of acoustic models taken out is calculated. Since the input speech signal is a read-out of a known sentence (text), the text is input to the phoneme information conversion unit 25 and converted into a phoneme or phoneme boundary information sequence, and each phoneme or phoneme boundary thus converted is converted. The corresponding acoustic model is read from the model storage unit 24 based on the information, and an acoustic model sequence for the input audio signal is supplied to the similarity calculation unit 23, and the correspondence between each acoustic model and the acoustic feature vector sequence The similarity is maximized. Each acoustic model of the acoustic model sequence may be associated with the acoustic feature vector sequence in that order. When the acoustic model is an HMM, this association is performed so that the likelihood is maximized by, for example, the Viterbi algorithm.
[0017]
When the acoustic model or acoustic model series having the maximum similarity is obtained in this way, the label attaching unit 26 assigns a label representing the phoneme or phoneme boundary of the acoustic model to the speech signal. That is, label information (phoneme or phoneme boundary information) indicating a label indicating a phoneme or phoneme boundary is assigned to each frame of the speech signal. The control unit 27 causes each unit to sequentially perform the processing of each unit described above.
Next, experimental examples showing that the present invention is superior will be described.
As a conventional technique, a mel frequency cepstrum coefficient (MFCC, which is calculated by the following equation) is an acoustic parameter generally used in speech recognition.
c _i = √ (2 / N) Σj _{= 1} ^N m _j cos ((πi / N) (j−0.5))
And its Δ component and ΔΔ component were used for automatic labeling. As an example of the present invention, automatic labeling is performed with a phoneme model of HMM having five states and four mixed distributions using Mel filter bank coefficient (FBANK) and energy E (Energy component), and their Δ component and ΔΔ component. It was. The parameter configuration used in this automatic phoneme labeling experiment is shown in FIG. For example, the 0th to 10th orders of the MFCC, the Δ component, and the ΔΔ component are referred to as the MFCC 10 for convenience, and similarly, the 1 to 30 channels of the FBANK, the E, each Δ component, and the ΔΔ component are referred to as the FBANK 30. The test data used for the experiment is shown in FIGS. FIG. 6 shows experimental results regarding the same utterance style and different utterance styles of the same speaker, and FIG. 7 shows experimental results regarding the same utterance style and different utterance styles of different speakers.
[0018]
In the figure, the bar graph indicates the average error (ms), and the line graph indicates the rate at which the error falls within the reference value. The smaller the value of the bar graph and the larger the value of the line graph, the better the performance. The shaded bar graph is when MFCC is used, the white bar graph is when FBANK is used, the solid line of the line graph is when MFCC is used, and the broken line of the line graph is when FBANK is used. A plot of Δ is a ratio (%) where the error is within 20 ms, and a plot of x is a ratio (%) where the error is within 10 ms. The error is a difference in boundary position between the result of automatic labeling and the manual label data. An example of error calculation is shown in FIG. When both the bar graph and the line graph show high values, the error variance is large. From the bar graphs of FIGS. 6 and 7, it can be seen that the FBANK parameter has less error due to automatic labeling. Similarly, it can be seen from the line graph that the FBANK parameter has a higher ratio of being within 20 ms / 10 ms. From these, it can be said that the FBANK parameter has higher labeling accuracy regardless of the speaker and the speech style.
[0019]
From the bar graph of FIG. 7, the difference in performance between MFCC and FBANK becomes extremely large when the order is 15th or higher. On the other hand, in the case of FIG. 6, the difference in performance between MFCC and FBANK does not change much even if the order is 15th or higher. As the order increases, a precise model in which the MFCC is suitable for the speaker X is constructed, and it is considered that the labeling accuracy deteriorates if the speakers are different. In other words, FBANK is less likely to be affected by the speaker. Further, it can be seen from the bar graph of FIG. 7 that the automatic labeling accuracy is saturated in FBANK10. Therefore, in FBANK, only about 10 bands are required.
[0020]
As the acoustic model, a standard pattern used for voice recognition by DP patching may be used. When this standard pattern is used as an acoustic model, the similarity between the acoustic feature quantity and the acoustic model (standard pattern) is calculated by calculating the distance between them, and the smaller the distance, the higher the similarity. Although the accuracy is somewhat reduced in the above, the zeroth order component of the mel frequency cepstrum coefficient may be used instead of energy.
The apparatus shown in FIG. 2 may be caused to function by causing a computer to execute a program. In this case, the phoneme determination program may be downloaded into a computer from a recording medium such as a CD-ROM or a magnetic disk or via a communication line and executed.
[0021]
【The invention's effect】
As described above, according to the present invention, automatic labeling can be performed with higher accuracy than before.
[Brief description of the drawings]
FIG. 1 is a flow chart illustrating an embodiment of the method of the present invention.
FIG. 2 is a diagram showing a functional configuration of an embodiment of the apparatus of the present invention.
FIG. 3 is a diagram showing a configuration of parameters used in an experiment.
FIG. 4 is a diagram showing test data for the same speaker labeling used in the experiment.
FIG. 5 is a diagram showing test data for other labeling used in the experiment.
FIG. 6 is a diagram showing an automatic voice labeling accuracy evaluation result of the same speaker.
FIG. 7 is a diagram showing a result of another person's voice automatic labeling accuracy evaluation.
FIG. 8 is a diagram illustrating an example of boundary error calculation for automatic labeling.

Claims (3)

A phoneme determination method for extracting a speech unit for speech synthesis,
The process of dividing the input audio signal into mel frequencies into multiple bands for each frame;
A process of calculating the power of the input audio signal for each of the divided bands as a Mel filter bank coefficient;
Calculating the energy of the input audio signal for each frame;
For each acoustic feature vector containing the mel filter bank coefficients and energy for the acoustic model associated with the sound rhyme boundary, a process of obtaining the similarity of the sound rhyme boundary acoustic feature vectors of the input speech signal, wherein The acoustic model is a text corresponding to the phoneme boundary converted from the text of the input speech signal,
Phoneme determination method having the steps of similarity calculated for each frame to impart a label indicating the highest ion rhyme boundaries. A phoneme determination device for extracting a speech unit for speech synthesis,
A band dividing unit that divides the input audio signal into a plurality of bands for each frame;
A bank coefficient calculator that calculates the power of the input audio signal in each of the divided bands as a Mel filter bank coefficient;
An energy calculator for obtaining the energy of the input audio signal for each frame;
A model storage unit in which the text of the input speech signal is converted into a series of phoneme boundaries, and an acoustic model corresponding to the converted phoneme boundaries is stored;
A similarity calculation unit for calculating a similarity between the acoustic feature sequence including the Mel filter bank coefficient and the energy and the acoustic model;
Phoneme decision apparatus comprising a label applying unit that applies a label indicating the sound rhyme boundaries the calculated degree of similarity corresponding to the highest acoustic model.   A phoneme determination program for causing a computer to execute each step of the phoneme determination method according to claim 1.

Images