JP3267659B2 - Japanese speech synthesis method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of synthesizing a Japanese phrase or a sentence in units of mora, which is a unit of prosody corresponding to a single syllable in Japanese speech.
In particular, an object of the present invention is to adjust the time length between mora and to facilitate natural synthesis of Japanese speech.

[0002]

2. Description of the Related Art Conventionally, as a technique in such a field, CO2 is used.
Prototyping of text synthesis board using C (Context Oriented Clustering) method (IEICE technical report: 1)
(November 30, 990). The COC method described in this document is a method in which a phoneme including various variations generated by a statistical method from a database of natural speech is used as a synthesis unit.

[0003]

However, according to the conventional Japanese speech synthesis method, the following problems occur. FIG. 12 is a diagram for explaining conventional control of the phoneme duration. As shown in this figure, the conventional method of controlling the length of syllables is a control method that gives the length of time for each phoneme.When words spoken by humans are analyzed, the time of each phoneme actually changes. It can be seen that it is rich. When performing syllable duration control using this method, the factors that must be considered include the type of the sound itself, the type of the previous sound, the type of the next sound, and the type of the previous sound. Kind, 2
The type of the next sound, the position in a word, the position in a sentence, the presence or absence of an accent, and the like can be cited (Umeki et al .: “Setting of phoneme duration in sentence speech” IEICE Technical Report, SP90-2 (Heisei 02 -04)). As described above, the length of time that appears differently depending on the type of syllables and how they are connected is called the rhythm of words. When synthesizing natural speech, it is necessary to give each syllable an accurate time length. In other words, since the rhythm differs depending on the syllables connected before and after a syllable, it is necessary to accumulate various syllable data and cut out a necessary portion from the data for use. However, the combination of these factors alone is an astronomical number and cannot be considered in all cases. We also know that these factors alone are not enough. For this reason, there has been a problem that a huge amount of data has to be accumulated, and it is difficult to improve the quality of Japanese speech synthesized sound because complicated processing and case classification are required.

Accordingly, an object of the present invention is to provide a Japanese speech synthesis method capable of easily synthesizing natural speech in view of the above problems.

[0005]

In order to solve the above-mentioned problems, the present invention provides a method for synthesizing Japanese speech using a mora, which is a unit of prosody corresponding to a single syllable, as a synthesis unit. The energy of the vowel part of the two mora is determined, the time integral of the energy of the vowel part of the two mora is obtained, and the time length between the vowel part energy centroid positions between the centroids of the time integral of the energy of the two mora And the mora interval is determined using the consonant between the two mora as a parameter. The phoneme duration of a sentence to be synthesized is adjusted at the mora interval.

[0006]

According to the Japanese speech synthesis method of the present invention, the energy of the speech waveform of any two adjacent mora is determined,
The time integral of the energy of the vowel part of the voice waveform is calculated, and the mora interval is determined from the time length between the vowel part energy centroid points between the centroid points of the time integral of the energy of the two moras. Does not substantially change the time length between the vowel part energy centroid positions.
Even if the preceding consonant changes, the time length between the vowel part energy centroid positions hardly changes. However, the length of time between the vowel energy barycenter positions changes due to the change in consonants between them. A consonant between the two moras is used as a parameter to determine a mora interval. By adjusting the phoneme duration of a sentence to be synthesized at the above-mentioned mora interval, a natural Japanese synthesized sound can be easily obtained.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of a Japanese speech synthesis system according to an embodiment of the present invention. The Japanese speech synthesis system shown in this figure analyzes the Japanese sentence to be synthesized when a synthesized sentence is input, and generates pronunciation information such as accent information, pause, and vowel de-voice necessary for speech synthesis processing. Analysis unit 1 for converting to a phoneme symbol string to which is added
A phoneme duration generator 2 for controlling the phoneme duration of the phoneme symbol string generated by the text analyzer 1 according to a rhythm rule described later; and a phoneme duration generator 2.
The sound amplitude pattern generator 3 determines the power pattern of the voice according to the power rule so as to observe the time length between the vowel part energy centroids given by the rhythm rule of the above, and the rhythm rule of the phoneme duration length generator 2. A pitch pattern generation unit 4 for determining pitch patterns based on the accent of each accent phrase from the prosody control rules and interpolating them to generate a continuous point pitch pattern so as to maintain the time length between the vowel part energy centroids; Sound source generation unit 5 for generating a sound source based on the pattern and the continuous point pitch pattern
And a phonological improvement rule, a phonological combination rule, determines a type of vowel such as a vowel or a consonant, and combines the spectra of each phonology to form a formant pattern. And a speech synthesizer 7 for creating a synthesized speech from the formant pattern.

Next, the rhythm rule of the phoneme duration generation unit 2 which is a feature of the present invention will be described in detail. The present inventor thought that in order to easily synthesize high-quality Japanese speech, it was essential to find a Japanese rhythm rule and apply it as a rhythm rule. This is because, despite the efforts of many researchers, rhythm rules have not been found. As explained below, the rhythm of the Japanese voice is given as the time length between the vowel part energy centroids, and furthermore, this time length is almost uniquely determined by the type of consonant existing between the vowels within a certain tempo range. It became clear that it was decided.

First, the explanation will be made based on the following hypothesis of Japanese rhythm. That is, even a very small child can correctly form a sense of rhythm unique to Japanese. Also, no matter what the mora series, a person can pronounce with a natural rhythm. From these phenomena, Japanese rhythm rules should be very simple.
About this, Kaneda Kazu (Japanese Phonological Research (196
7) Tokyodo Shuppan) points out the following rules.

[Rule] Japanese rhythm is taken in mora units and is basically governed by a single concept of isochronism. In addition, when considering the case of uttering slowly, it is understood that the rhythm is taken in the vowel part in the mora. Furthermore, it is believed that one is hearing the energy of sound. Therefore, the physical rhythm can be defined as follows.

[Definition] The rhythm of Japanese is given by the time length between vowel energy centroids in the mora. The Japanese rhythm is basically isochronous, but the biggest factor that disturbs it is thought to be the additional physical constraints imposed by the structure of the vocal organs. This constraint is clearly larger when consonants are uttered than when vowels are uttered, and the time length between the vowel energy centroids seems to be largely dependent on the difference in consonants.

FIG. 2 is a diagram showing time length control focusing on the time length between the vowel energy center of gravity. Explaining in detail using this figure, Japanese syllables based on the audio waveform in the figure are:
Generally, it consists of a consonant part and a vowel part. As described above, since the rhythm is considered to be taken in the vowel part in the mora, attention is paid particularly to the energy of the vowel part in the syllable. That is, the energy of the vowel part is determined, and the point at which the center of gravity of the part is given is defined as the vowel part energy centroid point. Since the vowel energy center of gravity is determined one by one for each syllable, the time length of the syllable can be determined by giving the time length between the vowel energy center of gravity as t1, t2, t3 and t4 as shown in the figure.

In the time length control method focusing on the vowel part energy centroid, it is not necessary to consider the difference in the type of vowel because the energy of the waveform is considered. Consider a consonant part existing between the vowel part energy centroids. Consonants have less energy than vowels, but they have great physical limitations such as ease of pronunciation and difficulty. I mentioned earlier that the Japanese rhythm rule is based on equidistantness in energy units. The time length between the vowel energy centroids should originally be at regular intervals, but the equidistantness is greatly disturbed by the ease and difficulty of vocalization of consonants existing between them. It can be thought that it appears as various time lengths.

Therefore, the following hypothesis is provided and verified through analysis of real speech. [Hypothesis] Time length between vowel energy center of gravity (rhythm)
Is determined only by the type of consonant between target vowels below a certain tempo, and does not depend on the type of vowels before and after. Next, audio materials and analysis methods will be described. As audio material, 7
One sentence containing the meaningless word part of Mora, "It's" This is a OO Menkai "" was used. There are two speakers, one male and one female.

If Ci is the i-th consonant and Vi is the i-th vowel, the circles in the meaningless word are C2 V2,
It can be represented as C3 V3. [Analysis 1] If the previous vowel V2 is different, [Analysis 2] if the rear vowel V3 is different, and [Analysis 3] the previous consonant C2 The analysis of the time length between the vowel part energy centroids was performed for four cases, that is, when the consonants C3 sandwiched between [Analysis 4] were different. Ten utterances are made for each phoneme type, and the average time length and standard deviation are obtained. At this time, the duration of the meaningless word portion was normalized by linear expansion and contraction to 7 mora / sec in order to absorb a subtle difference in tempo generated for each utterance. For a tempo of 6 to 8 mora / sec, it has been separately confirmed that the time length between the vowel part energy centroids changes linearly.

Since the same tendency is observed in the analysis results of men and women, only the case of one person (female voice) will be described below. FIG. 3 is a diagram showing a case where the previous vowel is different,
FIG. 4 is a diagram showing the effect of the time length between the vowel energy center points in the case of FIG. As shown in FIG. 3, consonants 1, vowels 1, consonants 2, and vowels 2 are arranged in this order, and consonants 1, 2, and vowels 2
Is fixed, and the vowel 1 is variously changed, and the time length between the vowel part energy centroids is measured. As shown in FIG. 4, ◆ is the average time length between the vowel energy centroids, the line segment extending up and down is the standard deviation, and the horizontal axis is the phoneme of the second and third mora, for example, “ baba, biba, buba, beba,
boba, Nba ". FIG. 4 shows the time length between the vowel part energy centroids when the previous vowel V2 is different, and the average time length is 139 to 145 ms.
It is distributed over a width of about ± 3.5 ms. From the fact that the discrimination limit of the phoneme duration is about 10 to 20 ms (see: Hashimoto et al., 7th International Conference on Acoustics, 129-132 (197)
1)), it can be said that there is almost no influence by the difference of the previous vowel V2.

FIG. 5 is a diagram showing the case where the rear vowel is different, and FIG. 6 is a diagram showing the effect of the time length between the gravities of the vowel part energy in the case of FIG. Similarly to the above, as shown in FIG. 5, consonants 1, 2, and vowel 1 are fixed and vowel 2, for example, as shown in FIG. 6, "baba, babi, babu, babe, babo, baN
(de) ”, and the time length between the vowel part energy centroids is measured with various changes. As shown in FIG. 6, the rear vowel V3
Even if the vowel part is different, the distribution of the average time length between the vowel part energy centroids is 140 to 154 ms, ± about 7 ms, and it can be seen that there is almost no influence by the difference of the rear vowel V3 as in the case of the preceding vowel.

FIG. 7 is a diagram showing the case where the preceding consonant is different, and FIG. 8 is a diagram showing the effect of the time length between the gravities of the vowel part energy in FIG. Similarly to the above, vowels 1 and 2 and consonants 2 are fixed as shown in FIG. 7 and consonants 1 are variously changed as shown in FIG. 8, for example, "byaba, baba, gyaba,..., Naba". The time length between the local energy centroids is measured. As shown in FIG. 8, when the consonant C2 is different, the average time between the vowel energy centroids is 121 to 153 ms.
It is distributed over a width of about ± 16 ms.

FIG. 9 is a diagram showing the case where the consonants are different, and FIG. 10 is a diagram showing the influence of the time length between the vowel energy center of gravity. The vowel 1 as shown in FIG.
2. Consonant 1 is fixed and consonant 2 is set to, for example, "bara, bada,
bawa,..., bahya ”, and the time length between the vowel part energy centroids is measured with various changes. As shown in FIG.
If the consonant 2 is different between the vowels to be analyzed, the average time length distribution width between the vowel energy centroids is
113 to 195 ms, ± about 41 ms, clearly exceeding the discrimination limit.

FIG. 11 is a diagram for explaining a method of deriving the time length between the vowel energy centroids. In the i-th mora and (i + 1) -th mora sound waveforms each having a consonant part and a vowel part as shown in FIG. 6A, a sound energy E representing the sound waveform as shown in FIG. For the vowel part, the sum (integration) of voice energies as shown in this figure (c) is taken, and for example, the sum of all energies of the i-th and (i + 1) -th mora is calculated as Etota
l (i), Etotal (i + 1), and 1 / 2Etotal (i), 1 / 2Eto of the i-th and (i + 1) -th mora
The time when tal (i + 1) is reached is defined as the vowel energy centroid of the i-th and (i + 1) -th mora, and the interval between them is defined as the time length between the vowel energy centroids.

The following table shows the time length between the vowel energy centroids for various consonants derived by the above method. From the above results, the average time length is concentrated around 140 ms in both the effect of the difference in the type of the preceding vowel (FIG. 4) and the effect of the difference in the type of the rear vowel (FIG. 6). Although there is a slight variation among vowel types, humans cannot recognize changes in the sound length of about 10 to 20 ms (Hashimoto et al .: "Prosody rules for speech synthesis", 7th International Conference on Acoustic Sound)
Considering this, it can be concluded that the effect of the type of vowel is negligible. On the other hand, the influence (FIG. 10) due to the difference between consonants was widely distributed at about 110 to 190 ms, and the magnitude of the influence due to the difference in type could be confirmed.

As shown in FIG. 10, the time length between the vowel part energy centroids and the various consonants are substantially straight lines, and this relationship is stored as the rhythm rule of the phoneme duration generation unit 2 in FIG.

[0023]

As described above, according to the present invention, the mora interval is determined using the consonant between the two mora as a parameter, and the phoneme duration of the text to be synthesized is adjusted at the mora interval. Natural Japanese synthesized sounds can be easily obtained.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a Japanese speech synthesis system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating time length control focusing on the time length between vowel energy center points.

FIG. 3 is a diagram showing a case where a previous vowel is different.

FIG. 4 is a diagram showing the influence of the time length between vowel energy center points in the case of FIG. 3;

FIG. 5 is a diagram showing a case where a back vowel is different.

6 is a diagram showing the effect of the time length between vowel energy center points in the case of FIG. 5;

FIG. 7 is a diagram showing a case where a previous consonant is different.

FIG. 8 is a diagram showing the effect of the time length between vowel energy center points in FIG. 7;

FIG. 9 is a diagram showing a case in which consonants are different.

FIG. 10 is a diagram showing the influence of the time length between vowel energy center points.

FIG. 11 is a diagram illustrating a method for deriving a time length between vowel energy center points.

FIG. 12 is a diagram for explaining conventional control of phoneme duration.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Text analysis part 2 ... Phoneme duration length generation part 3 ... Sound source amplitude pattern generation part 4 ... Pitch pattern generation part 5 ... Sound source generation part 6 ... Spectrum pattern generation part 7 ... Speech synthesizer

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-56-11118 (JP, A) JP-A-61-84697 (JP, A) JP-A-61-84699 (JP, A) JP-A-62 284397 (JP, A) JP-A-3-2799 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G10L 13/00-13/08

Claims (2)

(57) [Claims] 1. A speech synthesis method for synthesizing Japanese using a mora, which is a unit of prosody corresponding to a single syllable, as a synthesis unit, wherein energy of a speech waveform of two adjacent mora is obtained, and energy of the speech waveform is calculated. Take the time integral of the energy of the vowel part, find the mora interval by the time length between the vowel part energy centroid positions between the centroids of the time integral of the energy of the two mora, and find the consonant between the two mora A Japanese speech synthesis method characterized by determining a mora interval as a parameter. 2. The Japanese speech synthesis method according to claim 1, wherein a phoneme duration of a sentence to be synthesized is adjusted at the mora interval.