JP3420964B2 - Pitch pattern generation method, device thereof, and program recording medium - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pitch pattern generating method, its apparatus and program recording medium in a text-to-speech synthesis system.

[0002]

2. Description of the Related Art A conventional pitch pattern generation method in text-to-speech synthesis analyzes the pitch pattern of natural speech, constructs a general pitch pattern generation model from the rules obtained from it, and uses the model to generate a text pattern. The pitch pattern of was generated. However, even if a large amount of data is analyzed, it is very difficult to construct a universal model including all the rules, and thus an unnatural pitch pattern may be generated depending on the content of the synthesized text. In addition, there is a problem in that the generated pitch pattern is mechanical and monotonous because general regularization is performed.

In order to solve such a problem, there is an approach using a pitch pattern database as a method of generating a pitch pattern of text-synthesized speech. this is,
A large amount of natural voice pitch patterns are stored in the pitch pattern database, and a pitch pattern suitable for the text to be synthesized is selected from the database to generate pitch patterns. In this method, since the pitch information of the real voice is used, if the pitch pattern data suitable for the text to be synthesized can be obtained from the database, it is possible to generate a highly natural pitch pattern, but select an appropriate pitch pattern data. Is difficult. If there is no pitch pattern data in the database that has the same prosodic category conditions as the input text to be synthesized (text conditions determined by the text attributes such as accent type and number of mora), the appropriate pitch pattern data There was a problem that selection could not be made.

[0004]

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and when pitch pattern generation in a text-to-speech synthesis system is performed, appropriate pitch pattern data is selected from a pitch pattern database. An object of the present invention is to provide a pitch pattern generation method that enables highly natural pitch pattern generation.

[0005]

In order to solve the above-mentioned problems, the method of the present invention uses a pitch pattern data when selecting pitch pattern data from a pitch pattern database consisting of a large amount of pitch pattern data extracted from natural speech. Match prosodic categories to the pattern database (accent type, number of mora, leading accent type, trailing accent type,
If there is no pitch pattern data (including a set of preceding boundary conditions and subsequent boundary conditions), the distance (similarity) between the prosody categories is obtained by statistical estimation, and from this distance, the pitch pattern database is obtained. Select the prosody category that is estimated to be the closest to the prosody category of the text you want to synthesize, select the pitch pattern data to be used for the text to be synthesized from the pitch pattern data belonging to the selected prosody category, and select that pitch. The pattern data is used for pitch pattern generation. The distance between the prosody categories is expressed as a vector of the pitch pattern data belonging to each prosody category, the average vector of the prosody categories is obtained, and the difference in pitch pattern shape between different prosody categories is expressed as a difference vector. It is the distance between the prosody categories. Further, using the difference vector between prosodic categories obtained from the estimation result, the shape of the selected pitch pattern is transformed so as to be closer to the pitch pattern shape of the prosody category of the text to be synthesized, and used for pitch pattern generation. .

[0006]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below. In the present invention, pitch pattern generation is performed using accent phrases as basic units. In this embodiment, all the accent phrases to be handled are information that can be extracted from the text with stable quality, such as accent type (classification of which syllable is placed in the accent, flat accent, etc.), the number of mora, Depending on the prosody category represented by the set of preceding accent type (accent type of previous accent phrase), subsequent accent type (accent type of subsequent accent phrase), preceding boundary condition, subsequent boundary condition, etc. Shall belong. That is, in this method, the accent phrase input to the text-to-speech system belongs to one of the above prosodic categories based on the result of morphological analysis.

A pitch pattern generating method in this embodiment will be described with reference to FIG. The pitch pattern database 11 stores pitch pattern data in units of accent phrases. This pitch pattern data has a pitch value for each frame. In this embodiment, one frame is 2.5 msec. In addition, each pitch pattern data also includes the prosody category (condition).
The text to be composed is given in units of accent phrases.

In step 101, it is searched whether pitch pattern data belonging to a prosody category equal to the prosody category of the text to be synthesized exists in the pitch pattern database 11. Here, if the pitch pattern data exists, the processing of the following step 102 is performed, and if it does not exist, the processing of the following steps 103 to 105 is performed.

In step 102, pitch pattern data from a prosody category equal to the prosody category of the text to be synthesized is selected. Here, when the selected prosody category has a plurality of pitch pattern data, it is necessary to determine the pitch pattern data to be used. In this example,
The pitch pattern data of the most average shape in the category is selected as the representative pitch pattern data of the prosody category.

In step 103, step 3 which will be described later is performed.
By the method described in 01 and step 302, the prosody category in which the shape of the pitch pattern is considered to be the closest to the prosody category of the text to be synthesized is estimated from the prosody categories included in the database 11. Step 104
Then, similarly to step 102, pitch pattern data is selected from the estimated prosody category.

In step 105, the selected prosodic category is selected.
Select the difference vector between the prosodic categories of text and text
It is transformed by applying it to the specified pitch pattern data. Pitchpa
For the vector representation of the tan data, see the steps below.
In 202 and step 203, the difference vector table
The present will be described in step 204 described later.
It Of the pitch pattern data to which this difference vector is applied
For the transformation method, see, for example, Isogai et al.
Forming method and its recording medium "(Japanese Patent Application No. 10-24755)
The method described in 5) can be used. Database
The prosody category K selected from the
When it is Gori E, the transformation from prosody category K to E is displayed.
Let the difference vector be D (K, E). D (K,
E) is the pitch pattern data in step 103.
Use the value when the prosody category K is selected from the base 11
To do. X is the vector representation of the selected pitch pattern_inWhen
Then, the difference vector D (K, E) is added and transformed.
Pitch pattern vector X ' _inX ′_in= X_in
Obtained from + D (K, E). Vector x_inBut transformed
Vector X ′_inTo be equal to the pitch pattern of
Then, the shape of the selected pitch pattern is transformed. This shape
The modification is to increase or decrease the pitch value for each frame.
By.

In step 106, the pitch pattern data is linearly expanded / contracted in the time axis direction in mora units,
The time length is corrected according to the given time length. In step 107, with respect to the height of each pitch pattern, the midpoint between the heights of the start point and the end point of the pitch pattern is set to be the average value of the voice tone component heights obtained by the voice tone component determination algorithm of the point pitch pattern. decide. Regarding the speech tone component determination algorithm of the point pitch pattern, see, for example, Hakoda et al.
63-D No. 9) can be used.

By the above processing, the pitch pattern data suitable for the text to be synthesized can be obtained from the pitch pattern database 11. Next, it becomes necessary in step 103. A learning method for estimating the distance (similarity) between prosody categories will be described with reference to FIG. Here, the distance between the prosody categories is determined in order to select the prosody category in which the shape of the pitch pattern is considered to be the closest to the prosody category of the text to be synthesized. In order to define the distance, it is necessary to express the difference in the shape of the pitch pattern, and therefore the shape of the pitch pattern is expressed by a vector. Then, the absolute value of the difference vector representing the shape difference of the pitch pattern vector between different prosody categories is regarded as the distance between the prosody categories. That is, estimating the difference vector between the prosody categories is synonymous with estimating the distance between the prosody categories.

Steps 201 to 204 are processing for creating learning data for learning the distance between prosody categories. In step 201, the pitch pattern data included in the pitch pattern database 11 is categorized (classified) into a prosody category represented by a set of accent type, number of mora, preceding accent type, succeeding accent type, preceding boundary condition, succeeding boundary condition and the like. ) Do. The categorization result is stored in the storage device 21.

At step 202, the shape of the pitch pattern is represented by a vector. As a vector expression method of this pitch pattern, for example, Isogai et al. “Pitch pattern deformation method and recording medium thereof” (Japanese Patent Application No. 10-247555) can be used. That is, the pitch value at the center of the vowel and the pitch value at the beginning and end of the accent phrase are used as model parameters of the pitch pattern. Each element of the pitch pattern vector is the difference between the pitch of the mora and the value of the pitch of the mora preceding the mora. When the mora number of a certain accent phrase a is m, the difference amount between the pitch values of the i-th mora and the i + 1-th mora is xi, and the maximum number of mora is M among the mora numbers of all prosody categories, the pitch pattern of the accent phrase a is The vector expression x _a is _converted into x _a = (x ₀ , x ₁ , ..., X _m-2 ,
0, ..., 0, x _Mm , x _m-1 , x _m ).

In step 203, an average pitch pattern vector of the prosody category is obtained. Let X _K be the average pitch pattern vector of the prosody category K, then X
_K = (1 / n) Σ _{i = 0} ⁿ⁻¹ xi (xiεK) Here, n is the number of pitch pattern data belonging to the prosody category K. This is the pitch pattern vector of the prosody category. The obtained pitch pattern vector is stored in the storage device 22.

In step 204, a difference vector between prosody categories is obtained. The difference vector D (K, L) from the prosody category K to the prosody category L is D (K, L)
= Obtained from the X _L -X _K. However, (K = 0, ..., N
-1, L = 0, ..., N-1, K ≠ L). here,
X _K is the pitch pattern vector of prosody category K, X
_L is the pitch pattern vector of the prosody category L, and N is the number of all prosody categories having pitch patterns in the database. The calculated difference vector is stored in the storage device 23.

At step 205, based on the difference vector between the prosody categories obtained at step 204, the contribution of each prosody category condition to the difference vector is learned using quantification type I. Quantification type I is an external standard quantitatively measured based on qualitative explanatory factors (quantitative variable of purpose).
Is a method for explaining or predicting the value of, for example, Tanaka et al. “Multivariate statistical analysis method”, (Hyundai Mathematics Co. Ltd., 19
83). The quantification type I is formulated by the following equation (1).

[0019] y ^ i = y 'in _{+ Σ f Σ c a fc δ} fc (i) (1) above (1), y ^ i is an estimate of the i th sample, y' is the total training data Mean value of δ _fc (i)
Is a dummy variable that takes 1 when the sample i falls under the category c of the factor f, and 0 otherwise. a
_fc is a quantity (called a category quantity) given to the category c of the factor f, and is determined so that the error between the external reference yi and the estimated value y ^ i is minimized. That is, yi
And y ^ i mean squared error Σ _{i = 1} ⁿ (yi-y ^ i) ² (2) (n is the number of samples of yi) is determined to be the minimum. Σ _c is a _fc δ _fc (i)
With respect to c, Σ _f is Σ _c a _fc δ _fc (i)
Represents adding for f.

The feature quantity, ie, yi, which is a quantitative variable of the quantification type I, is the difference vector D obtained in step 204.
It is each element di of (K, L). The factors and categories that are qualitative explanation factors are the same as the prosody category conditions used in step 201. For example, the category of the factor'accent type 'is' 0 type 'or' 1 type ', and the category of the factor'leading boundary condition'is'strong coupling 'or'weak coupling'. Here, these factors and categories are the prosody category conditions of the prosody category L that is the end point of the difference vector. These factors and categories are used as numerical expressions.

From these, the difference vector D ^ (Kp,
Category quantity ak for estimating the value of element k of Eq)
_fc (Kp) and average value y'k (Kp) of learning data
Ask for. Here, Kp is a prosody category (p = 0, 0) of learning data having pitch pattern data in the database.
, N _K-1 : N _K is the number of all prosody categories in the database). Eq is a prosody category (q = 0, ..., N _E-1 : Pitch pattern data that does not have pitch pattern data in the database).
N _E is the number of prosodic categories not included in the database)
Is. Further, (k = 0, ..., M: M is the maximum number of mora in all prosody categories).

Learning is performed in an arbitrary prosody category Kp (0≤p≤
Focusing on N _K-1 , Kp to Kr (r is 0, ...,
The average value of the category quantity and the learning data is obtained by using N _K−1 difference values to N _K−1 , r ≠ p) as external criteria.
The categorical quantity is obtained by solving the normal equation obtained by partially differentiating equation (2) with a _fc and setting it to 0. The category quantity is standardized so that the average of the category quantities in each factor is zero. The category quantity and the average value of the learning data are calculated for each prosody category Kp. Further, the category quantity and the average value of the learning data are obtained for each element k of the difference vector D (Kp, Kr).

By the above procedure, the category quantity necessary for estimating the unknown prosody category and the average value of the learning data were obtained. The category quantity and the average value of the learning data are stored in the storage device 12. Next, the procedure for obtaining the prosody category to be used by estimation in step 103 will be described with reference to FIG.

Step 301 estimates a difference vector from all the prosody categories in the database to the prosody category E of the text to be synthesized which is not in the database 11. A difference vector D ^ (Kp, E) from the prosody category Kp to the prosody category E is estimated. Here, E is a prosody category of the text to be synthesized, Kp is a prosody category (p = 0, ..., N-1) in the database (N is the number of all prosody categories in the database).

Each element k of the difference vector can be estimated by the following equation (3). d ^ k (Kp, E) = the _{y'k (Kp) + Σ f Σ} c ak fc (Kp) δ fc (E) (3) the above equation (3), d ^ k ( Kp, E) is determined An estimated value of the element k of the difference vector D ^ (Kp, E),
y′k (Kp) is an average value of all learning data regarding the element k of the prosody category Kp, and δ _fc (E) is a dummy that takes 1 when the prosody category E corresponds to the category c of the factor f, and 0 otherwise. The variable, ak _fc (Kp), is the category quantity assigned to the factor f category c of the element k of the prosody category Kp. ak _fc (Kp) and y'k (Kp) are
Obtained from the storage device 12. That is, the difference vector D ^ (Kp, E) can be obtained from the category quantity related to the prosody category Kp, the average value of the learning data, and the prosody category condition of the prosody category E that is the end point of the difference vector.

In step 302, the prosody category that minimizes the magnitude of the difference vector is obtained from all the difference vectors obtained in step 301. That is, Kp estimated to be closest to E is obtained. By the above processing,
The prosody category estimated to be the closest to the pitch pattern shape of the prosody category of the text to be synthesized can be selected from the database.

[0027]

As described above, according to the present invention, in the text-to-speech synthesis system using the pitch pattern database, the prosody category conditions of all texts that may be synthesized are included in the pitch pattern database. Even without preparing the corresponding pitch pattern data,
It is possible to obtain a natural pitch pattern by automatically estimating the optimum alternative pitch pattern from the database and deforming the selected pitch pattern.

Examples will be shown below. A pitch pattern database was constructed from 503 sentences of the ATR continuous speech database, but the sentences for evaluation were excluded from the database. For the five sentences, the following five types of synthetic speech were created. (A) The present invention (1). The pitch pattern is generated by the method of the present invention under the condition that all the prosody categories included in the evaluation text are in the database. (B) The present invention (2). All the prosody categories included in the sentences for evaluation are removed from the database, and pitch patterns are generated by the method of the present invention. (C) The present invention (3). Similar to (b), but the shape transformation process after selecting the pitch pattern is not performed. (D) Conventional method. A pitch pattern is generated by the point pitch model (reference of Hakoda et al.). (E) Original voice. Uses the pitch pattern of the original voice.

The values of the original speech were used for the phoneme duration, accent information, and the like. Speech by the method of the present invention (a)
In the pitch pattern heights of (b) and (c), the midpoints of the heights of the starting point and the ending point of the pitch pattern are the tone-tone components obtained by the tone-tone component generation algorithm of the point-pitch model (Hakoda et al.). The height was adjusted to be the average value. These were presented to 10 subjects, and the naturalness of intonation was evaluated on a 5-grade scale. The results are shown below.

Voice type (a) present invention (b) present invention (c) present invention (d) conventional (e) original sound evaluation 3.28 3.32 3.16 3.00 3.98 This invention method (a) (b) (c) is all conventional Method (d)
It is more natural than that. Also, if there is no prosody category that matches the input text in the database (b)
Since the same naturalness was achieved, it was confirmed that the estimation was performed appropriately. Further, it was confirmed from the evaluation results of (b) and (c) that the naturalness was improved by the shape deformation of the pitch pattern.

[Brief description of drawings]

FIG. 1 is a flowchart illustrating a method of generating a pitch pattern of text to be combined in an embodiment of the present invention.

FIG. 2 is a flowchart for generating a category quantity data table necessary for estimating a prosody category to be used from a pitch pattern database in an embodiment of the present invention.

FIG. 3 is a flowchart illustrating a method of estimating a prosody category to be used in the embodiment of the present invention.

─────────────────────────────────────────────────── --- Continuation of front page (56) References JP-A-6-236197 (JP, A) JP-A-10-143196 (JP, A) JP-A-9-44191 (JP, A) JP-A-8- 44391 (JP, A) JP 63-46497 (JP, A) JP 11-95783 (JP, A) JP 2000-75880 (JP, A) JP 7-219599 (JP, A) Kaihei 7-261778 (JP, A) (58) Fields surveyed (Int.Cl. ⁷ , DB name) G10L 13/08

Claims (8)

(57) [Claims] 1. A method for generating pitch patterns by selecting pitch pattern data suitable for text from a pitch pattern database consisting of a large amount of pitch pattern data extracted from natural speech in a text-to-speech system. If, in the pitch pattern database, pitch pattern data of a prosody category that matches a prosody category consisting of a set of accent type, number of mora, leading accent type, trailing accent type, leading boundary condition, trailing boundary condition, etc. From the pitch pattern data belonging to the prosody category, select the pitch pattern data used for synthesis, and if there is no matching pitch pattern data in the prosody category in the pitch pattern database, the Pitch pattern database The distance (similarity) to each prosody category is obtained by statistical estimation, and from this distance, the prosody category estimated to be closest to the prosody category of the text to be synthesized is selected in the pitch pattern database, A pitch pattern generation method characterized in that pitch pattern data used for synthesis is selected from the pitch pattern data belonging to the selected prosody category, and the selected pitch pattern data is used for pitch pattern generation. 2. The pitch pattern generation method according to claim 1, wherein the distance between the prosodic categories is represented by a vector of pitch pattern data belonging to each prosody category, an average vector of the prosody categories is calculated, and the distance between different prosody categories is calculated. A pitch pattern generation method, characterized in that the difference in shape of pitch patterns is represented by a difference vector, and the difference vector is used as a distance between prosodic categories. 3. The pitch pattern generation method according to claim 1, wherein the shape of the selected pitch pattern is brought closer to the pitch pattern shape of the prosody category of the text to be synthesized by using a difference vector between the prosody categories. A pitch pattern generation method characterized by generating a pitch pattern by performing a transformation for 4. An apparatus for generating a pitch pattern by selecting pitch pattern data suitable for text from a pitch pattern database composed of a large amount of pitch pattern data extracted from natural speech in a text-to-speech system. Type pattern, number of mora, leading accent type, trailing accent type, leading boundary condition, trailing boundary condition, and so on. Pitch pattern database in which pitch pattern data is recorded in association with pitch pattern data, and matches the text prosody category. A means for searching the pitch pattern data of the prosody category in the pitch pattern database, and if there is pitch pattern data of the prosody category that matches this search, the pitch pattern data used for synthesis is selected from the pitch pattern data belonging to the prosody category. Means to choose , If there is no pitch pattern data of the prosody category that matches the pitch pattern database in the above search, the distance (similarity) between the prosody category of the text and each prosody category of the pitch pattern database is statistically estimated. And a means for selecting the prosody category that is estimated to be the closest to the prosody category of the text to be synthesized in the pitch pattern database from this distance, and from the pitch pattern data belonging to the selected prosody category. A pitch pattern generation device comprising: a means for selecting pitch pattern data to be used for synthesis; and a means for generating a pitch pattern using the selected pitch pattern data. 5. The pitch pattern generation device according to claim 4, wherein the difference vector between the prosodic categories is used to transform the shape of the selected pitch pattern so as to approximate the pitch pattern shape of the prosody category of the text to be synthesized. A pitch pattern generation device comprising means for applying the pitch pattern to generate a pitch pattern. 6. In a text-to-speech synthesis system, pitch pattern data suitable for text is selected from a pitch pattern database consisting of a large amount of pitch pattern data extracted from natural speech, and a text is stored in a computer of a device for generating a pitch pattern. Processing for searching the pitch pattern database for pitch pattern data of a prosody category that matches a prosody category consisting of a set of accent type, number of mora, leading accent type, trailing accent type, leading boundary condition, trailing boundary condition, etc. If pitch pattern data of the prosody category that matches in the search exists, the process of selecting the pitch pattern data to be used for synthesis from the pitch pattern data belonging to that prosody category, and the process of selecting the prosody category that matches the pitch pattern database in the above search Pitch pattern If data does not exist, the process of statistically estimating the distance (similarity) between the prosody category of the text and each prosody category of the pitch pattern database, and from this distance, in the pitch pattern database , The process of selecting the prosody category that is estimated to be the closest to the prosody category of the text to be synthesized, the process of selecting the pitch pattern data used for synthesis from the pitch pattern data belonging to the selected prosody category, and the selected A recording medium in which a process for generating a pitch pattern using pitch pattern data and a program for executing the process are recorded. 7. The recording medium according to claim 6, wherein the distance between prosodic categories is represented by a vector of pitch pattern data belonging to each prosody category, an average vector of the prosody categories is calculated, and pitch patterns between different prosody categories are calculated. A recording medium characterized in that the difference in shape of the is expressed as a difference vector, and the difference vector is used as a distance between prosody categories. 8. The recording medium according to claim 6 or 7, wherein a variation vector between prosodic categories is used to modify the shape of the selected pitch pattern to approximate the pitch pattern shape of the prosody category of the text to be synthesized. A recording medium, wherein the program includes a program that causes the computer to execute a process of generating a pitch pattern.