JP5925082B2 - Speech synthesis apparatus, method and program - Google Patents

Description

  The present invention relates to a speech synthesis technique using an HMM (Hidden Markov Model).

  The configuration of a conventional HMM speech synthesizer is shown in FIG. The conventional HMM speech synthesizer includes a phoneme HMM sequence generation unit 1, a vocal tract spectrum parameter calculation unit 2, a speech synthesis unit 3, and a phoneme HMM storage unit 4.

  The phoneme HMM sequence generation unit 1 connects a phoneme HMM (Hidden Markov Model) previously learned from a speech database for learning according to the input phoneme string of the text to be synthesized. Generate a series of states. The generated sequence of phoneme HMM states is transmitted to the vocal tract spectrum parameter calculation unit 2.

  Here, the sequence of phoneme HMM states is a vector composed of the average value of the vocal tract spectral parameters and the average value of the velocity of the vocal tract spectral parameters, and the variance of the vocal tract spectral parameters and the velocity dispersion of the vocal tract spectral parameters. It is expressed as a matrix consisting of Note that the average value of the vocal tract spectrum parameter, the average value of the velocity of the vocal tract spectrum parameter, the variance of the vocal tract spectrum parameter, and the variance of the velocity of the vocal tract spectrum parameter are respectively the phoneme HMMs constituting the sequence of the phoneme HMM states. It is calculated for each state.

  The vocal tract spectrum parameter calculation unit 2 calculates a vocal tract spectrum parameter that maximizes the output probability.

  Finally, based on the calculated vocal tract spectrum parameters, the speech synthesizer 3 inputs the input sound source signal (a pulse train corresponding to the basic period if voiced sound, or a sound source signal consisting of white noise if unvoiced sound). A voice is synthesized by convolution (see, for example, Non-Patent Document 1).

As the vocal tract spectrum parameter, an LSP parameter ω _i (i = 1, 2,..., P) that is an all-pole filter expression of the vocal tract spectrum of speech is widely used. Requirements LSP parameters for the filter is stable, ascending characteristic LSP parameter omega _i, that is, is to satisfy _{0 <ω 1 <ω 2 <} ... <ω p. When speech is synthesized from LSP parameters that do not satisfy the ascending characteristics, the synthesized speech signal may diverge.

Now, let the LSP parameter at time t be ω _i (t) and the speed of the LSP parameter be

The vector ω _static consisting only of LSP parameters ω _i (t)

The vector ω _dynamic consisting of LSP parameter ω _i (t) and its velocity Δω _i (t)

The transformation matrix R into

Can be defined. For example, a is 0.5. In this specification, T superscript matrix · ^T means transpose. Note that T other than the superscript ^T of the matrix T means a predetermined time T. 0 _{a × b} means an a × b matrix where all elements are 0, I _{a × a} means an a × a square matrix whose diagonal elements are 1 and all other elements are 0 To do.

Average value of the speed of the average value .omega.i _^(t) and LSP parameters LSP parameter Δω ^- _i (t) from the composed vector omega ^-, and the variance of the velocity of the variance σ _{ωi (t)} and LSP parameters LSP parameters A matrix σ composed of σ _{Δωi (t)} can be expressed as follows. diag means a diagonal matrix.

  Here, the LSP parameter vector ω ^ estimated by calculation in the vocal tract spectrum parameter calculation unit 2 is defined as follows.

  Then, in order to obtain the LSP parameter that maximizes the output probability in the vocal tract spectrum parameter calculation unit 2, the following may be minimized.

  That means

(For example, see Non-Patent Document 2).

  However, since the value of the LSP parameter obtained by the above calculation exists near the average value of the HMM, compared to the value of the original LSP parameter uttered by humans, the dynamic range of the LSP parameter is narrow, and it can be heard that it is muffled There's a problem. Therefore, it is considered to define a parameter called Global Variance (GV) and evaluate the dynamic range of the LSP parameter (see, for example, Non-Patent Document 3).

Ν _{i, n,} which is a GV for one sentence n, is defined as follows using the LSP parameter for the utterance of the sentence n. T _n is the utterance length of sentence n.

Also, μ and U are defined as follows using LSP parameters for the utterance of N sentences n (n = 1,..., N) of learning data. μ is the average of ν _{i, n} (n = 1,..., N), and U is the variance of ν _{i, n} (n = 1,..., N). Note that N is a predetermined positive number. It is assumed that μ and U are stored in the phoneme HMM storage unit 4.

  The estimated GV ν (ω) is defined as follows.

To find the LSP parameter that maximizes the output probability of GV, you can minimize:

  Finally, in Non-Patent Document 2, the vocal tract spectrum parameter calculation unit 2 minimizes Expression (3), which is an evaluation expression combining Expression (1) and Expression (2), instead of Expression (1). The LSP parameter ω ^ that can be obtained is calculated using an algorithm such as the steepest descent method.

Here, 1 / (2T) is a weight for aligning the order of the values of the expressions (1) and (2).

  In HMM speech synthesis, it is necessary to use a relatively high-order LSP parameter (for example, p = 40), and the above-mentioned method does not necessarily satisfy the ascending characteristics of the LSP parameter by introducing GV. is there. Therefore, a method for introducing a penalty function has been proposed. That means

This is a method for introducing a constraint by a penalty function (for example, see Non-Patent Document 4). Here, α is a positive value that is not so large, M is an even number, and ω _{p + 1} = π. It can be seen that the value of the penalty function becomes large when ω _i-1 > ω _i , but this penalty function does not guarantee that the ascending characteristics are satisfied, and the maximum output probability is not guaranteed due to the effect of the penalty function. There's a problem.

L.R.Rabiner, R.W.Schafer, “Digital processing of speech signals”, Prentice Hall, 1978. Tokuda, Mashiko, Kobayashi, Imai, “Speech parameter generation algorithm from HMM using dynamic features”, Acoustical Society of Japan, Vol.53, No.3, 192-200, 1997. T. Toda, K. Tokuda, “A speech parameter generation algorithms considering global variance for HMM-based speech synthesis”, IEICE Trans. Inf. & Syst., 816-824, 2007. M. Lei, Z.-H. Ling, L.-R. Dai, “Preserve ordering property of generated LSPs for minimum generation error training in HMM-based speech synthesis”, Proc. ICASSP, 4712-4715, 2011.

  As described above, the conventional method cannot maintain the ascending order characteristic of the LSP parameter.

  An object of the present invention is to provide a speech synthesizer, a method, and a program that can maintain the ascending characteristics of LSP parameters.

A speech synthesizer according to an aspect of the present invention includes a phoneme HMM storage unit that stores phoneme HMMs of a plurality of phonemes, and a phoneme HMM corresponding to each phoneme of the input text from the phoneme HMM storage unit. And a phoneme HMM state sequence generator that generates a sequence of phoneme HMM states, and p is the order of the LSP parameter, and ω ^ _i (t) (i = 1,2, .... p) Is the LSP parameter, and the differential LSP parameter d _i (t) is d ₁ (t) = ω ^ ₁ (t)> 0 and d _j (t) = ω ^ _j (t) -ω ^ _j-1 (t) > 0 (2 ≤ j ≤ p), using the LSP parameters that make up the sequence of phoneme HMM states, the differential LSP parameter calculated by repeatedly updating the differential LSP parameter d _i (t) with a multiplication-type update equation a calculation unit, using the calculated difference LSP parameters d _i (t), and the LSP parameter calculating unit that calculates an LSP parameter ω ^ _i (t) which is defined by the following equation, the calculated LSP parameter Based on over data ω ^ _i (t), and a speech synthesis unit for generating a sound corresponding to the input text, provided.

  The ascending characteristics of LSP parameters can be retained.

The figure for demonstrating the example of a speech synthesizer. The figure for demonstrating the example of the speech synthesis method. The figure for demonstrating the example of the conventional speech synthesizer.

  Hereinafter, embodiments of a speech synthesis apparatus and method will be described with reference to the drawings.

  As shown in FIG. 1, the speech synthesizer includes a phoneme HMM sequence generation unit 1, a differential LSP parameter calculation unit 5, an LSP parameter calculation unit 6, a speech synthesis unit 3, and a phoneme HMM storage unit 4. In the embodiment of the present invention, the differential LSP parameter calculation unit 5 and the LSP parameter calculation unit 6 calculate LSP parameters instead of the vocal tract spectrum parameter calculation unit 2 described in the background art. Other parts are the same as in the background art.

  The phoneme HMM storage unit 4 stores a plurality of phoneme HMMs. Further, with N as a predetermined positive number, the phoneme HMM storage unit 4 stores GV (μ and U) obtained from the LSP parameters for the utterances of N sentences n (n = 1,..., N). ) Is stored.

Each state of the phoneme HMM consists of an LSP parameter average value ω ^- _i (t), an LSP parameter speed average value Δω ^- _i (t), an LSP parameter variance σ _{ωi (t),} and an LSP parameter velocity variance σ It is expressed by _{Δωi (t)} . LSP parameter average value ω ^- _i (t), LSP parameter speed average value Δω ^- _i (t), LSP parameter variance σ _{ωi (t)} and LSP parameter velocity variance σ _{Δωi (t)} And stored in the phoneme HMM storage unit 4.

  The phoneme HMM sequence generation unit 1 performs the same processing as the phoneme HMM sequence generation unit 1 described in the background art. That is, the phoneme HMM sequence generation unit 1 reads a phoneme HMM corresponding to each phoneme of the input text from the phoneme HMM storage unit 4 and connects the read phoneme HMM to generate a sequence of phoneme HMM states. (Step S1). The generated sequence of phoneme HMM states is transmitted to the differential LSP parameter calculation unit 5.

Each state of the phoneme HMM consists of an LSP parameter average value ω ^- _i (t), an LSP parameter speed average value Δω ^- _i (t), an LSP parameter variance σ _{ωi (t),} and an LSP parameter velocity variance σ _Since it is expressed by _{Δωi (t)} , the sequence of phoneme HMM states is also a vector ω ⁻ composed of the average value ω ⁻ _i (t) of the LSP parameter and the average value Δω ⁻ _i (t) of the speed of the LSP parameter, In addition, it is expressed by a matrix σ composed of LSP parameter variance σ _{ω i (t)} and LSP parameter velocity variance σ _{Δω i (t)} .

The difference LSP parameter calculation unit 5 calculates the difference LSP parameter d _j by repeatedly updating the difference LSP parameter dj using a multiplication type update equation using the LSP parameters constituting the sequence of phoneme HMM states (step S2). The calculated differential LSP parameter d _j is transmitted to the LSP parameter calculation unit 6.

Specifically, the differential LSP parameter calculation unit 5 uses the parameters such as ω ^- _i (t), Δω ^- _i (t), σ _{ωi (t)} , σ _{Δωi (t)} , μ, U, etc. The parameter d _j is calculated. Here, parameters such as ω ^- _i (t), Δω ^- _i (t), σ _{ωi (t)} , σ _{Δωi (t)} , μ, U, etc. are LSP parameters ω _i that constitute a sequence of states of the phoneme HMM. This is a parameter calculated from (t). Therefore, it can be said that the differential LSP parameter calculation unit 5 calculates the differential LSP parameter d _j by using the LSP parameters constituting the sequence of phoneme HMM states.

With p as the order of the LSP parameter and LSP parameter determined by the differential LSP parameter d _i (t) as ω ^ _i (t), d _i (t) and ω ^ _i (t) have the following relationship.

The differential LSP parameter calculation unit 5 calculates d _j (t) that minimizes the expression (3) described in the background art section.

  For example, in equation (3)

The evaluation formula obtained by substituting defined as _{F (d i (t))} , the positive term of F (d _i (t)) was partially differentiated by d _i (t) polynomial _{▽ F (d i (t)} ) a polynomial consisting of ▽ F ⁺ and (d _i (t)), a polynomial consisting of negative term of the polynomial _{▽ F (d i (t)} ) ▽ F - and (d _i (t)). At this time, by repeatedly updated by the following multiplicative update equation d _j, it is possible to calculate the d _j (e.g., see reference 1.).

[Reference 1] Virtanen, IEEE Trans. Audio Speech Lang. Process., 15 (3), 1066-1074, 2007.
Further, when the speed of the LSP parameter is defined as in Expression (0) described in the background art, the multiplication type update expression of Expression (4) is specifically as follows.

  Further, when the constant “a” of the equation (0) described in the background art is set to 0.5, the multiplication type update equation of the equation (4) is specifically as follows.

  If the speed of the LSP parameter is defined as follows,

Specifically, the multiplication type update formula of Formula (4) is as follows.

  If the speed of the LSP parameter is defined as follows,

Specifically, the multiplication type update formula of Formula (4) is as follows.

In the above-described multiplication type updating formula, the case of using GV has been described, but the present invention can also be used in an embodiment that does not use GV. For example, define the speed of the LSP parameter as follows:

Specifically, the multiplication type update formula is as follows. Note that when using a multiplication type update equation that does not use GV, μ and U need not be stored in the phoneme HMM storage unit 4 in advance.

  T in the above-described multiplication type update formula is the sum of the time lengths of the phoneme HMM state sequences, and is calculated by the phoneme HMM state sequence generation unit 1.

In addition, in the above-described multiplication type update equation, d _i (t) and Δω ⁻ _j (t) are not defined when t ≦ 0 or t> T, and therefore d _i (t when t ≦ 0 or t> T The value of the term including t), Δω ⁻ _j (t) is set to 0. For example, in the case of d _l (0) in the second term on the right side of A in Expression (5) of the multiplication type update formula given as a specific example first, the value of the second term in this case is set to 0. Further, in the case of d _l (T) in the third term on the right side of A in Equation (5), the value of the third term in this case is set to 0.

The initial value of d _i (t) is, for example, a non-negative random number. The number of times of updating d _i (t) is appropriately determined according to the required accuracy and specifications. In general, the higher the number of updates of d _i (t) is large, the accuracy of d _i (t) is higher. For example, the update is performed until the difference between the value of Equation (3) after the K-1th update and the value of Equation (3) after the ^Kth update is equal to or less than a predetermined threshold (for example, 10 ⁻⁷ ).

Since d _i (t) converges to a non-negative value, 0 <ω ₁ (t) = d _i (t) <ω ₂ (t) = d _i (t) + d ₂ (t) <… <ω _p ( t) = d _i (t) +... + d _p (t). Thus, by calculating d _i (t) instead of ω _i (t) and calculating ω _i (t) from d _i (t), the obtained ω _i (t) always satisfies the ascending characteristics. , And minimize the error in equation (3).

The LSP parameter calculation unit 6 calculates the LSP parameter ω ^ _i (t) by the following equation using d _i (t) obtained by the differential LSP parameter calculation unit 5 (step S3). The calculated LSP parameter ω ^ _i (t) is transmitted to the speech synthesizer 3.

The speech synthesizer 3 performs the same processing as the speech synthesizer 3 described in the background art. That is, the speech synthesizer 3 generates speech corresponding to the input text based on the calculated LSP parameter ω ^ _i (t) (step S4).

  Specifically, the speech synthesizer 3 receives the input sound source signal u (t) (a pulse train corresponding to the basic period if it is a voiced sound) based on the calculated vocal tract spectrum parameters as shown in the following equation. The voice signal s˜ (t) is generated by convolving a sound source signal composed of white noise if it is an unvoiced sound.

Note that the following parameters are added to the phoneme HMM stored in the phoneme HMM storage unit 4 to compensate for d _i (t) <π. This further reduces the possibility that the synthesized audio signal will diverge.

The phoneme HMM storage unit 4 stores the LSP parameter average value ω ^- _i (t), the LSP parameter speed average value Δω ^- _i (t), the LSP parameter variance σ _{ωi (t),} and the LSP parameter speed. _{Instead of the} variance σ _{Δωi (t)} , a phoneme HMM expressed by the LSP parameter ω _i (t) may be stored. In this case, the phoneme HMM sequence generation unit 1 uses the LSP parameter ω _i (t) read from the phoneme HMM storage unit 4, and uses the LSP parameter average value ω ⁻ _i (t) and the LSP parameter speed average value Δω. ^- _i (t), and transmits the calculated variance sigma _.omega.i the LSP parameter _(t) and the variance sigma _{Derutaomegaai} speed LSP parameter _(t) to the difference LSP parameter calculating part 5.

  Note that μ and U may not be stored in the phoneme HMM storage unit 4. In this case, when a multiplication type update formula using GV is used, the phoneme HMM sequence generation unit 1 uses the LSP parameters for the utterances of N sentences n (n = 1,..., N). μ and U are calculated and transmitted to the differential LSP parameter calculation unit 5.

  The processes described in the above apparatus and method are not only executed in time series according to the order of description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the process.

  Further, when the processing means in the apparatus is realized by a computer, the processing contents of the functions that the apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Phoneme HMM sequence generation part 2 Vocal tract spectrum parameter calculation part 3 Speech synthesis part 4 Phoneme HMM storage part 5 Difference LSP parameter calculation part 6 LSP parameter calculation part

Claims (4)

A phoneme HMM storage unit for storing a plurality of phoneme HMMs;
A phoneme HMM state sequence generation unit that reads a phoneme HMM corresponding to each phoneme of the input text from the phoneme HMM storage unit and generates a sequence of phoneme HMM states by connecting the read phoneme HMM;
p is the order of the LSP parameter, ω ^ _i (t) (i = 1,2, .... p) is the LSP parameter, and the differential LSP parameter d _i (t) is d ₁ (t) = ω ^ ₁ (t)> 0 and d _j (t) = ω ^ _j (t) -ω ^ _j-1 (t)> 0 (2 ≦ j ≦ p) A differential LSP parameter calculation unit that calculates the difference LSP parameter d _i (t) by repeatedly updating with a multiplication type update equation,
An LSP parameter calculation unit that calculates an LSP parameter ω ^ _i (t) defined by the following formula using the calculated differential LSP parameter d _i (t) ,

Based on the calculated LSP parameter ω ^ _i (t), a speech synthesizer that generates speech corresponding to the input text;
A speech synthesizer. The speech synthesis apparatus according to claim 1,
Positive expression of polynomial ▽ F (d _i (t)) obtained by partial differentiation of evaluation formula F (d _i (t)) with respect to d _i (t) to obtain LSP parameter ω ^ _i (t) that maximizes output probability a polynomial consisting of terms ▽ F ⁺ and (d _i (t)), a polynomial consisting of negative term of the polynomial _{▽ F (d i (t)} ) ▽ F - as (d _i (t)),
The multiplication type update equation is the following equation:

Speech synthesizer. The phoneme HMM state sequence generation unit reads the phoneme HMM corresponding to each phoneme of the input text from the phoneme HMM storage unit that stores the phoneme HMMs of a plurality of phonemes, and connects the read phoneme HMMs to connect the phoneme HMM A phoneme HMM state sequence generation step for generating a sequence of states;
The differential LSP parameter calculation unit sets p as the order of the LSP parameter, ω ^ _i (t) (i = 1, 2,... P) as the LSP parameter, and sets the differential LSP parameter d _i (t) as d ₁ (t) = ω ^ ₁ (t)> 0 and d _j (t) = ω ^ _j (t) -ω ^ _j-1 (t)> 0 (2 ≦ j ≦ p) A differential LSP parameter calculation step for calculating the differential LSP parameter d _i (t) by repeatedly updating the differential LSP parameter d _i (t) using a multiplication-type update equation,
The LSP parameter calculation unit calculates an LSP parameter ω ^ _i (t) defined by the following formula using the calculated difference LSP parameter d _i (t) ,

A speech synthesis unit that generates speech corresponding to the input text based on the calculated LSP parameter ω ^ _i (t);
A speech synthesis method including:   The program for functioning a computer as each part of the speech synthesizer of Claim 1 or 2.

Images