JP4816144B2 - Speech synthesis apparatus, speech synthesis method, and program - Google Patents

Description

  The present invention relates to a speech synthesizer, a speech synthesis method, and a program for synthesizing speech from a given text string.

  HMM (Hidden Markov Model) is used in various forms in technology for synthesizing speech from text strings.

  For example, in the technique of Patent Document 1, LSP (Line Spectrum Pair) coefficients are extracted from speech data, and each phoneme is modeled by an HMM. Then, the HMM corresponding to the given character string is selected. The HMM is driven to output LSP coefficients, and a voice is synthesized using the output LSP coefficients.

  The speech synthesizer of Patent Document 1 drives an HMM at a constant output frame period and outputs LSP coefficients. In such a case, if an attempt is made to synthesize and output smooth speech, the output frame period for outputting the LSP coefficient from the HMM must be shortened. If it does so, the processing burden of a speech synthesizer will become large and processing speed will fall.

JP 2002-62890 A

  The present invention has been made in view of the above problems, and an object thereof is to provide a speech synthesizer, a speech synthesis method, and a program that synthesize high-quality speech.

In order to achieve the above object, a speech synthesizer according to the first aspect of the present invention provides:
Storage means for storing phoneme HMM (Hidden Markov Model) data for generating LSP (Line Spectrum Pair) coefficients, which are parameters for synthesizing speech, and phoneme labels in association with each other;
Phoneme HMM data conversion means for generating a phoneme label from given text data, referring to the storage information of the storage means, and converting the generated phoneme label into corresponding phoneme HMM data;
A period setting means for setting, for each state position, a period for outputting an LSP coefficient from the phoneme HMM data converted by the phoneme HMM data conversion means;
LSP coefficient output means for outputting LSP coefficients at a period set by the period setting means from the phoneme HMM data converted by the phoneme HMM data conversion means;
With
The period setting means includes
Determine the size of the dispersed values of the LSP coefficients for each state position of the phoneme HMM data,
If the variance value is less than or equal to the first threshold and greater than or equal to the second threshold smaller than the first threshold, the cycle for outputting the LSP coefficient is set to the first cycle,
If the variance value is greater than the first threshold, set the period for outputting the LSP coefficient to a second period smaller than the first period,
When the variance value is smaller than the second threshold value, the period for outputting the LSP coefficient is set to a third period that is larger than the first period .

The speech synthesis method according to the second aspect of the present invention provides:
A storage step of storing phoneme HMM (Hidden Markov Model) data for generating LSP (Line Spectrum Pair) coefficients, which are parameters for synthesizing speech, and phoneme labels in association with each other;
A phoneme HMM data conversion step of generating a phoneme label from given text data, referring to the information stored in the storage step, and converting the generated phoneme label into corresponding phoneme HMM data;
A cycle setting step for setting, for each state position, a cycle for outputting LSP coefficients from the phoneme HMM data converted in the phoneme HMM data conversion step;
An LSP coefficient output step of outputting an LSP coefficient at a cycle set in the cycle setting step from the phoneme HMM data converted in the phoneme HMM data conversion step;
With
In the period setting step,
Determine the size of the dispersed values of the LSP coefficients for each state position of the phoneme HMM data,
If the variance value is less than or equal to the first threshold and greater than or equal to the second threshold smaller than the first threshold, the cycle for outputting the LSP coefficient is set to the first cycle,
If the variance value is greater than the first threshold, set the period for outputting the LSP coefficient to a second period smaller than the first period,
When the variance value is smaller than the second threshold value, the period for outputting the LSP coefficient is set to a third period that is larger than the first period .

A computer program according to the third aspect of the present invention provides:
Computer
Storage means for storing phoneme HMM (Hidden Markov Model) data for generating LSP (Line Spectrum Pair) coefficients, which are parameters for synthesizing speech, and phoneme labels in association with each other;
Phoneme HMM data conversion means for generating a phoneme label from given text data and converting the generated phoneme label into corresponding phoneme HMM data by referring to the storage information of the storage means;
A period setting means for setting a period for outputting an LSP coefficient from the phoneme HMM data converted by the phoneme HMM data conversion means for each state position;
LSP coefficient output means for outputting an LSP coefficient at a period set by the period setting means from the phoneme HMM data converted by the phoneme HMM data conversion means;
Function as
The period setting means includes
Determine the size of the dispersed values of the LSP coefficients for each state position of the phoneme HMM data,
If the variance value is less than or equal to the first threshold and greater than or equal to the second threshold smaller than the first threshold, the cycle for outputting the LSP coefficient is set to the first cycle,
If the variance value is greater than the first threshold, set the period for outputting the LSP coefficient to a second period smaller than the first period,
When the variance value is smaller than the second threshold value, the period for outputting the LSP coefficient is set to a third period that is larger than the first period .

  According to the present invention, it is possible to synthesize high-quality sound while maintaining the processing speed by appropriately setting the cycle of outputting the LSP coefficient for each state position of the phoneme HMM data.

A speech synthesizer 100 according to an embodiment of the present invention will be described with reference to the drawings.
The speech synthesizer 100 is a device that, when given an arbitrary text string, synthesizes and outputs the speech of the text string.

  As shown in FIG. 1, the speech synthesizer 100 includes an input conversion unit 10, a speech synthesis dictionary 20, a phoneme HMM sequence conversion unit 30, a parameter generation unit 40, an excitation sound source generation unit 50, and an LSP coefficient interpolation unit. 60 and an LSP synthesis filter 70.

  The input conversion unit 10 receives input of text character string data from the user. Then, the input conversion unit 10 converts the input text character string data into phoneme label string data that is an array of labels in phoneme units.

  The speech synthesis dictionary 20 is used when converting phoneme label string data into phoneme HMM string data. The speech synthesis dictionary 20 stores phoneme HMM data related to LSP coefficients and phoneme HMM data related to pitch. Each phoneme HMM data is created by learning from LSP coefficients extracted from a large number of speech data and phoneme label string data corresponding to the speech data.

  As shown in FIG. 5A, the phoneme HMM data related to the LSP coefficient and the phoneme HMM data related to the pitch each have five states, and each of the state positions S1 to S3 (S0 is an initial state, S4 is an end state) In S0 and S4, the LSP coefficient and pitch data are not output), and the LSP coefficient and pitch data are output. Note that an average value and a variance value are held as parameters for each state.

  The LSP coefficient is a feature vector that represents a feature of speech and is used as a parameter for synthesizing speech.

  The phoneme HMM string converter 30 receives phoneme label string data from the input converter 10. Then, referring to the speech synthesis dictionary 20, the received phoneme label string data is converted into phoneme HMM string data related to LSP coefficients and phoneme HMM string data related to pitch. The phoneme HMM sequence data is obtained by connecting the phoneme HMM data shown in FIG. 5A into the sequence data as shown in FIG. 5B.

  The parameter generation unit 40 receives phoneme HMM sequence data related to LSP coefficients from the phoneme HMM conversion unit 30, and generates LSP coefficient series data as a parameter for synthesizing speech. The LSP coefficient series data is obtained by arranging and connecting LSP coefficients that change in a time series indicated by white circles in a predetermined cycle, as shown by a line graph in the lower part of FIG. Here, in order to simplify the drawing, it is assumed that each LSP coefficient is composed of a five-dimensional feature vector.

  Further, the parameter generation unit 40 receives phoneme HMM sequence data related to pitch, and generates pitch sequence data as shown in FIG. 7 as a parameter for synthesizing speech.

The parameter generation unit 40 generates parameters so that the likelihood of the phoneme HMM sequence data with respect to each phoneme HMM data is maximized.
The parameter that maximizes the likelihood for each phoneme HMM data can be obtained by solving the following equation.

  However, P is the probability that the parameter O created at the state position Q is observed from the phoneme HMM data λ (the likelihood of λ at Q with respect to O), and C is the parameter that maximizes P.

  By generating a parameter that maximizes the likelihood, variations in LSP coefficient sequence data and pitch sequence data are reduced, discontinuous changes can be suppressed, and speech closer to an actual utterance can be synthesized.

The period (frame period) at which the parameter generation unit 40 outputs the LSP coefficient is set for each state position of the phoneme HMM data.
Assuming that a frame period for synthesizing speech with an LSP synthesis filter 70, which will be described later, is FPRD, the parameter generator 40 normally outputs LSP coefficients with a predetermined frame period PRD longer than FPRD.

  The parameter generation unit 40 switches the frame period according to the size of the variance value at each state position. That is, when the variance value at a certain state position is smaller than the first predetermined value, the frame period is switched to twice the length of the PRD. Further, when the variance value at a certain state position is larger than the second predetermined value, the state is switched to FPRD which is the shortest cycle among which the frame cycle can be set.

In the state position where the variance value is small, even if the parameter is output with a longer cycle than usual, there is little variation, and thus it is difficult to produce discontinuous data. Therefore, it is possible to improve the processing speed by setting the frame period to twice the normal period.
In a state position where the variance value is large, discontinuous data is obtained unless parameters are output in a short cycle. Therefore, the frame period is set to FPRD to suppress discontinuous data.

  The excitation sound source generator 50 receives time series pitch string data as shown in FIG. 7 from the parameter generator 40, and generates excitation sound source data from the pitch string data.

  The LSP coefficient interpolation unit 60 receives LSP coefficient series data from the parameter generation unit 40. The coefficients of the received LSP coefficient series data are interpolated as shown by the black circles shown in FIG. 6 using adjacent LSP coefficients to generate LSP coefficient series data of the frame period FPRD. The interpolation is performed by linear interpolation using adjacent LSP coefficients of the LSP coefficient series data.

  The LSP synthesis filter 70 receives excitation sound source data from the excitation sound source generation unit 50. In addition, the LSP coefficient series data of the frame period FPRD is received from the LSP coefficient interpolation unit 60. Then, they are synthesized to generate synthesized speech. Then, the generated synthesized speech is output.

  Next, an operation of speech synthesis processing for synthesizing speech of the speech synthesizer 100 having the above configuration will be described with reference to FIG.

  First, the input conversion unit 10 of the speech synthesizer 100 receives an input of text character string data from the user (step S11).

  When receiving input of text character string data, the input conversion unit 10 converts the text character string data into phoneme label string data (step S12). Then, the converted phoneme label string data is delivered to the phoneme HMM string converter 30.

  Next, the phoneme HMM sequence converter 30 receives the phoneme label sequence data converted in step S12, refers to the speech synthesis dictionary 20, and converts it into phoneme HMM sequence data related to LSP coefficients and phoneme HMM sequence data related to pitch sequences. (Step S13). Then, the phoneme HMM string data related to the converted LSP coefficient and the phoneme HMM string data related to the pitch string are delivered to the parameter generation unit 40.

  When the parameter generation unit 40 receives the phoneme HMM string data related to the LSP coefficient and the phoneme HMM string data related to the pitch string, the parameter generation unit 40 executes the parameter generation process shown in FIG. 3 (step S14).

  In the parameter generation process (step S14), the parameter generation unit 40 generates pitch sequence data as shown in FIG. 7 from the phoneme HMM sequence data related to the received pitch sequence (step S21).

  At the same time, an LSP coefficient sequence data generation process (step S22) is executed to generate LSP coefficient sequence data from the phoneme HMM sequence data related to the received LSP coefficient.

  The operation of the LSP coefficient series data generation process (step S22) is shown in FIG.

  In the LSP coefficient series data generation process (step S22), the parameter generation unit 40 sets the phoneme HMM data λi of the phoneme HMM string data {λi | 1 ≦ i ≦ N} related to the LSP coefficient as shown in the upper part of FIG. LSP coefficients are output for all state positions S1 to S3. Then, LSP coefficient series data (white circles) as shown in the lower part of FIG. 6 is generated.

First, the frame period for outputting the LSP coefficient is determined. Therefore, the parameter generation unit 40 determines whether or not the variance value at a certain state position is smaller than a predetermined threshold value V1 (step S31).
The threshold V1 is set to a value at which a stable parameter is output if the variance value is smaller than this.

  If it is determined that the variance value is smaller than the threshold value V1 (step S31; Yes), the frame period at the state position is set to twice the normal output period PRD (step S32).

If it is determined that the variance value is not smaller than the threshold value V1 (step S31; No), it is determined whether the variance value is greater than a predetermined threshold value V2 (step S33).
The threshold value V2 is set to a value that outputs a parameter with a variation when the variance value is greater than this value.

  If it is determined that the variance value is larger than the threshold value V2 (step S33; Yes), the frame period at the state position is set to FPRD, which is the minimum period that can be set (step S34).

  On the other hand, if it is determined that the variance value is not greater than the threshold value V2 (step S33; No), the frame period is not reset and the frame period is the normal output period PRD.

  When the frame period is determined in steps S31 to S34, the LSP coefficient having the maximum likelihood for the phoneme HMM data is output in the frame period (step S35).

  The processes in steps S31 to S35 described above are repeated for all the state positions S1 to S3 of each phoneme HMM data λi of the phoneme HMM sequence data {λi | 1 ≦ i ≦ N}. It is possible to generate LSP coefficient series data in which LSP coefficients are output at a frame period.

In the example of FIG. 6, the frame periods PRD _{λi, S1} and PRD _{λi, S3} at the state positions S1, S3 in the phoneme HMM data λi are output at the normal frame period PRD. Further, since the variance value is sufficiently small at the state position S2, the frame period PRD _{λi, S2} at the state position _S2 is output at a frame period twice that of the PRD. At this time, the frame period between each state position is set to FPRD.

  When the LSP coefficient series data generation process (step S22) ends and the parameter generation process (step S14) ends, the process returns to the speech synthesis process shown in FIG. 2, and the parameter generation unit 40 uses the generated pitch string data as the excitation sound source generation unit. Hand over to 50. The generated LSP coefficient series data is delivered to the LSP coefficient interpolation unit 60.

  Upon receiving the pitch string data, the excitation sound source generator 50 generates excitation sound source data from the pitch string data (step S15). Then, the generated excitation sound source data is delivered to the LSP synthesis filter 70.

  Further, upon receiving the LSP coefficient series data, the LSP coefficient interpolation unit 60 performs linear interpolation using adjacent LSP coefficients of the LSP coefficient series data to generate LSP coefficient series data of the frame period FPRD (step S16). (Black circle in FIG. 6). The generated LSP coefficient series data of the frame period FPRD is delivered to the LSP synthesis filter 70.

  When the LSP synthesis filter 70 receives the excitation sound source data and the LSP coefficient series data having the frame period FPRD, the LSP synthesis filter 70 synthesizes speech from both (step S17).

  Subsequently, the LSP synthesis filter 70 outputs the synthesized voice (step S18).

  As described above, the speech synthesizer 100 can synthesize and output high-quality speech from given text character string data while preventing a decrease in processing speed.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation and application are possible.

  In the above embodiment, the LSP coefficient series data is interpolated by linear interpolation, but the present invention is not limited to this. For example, in order to synthesize higher-quality sound, the interpolation method may be switched for each state position.

  For example, the sound quality of the synthesized speech is further improved by further including an LSP coefficient correction unit 80 that determines the stability of LSP coefficient series data and corrects the data to normal data when the data becomes unstable. You may make it do. The correction by the LSP coefficient correction unit 80 may be performed on the LSP coefficient series data after the interpolation by the LSP coefficient interpolation unit 60 as shown in FIG. 8, or the LSP coefficient before the interpolation as shown in FIG. It may be for series data.

The speech synthesizer 100 is not limited to a dedicated device as in the above embodiment.
For example, the computer may function as the speech synthesizer 100 by a program, or the speech synthesizer 100 may be operated by reading a program into a DSP (Digital Signal Processor) or the like.

It is a block diagram which shows the structure of the speech synthesizer which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement of a speech synthesis process. It is a flowchart which shows the operation | movement of the parameter production | generation process of FIG. It is a flowchart which shows the operation | movement of the LSP coefficient series data generation process of FIG. (A) It is a figure which shows the example of phoneme HMM data. (B) It is a figure which shows the example of phoneme HMM row | line | column data. It is a figure which shows the example of LSP coefficient series data. It is a figure which shows the example of pitch row data. It is a block diagram which shows the structure of the 1st modification of a speech synthesizer. It is a block diagram which shows the structure of the 2nd modification of a speech synthesizer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Input conversion part, 20 ... Speech synthesis dictionary, 30 ... Phoneme HMM sequence conversion part, 40 ... Parameter generation part, 50 ... Excitation sound source generation part, 60 ... LSP coefficient interpolation part, 70 ... LSP synthesis filter, 80 ... LSP coefficient Correction unit, 100 ... speech synthesizer

Claims (5)

Storage means for storing phoneme HMM (Hidden Markov Model) data for generating LSP (Line Spectrum Pair) coefficients, which are parameters for synthesizing speech, and phoneme labels in association with each other;
Phoneme HMM data conversion means for generating a phoneme label from given text data, referring to the storage information of the storage means, and converting the generated phoneme label into corresponding phoneme HMM data;
A period setting means for setting, for each state position, a period for outputting an LSP coefficient from the phoneme HMM data converted by the phoneme HMM data conversion means;
LSP coefficient output means for outputting LSP coefficients at a period set by the period setting means from the phoneme HMM data converted by the phoneme HMM data conversion means;
With
The period setting means includes
Determine the size of the dispersed values of the LSP coefficients for each state position of the phoneme HMM data,
If the variance value is less than or equal to the first threshold and greater than or equal to the second threshold smaller than the first threshold, the cycle for outputting the LSP coefficient is set to the first cycle,
If the variance value is greater than the first threshold, set the period for outputting the LSP coefficient to a second period smaller than the first period,
When the variance value is smaller than the second threshold value , the speech synthesizer is characterized in that the period for outputting the LSP coefficient is set to a third period that is larger than the first period . 2. The voice according to claim 1 , further comprising means for interpolating between LSP coefficients output by the LSP coefficient output means at a period set by the period setting means using LSP coefficients adjacent in time series. Synthesizer. If it is determined that the low stability of the LSP coefficients LSP coefficient output unit outputs, according to claim 1, characterized by further comprising means for correcting the LSP coefficients as stability of the LSP coefficients is increased or The speech synthesizer according to 2 . A storage step of storing phoneme HMM (Hidden Markov Model) data for generating LSP (Line Spectrum Pair) coefficients, which are parameters for synthesizing speech, and phoneme labels in association with each other;
A phoneme HMM data conversion step of generating a phoneme label from given text data, referring to the information stored in the storage step, and converting the generated phoneme label into corresponding phoneme HMM data;
A cycle setting step for setting, for each state position, a cycle for outputting LSP coefficients from the phoneme HMM data converted in the phoneme HMM data conversion step;
An LSP coefficient output step of outputting an LSP coefficient at a cycle set in the cycle setting step from the phoneme HMM data converted in the phoneme HMM data conversion step;
With
In the period setting step,
Determine the size of the dispersed values of the LSP coefficients for each state position of the phoneme HMM data,
If the variance value is less than or equal to the first threshold and greater than or equal to the second threshold smaller than the first threshold, the cycle for outputting the LSP coefficient is set to the first cycle,
If the variance value is greater than the first threshold, set the period for outputting the LSP coefficient to a second period smaller than the first period,
When the variance value is smaller than the second threshold value, the period for outputting the LSP coefficient is set to a third period larger than the first period . Computer
Storage means for storing phoneme HMM (Hidden Markov Model) data for generating LSP (Line Spectrum Pair) coefficients, which are parameters for synthesizing speech, and phoneme labels in association with each other;
Phoneme HMM data conversion means for generating a phoneme label from given text data and converting the generated phoneme label into corresponding phoneme HMM data by referring to the storage information of the storage means;
A period setting means for setting a period for outputting an LSP coefficient from the phoneme HMM data converted by the phoneme HMM data conversion means for each state position;
LSP coefficient output means for outputting an LSP coefficient at a period set by the period setting means from the phoneme HMM data converted by the phoneme HMM data conversion means;
Function as
The period setting means includes
Determine the size of the dispersed values of the LSP coefficients for each state position of the phoneme HMM data,
If the variance value is less than or equal to the first threshold and greater than or equal to the second threshold smaller than the first threshold, the cycle for outputting the LSP coefficient is set to the first cycle,
If the variance value is greater than the first threshold, set the period for outputting the LSP coefficient to a second period smaller than the first period,
A computer program for setting a period for outputting an LSP coefficient to a third period larger than the first period when the variance value is smaller than the second threshold value .

Images