JPH0330880B2 - - Google Patents

Description

[Detailed Description of the Invention] (A) Technical Field of the Invention The present invention relates to a speech analysis and synthesis device, particularly a speech analysis and synthesis device that extracts parameters from input speech using a linear prediction method. When extracting the pitch cycle based on autocorrelation, multiple components with larger autocorrelation coefficients are extracted as candidates, and the pitch cycle of the more reliable cycle is used as a reference to calculate the reliability. This invention relates to a speech analysis/synthesis device that sets pitch cycles for lower periods.

(B) Technical background and problems Conventionally, in speech analysis and synthesis devices using linear prediction methods (including the PARCOR method), the prediction residual as a result of linear prediction is examined, and the autocorrelation coefficient sequence of the prediction residual is calculated. The pitch period is extracted by focusing on the fact that the change with respect to the time delay takes a peak at the time delay position corresponding to the pitch period. That is, when extracting the pitch period,
() Find the time delay that gives the maximum value of the autocorrelation coefficient sequence of the prediction residual, () check the fluctuation from the pitch period for the previous and subsequent frames, and () find the half period or A method is adopted in which it is checked whether there is a peak at a position corresponding to the double period, and if there is (), the time delay giving that peak is taken as the pitch period.

However, in the case of this method, if the pitch period of the original speech fluctuates or the pitch structure collapses at speech boundaries, the autocorrelation coefficient of the prediction residual will no longer have a sharp peak, and the pitch period will change. This includes the problem that it cannot be discovered correctly.

(C) Object and Structure of the Invention The present invention aims to solve the above-mentioned problems, and by extracting the pitch period in a highly reliable manner, it eliminates manual error correction work and improves speech analysis. The aim is to fully automate the synthesis. Therefore, the speech analysis and synthesis device of the present invention analyzes the input speech using the linear prediction method and converts it into parameters, and extracts the pitch period using the autocorrelation coefficient of the prediction residual. A speech analysis and synthesis device that performs speech synthesis based on the results examines the autocorrelation coefficient sequence of the prediction residual or the autocorrelation coefficient sequence of the autocorrelation coefficient of the prediction residual that is obtained by taking a weighted moving average. A pitch cycle determination candidate extraction unit that extracts a plurality of candidates with larger correlation values, a time delay component extraction unit that extracts a time delay corresponding to each of the pitch cycle determination candidates, and the pitch cycle determination candidate extraction unit. It is equipped with a logical judgment processing section that determines the pitch period based on the extracted pitch period determination candidate and the corresponding time delay component, and the logical judgment processing section determines the pitch period based on the value of the pitch period determination candidate. Examine the reliability with which the time delay component can set the pitch period, and use the time delay component corresponding to the pitch period determination candidate with reliability above the threshold to determine the pitch period for the period with reliability above the threshold. The present invention is characterized in that pitch periods are determined for periods having reliability equal to or lower than a threshold value in order to maintain continuity with previously determined pitch periods. This will be explained below with reference to the drawings.

(D) Embodiments of the Invention FIG. 1 is an explanatory diagram explaining the concept of a frame referred to in this specification, FIG. 2 is a block diagram of an embodiment of the speech analysis and synthesis device of the present invention, and FIGS. 3 to 7 are explanatory diagrams illustrating the processing mode in the logical judgment processing section shown in FIG. 2, FIG. 8 is a flowchart of an embodiment of the processing related to the first pass in the logical judgment processing section, and FIG. 9 is the logical judgment processing section. 10A and 10B are together a flowchart of an embodiment of the processing related to the third pass in the logic judgment processing unit, and FIG.
Flowchart of an embodiment of the process related to the third pass in the logical judgment processing section shown in FIG.
FIG. 2 shows a flowchart of an embodiment of processing related to the fourth pass in the logic judgment processing section.

When extracting various parameters for voice synthesis, as shown in Figure 1, the input voice is cut out at a predetermined time length (frame length), and adjacent frames are created so that they partially overlap. A frame number is assigned to each frame, and parameters for each frame are extracted.

When the input speech extracted in this way is {x _(o) } ^N-1 _o=0 = {x ₍ o ₎ , x ₍₁₎ , ..., x _(N-1) }, Fig. 2 The information is input and processed as shown in the figure.

In FIG. 2, 1 is an input speech correlation processing section;
3 is a linear prediction processing section, 3 is a residual power extraction section, 4 is a residual correlation processing section, 5 is a moving average processing section, 6 is a maximum value extraction processing section, 7 is a pitch period determination candidate holding section, 8
9 is a time delay component holding unit, 9 is a logical judgment processing unit, 1
0 represents a voiced/unvoiced determination section, 11 represents a voiced section driven sound source generation section, 12 represents an unvoiced (including silent) section driven sound source generation section, 13 represents a switching processing section, and 14 represents a linear predictive synthesis section.

The input voice {x _(o) } extracted as described above is processed by the processing unit 1 using the autocorrelation {r _i } for the input voice.
will be taken. Then, parameters related to the vocal tract are extracted in the linear prediction processing unit 2, and the prediction residual {e _(o) } is supplied to the residual power extraction unit 3, the residual correlation processing unit 4, and the voiced/unvoiced determination unit 10. be done.

The illustrated units 3, 4, 5, 6, 7, 8, 9 are:
This corresponds to the pitch period extraction section when the present invention is applied. The pitch period extracted by the pitch period extraction section is supplied to the voiced section drive sound source generation section 11 during synthesis. Then, a vibration component corresponding to the pitch period and the electric power from the extractor 3 is generated in the form of a pulse or a triangular wave. Furthermore, the voice section driven sound source generating section 12 generates power corresponding to the unvoiced section in the form of white noise.

The voice/unvoiced determining section 10 determines voiced sections and unvoiced sections, and performs switching processing in the switching processing section 13 during synthesis. Then, the linear prediction synthesis unit 14
adds parameters related to the vocal tract to the output from the switching processing unit 13 to generate synthesized speech {y _(o) }
Output as .

An important feature of the present invention resides in the pitch period extraction section shown in units 3, 4, 5, 6, 7, 8, and 9 in FIG. 2, and will be specifically described below.

When the prediction residual {e _(o) } is extracted as shown in FIG. 2, the residual power extraction unit 3 extracts the residual {e _(o) }
of electric power is extracted. On the other hand, the residual correlation processing section 4 calculates autocorrelation for the residuals. That is, the prediction residual is set as {e _(o) } ^N-1 _o=0 , and the autocorrelation coefficient for the time delay i of the prediction residual of the m-th frame is ρ _i (m)={1/N _Ni-1 〓 ⁿ⁼⁰ e _(o)・e _(o+i) }/{1/N _N-1 〓 ⁿ⁼⁰ e ² _(o) } -(1) (However, i=0, 1, ..., N-1), a series of autocorrelation coefficients ρ _i (m) is extracted.

Each coefficient ρ _i (m) may undesirably have extreme unevenness when viewed in time series, and the moving average processing unit 5 calculates, for example, ρ′ _i (m)=1/3 {ρ _{i -1} (m)+ρ _i (m) +ρ _i+1 (m)} -(2) However, the moving average is taken in the form iε[Tmin, 2T.max]. Note that the above Tmin and 2Tmax will be described later. In the configuration shown in FIG. 2, the autocorrelation coefficient ρ′ _i (m) obtained by taking the above-mentioned moving average is used, but in the following explanation, for the sake of simplicity, the coefficient ρ′ _i (m) is simply used as a coefficient. It is written as ρ _i (m).

The maximum value extraction processing section 6 shown in FIG. 2 selects, for example, five larger values of the coefficient ρ _i (m) for each frame, and stores them in the pitch period determination candidate holding section 7. At the same time, the time delay i corresponding to each selected coefficient is stored in the time delay component holding unit 8. That is, when the range of time delay i in which the pitch period is considered to exist in the correct form is [Tmin, Tmax], in order to deal with the case where the pitch in the original voice is missing, etc.
The search range of time delay i in equation (1) above is [Tmin,
2Tmax], but within this time delay i range, the autocorrelation coefficient of the prediction residual {ρ _i (m)} ^2Tmax _i=Tnio, five candidates in descending order of the value {ρ ^(j) (m)} ⁵ Select _j=1 . Then, the time delay {τ ^(j) (m)} ⁵ _j=1 corresponding to this is selected. Expressing this selection in a formula, in general, ρ ^(j) (m) = max ρ _i (m) −(3) iε[Tmin, 2Tmax] i≠τ ^(k) (m), 1≦k≦j− 1 1≦j≦5 τ ^(j) (m)=arg i max ρ _i (m) −(4) iε[Tmin, 2Tmax] i≠τ ^(k) (m), 1≦k≦j−1 1 It is expressed as ≦j≦5. Note that the above arg i is a function that takes the time delay i that gives ρ ^(j) as a value.

As above, {ρ ₍₁₎ (m), ρ ₍₂₎ (m), ..., ρ ₍₅₎ (m)
} {τ ₍₁₎ (m), τ ₍₂₎ (m), ..., τ ₍₅₎ (m)
} is chosen, and each ρ ^(j) (m) above is the corresponding time delay τ ^(j)
It can be considered that this corresponds to the reliability when (m) is regarded as the pitch period.

The logic judgment processing unit 9 shown in FIG. 2 determines the pitch period for each frame based on the selected ρ ^(j) (m) and τ ^(j) (m). The processing in the processing unit 9 is as shown in FIG. 3, and as shown schematically in FIGS. 4 to 7, it consists of each process of the first pass, second pass, third pass, and fourth pass. It's on. This will be explained below.

Assume that the current input audio is as shown in FIG. 3A when the frame number is plotted on the horizontal axis and the above-mentioned first reliability value is plotted on the vertical axis. As shown in FIG. 3B, the input audio includes a silent section, a voiceless section, a voiced section, a silent section,
Assume that there are voiced sections and unvoiced sections.

In the logic judgment processing unit 9, in the first pass, as shown in ^{FIG .} For each frame that satisfies m)≧θ〓 −(5), the pitch period is tentatively determined. Such decisions are made in chronological order as indicated by the arrows in the figure, and then also in the opposite direction of the arrows. FIG. 3C shows the processing corresponding to the first pass. That is, if we consider processing that corresponds to the order of time series, we set the pitch period corresponding to the first frame as τ (m) = τ ⁽¹⁾ (m) − (6), and then is such that it satisfies the condition that the pitch period deviation is within a predetermined width θ〓 (referred to as continuous) |τ ^F (m)−τ ^F (m−1)|≦θ〓 −(7) Let the pitch period be {τ ⁽¹⁾ (m), ..., τ ^(j) (m), ..., τ ⁽⁵⁾ (
m)} are selected in descending order of j. If there is no frame that satisfies Equation (7), the pitch period is not determined for the frame and the process proceeds to the next frame. The pitch period for the next frame is determined according to equation (6). To process in the opposite direction to the time series, consider τ ^B (m) instead of τ _F (m) and convert the time as m'=M−m+1. Then, for the frame in which τ ^F (m) and τ ^B (m) are sought,
The pitch period τ(m) is found from τ(m)=min{τ ^F (m), τ ^B (m)} −(8), and for frames for which only one is found, the pitch of the found and current one is Let the period be τ(m). In other cases, the pitch period τ(m) is not determined for the frame.

In the second pass, the pitch period of each frame determined in the first pass is checked, and if there is a pitch period undetermined portion in three frames or less as shown in FIG.
The pitch period of point Q ((m+n+1)th frame) with respect to the pitch period of (mth frame) is |τ(m+n+1)−τ(m)|≦√(+1)・θ〓
-(9) is selected from among the five candidates mentioned above. If there is no corresponding one for the five items, the value of τ ^(j) (m)/2 (corresponding to half the pitch period of the previously obtained candidate)
I'll also look into it.

The above processing can also be performed in the opposite direction to the time series by converting the time as m' = M - m + 1, and then converting the time to point P and point Q.
If a smaller value can be found as the pitch period of the frame, it is extracted.

FIG. 4 and FIG. 5 are explanatory diagrams showing the processing for the first pass and the second pass, respectively, in the form of a schematic flowchart. In the flowchart shown in FIG. 4 corresponding to the first pass, for a voiced frame with high reliability,
The pitch period of the first frame is set as the first candidate, and candidates that maintain continuity for subsequent frames are selected as the pitch period of the frame. In the flowchart shown in FIG. 5 corresponding to the second pass, frames that are reliable and have a distance of 3 frames or less are processed to maintain continuity.

In the third pass, for a voiced frame whose pitch period has not yet been determined, as shown in FIG. The pitch period for each frame is determined by selecting from among the five candidates selected with low reliability so that the pitch period is continuous as shown in FIG.

FIG. 6 is an explanatory diagram showing the processing for the third pass in the form of a general flowchart. Find the beginning and end of a group of frames with low reliability,
The pitch period is determined for each frame within the frame group while maintaining continuity. During this time, taking into consideration that the value corresponding to the original pitch period may be dropped, the value of τ ^(j) (m)/2 is also considered as a candidate, as in the case of the second pass above. be made to investigate.

In the fourth pass, pitch continuity is checked for the entire voiced frame, and if it is discontinuous, linear interpolation, for example, is performed from the pitch periods of the previous and subsequent frames. FIG. 7 shows a schematic flowchart in that case.

Needless to say, in the above process, the constant
Tmin, Tmix, θ〓, θ〓, etc. are determined in preliminary experiments.

FIG. 8 shows details of the first pass process shown in FIG. 4. Reference numeral 15 in the figure corresponds to processing related to voiced frames, 16 corresponds to processing related to frames with high reliability, 17 corresponds to processing when the previous frame is silent/unvoiced, and 18 corresponds to processing related to continuous pitch periods. 19 corresponds to processing when the reliability of the previous frame is low.

FIG. 9 shows details of the second pass process shown in FIG. The reference numeral 20 in the figure corresponds to processing for the case where the distance is within three frames, 21 corresponds to processing for checking extended continuity as shown in equation (9), and 22
corresponds to the process of correcting the pitch period so as to maintain the extended continuity.

FIGS. 10A and 10B show details of the third pass process shown in FIG. Code 23 in the diagram
24 corresponds to the process of detecting a switch from a silent/silent frame to a frame with low reliability, and 24 corresponds to the process of detecting a switch from a frame with high reliability to a frame with low reliability, Both processes 23 and 24 detect the beginning of a frame with low reliability. 25 corresponds to a process of detecting a switch from a frame with low reliability to a silent/unvoiced frame. Further, 26 corresponds to the process corresponding to process A shown in FIG. 27 is the 6th
This corresponds to the process corresponding to process B shown in the figure.
Further, 28 corresponds to a process for detecting a switch from a frame with low reliability to a frame with high reliability, 29 corresponds to process C shown in FIG. 6, and 30 corresponds to process C shown in FIG. ing.

FIG. 11 shows the processes 26, 27, shown in FIG.
29 and 30 show details of sub-passes that perform each process. 31 in the figure corresponds to the process of selecting from candidates, and 32 is the above-mentioned τ ^(j)
(m)/2, 33 corresponds to the process when the previous frame was silent/unvoiced, 34 corresponds to the process when the previous frame was voiced, and 35 corresponds to the process when the previous frame was voiced. 36 corresponds to a process for checking whether τ ⁽¹⁾ /2 maintains continuity.

FIG. 12 shows details of the fourth pass process shown in FIG. Code 37 in the diagram
corresponds to processing for a voiced frame, 38 corresponds to processing when the previous frame is silent/unvoiced, 39 corresponds to processing when the subsequent frame is also silent/unvoiced, and 40 corresponds to processing when the subsequent frame is silent/unvoiced. 41 corresponds to processing when the previous frame is voiced, 42 corresponds to processing when the subsequent frame is silent/unvoiced, and 43 corresponds to processing when the subsequent frame is also voiced. It supports processing (such as interpolation).

(E) Effect of the Invention As explained above, according to the present invention, when determining the pitch period, it is possible to rationally select the pitch period from among a plurality of candidates even if the pitch fluctuates or falls off. It becomes possible to select and decide, and it becomes possible to eliminate the need for manual supplementary processing to determine the pitch period, making it possible to completely automate the speech analysis and synthesis processing.

In addition, in the above description, although the moving average of the autocorrelation coefficient sequence of the prediction residual was considered, the moving average is not limited to this, and other weighted averages may be considered.

[Brief explanation of drawings]

FIG. 1 is an explanatory diagram for explaining the concept of a frame referred to in this specification, FIG. 2 is a block diagram of an embodiment of the speech analysis and synthesis apparatus of the present invention, and FIGS. An explanatory diagram illustrating the processing mode in the logical judgment processing section, FIG. 8 is a flowchart of an embodiment of the processing related to the first pass in the logical judgment processing section, and FIG. 9 is the processing related to the second pass in the logical judgment processing section. A flowchart of an embodiment; FIGS. 10A and 10B together show a flowchart of an embodiment of the processing related to the third pass in the logical judgment processing unit; FIG. 11 is a flowchart of the processing related to the third pass in the logic judgment processing unit FIG. 12 shows a flowchart of an embodiment of the process related to the fourth pass in the logic judgment processing section. In the figure, 1 is an input audio correlation processing section, 2 is a linear prediction processing section, 3 is a residual power extraction section, 4 is a residual correlation processing section, 5 is a moving average processing section, 6 is a maximum value extraction processing section, 7 8 is a pitch period determination candidate holding unit, 8 is a time delay component holding unit, 9 is a logical judgment processing unit, 10 is a voiced/unvoiced judgment unit, 11 is a voiced section drive sound source generation unit, and 12 is a silent (including silent) section drive Reference numeral 13 represents a sound source generation section, numeral 13 represents a switching processing section, and 14 represents a linear prediction synthesis section.

Claims (1)

[Claims] 1 A speech analysis and synthesis device that analyzes input speech using a linear prediction method and converts it into parameters, extracts pitch periods using the autocorrelation coefficient of the prediction residual, and synthesizes speech based on these results. In this step, examine the autocorrelation coefficient sequence of the prediction residual or the autocorrelation coefficient sequence of the weighted moving average of the autocorrelation coefficient of the prediction residual and extract multiple candidates with larger correlation values. A pitch cycle determination candidate extraction unit, a time delay component extraction unit that extracts a time delay corresponding to each pitch cycle determination candidate, and a pitch cycle determination candidate extracted by the pitch cycle determination candidate extraction unit and the corresponding time delay. and a logical judgment processing unit that determines the pitch period based on the pitch period, and the logical judgment processing unit determines the reliability with which the corresponding time delay component can set the pitch period based on the value of the pitch period determination candidate. , and using the time delay component corresponding to the pitch period determination candidate with reliability above the threshold, determine the pitch period for the period with reliability above the threshold, and then determine the pitch period for the period with reliability below the threshold. 1. A speech analysis and synthesis device, characterized in that the pitch period of the voice is determined in order to maintain continuity with the previously determined pitch period.