JP6260227B2 - Speech synthesis apparatus and method - Google Patents

Description

The present invention relates to a voice generation apparatus capable of reproducing a voice in which the degree of familiarity felt by a person who has heard the voice is increased, and a voice synthesizer capable of increasing or decreasing the degree of familiarity in which the person who has heard the voice feels. And a method.

  In recent years, voice guidance that explains an operation method of an electronic device to an operator by voice has become widespread. The voice used for voice guidance is often a voice that has a flat prosody and no emotion. Patent Document 1 describes a speech synthesizer that adds emotion to emotionless speech.

JP-A-7-72900

A speech synthesizer described in Patent Document 1 which is a conventional speech synthesizer makes an emotional voice by learning to convert an emotional voice parameter into an emotional parameter using a neural network. Give emotions. Therefore, the speech synthesizer described in Patent Document 1 has a problem that a complicated configuration and procedure are required.

The present invention has been made in view of such problems, and an object thereof is to provide a voice synthesizing apparatus and method capable of effectively increasing size of the degree of familiarity of the voice easy easy Do construction and procedure.

The present invention, by a voice including prosody information the frequency was above temperature by a predetermined frequency only the last clause in the text consisting of a plurality of clauses, causes increase of the degree of familiarity of a sound felt by the human.

According to the speech synthesis apparatus and method of the present invention, it is possible to effectively increase large degree of familiarity of the speech with a simple configuration and procedure.

It is a block diagram which shows the speech generator and speech synthesizer of 1st Embodiment. It is a block diagram which shows the speech generator and speech synthesizer of 2nd Embodiment. It is a block diagram which shows the speech generator and speech synthesizer of 3rd Embodiment. It is a figure which shows the phrase and phoneme which comprise the amplitude waveform of the 1st example sentence for demonstrating the speech synthesis method of embodiment, and a 1st example sentence. It is a figure which shows the phrase and phoneme which comprise the amplitude waveform of the 2nd example sentence for demonstrating the speech synthesis method of embodiment, and a 2nd example sentence. It is a figure which shows the example which raises the frequency after the vowel of the first phoneme of the last clause in a sentence. It is a figure which shows the characteristic of an amplitude and frequency when raising the frequency of the last clause in a text. It is a figure for demonstrating the case where the pole or inflection point nearest to the first phoneme of the last phrase in the frequency characteristic indicated by the prosody of the sentence is set as a start position for increasing the frequency. It is a figure which shows the evaluation result for demonstrating the relationship between the grade which raises a frequency only by a fixed frequency, and an effect.

Hereinafter, a speech generation device, a speech synthesis device, and a method according to each embodiment will be described with reference to the accompanying drawings. The speech generation device, speech synthesis device, and method of each embodiment can increase the degree of familiarity of speech by adding familiarity to speech. In addition, the speech generation device, speech synthesis device, and method of each embodiment can reduce the degree of familiarity of speech. In the following description, the operation for increasing the degree of familiarity of sound will be mainly described.

<Speech generator and speech synthesizer of first embodiment>
The speech generator and speech synthesizer of the first embodiment shown in FIG. 1 are configuration examples that increase the degree of familiarity of speech when generating speech data by speech synthesis. The speech generator and speech synthesizer of the first embodiment are speech synthesizers that generate speech data based on text data indicating a sentence.

  In FIG. 1, text data indicating a predetermined sentence is input to a prosody information generation unit 11, a prosody information correction unit 13, and a synthesis unit 14. The text data is, for example, an ASCII code.

  The prosodic dictionary 12 holds a plurality of prosodic information patterns. Prosodic information is a part other than the voice quality in speech, and is a part that forms accents, rhythms, and the like. The prosody information generation unit 11 reads prosody information of a pattern suitable for each clause of the sentence of the input text data from the prosody dictionary 12, and generates the prosody information of the sentence. The prosody information is input to the prosody information correction unit 13.

  For example, the sentence indicated by the text data is a sentence “I will set…”, and between the phrase “…” and the phrase “setting”, between the phrase “setting” and the phrase “I will” It is assumed that a short interval corresponding to this time is provided. In this case, the text data itself may be provided with an interval, or the prosodic information generation unit 11 may generate prosodic information with the interval provided.

  The prosodic information correction unit 13 corrects the prosodic information so as to increase the degree of familiarity of the speech. A specific method of correcting the prosodic information in the prosodic information correcting unit 13 will be described in detail later. The modified prosody information is input to the synthesis unit 14.

  The sound path dictionary 15 holds a plurality of sound path information patterns. The sound path information is a part of voice quality in speech. The sound path dictionary 15 may hold a pattern of sound path information in units of sentences, may be stored in units of words, or may be stored in units of phonemes.

  The synthesizing unit 14 reads out the sound path information of a pattern suitable for the text of the input text data, and synthesizes the corrected prosodic information and the sound path information to generate sound data of a digital signal. The audio data is converted into an analog signal by the D / A converter 16 and output as sound from the speaker 17.

Sound generating equipment constructed in accordance with the speech synthesis device shown in FIG. 1, a portion of the prosodic information generation unit 11 to the sound path dictionary 15, be constituted by a microcomputer including a processing unit (microprocessor) and a memory device it can.

<Speech generator and speech synthesizer of second embodiment>
The speech generation device and speech synthesis device according to the second embodiment shown in FIG. 2 is a configuration example that increases the degree of familiarity of speech when speech data is previously formed as a speech file. The speech generator and speech synthesizer of the second embodiment are speech that corrects the prosodic information of speech data based on a speech file that includes speech data of a sentence, text data that indicates the sentence, and timing data of the text data. It is a processing device.

  In FIG. 2, the prosody / vocal tract separation unit 21 receives an audio file including audio data of sentences. The audio file is, for example, in the WAV format. The audio file is not limited to the WAV format.

  The prosody / vocal tract separation unit 21 separates audio data of the audio file into prosody information and vocal tract information. The vocal tract information holding unit 22 holds the vocal tract information. The prosodic information holding unit 23 holds prosodic information.

  Text data indicating the voice data of the voice file and timing data are input to the correction position detection unit 26. The timing data indicates the time position of the audio data. With the timing data, it is possible to set an interval corresponding to the breathing time between phoneme utterance start positions and phrases. The correction position detection unit 26 detects a correction position when the prosody information correction unit 24 corrects the prosody information based on the text data and the timing data.

  The prosodic information correction unit 24 increases the degree of familiarity of the voice by correcting the prosodic information from the correction position detected by the correction position detection unit 26. The modified prosody information is input to the synthesis unit 14. A method of detecting the correction position in the correction position detection unit 26 and a specific correction method in the prosody information correction unit 24 will be described in detail later.

  The synthesizer 14 synthesizes the modified prosody information and the vocal tract information held in the vocal tract information holding unit 22 to generate voice data of a digital signal. The audio data is converted into an analog signal by the D / A converter 27 and output as audio from the speaker 28.

The speech generator and speech synthesizer configured by the speech processing device shown in FIG. 2 includes the prosody / vocal tract separation unit 21 to the correction position detection unit 26 as a microcomputer including an arithmetic processing unit and a storage device. be able to.

<Speech Generator and Speech Synthesizer of Third Embodiment>
The speech generator and speech synthesizer of the third embodiment shown in FIG. 3 is a configuration example that increases the degree of familiarity of speech spoken by a person. 3, the same parts as those in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted as appropriate.

The speech generator and speech synthesizer of the third embodiment are based on speech data obtained by collecting speech of a sentence uttered by a person with a microphone, and text data indicating a sentence generated by speech recognition of the speech data. This is a speech processing device for correcting prosodic information of speech data.

  In FIG. 3, a microphone 31 picks up a voice uttered by a person and outputs an analog voice signal. The A / D converter 32 converts an analog audio signal into digital audio data. The voice data is input to the prosody / vocal tract separation unit 21 and the voice recognition unit 33.

  The voice recognition unit 33 recognizes the voice of the input voice data and outputs text data. The text data is input to the correction position detector 34. The correction position detection unit 34 detects a correction position when correcting the prosodic information using, for example, a morphological analysis technique. The prosodic information correction unit 24 increases the degree of familiarity of the voice by correcting the prosodic information from the correction position detected by the correction position detection unit 34.

The speech generator and speech synthesizer configured by the speech processing device shown in FIG. 3 includes parts other than the microphone 31, the A / D converter 32, the D / A converter 27, and the speaker 28, an arithmetic processing device, a storage device, and the like. It can comprise with the microcomputer containing.

<Speech Synthesis Method of Embodiment>
The prosody information correction unit 13 in FIG. 1, the prosody information correction method in the prosody information correction unit 24 in FIGS. 2 and 3, and the correction position detection unit 26 in FIG. 2 and the correction position in the correction position detection unit 34 in FIG. A detection method will be described.

  FIG. 4A shows an amplitude waveform when the first example sentence, “I will make a stopover”, is generated. As shown in Fig. 4 (b), Ke, I, Yu, Chi, Ni, Shi, Ma, and Su of "KeIYuChiNiShiMaSu" with "Make a stopover" in Roman letters are phonemes numbered 1-8. Show. The phonemes with phoneme numbers 1 to 8 are located at respective time positions from 2.22 seconds to 2.85 seconds, for example.

“KeIYuChi” is the phrase Ph1, “Ni” is the phrase Ph2, and “ShiMaSu” is the phrase Ph3. In the speech synthesis method of the embodiment, when generating speech of a sentence having a plurality of phrases, only the last phrase in the sentence is increased in frequency by a certain frequency, thereby increasing the degree of familiarity of the speech.

  In the example shown in FIG. 4B, the prosody information correction unit 13 in FIG. 1 and the prosody information correction unit 24 in FIGS. 2 and 3 increase the frequency of the part “ShiMaSu” that is the phrase Ph3 by a certain frequency. Let The correction position detection unit 26 in FIG. 2 and the correction position detection unit 34 in FIG. 3 detect the phrase Ph3 that is the last phrase.

  FIGS. 5A and 5B show another example. FIG. 5A shows an amplitude waveform when a voice “Please enjoy yourself slowly” is generated as a second example sentence.

  As shown in Fig. 5B, "YuKkuRiTaNoShiNDeKiTeKuDaSaINe" Yu, Kku, Ri, Ta, No, Shi, N, De, Ki, Te, Ku, Da , Sa, I, Ne indicate phonemes with phoneme numbers 1 to 15, respectively. The phonemes with the phoneme numbers 1 to 15 are located at the respective time positions from 2.22 seconds to 3.49 seconds, for example.

  “YuKkuRi” is the phrase Ph1, “TaNoShiNDe” is the phrase Ph2, “KiTe” is the phrase Ph3, and “KuDaSaINe” is the phrase Ph4. In the example shown in FIG. 5B, the prosody information correction unit 13 in FIG. 1 and the prosody information correction unit 24 in FIGS. 2 and 3 perform the frequency of the “KuDaSaINe” portion of the phrase Ph4 which is the last phrase in the sentence. Is increased by a certain frequency to increase the degree of familiarity of the voice.

  In FIG. 5B, the time position of the phoneme “Ku” in the phrase Ph4 is set to 3.12. This time position 3.12 indicates the position of the consonant K. Therefore, the frequency after the consonant K rises in the “KuDaSaINe” portion of the phrase Ph4.

  In the example shown in FIG. 6, the time position of the consonant K of the phoneme “Ku” is set separately as 3.12 and the time position of the vowel u is set as 3.13 separately. In such a case, the frequency after the vowel u in the phoneme “Ku” in the “KuDaSaINe” portion of the phrase Ph4 may be increased.

  To increase the frequency by a certain frequency only for the last phrase in the sentence means to increase the frequency after the first phoneme consonant of the last phrase in the sentence and to increase the frequency after the vowel. Including.

  FIGS. 7A and 7B show amplitude and frequency characteristics when the frequency of the phrase Ph4 in FIG. 5B is increased. Black dots shown in FIG. 7B indicate extreme points or inflection points indicating the maximum value or the minimum value in the frequency characteristics. The position of the black dot does not necessarily match the position of the phoneme. However, in many cases, the position of the phoneme is a pole or an inflection point.

  (A), (b) of FIG. 7 has shown the case where the frequency after the consonant K in phoneme "Ku" is raised. In FIG. 7B, the broken line indicates the characteristic when the frequency is not increased, and the solid line indicates the characteristic when the frequency is increased. Here, an example is shown in which the frequency of the prosody of “KuDaSaINe” is uniformly increased by 40 Hz.

  Increasing the frequency does not affect the amplitude characteristics shown in FIG. Therefore, there is no change in the amplitude characteristics between the state where the frequency is not increased and the state where the frequency is increased.

A more detailed and preferred speech synthesis method for increasing the frequency of the last phrase in a sentence will be described with reference to FIG. In this example, the part of “... it was ...” in the sentence “I will set…” is taken as an example. It is assumed that the sound prosody of the portion “... Was ...” has a frequency characteristic as shown in FIG. Here, frequency characteristics are schematically shown for simplification.

  8A to 8C, black circle points p1 to p6 indicate pole points or inflection points as in FIG. The positions of the points p1 to p6 do not necessarily coincide with the positions of the phonemes, but the positions of the phonemes are often extreme points or inflection points.

  The prosodic information correcting unit 13 in FIG. 1 and the prosodic information correcting unit 24 in FIGS. 2 and 3 are positions closest to the first phoneme among a plurality of phonemes constituting the last phrase, and the frequency indicated by the prosody of the sentence The characteristic pole or inflection point can be the starting position for increasing the frequency by a certain frequency.

  In the example of FIG. 8A, the pole or inflection point closest to “I”, which is the first phoneme of “Ita ...”, is the point p6. The prosodic information correcting unit 13 in FIG. 1 and the prosodic information correcting unit 24 in FIGS. 2 and 3 increase the frequency from the point p6 with the point p6 as the starting position of the frequency increase.

  FIG. 8B shows a state in which the frequency after the point p6 is increased. As the frequency increases, point p6 moves to point p6 '.

  In the frequency characteristics as shown in FIG. 8B, the frequency changes abruptly. Therefore, it is preferable to continuously change the frequency from a position a predetermined time before the start position (here, point p6 (p6 ')) where the frequency is increased by a certain frequency to the start position. The position a predetermined time before the start position may also be a pole or an inflection point.

  In the example of FIG. 8C, the position before the predetermined time is a point p5 that is a pole or an inflection point located before the start position. The frequency may be linearly changed so that the frequency continuously increases from the point p5 to the point p6 ', or may be changed to an upward convex curve or a downward convex curve.

  Here, if there is a time of 0.05 seconds or more from the point p5 to the point p6 ', it is easy to hear naturally. Therefore, when the time interval between the pole or inflection point that is the start position and the pole or inflection point located immediately before the start position is less than 0.05 seconds, A pole or inflection point having a time interval of 0.05 seconds or more is preferably selected.

  To increase the frequency by a certain frequency only for the last phrase in a sentence means to increase the frequency after a pole or inflection point closest to the first phoneme among the phonemes making up the last phrase by a certain frequency including.

  When the sentence is composed of only one phrase such as “Yes” or “Sorry”, only the last phrase described above is not set as a target for increasing the frequency.

  The relationship between the effect of increasing the frequency by a certain frequency and the effect will be described with reference to FIG. The experimental results shown in FIG. 9 show that only the last phrase in a sentence is heard by a plurality of people listening to voices whose frequencies are increased from +10 Hz to +100 Hz, and the degree of familiarity is evaluated. It is the total. In FIG. 9, ◯ indicates that the effect of increasing the degree of familiarity is high, Δ indicates that the effect is moderate, and x indicates that there is no effect.

  When the frequency is increased by +10 Hz, the sound becomes somewhat familiar, but the effect is slight. If the frequency is lower than +10 Hz, it is difficult for a person to feel the effect. Therefore, the lower limit value for increasing the frequency by a certain frequency is +10 Hz. On the other hand, when the frequency is increased by +90 Hz, it becomes a mechanical sound, and on the contrary, the familiarity is lowered. Therefore, the upper limit value for increasing the frequency by a certain frequency is +80 Hz.

  Thus, +10 Hz to +80 Hz is preferable for increasing the frequency of the last phrase in the sentence by a certain frequency. At +10 Hz to +20 Hz, the sound becomes somewhat familiar, and the effect is limited. At +60 Hz to +80 Hz, the sound becomes close, but some unnaturalness occurs. Therefore, +30 Hz to +50 Hz is particularly preferable for increasing the frequency of the last clause in the sentence by a certain frequency.

By the way, it has also been confirmed that when the frequency of only the last phrase in the sentence is lowered by a certain frequency, the degree of familiarity of the voice decreases. The speech generator, speech synthesizer, and method of each embodiment decrease the frequency of only the last phrase in a sentence by a certain frequency in order to intentionally reduce the degree of familiarity that a person feels when listening to the voice. It is also possible to make it.

As described above, the speech generation device and the speech synthesis device according to each embodiment include the prosodic information correction units 13 and 24 and the synthesis units 14 and 25. The prosodic information correction units 13 and 24 correct the prosodic information so that the frequency of only the last phrase of the sentence is increased or decreased by a certain frequency when the sentence composed of a plurality of phrases is expressed by speech. The synthesizing units 14 and 25 synthesize the prosodic information corrected by the prosody information correcting units 13 and 24 and the sound path information, thereby generating sentence voice data.

According to the speech generator and speech synthesizer of each embodiment, the degree of familiarity of speech can be effectively increased or decreased with a simple configuration.

  The prosodic information correction units 13 and 24 set the position of the consonant or vowel of the first phoneme among the plurality of phonemes constituting the last phrase as a start position for increasing or decreasing the frequency by a certain frequency. Thereby, the prosodic information can be corrected only for the last phrase.

  The prosodic information correction units 13 and 24 are positions closest to the first phoneme among a plurality of phonemes constituting the last phrase, and the frequency characteristic pole or inflection point indicated by the prosody of the sentence is set to a fixed frequency. It may be a starting position where the frequency is increased or decreased. Thereby, the prosodic information can be corrected only for the last phrase.

  At this time, it is preferable that the prosodic information correction units 13 and 24 continuously change the frequency from a position a predetermined time before the start position to the start position. In this way, there is almost no sense of incongruity, and the degree of familiarity of the voice can be increased or decreased.

  The prosodic information correction units 13 and 24 may set a position before a predetermined time as a pole or an inflection point located before the start position. In this way, it is possible to continuously change the frequency according to the change of the frequency characteristic.

The speech generator and speech synthesizer of the embodiment may be a speech synthesizer that generates speech data based on text data indicating a sentence. Speech synthesizer can generate voice data to increase or decrease the degree of familiarity of the speech by the speech synthesis.

The speech generation device and the speech synthesis device according to the embodiment include a speech processing device that corrects prosodic information of speech data based on a speech file including speech data of a sentence, text data indicating the sentence, and timing data of the text data. It may be. If the voice generator and the voice synthesizer are configured by a voice processing device that operates in this manner, the degree of familiarity of the voice data recorded as a voice file can be increased or decreased.

The speech generation device and the speech synthesis device according to the embodiment are based on speech data in which speech of a sentence uttered by a person is collected by a microphone and text data indicating a sentence generated by speech recognition of the speech data. It may be a speech processing device that corrects the prosodic information. If the voice generator and the voice synthesizer are configured by a voice processing device that operates in this way, it is possible to increase or decrease the degree of familiarity of voice uttered by a person.

The speech synthesis method according to the embodiment includes a prosodic information correction step and a synthesis step. The prosodic information correction step corrects the frequency of only the last phrase of the sentence in the prosodic information to be increased or decreased by a certain frequency out of the prosodic information and the sound path information constituting the speech data of the sentence composed of a plurality of phrases. .

  In the synthesis step, the prosody information corrected in the prosody information correction step and the sound path information are synthesized to generate voice data in which the degree of familiarity of the voice when the voice data of the sentence is pronounced is changed. To do.

According to the speech synthesis method of the embodiment, the degree of familiarity of speech can be effectively increased or decreased by a simple procedure.

The present invention is not limited to the speech generator, speech synthesizer, and method of each embodiment described above, and various modifications can be made without departing from the scope of the present invention.

The voice generator shown in FIGS. 1 to 3 has a configuration including a voice synthesizer . Sound generation device is not equipped with a voice synthesizer may be provided a voice synthesizing apparatus to an external sound generation device. The speech generator includes a storage unit that stores speech data generated to include prosodic information in which the frequency of only the last phrase of a sentence is increased or decreased by a certain frequency, and the speech data read from the storage unit The structure provided with the audio | voice reproduction | regeneration part to reproduce | regenerate may be sufficient.

  The D / A converter 16 and the speaker 17 in FIG. 1 and the D / A converter 27 and the speaker 28 in FIGS. 2 and 3 constitute at least a part of the sound reproducing unit. When the sound generation device includes a storage unit that stores sound data, a reading unit that reads out sound data from the storage unit may be part of the sound reproduction unit.

  As described above, the sound generation device includes a sound reproducing unit that reproduces sound data generated so as to include prosodic information in which only the last phrase of a sentence including a plurality of phrases is increased or decreased by a certain frequency. Just do it.

The voice generation device and the voice synthesis device may be configured by hardware, may be configured by software, or may be configured by mixing both.

  When separating and synthesizing speech into prosodic information and vocal tract, for example, a general speech analysis / synthesis system called vocoder can be used. As software, voice analysis software Praat can be used. Use the T-SOLA algorithm used in Praat.

The present invention, in the computer, equivalent to the prosody information correction step and the synthesis step in the speech synthesis method can also be implemented by a speech synthesis program for executing the synthetic steps with prosody information correction step.

13, 24 Prosodic information correction unit 14, 25 Compositing unit 16, 27 D / A converter (voice reproduction unit)
17, 28 Speaker (Audio playback unit)

Claims (11)

Upon a sentence including a plurality of clauses representing voice, and prosody information correction unit for correcting the prosodic information so as to upper rise by a predetermined frequency frequency only the last clause in the sentence,
A synthesizing unit that generates speech data of the sentence by synthesizing the prosodic information and the sound path information corrected by the prosody information correcting unit;
Equipped with a,
The prosody information correcting unit sets a position of a consonant or a vowel of the first phoneme among a plurality of phonemes constituting the last clause as a start position for increasing the frequency by a certain frequency.
Speech synthesis apparatus characterized by and this. An audio data input unit to which input audio data is input;
A speech recognition unit that recognizes speech from the input speech data and outputs text data;
A start position detecting unit for detecting the start position from the text data;
The speech synthesizer according to claim 1, further comprising: The speech synthesis apparatus according to claim 2, wherein the start position detection unit detects a phrase included in the text data. The speech synthesizer according to any one of claims 1 to 3, wherein the constant frequency is in a range of 10 to 80 Hz. Upon a sentence including a plurality of clauses representing voice, and prosody information correction unit for correcting the prosodic information so as to upper rise by a predetermined frequency frequency only the last clause in the sentence,
A synthesizing unit that generates speech data of the sentence by synthesizing the prosodic information and the sound path information corrected by the prosody information correcting unit;
Equipped with a,
The prosodic information correction unit is a position closest to the first phoneme of the plurality of phonemes constituting the last clause, and the frequency characteristic extreme point or inflection point indicated by the prosody of the sentence is set to a constant frequency. Start position to increase by frequency
Speech synthesis apparatus characterized by and this. The speech synthesizer according to claim 5, wherein the prosodic information correction unit continuously changes the frequency from a position a predetermined time before the start position to the start position. The speech synthesizer according to claim 6, wherein the prosodic information correction unit sets the position before the predetermined time as a pole or an inflection point positioned before the start position. The speech synthesizer, speech synthesis apparatus according to any one of claims 1,5～7 which is a speech synthesizer for generating the audio data based on the text data indicating the sentence. The speech synthesizer is a speech processing device that corrects prosodic information of the speech data based on a speech file including speech data of the sentence, text data indicating the sentence, and timing data of the text data. The speech synthesizer according to claim 1, wherein the speech synthesizer is configured as described above. The voice synthesizer is based on voice data obtained by collecting voice of a sentence uttered by a person with a microphone, and text data indicating the sentence generated by voice recognition of voice data collected by the microphone. The speech synthesizer according to claim 1, wherein the speech synthesizer is a speech processing device for correcting prosodic information of speech data. Of the prosody information and sound path information constituting the audio data of a sentence including a plurality of clauses, prosody information correction step of modifying so as to only the upper rise last constant frequency frequency only clause of the sentence in the prosodic information When,
Synthesizing the prosodic information corrected in the prosodic information correcting step and the sound path information to generate speech data in which the degree of familiarity of the speech when the speech data of the sentence is pronounced is increased A synthesis process;
A speech synthesis method comprising:

Images