JP5229738B2 - Speech recognition device and speech conversion device - Google Patents

Description

  The present invention relates to a speech recognition device and a speech conversion device.

  A voice recognition device that performs voice recognition on a voice signal collected by a microphone and a voice conversion device that emits a voice synthesis signal corresponding to a recognition result by the voice recognition device from a speaker are known.

  In speech recognition, a method of modeling using a probabilistic model such as a Hidden Markov Model (hereinafter referred to as HMM) for each recognition category of units such as syllables and words constituting recognition result candidates is a recognition performance. Is the mainstream of current speech recognition technology. A conventional speech recognition apparatus using an HMM will be briefly described with reference to FIG. The audio signal input from the input terminal 101 is converted into a digital signal by the A / D converter 102. The feature vector extraction unit 103 extracts a voice feature vector from the digital signal. Thereafter, for each recognition category, the HMM created for the speech unit is read from the model memory 104 in advance, and the likelihood calculation unit 105 calculates the matching likelihood of each model with respect to the extracted speech feature vector. Then, the speech unit (recognition category) expressed by the model having the largest matching likelihood is output from the output unit 106 as a recognition result.

  However, in the conventional speech recognition, since speech recognition is performed after the utterance is finished, there is a problem that a time lag occurs between the speech timing and the speech recognition timing.

  In addition, when a synthesized speech signal corresponding to the recognition result obtained by the conventional speech recognition method is emitted from the speaker, the synthesized speech is emitted at a timing considerably later than the timing when the speaker uttered. The problem arises that it becomes difficult for the speaker to proceed with the conversation well under the influence of the synthesized speech generated late.

  The present invention has been made to solve such a problem, and an object thereof is to provide a speech recognition device and a speech conversion device that can perform speech recognition at high speed.

  The object of the present invention is from voice input means for inputting voice, conversion means for converting a voice signal input to the voice input means to a digital voice waveform signal, and a digital voice waveform signal converted by the conversion means. A speech recognition apparatus comprising: analysis means for analyzing speech input to the speech input means, wherein the analysis means analyzes the digital speech waveform signal in units of frames and represents a feature amount of speech A feature vector extracting unit for extracting a frame, a feature vector storage unit for storing the feature vectors extracted in units of frames in a time series for a plurality of frames, and a recognition candidate speech storage unit for storing a plurality of voices as speech recognition candidates And the likelihood of each speech that is a speech recognition candidate based on the feature vectors for a plurality of frames stored in the feature vector storage unit A first analysis unit that calculates an average feature vector per frame from the feature vectors for the plurality of frames, and calculates a likelihood of speech that is a speech recognition candidate from the average feature vector; The speech for determining one speech based on the likelihood of each speech to be a speech recognition candidate calculated in the first analysis unit and the likelihood of each speech to be a speech recognition candidate calculated in the second analysis unit And a determination unit.

  In this speech recognition apparatus, at least one of the first analysis unit and the second analysis unit calculates the likelihood of speech that is a speech recognition candidate using a Viterbi algorithm or a neural network. Is preferred.

  In addition, the speech determination unit calculates the likelihood of each speech that is a speech recognition candidate calculated in the first analysis unit and the likelihood of each speech that is a speech recognition candidate calculated in the second analysis unit for each speech. It is preferable that the voice corresponding to the maximum value of the added value for each voice is used as the recognition result.

  In addition, the object of the present invention is achieved by a speech conversion device that includes the speech recognition device and a speech generation device that generates a synthesized speech corresponding to the speech that is determined as a recognition result by the speech determination unit.

  ADVANTAGE OF THE INVENTION According to this invention, the speech recognition apparatus and speech conversion apparatus which can perform speech recognition at high speed can be provided.

It is a block diagram which shows the audio | voice conversion apparatus which concerns on one Embodiment of this invention. It is a graph which shows an example of the waveform of the digital audio | voice waveform signal acquired when the sound of "Asahi" is uttered. It is the figure which represented typically the recognition result for every frame. It is a block diagram which shows the speech recognition apparatus using the conventional HMM.

  Hereinafter, actual forms of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing an audio conversion device 1 according to an embodiment of the present invention. As shown in FIG. 1, the speech conversion device 1 according to the present embodiment includes a speech recognition device 2 and a speech generation device 3. The voice generation device 3 is a device that generates synthesized voice corresponding to the voice recognized by the voice recognition device 2.

  The voice recognition device 2 includes a voice input unit 21, a conversion unit 22, and an analysis unit 23. The voice input means 21 is an input device that picks up the voice uttered by the speaker, and examples thereof include an acceleration pickup and a microphone that extract a solid propagation signal such as a body conduction sound including a bone conduction sound. it can.

  The conversion unit 22 is a device that converts the audio signal input to the audio input unit 21 into a digital audio waveform signal. Specifically, for example, it is a device that obtains a waveform signal in PCM (pulse code modulation) format by AD-converting an audio signal.

  The analysis unit 23 analyzes the voice input to the voice input unit 21, and includes a feature vector extraction unit 231, a feature vector storage unit 232, a recognition candidate speech storage unit 233, a first analysis unit 234, A second analysis unit 235 and a voice determination unit 236 are provided.

  The feature vector extraction unit 231 extracts a feature vector representing a feature amount of speech by analyzing the digital speech waveform signal in units of frames. The feature vector used in speech recognition includes a feature amount (MFCC: Mel Frequency Cepstrum Coefficient) and power of a cepstrum region. MFCC is extracted by applying a Mel-scale filter bank to the power spectrum obtained by FFT analysis of audio data for each frame, and performing discrete cosine transform (DCT) on the frequency spectrum converted power spectrum This is a parameter that represents the spectral envelope, and the details are “speech recognition system” (edited by Kiyoshi Hiroshi, Katsunobu Ito, Tatsuya Kawahara, Kazuya Takeda, Mikio Yamamoto, Ohm Publishing Co., Ltd .; ISBN4-274-13228-5) Explained.

  In speech recognition, 12-dimensional MFCC (MFCC1, MFCC2) obtained by performing discrete cosine transform on spectral features of input speech and executing three processes in the cepstrum domain (DC component removal, liftering processing, and cepstrum average removal) ,... MFCC12) and its first-order time derivative (ΔMFCC1, ΔMFCC2,. The

  The feature vector storage unit 232 stores feature vectors extracted in units of frames for a plurality of frames in time series. In the present embodiment, it is configured so that feature vectors for three frames can be sequentially stored in time series.

  The recognition candidate voice storage unit 233 is a storage unit that stores a plurality of voices that are voice recognition candidates in advance. For example, “a”, “i”, “u”, “e”, “o”, and the like. Information on voices and feature vectors corresponding to the respective voices is stored.

  The first analysis unit 234 is an analysis unit that calculates the likelihood of each speech that is a speech recognition candidate based on the feature vectors for a plurality of frames stored in the feature vector storage unit 232. More specifically, feature vectors for three frames (26 dimensions × 3 frames = 78 dimensions) stored in the feature vector storage unit 232 and feature vectors corresponding to each speech stored in the recognition candidate speech storage unit 233 Based on the information, a likelihood calculation for calculating which voice the input voice corresponds to is performed. As the likelihood calculation method, for example, a maximum likelihood path search using a Viterbi algorithm or a neural network can be exemplified.

The second analysis unit 235 is an analysis unit that calculates an average feature vector per frame unit from feature vectors for a plurality of frames, and calculates the likelihood of speech that is a speech recognition candidate from the average feature vector. The average feature vector is calculated based on Equations 1 to 4 shown below. Equation 1 is an equation for calculating an average cepstrum, Equation 2 is an equation for calculating an average Δ cepstrum, Equation 3 is an equation for calculating an average ΔPOW, and Equation 4 is an equation for calculating an average ΔΔPOW. Here, M in Equations 1 to 4 represents the number of frames. W _MFCC (j), WΔ _MFCC (j), WΔ _POW (j), and WΔΔ _POW (j) are weighting coefficients attached to each frame, for example, arbitrary values from 0.0 to 1.0 Constants can be set.

  In order to calculate the likelihood of speech as a speech recognition candidate from the average feature vector calculated from the above mathematical formula, the average feature vector (26 dimensions) and the recognition candidate speech storage unit 233 are calculated in the same manner as the first analysis unit 234. Based on the feature vector information corresponding to each voice stored in, a likelihood calculation for calculating which voice the input voice corresponds to is performed. As the likelihood calculation method, for example, a maximum likelihood path search using a Viterbi algorithm or a neural network can be exemplified. The second analysis unit 235 is preferably configured to perform likelihood calculation on the top N recognition results recognized by the first analysis unit 234. For example, the number of N is “2”, and likelihood calculation is performed based on the top two voices among the voices recognized by the first analysis unit 234.

  The voice determination unit 236 uses one likelihood based on the likelihood of each voice to be a voice recognition candidate calculated in the first analysis unit 234 and the likelihood of each voice to be a voice recognition candidate calculated in the second analysis unit 235. Determine the voice. Specifically, the speech is determined according to the following Formula 5, and the likelihood of each speech that is a speech recognition candidate calculated by the first analysis unit 234 and the second analysis unit 235 calculate the speech. The likelihood of each voice that is a voice recognition candidate is added for each voice, and a voice corresponding to the maximum value of the added value for each voice is determined as a recognition result.

Formula 5 is a formula for calculating a speech having the largest value calculated by {L1 _s · λ + L2 _s } as a solution, and L1 is a speech likelihood representing a recognition result recognized by the first analysis unit 234. Yes, L2 is the likelihood obtained by the second analysis unit 235 for the same speech as the speech recognized by the first analysis unit 234. Λ is a weighting factor.

  The operation of the speech conversion apparatus 1 configured as described above will be described below. First, after the voice generated by the speaker is picked up by the voice input means 21, the conversion means 22 performs AD conversion on the picked-up voice signal and converts it into a PCM (pulse code modulation) format digital voice waveform signal. . For example, FIG. 2 shows a waveform of a digital speech waveform signal acquired when a speaker utters a voice “Asahi”.

  Next, the feature vector extraction unit 231 of the analysis unit 23 analyzes the digital speech waveform signal in units of frames and extracts a total 26-dimensional feature vector representing the feature amount of speech. The extracted feature vectors are stored in the feature vector storage unit 232 for a plurality of feature vectors, for example, three frames, for each frame.

  Then, the first analysis unit 234 corresponds to the feature vectors for three frames (26 dimensions × 3 frames = 78 dimensions) stored in the feature vector storage unit 232 and each voice stored in the recognition candidate speech storage unit 233. Based on the feature vector information, the likelihood of which input speech corresponds is calculated by, for example, the Viterbi algorithm. As a result of the likelihood calculation, the likelihood of the voice “a” is 0.80, the likelihood of the voice “i” is 0.20, the likelihood of the voice “u” is 0.30, and the voice “e”. , The likelihood of the speech “o” is 0.40, and the likelihood of silence is 0.10.

  Next, the second analysis unit 235 calculates an average feature vector from the feature vectors for three frames based on the above formulas 1 to 4. Then, based on the average feature vector (26 dimensions) and the feature vector information corresponding to each speech stored in the recognition candidate speech storage unit 233, which speech the input speech corresponds to, for example, a Viterbi algorithm The likelihood is calculated by In the likelihood calculation, for example, the second analysis unit 235 performs likelihood calculation for the top two recognition results recognized by the first analysis unit 234. Since the top two results recognized by the first analysis unit 234 are “a” and “e”, likelihood calculation is performed on these. As a result of the likelihood calculation, it is assumed that the likelihood of the speech “a” is 0.90 and the likelihood of the speech “e” is 0.60.

After that, the speech determination unit 236 is based on the likelihood of each speech that is a speech recognition candidate calculated by the first analysis unit 234 and the likelihood of each speech that is a speech recognition candidate calculated by the second analysis unit 235. Then, one voice is determined from Equation 5 above. When the weighting factor λ in Equation 5 is “1”, {L1 _s · λ + L2 _s } in Equation 5 is such that “a” is {0.80 × 1 + 0.90} = 1.70. On the other hand, “e” is {0.65 × 1 + 0.6} = 1.25. Formula 5 calculates a voice having a large numerical value among 1.70 and 1.25, and therefore calculates “a” as a final recognition result.

  The calculation is repeated while shifting such speech recognition frame by frame, and speech recognition is performed for all the frames of the digital speech waveform signal. FIG. 3 schematically shows the recognition result for each frame. In FIG. 3, □ represents one frame, and the notation “none” represents silence.

  Thereafter, the voice generation device 3 generates a synthesized voice corresponding to “a” calculated by the voice determination unit 236 as a recognition result, and emits the sound from the speaker.

  Since the speech recognition apparatus 2 according to the present embodiment can perform speech recognition on a frame basis as described above, at the initial stage of the time when the sound “a” of the word “Asahi” is being emitted, Since the uttered voice can be recognized as “A”, the time lag between the utterance timing and the voice recognition timing can be shortened, and the grace can be recognized at high speed.

  Further, when a speech synthesis signal corresponding to the recognition result obtained by the speech recognition apparatus 2 according to the present embodiment is emitted from the speaker, the time lag can be shortened between the speech timing and the speech recognition timing. , It will not be emitted at a timing much later than the time when the speaker uttered, so that it will be difficult for the speaker to be able to advance the conversation well under the influence of synthesized speech generated late It can be effectively suppressed.

DESCRIPTION OF SYMBOLS 1 Voice conversion apparatus 2 Voice recognition apparatus 21 Voice input means 22 Conversion means 23 Analysis means 231 Feature vector extraction part 232 Feature vector storage part 233 Recognition candidate voice storage part 234 First analysis part 235 Second analysis part 236 Voice determination part 3 Voice Generator

Claims (4)

Voice input means for inputting voice, conversion means for converting a voice signal input to the voice input means into a digital voice waveform signal, and input to the voice input means from the digital voice waveform signal converted by the conversion means A speech recognition device comprising an analysis means for analyzing the generated speech,
The analysis means includes
A feature vector extraction unit that analyzes the digital speech waveform signal in units of frames and extracts a feature vector representing a feature amount of speech;
A feature vector storage unit for storing the feature vectors extracted in units of frames for a plurality of frames in a time series;
A recognition candidate voice storage unit that stores a plurality of voices as voice recognition candidates;
A first analysis unit that calculates the likelihood of each speech that is a speech recognition candidate based on the feature vectors for a plurality of frames stored in the feature vector storage unit;
A second analysis unit that calculates an average feature vector per frame from the feature vectors for the plurality of frames, and calculates a likelihood of speech that is a speech recognition candidate from the average feature vector;
Speech determination for determining one speech based on the likelihood of each speech to be a speech recognition candidate calculated in the first analysis unit and the likelihood of each speech to be a speech recognition candidate calculated in the second analysis unit A voice recognition device.   2. The speech according to claim 1, wherein at least one of the first analysis unit and the second analysis unit calculates the likelihood of speech that is a speech recognition candidate using a Viterbi algorithm or a neural network. Recognition device.   The speech determination unit adds, for each speech, the likelihood of each speech that is a speech recognition candidate calculated by the first analysis unit and the likelihood of each speech that is a speech recognition candidate calculated by the second analysis unit. The speech recognition apparatus according to claim 1, wherein the speech corresponding to the maximum value of the addition value for each speech is a recognition result. The speech recognition device according to any one of claims 1 to 3,
A voice conversion device comprising: a voice generation device that generates a synthesized voice corresponding to the voice determined by the voice determination unit as a recognition result.

Images