JP6728083B2 - Intermediate feature amount calculation device, acoustic model learning device, speech recognition device, intermediate feature amount calculation method, acoustic model learning method, speech recognition method, program - Google Patents

Description

The present invention relates to a voice recognition technique, and particularly to a technique for performing voice recognition using an acoustic model learned by using a neural network.

Model learning using neural networks is performed in various fields such as voice recognition and image recognition. First, an outline of a deep neural network (DNN: Deep Neural Networks) that is a typical model of a neural network will be described (see FIG. 1).

The DNN is composed of an input layer (0th layer), K (K is an integer of 1 or more) hidden layers (1st to Kth layers), and an output layer (K+1th layer). The feature quantity x ^k input to the k-th layer (0≦k≦K+1) of the DNN is an n _k -dimensional vector. Also, the feature quantity x ^K+2 output from the K+1th layer (that is, the output layer) is an n _K+2 dimensional vector. In this case, the k-th layer characteristic amount x ^k is n _{k-dimensional} vector is inputted, the feature x ^{k + 1} is n _{k + 1-dimensional} vector is outputted. Here, the output feature quantity x ^k+1 of the k-th layer becomes the input feature quantity of the k+1-th layer. The Kth layer is called the final hidden layer.

Generally, in DNN, linear transformation and nonlinear transformation are performed in each layer. If the weight matrix characterizing the linear transformation in the k-th layer (0≦k≦K+1) is W ^k , the bias vector is b ^k , and the nonlinear transformation is f (f is also called an activation function), the output feature amount x ^k+1 is expressed by the following equation using the input feature amount x ^k .

Here, W ^k is an n _k+1 ×n _k matrix, and b ^k is an n _k+1 dimensional vector.

Further, f is a non-linear transformation from n _{k + 1-dimensional} vector into n _{k + 1-dimensional} vector. In the hidden layer, a sigmoid function is used. In this case, if the i-th component of the linearly transformed feature quantity x ^k+1/2 is x _i ^k+1/2 and the i-th component of the output feature quantity x ^k+1 is x _i ^k+1 , Holds (1≦i≦n _k+1 ).

On the other hand, in the output layer, the softmax function is used. In this case, the i-th component of the linearly transformed feature x ^(K+1)+1/2 is x _i ^(K+1)+1/2 , and the i-th component of the output feature x ^K+2 is x _i ^{If K+2} , the following equation holds (1≦i≦n _K+2 ).

The i-th component x _i ^K+2 of the output feature value x ^K+2 is the output from the i-th unit forming the output layer. 0≦x _i ^K+2 ≦1 for each i, and Σx _i ^K+2 =1 holds. Therefore, the output feature amount x ^K+2 from the output layer will be referred to as the output probability distribution. Further, since the output characteristic amounts x ^{K + 2} is the probability distribution, may represent an output characteristic amounts x ^{K + 2} and the output probability distributions p. That is, p=(p ₁ ,...,p _{n_K+2} )=(x ₁ ^K+2 ,...,x _{n_K+2} ^K+2 ) (where _ (underscore) represents a subscript, For example, x ^y_z means that y _z is a subscript to x and x _{y_z} means that y _z is a subscript to x).

Learning a model by the DNN is learning a weight matrix and a bias vector. That is, the model learned by the DNN becomes a parameter that characterizes the DNN.

When learning the acoustic model used for speech recognition, the input feature amount to the DNN (that is, the feature amount that is the input to the input layer) x ⁰ is the voice feature amount of the voice data, and the output feature amount from the DNN (that is, the output amount). The feature quantity output from the layer) x ^K+2 is the probability distribution p of the phoneme (phoneme state) that is the output symbol of the voice. At this time, the number of units forming the output layer is equal to the number of phonemes, and the probability output from the i-th unit represents the probability that the input speech feature amount is the i-th phoneme.

If the phoneme corresponding to the unit number of the output layer with the maximum probability is the phoneme corresponding to the input speech feature amount, DNN will learn the model that outputs the phoneme with the speech feature amount as an acoustic model. Become. Here, the learned acoustic model is the weight matrix and bias vector of each layer.

Therefore, the voice recognition device configured by using the acoustic model generates a phoneme sequence from the voice feature amount of the voice data (recognition voice data) to be recognized, and generates the voice recognition result.

The outline of DNN has been described above, including the case of applying it to speech recognition.

In general, a neural network can be regarded as a function that inputs a vector and outputs the vector.

Hereinafter, a method in which the neural network described in Non-Patent Documents 1 to 3 learns an acoustic model (that is, learns parameters that characterize the neural network) will be described. First, the acoustic model learning device 700 of Non-Patent Document 1 will be described with reference to FIGS. 12 to 13. FIG. 12 is a block diagram showing the configuration of the acoustic model learning device 700. FIG. 13 is a flowchart showing the operation of the acoustic model learning device 700. As shown in FIG. 12, the acoustic model learning device 700 includes a speech intermediate feature amount calculation unit 710, a phoneme probability distribution calculation unit 720, a parameter optimization unit 730, and a recording unit 790.

The recording unit 790 is a component that appropriately records information necessary for the processing of the acoustic model learning device 700. For example, the initial values of the parameters (acoustic model parameters) forming the acoustic model are recorded in advance. Also, the acoustic model parameters generated in the learning process are recorded appropriately.

The initial value of the acoustic model parameter may be generated by using a random number, or a parameter generated by using, as learning data, different voice data different from the voice data used for this learning may be used.

The number of phonemes is M, each phoneme is numbered from 1 to M (hereinafter referred to as phoneme number), and each phoneme is identified using the phoneme number m (1 ≤ m ≤ M). .. The phoneme with phoneme number m is called the mth phoneme.

Before the learning is started, the voice feature amount is extracted from the voice data (learning voice data) to be the learning data. The voice feature amount is represented as a vector. In addition, a correct phoneme number, which is a phoneme number for identifying a phoneme (correct phoneme) corresponding to the voice feature amount, is also prepared. That is, the set of the voice feature amount and the correct phoneme number is input to the acoustic model learning device 700. The set of speech feature quantities and correct phoneme numbers is called training data.

The acoustic model learning device 700 learns an acoustic model from a set of a voice feature amount and a correct phoneme number, which is training data.

The voice intermediate feature amount calculation unit 710 is a configuration unit that executes calculation from the input layer of the DNN to the final hidden layer. The phoneme probability distribution calculation unit 720 is a component that executes calculation in the output layer of the DNN. Therefore, the acoustic model learned by the acoustic model learning device 700 becomes a DNN parameter that characterizes the speech intermediate feature amount calculation unit 710 and the phoneme probability distribution calculation unit 720.

The acoustic model learning device 700 sets the initial values of the acoustic model parameters recorded in the recording unit 790 in the speech intermediate feature amount calculation unit 710 and the phoneme probability distribution calculation unit 720 by the start of learning. In addition, the acoustic model learning device 700 sets the calculated acoustic model parameters to the speech intermediate feature amount calculation unit 710 and the phoneme probability distribution calculation unit 720 every time the parameter optimization unit 730 optimizes the acoustic model parameters during learning. To do. As a result, the following training data will be processed using the DNN (speech intermediate feature amount calculation unit 710 and phoneme probability distribution calculation unit 720) characterized by the newly calculated acoustic model parameter.

The operation of the acoustic model learning device 700 will be described with reference to FIG. The voice intermediate feature amount calculation unit 710 calculates a voice intermediate feature amount, which is an intermediate feature amount for phoneme identification, from the input voice feature amount (S710). The phoneme probability distribution p=(p ₁ ,...,p) is a distribution of the probability p _m that the phoneme corresponding to the input voice feature is the phoneme of the phoneme number m (1≦m≦M). It is the feature quantity used to calculate _M ). Further, as described above, since the voice intermediate feature amount calculation unit 710 is a component unit that executes the calculation from the input layer of the DNN to the final hidden layer, the voice intermediate feature amount is the output feature of the final hidden layer of the DNN being learned. Is the amount.

The phoneme probability distribution calculation unit 720 calculates a phoneme probability distribution from the speech intermediate feature amount calculated in S710 (S720). As described above, the phoneme probability distribution calculation unit 720 is a component that executes the calculation in the output layer of the DNN, and thus the phoneme probability distribution is the output feature amount of the output layer of the DNN being learned. Here, assuming that the phoneme with the phoneme number m (the m-th phoneme) corresponds to the m-th unit forming the output layer, the phoneme probability distribution is the probability p _m (the output value from the m-th unit in the output layer). The distribution p = (p ₁ , ..., p _M ) in which the values calculated by equation (3) are arranged.

The parameter optimizing unit 730 optimizes the acoustic model parameters using the phoneme probability distribution calculated in S720 and the input correct phoneme number (S730). For example, the acoustic model parameters are optimized and calculated so as to reduce the value of the loss function C defined by the following equation.

However, p=(p ₁ ,..., P _M ) is a phoneme probability distribution, and d=(d ₁ ,..., D _M ) is a correct probability distribution defined by the following equation.

Note that the loss function C is called cross entropy, and is a scale defined between two probability distributions and is a measure for measuring the deviation between the distributions.

Generally, the number of training data, which is a set of a voice feature amount and a correct phoneme number, is very large, about tens of millions to hundreds of millions of times. In order to efficiently optimize the acoustic model parameters from the enormous amount of training data, for example, Equation (4) in Non-Patent Document 1 may be used.

The acoustic model learning device 700 repeats the processing of S710 to S730 for the number of training data, and outputs the finally calculated acoustic model parameter as a learning result.

Next, the acoustic model learning device 800 of Non-Patent Document 2 will be described with reference to FIGS. 14 to 15. In the learning of the acoustic model learning device 800, in addition to the DNN used in the learning of the acoustic model learning device 700, a convolutional neural network (CNN) which is a neural network often used in image recognition is also used. CNN consists of input layer, convolutional layer and pooling layer.

FIG. 14 is a block diagram showing the configuration of the acoustic model learning device 800. FIG. 15 is a flowchart showing the operation of the acoustic model learning device 800. As shown in FIG. 14, the acoustic model learning device 800 includes a noise resistant intermediate feature amount calculation unit 810, a speech intermediate feature amount calculation unit 710, a phoneme probability distribution calculation unit 720, a parameter optimization unit 730, and a recording unit 790.

The acoustic model learning device 800 learns an acoustic model from a set of a voice feature amount and a correct phoneme number, which is training data.

The voice intermediate feature amount calculation unit 710 is a configuration unit that executes calculation from the input layer of the DNN to the final hidden layer. The phoneme probability distribution calculator 720 is a component that executes calculation in the output layer of the DNN. The noise-proof intermediate feature amount calculation unit 810 is a component unit that executes CNN calculation. Therefore, the acoustic model learned by the acoustic model learning device 800 includes the DNN parameters that characterize the speech intermediate feature amount calculation unit 710 and the phoneme probability distribution calculation unit 720 and the CNN parameters that characterize the target sound feature amount calculation unit 810.

The acoustic model learning device 800 sets the initial values of the acoustic model parameters recorded in the recording unit 790 in the noise-resistant intermediate feature amount calculation unit 810, the speech intermediate feature amount calculation unit 710, and the phoneme probability distribution calculation unit 720 by the start of learning. To do. In the acoustic model learning apparatus 800, each time the parameter optimizing unit 730 optimizes the acoustic model parameters during learning, the acoustic model parameters are calculated by using the noise-resistant intermediate feature amount calculating unit 810 and the speech intermediate feature amount calculating unit 710. , And phoneme probability distribution calculation unit 720.

The operation of the acoustic model learning device 800 will be described with reference to FIG. The noise resistance intermediate feature amount calculation unit 810 calculates the noise resistance intermediate feature amount, which is the feature amount of the target sound included in the voice data from which the voice feature amount is extracted, from the input voice feature amount (S810). The noise resistant intermediate feature amount is a feature amount calculated from the voice feature amount corresponding to the target sound of the voice data on which noise is superimposed, and is specifically calculated as follows. First, a two-dimensional image of time and logarithmic power spectrum is generated from the voice feature amount. Next, the noise-resistant intermediate feature amount is calculated from the two-dimensional image using CNN.

The speech intermediate feature amount calculation unit 710 calculates a speech intermediate feature amount, which is an intermediate feature amount for phoneme identification, from the noise resistant intermediate feature amount calculated in S810 (S710). The speech intermediate feature amount calculation unit 710 differs from that of the acoustic model learning device 700 in that the noise-resistant intermediate feature amount in which the influence of noise is removed is input instead of the voice feature amount extracted while the influence of noise remains. However, the operation is similar. That is, the speech intermediate feature amount calculation unit 710 uses the neural network configured from the input layer of the DNN to the final hidden layer to extract the speech intermediate feature amount, which is the output feature amount of the final hidden layer, from the noise resistant intermediate feature amount. calculate.

The phoneme probability distribution calculation unit 720 calculates a phoneme probability distribution from the speech intermediate feature amount calculated in S710 (S720).

The parameter optimizing unit 730 optimizes the acoustic model parameters using the phoneme probability distribution calculated in S720 and the input correct phoneme number (S730).

It has been confirmed that the voice recognition using the acoustic model learned by the acoustic model learning device 800 has high accuracy of the voice recognition for the voice data including noise. That is, the acoustic model learned by the acoustic model learning device 800 is a noise resistant acoustic model.

Next, the acoustic model learning device 900 of Non-Patent Document 3 will be described with reference to FIGS. Also in the learning of the acoustic model learning device 900, DNN and CNN are used as in the learning of the acoustic model learning device 800. The difference from the acoustic model learning device 800 is the way of connecting these neural networks. The acoustic model learning device 800 uses a neural network in which CNN and DNN are connected in series in this order, while the acoustic model learning device 900 uses a neural network in which CNN and DNN are connected in parallel.

FIG. 16 is a block diagram showing the configuration of the acoustic model learning device 900. FIG. 17 is a flowchart showing the operation of the acoustic model learning device 900. As shown in FIG. 16, the acoustic model learning device 900 includes a speech intermediate feature amount calculation unit 710, a noise resistant intermediate feature amount calculation unit 810, an intermediate feature amount combination unit 910, a phoneme probability distribution calculation unit 720, a parameter optimization unit 730, The recording unit 790 is included.

The acoustic model learning device 900 learns an acoustic model from a set of a speech feature amount and a correct phoneme number, which is training data.

The acoustic model learned by the acoustic model learning device 900 is the same as the acoustic model parameters learned by the acoustic model learning device 800, and the parameters of the DNN and the target sound feature amount that characterize the speech intermediate feature amount calculation unit 710 and the phoneme probability distribution calculation unit 720. It includes the CNN parameters that characterize the calculator 810.

The acoustic model learning device 900 sets the initial values of the acoustic model parameters recorded in the recording unit 790 in the noise resistant intermediate feature amount calculation unit 810, the speech intermediate feature amount calculation unit 710, and the phoneme probability distribution calculation unit 720 by the start of learning. To do. In the acoustic model learning device 900, each time the parameter optimizing unit 730 optimizes the acoustic model parameters during learning, the acoustic model parameters are calculated by using the calculated acoustic model parameters as a noise resistant intermediate feature amount calculating unit 810 and a voice intermediate feature amount calculating unit 710. , And phoneme probability distribution calculation unit 720.

The operation of the acoustic model learning device 900 will be described with reference to FIG. The voice intermediate feature amount calculation unit 710 calculates a voice intermediate feature amount that is a part of the phoneme identification intermediate feature amount from the input voice feature amount (S710).

The noise resistant intermediate feature amount calculation unit 810 calculates a noise resistant intermediate feature amount that is a part of the phoneme identification intermediate feature amount from the input speech feature amount (S810).

The intermediate feature amount combining unit 910 generates a combined intermediate feature amount from the voice intermediate feature amount calculated in S710 and the noise resistant intermediate feature amount calculated in S810 (S910). The combined intermediate feature amount is a vector obtained by combining the voice intermediate feature amount and the noise resistant intermediate feature amount, which are vectors, as a vector.

The phoneme probability distribution calculation unit 720 calculates a phoneme probability distribution from the combined intermediate feature amount generated in S910 (S720).

The parameter optimizing unit 730 optimizes the acoustic model parameters using the phoneme probability distribution calculated in S720 and the input correct phoneme number (S730).

In the voice recognition using the acoustic model learned by the acoustic model learning device 900, when the number of learning data is the same, as compared with the voice recognition using the acoustic model learned by the acoustic model learning device 700 or the acoustic model learning device 800, It has been confirmed that the accuracy is high.

Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patric Nguyen, Tara Sainath, Brian Kingsbury, “Deep Neural Networks for Acoustic Modeling in Speech Recognition”, IEEE Signal Processing Magazine, Vol.29, No.6, pp.82-97, 2012. Ossama Abdel-Hamid, Adbel-rahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, Dong Yu, “Convolutional Neural Networks for Speech Recognition”, IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol.22, No. 10, pp.1533-1545, 2014. Hagen Soltau, George Saon, Tara N. Sainath, “Joint Training of Convolutional and Non-Convolutional Neural Networks”, IEEE ICASSP, 2014.

In the voice recognition using the acoustic model learned by the method of Non-Patent Document 1, there is a problem that the recognition accuracy of a voice on which noise is superimposed is significantly lower than the recognition accuracy of a voice without noise.

Speech recognition using an acoustic model learned by the method of Non-Patent Document 2 is superior to that of Non-Patent Document 1 in terms of noise resistance. However, there is a problem that the calculation time required for learning the acoustic model is longer than that of the method of Non-Patent Document 1. In addition, speech recognition of speech with less noise has a problem that the calculation time required for the recognition is long, although the accuracy is not so different from that of the speech recognition using the acoustic model learned by the method of Non-Patent Document 1. ..

According to the method of Non-Patent Document 3, whether or not noise is included in the speech to be learned, the speech feature amount corresponds to the learning of the DNN, and the speech intermediate feature amount calculation unit 710 corresponding to the learning of the CNN corresponds to the tolerance. It is input to both of the noise intermediate feature amount calculation units 810 and used for learning. Therefore, although the recognition accuracy of the voice recognition using the acoustic model obtained as the learning result is high, the calculation time required for the learning is long. In addition, the calculation time required for recognition becomes long.

In addition, although the acoustic model is learned by using a voice that does not include noise, there may be a case where the voice to be recognized includes noise at the voice recognition stage. If an attempt is made to recognize a voice of a different type from the voice used for learning as in this case, the recognition accuracy may decrease. Therefore, in order to solve this problem, a method in which a plurality of types of voices are prepared as learning data and voice recognition is performed using an acoustic model learned using the methods of Non-Patent Document 1 and Non-Patent Document 2 can be considered. However, there is a problem that the recognition accuracy is lower than that in the case of performing voice recognition using an acoustic model learned for each type of voice.

As described above, with the methods of Non-Patent Documents 1 to 3, it is difficult to learn an acoustic model that realizes highly accurate speech recognition regardless of the presence or absence of noise while suppressing the calculation time required for recognition processing.

Therefore, the present invention provides a technique for calculating an intermediate feature amount used for learning an acoustic model that realizes highly accurate voice recognition regardless of the type of learning voice data while suppressing the calculation time required for recognition processing. The purpose is to do.

One aspect of the present invention, the number of types to classify the voice data is J, the number for identifying the type is a type number, the number of phonemes is M, and the number for identifying the phoneme is a phoneme number, A type identification unit that determines a type number j′ (where j′ is an integer that satisfies 1≦j′≦J) corresponding to the type of voice data from which the voice feature amount is extracted, and the voice feature amount And the type number j′, the phoneme probability distribution p=(p ₁ ,...,p) which is a distribution of the probability p _m that the phoneme corresponding to the voice feature quantity is the phoneme of the phoneme number m (1≦m≦M) _M ) is an intermediate feature amount calculation device including a phoneme intermediate feature amount calculation unit that calculates a j'th type intermediate feature amount that is a feature amount used as a phoneme intermediate feature amount, wherein the phoneme intermediate feature amount is For each integer j satisfying 1≦j≦J, the calculating unit calculates the j-th type intermediate feature amount from the voice feature amount extracted from the voice data of the type number j by using the neural network. And a feature quantity calculation unit.

According to the present invention, it is possible to calculate an intermediate feature amount used for learning an acoustic model that realizes highly accurate voice recognition regardless of the type of learning voice data while suppressing the calculation time required for recognition processing. it can.

The figure which shows an example of DNN. The figure which shows an example of a structure of the acoustic model learning apparatus 100. The figure which shows an example of operation|movement of the acoustic model learning apparatus 100. The figure which shows an example of a structure of the type identification part 110. The figure which shows an example of operation|movement of the type identification part 110. The figure which shows an example of a structure of the phoneme intermediate feature-value calculation part 120. The figure which shows an example of operation|movement of the phoneme intermediate feature-value calculation part 120. The figure which shows an example of a structure of the speech recognition apparatus 200. The figure which shows an example of operation|movement of the speech recognition apparatus 200. The figure which shows an example of a structure of the voice recognition part 220. The figure which shows an example of operation|movement of the voice recognition part 220. The figure which shows an example of a structure of the acoustic model learning apparatus 700. The figure which shows an example of operation|movement of the acoustic model learning apparatus 700. The figure which shows an example of a structure of the acoustic model learning apparatus 800. The figure which shows an example of operation|movement of the acoustic model learning apparatus 800. The figure which shows an example of a structure of the acoustic model learning apparatus 900. The figure which shows an example of operation|movement of the acoustic model learning apparatus 900.

Hereinafter, embodiments of the present invention will be described in detail. It should be noted that components having the same function are denoted by the same reference numeral, and redundant description will be omitted.

Hereinafter, terms used in each embodiment will be briefly described.

The voice data is voice data that is recorded in advance because it is used for learning an acoustic model and voice recognition. The voice data is, for example, the voice of a sentence spoken by the speaker. Further, the audio data is, for example, digital data digitized at a sampling frequency of 16 kHz.

The voice feature amount is a feature amount extracted from voice data, and includes, for example, FBANK (filter bank logarithmic power) extracted for each frame (normally about 20 ms to 40 ms) into which voice data is divided. The voice feature amount is generally expressed as a vector.

The type of voice is a category for classifying learning voice data used when learning an acoustic model. For example, when classification is performed based on the presence or absence of noise, the learning voice data is classified into two types, that is, a voice including noise and a voice not including noise. Further, the learning voice data can be classified into three types, that is, spoken language voice, read-aloud voice, and conference voice. It is also possible to classify the learning voice data into two types, a male voice and a female voice. Furthermore, regarding the noise, it is also possible to classify the learning voice data into three types when the noise is not divided into two types, ie, high level, medium level, low level and noise level.

<First embodiment>
[Acoustic model learning device 100]
Hereinafter, the acoustic model learning device 100 will be described with reference to FIGS. 2 to 7. As shown in FIG. 2, the acoustic model learning device 100 includes a type identification unit 110, a phoneme intermediate feature amount calculation unit 120, a phoneme probability distribution calculation unit 130, a parameter optimization unit 140, and a recording unit 790. The recording unit 790 is a component that appropriately records information necessary for the processing of the acoustic model learning device 100.

The voice data for learning shall be classified into J types of voice. Further, numbers (hereinafter referred to as type numbers) are assigned to the respective categories from 1 to J, and each type is identified using the type number j (1≦j≦J). The voice of type number j is called the j-th voice. For example, when classifying on the basis of presence or absence of noise and gender, a male voice without noise is type 1 voice, a female voice without noise is type 2 voice, and a male voice containing noise is type 3 The voices of female and voices including noise can be classified into four types with the type 4 voice.

In addition, the number of phonemes is M, each phoneme is numbered from 1 to M (hereinafter referred to as phoneme number), and each phoneme is identified using the phoneme number m (1 ≤ m ≤ M). To The phoneme with phoneme number m is called the mth phoneme.

It is the same as the acoustic model learning device 700 that training data is prepared from the learning voice data before the learning is started.

The acoustic model learning device 100 learns an acoustic model from a set of a voice feature amount and a correct phoneme number, which is training data. The correct phoneme number is a phoneme number for identifying a phoneme (correct phoneme) corresponding to a voice feature amount.

The type identification unit 110, the phoneme intermediate feature amount calculation unit 120, and the phoneme probability distribution calculation unit 130 include a configuration for executing calculation by a neural network. Therefore, the acoustic model learning device 100 sets the initial values of the acoustic model parameters recorded in the recording unit 790 in the type identification unit 110, the phoneme intermediate feature amount calculation unit 120, and the phoneme probability distribution calculation unit 130 before the learning starts. In the acoustic model learning apparatus 100, each time the parameter optimizing unit 140 optimizes the acoustic model parameters during learning, the acoustic model parameters are calculated by the type identifying unit 110, the phoneme intermediate feature amount calculating unit 120, and the phoneme probability distribution. The calculation unit 130 is set.

The operation of the acoustic model learning device 100 will be described with reference to FIG. The type identifying unit 110 determines a type number j′ (where j′ is an integer satisfying 1≦j′≦J) corresponding to the type of audio data from which the audio feature amount is extracted, from the input audio feature amount. Yes (S110). The type identifying unit 110 will be described below with reference to FIGS. 4 to 5.

As shown in FIG. 4, the type identification unit 110 includes a type intermediate feature amount calculation unit 111, a type probability distribution calculation unit 112, and a type number determination unit 113. The type feature amount calculation unit 111 is a configuration unit corresponding to the neural network from the input layer of the DNN to the final hidden layer. The type probability distribution calculator 112 is a component corresponding to the neural network in the output layer of the DNN. The number of units included in the output layer of the type probability distribution calculation unit 112 is equal to the number J of types. The j-th unit (1.ltoreq.j.ltoreq.J) is a unit that outputs the probability that the type number of the speech from which the speech feature amount is extracted is j (is the j-th type).

The type intermediate feature amount calculation unit 111 and the type probability distribution calculation unit 112 may be configured by using a neural network other than DNN. However, the output of the type probability distribution calculation unit 112 is configured to be a J-dimensional vector.

The operation of the type identifying unit 110 will be described with reference to FIG. The type intermediate feature amount calculation unit 111 calculates a type feature amount, which is an intermediate feature amount for type identification, from the input voice feature amount (S111). The type intermediate feature amount is a type probability distribution q=(q ₁ ,..., Which is a distribution of the probability q _j that the type corresponding to the speech from which the input speech feature amount is extracted is the j-th type (1≦j≦J) , q _J ) is a feature quantity used to calculate. Further, as described above, when the type intermediate feature amount calculation unit 111 is a configuration unit corresponding to the neural network from the input layer of the DNN to the final hidden layer, the type intermediate feature amount is the output of the final hidden layer of the DNN being learned. It becomes a feature quantity.

The type probability distribution calculation unit 112 calculates a type probability distribution from the type intermediate feature amount calculated in S111 (S112). As described above, when the type probability distribution calculation unit 112 is a configuration unit corresponding to the neural network of the output layer of the DNN, the type probability distribution becomes the output feature amount of the output layer of the DNN under learning. Here, the type of the type number j (a j species) because corresponds to the j units constituting the output layer, the type probability distribution arranged the probability q _j is the output value from the j-th unit of the output layer distribution q=(q ₁ ,...,q _J ).

The type determining unit 113 determines the type number j′ having the maximum probability from the type probability distribution calculated in S112 (S113).

The phoneme intermediate feature amount calculation unit 120 calculates a phoneme intermediate feature amount, which is an intermediate feature amount for phoneme identification, from the input voice feature amount and the type number j′ determined in S110 (S120). The phoneme intermediate feature amount is a phoneme probability distribution p=(p ₁ ,...,p) that is a distribution of the probability p _m that the phoneme corresponding to the input voice feature amount is the phoneme of the phoneme number m (1≦m≦M). It is the feature quantity used to calculate _M ). The phoneme intermediate feature amount calculation unit 120 will be described below with reference to FIGS. 6 to 7.

As shown in FIG. 6, the phoneme intermediate feature amount calculation unit 120 includes a voice feature amount input unit 121, a first type intermediate feature amount calculation unit 122 ₁ ,..., A J type intermediate feature amount calculation unit 122 _J , and a phoneme intermediate feature amount. The output unit 123 is included. The first-type intermediate feature amount calculation unit 122 ₁ ,..., And the J-type intermediate feature amount calculation unit 122 _J respectively correspond to the neural network from the input layer to the final hidden layer of the DNN, and the configuration unit corresponding to the CNN. Is either. When the voice of the type number j (the voice of the j-th type) is a voice including noise, it is preferable that the j-th type intermediate feature amount calculation unit 122 _j is a component corresponding to the CNN. In the j-th type intermediate feature amount, assuming that the input voice feature amount is that of the type number j (1≦j≦J), the phoneme corresponding to the voice feature amount is the phoneme number m (1≦m≦ It is a feature quantity for calculating the phoneme probability distribution p=(p ₁ ,..., P _M ) which is the distribution of the probability p _m that is the phoneme of _M ). Further, the dimensions of the first-type intermediate feature amount,..., And the J-type intermediate feature amount as vectors are the same.

The first-type intermediate feature amount calculation unit 122 ₁ ,..., And the J-type intermediate feature amount calculation unit 122 _J may be configured by using a neural network other than DNN or CNN. However, in this case as well, it is preferable to prepare a neural network that generates a noise-resistant acoustic model and a neural network that does not, such as a neural network depending on the type of voice.

For example, when classifying on the basis of the presence or absence of noise and gender, J=4, and the first type intermediate feature amount calculation unit 1221 sets the _first feature amount of the noise-free male voice (voice of type number 1) as the first feature amount. The type 2 intermediate feature amount/type 2 intermediate feature amount calculation unit 122 ₂ calculates the feature amount of a female voice (voice of type number 2) without noise as the type 2 intermediate feature amount and type 3 intermediate feature amount calculation unit 122 ₃ type 3 intermediate feature quantity the feature quantity of the speech (voice type number 3) of the men including noise, the four intermediate feature quantity calculator 122 ₄ female noisy speech (voice type No. 4) The feature amount is calculated as the fourth type intermediate feature amount. In this case, the first type intermediate feature amount calculation unit 122 ₁ and the second type intermediate feature amount calculation unit 122 ₂ are components corresponding to the neural network from the input layer of the DNN to the final hidden layer, and the third type intermediate feature amount calculation unit. The unit 122 ₃ and the fourth type intermediate feature amount calculation unit 122 ₄ are configured as a configuration unit corresponding to CNN.

The operation of the phoneme intermediate feature amount calculation unit 120 will be described with reference to FIG. The voice feature amount input unit 121 outputs the input voice feature amount to the j'th type intermediate feature amount calculation unit 122 _j'using the type number j'determined in S110 (S121).

The j'th type intermediate feature amount calculation unit 122 _j'calculates the j'th type intermediate feature amount from the voice feature amount input from the voice feature amount input unit 121 (S122).

The phoneme intermediate feature amount output unit 123 outputs the j'th type intermediate feature amount calculated in S122 as a phoneme intermediate feature amount (S123).

The phoneme probability distribution calculation unit 130 calculates a phoneme probability distribution from the phoneme intermediate feature amount calculated in S120 (S130). The phoneme probability distribution calculator 130 is a component corresponding to the neural network in the output layer of the DNN. The number of units included in the output layer of the phoneme probability distribution calculation unit 130 is equal to the number of phonemes. The m-th unit is a unit that outputs the probability that the phoneme number of the phoneme corresponding to the voice feature amount is m (the phoneme corresponding to the voice feature amount is the m-th phoneme).

The phoneme probability distribution calculation unit 130 may be configured using a neural network other than DNN. However, the output of the phoneme probability distribution calculation unit 130 is configured to be an M-dimensional vector.

The parameter optimizing unit 140 optimizes the acoustic model parameters using the phoneme probability distribution calculated in S130 and the input correct phoneme number (S140). A specific optimization calculation method may be the same as that of the parameter optimization unit 730 of the acoustic model learning device 700.

The calculated acoustic model parameters are the type intermediate feature amount calculation unit 111, the type probability distribution calculation unit 112, the j'th type intermediate feature amount calculation unit 122 _j' (where j'is the type number determined in S110), the phoneme probability. It is fed back to the distribution calculation unit 130 and used for learning using the next training data.

The acoustic model learning device 100 repeats the processing of S110 to S140 for the number of training data, and outputs the finally calculated acoustic model parameter as a learning result.

The type identification unit 110 and the phoneme intermediate feature amount calculation unit 120 are collectively referred to as an intermediate feature amount calculation unit 105 (see FIG. 2). In addition, when the intermediate feature amount calculation unit is treated not as a part of the acoustic model learning device 100 but as an independent device, it is referred to as an intermediate feature amount calculation device. The intermediate feature amount calculation device receives the voice feature amount as an input, identifies the type of voice data from which the voice feature amount is extracted, and then calculates and outputs the intermediate feature amount.

[Voice recognition device 200]
Hereinafter, the voice recognition device 200 will be described with reference to FIGS. As shown in FIG. 8, the voice recognition device 200 includes a voice feature amount extraction unit 210 and a voice recognition unit 220.

The voice recognition device 200 is also connected to the learning result recording unit 290. The learning result recording unit 290 records the acoustic model learned by the acoustic model learning device 100. The learning result recording unit 290 may be a component included in the voice recognition device 200.

As shown in FIG. 10, the voice recognition unit 220 includes a type identification unit 110, a phoneme intermediate feature amount calculation unit 120, a phoneme probability distribution calculation unit 130, and a voice recognition result generation unit 221. The type identification unit 110, the phoneme intermediate feature amount calculation unit 120, and the phoneme probability distribution calculation unit 130 are the same configuration units as those of the acoustic model learning device 100.

The voice recognition device 200 generates a voice recognition result, which is a recognition result of the recognition voice data, from the recognition voice data.

The speech recognition apparatus 200 recognizes the acoustic model parameters recorded in the learning result recording unit 290 by the speech recognition unit 220 (specifically, the type identification unit 110, the phoneme intermediate feature amount calculation unit 120, and the phoneme probability distribution) before starting the speech recognition. The calculation unit 130) is set.

The operation of the voice recognition device 200 will be described with reference to FIG. The voice feature amount extraction unit 210 extracts the voice feature amount of the recognition voice data from the recognition voice data (S210). The voice feature amount extraction unit 210 extracts the voice feature amount under the same condition as the generation of the voice feature amount which is the input of the acoustic model learning device 100.

The voice recognition unit 220 generates a voice recognition result which is a result of recognizing the recognition voice data from the voice feature amount extracted in S210 (S220). Specific processing will be described with reference to FIG. The type identifying unit 110 determines a type number j′ corresponding to the type of voice data from which the voice feature amount is extracted, from the voice feature amount extracted in S210 (S110). The phoneme intermediate feature amount calculation unit 120 calculates a phoneme intermediate feature amount, which is an intermediate feature amount for phoneme identification, from the voice feature amount extracted in S210 and the type number j′ determined in S110 (S120). The phoneme probability distribution calculation unit 130 calculates a phoneme probability distribution from the phoneme intermediate feature amount calculated in S120 (S130). The speech recognition result generation unit 221 determines a phoneme having the maximum probability from the phoneme probability distribution calculated in S130, and generates a speech recognition result from the determined phoneme sequence (S221). The length of the phoneme sequence is equal to the number of voice feature amounts extracted from the recognition voice data.

According to the invention of this embodiment, the phoneme intermediate feature amount is calculated after identifying the type of voice. Further, one acoustic model corresponding to a combination of acoustic models for recognizing each type of speech is learned using the phoneme intermediate feature amount calculated by reflecting the type of speech. As a result, it is possible to learn an acoustic model that realizes highly accurate voice recognition regardless of the type of learning voice data while suppressing the calculation time required for recognition processing.

By performing voice recognition using the acoustic model according to the present invention, the presence or absence of noise is not affected as compared with the voice recognition using the acoustic model by the acoustic model learning device 700 or the acoustic model learning device 800. Highly accurate voice recognition is possible. Further, it is possible to suppress the calculation time required for the recognition processing, as compared with the voice recognition using the acoustic model by the acoustic model learning device 900.

<Modification>
It is needless to say that the present invention is not limited to the above-described embodiments and can be appropriately modified without departing from the spirit of the present invention. The various kinds of processing described in the above embodiments may be executed not only in time series according to the order described, but also in parallel or individually according to the processing capability of the device that executes the processing or the need.

<Additional notes>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating with the outside of the hardware entity. Connectable communication unit, CPU (Central Processing Unit, cache memory and registers may be provided), RAM or ROM that is memory, external storage device that is a hard disk, and their input unit, output unit, and communication unit , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged among external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

The external storage device of the hardware entity stores a program necessary to realize the above-described functions and data necessary for the processing of this program (not limited to the external storage device, for example, the program is read). It may be stored in a ROM that is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in the RAM, the external storage device, or the like.

In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data necessary for the processing of each program are read into the memory as necessary, and interpreted and executed/processed by the CPU as appropriate. .. As a result, the CPU realizes a predetermined function (each constituent element represented by the above,... Unit,... Means, etc.).

The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. Further, the processes described in the above-described embodiments are not only executed in time series in the order described, but may be executed in parallel or individually according to the processing capability of the device that executes the processes or as necessary. ..

As described above, when the processing functions of the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on the computer, the processing functions of the hardware entity are realized on the computer.

The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, or the like. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disc. Memory), CD-R (Recordable)/RW (ReWritable), etc. as a magneto-optical recording medium, MO (Magneto-Optical disc) etc., and semiconductor memory EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) etc. Can be used.

The distribution of this program is performed by selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network to distribute the program.

A computer that executes such a program temporarily stores, for example, the program recorded on a portable recording medium or the program transferred from the server computer in its own storage device. Then, when executing the processing, this computer reads the program stored in its own recording medium and executes the processing according to the read program. As another execution form of this program, a computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be sequentially executed. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by the execution instruction and result acquisition without transferring the program from the server computer to this computer May be It should be noted that the program in this embodiment includes information that is used for processing by an electronic computer and that is equivalent to the program (data that is not a direct command to a computer, but has the property of defining computer processing).

Further, in this embodiment, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of these processing contents may be implemented by hardware.

Claims (7)

The number of types for classifying the voice data is J, the number for identifying the type is a type number, the number of phonemes is M, and the number for identifying the phoneme is a phoneme number,
A type identification unit that determines a type number j′ (where j′ is an integer satisfying 1≦j′≦J) corresponding to the type of audio data from which the audio feature amount is extracted, from the audio feature amount;
A phoneme probability distribution p=(p ₁ that is a distribution of the probability p _m that the phoneme corresponding to the phonetic feature amount is a phoneme with a phoneme number m (1≦m≦M) from the phonetic feature amount and the type number j′. ,...,P _M ), which is a feature amount used for calculating the j′-type intermediate feature amount as a phoneme intermediate feature amount.
The phoneme intermediate feature amount calculation unit, for each integer j that satisfies 1≦j≦J,
A j-type intermediate feature amount calculation unit that calculates a j-type intermediate feature amount from the voice feature amount extracted from the voice data of type number j using a neural network;
Including
The type identification unit,
A type intermediate feature amount calculation unit that calculates a type intermediate feature amount, which is an intermediate feature amount for type identification, from the voice feature amount;
From the type intermediate feature amount, a type probability distribution calculation unit that calculates a type probability distribution,
An intermediate feature quantity calculation device, comprising : a type determination unit that determines the type number having the maximum probability as the type number j′ from the type probability distribution . An acoustic model learning device for learning an acoustic model used for speech recognition from a correct phoneme number that is a phoneme number for identifying a phoneme corresponding to a voice feature and a phoneme corresponding to the voice feature,
Using the intermediate feature amount calculation device according to claim 1, a distribution of probability p _m from the voice feature amount that a phoneme to which the voice feature amount corresponds is a phoneme with a phoneme number m (1≦m≦M). A j'th type intermediate feature amount (where j'is the type of voice data from which the voice feature amount is extracted) that is a feature amount used to calculate a certain phoneme probability distribution p=(p ₁ , ..., p _M ). An intermediate feature amount calculation unit that calculates a phoneme intermediate feature amount that is a corresponding type number and that satisfies 1≦j′≦J),
From the phoneme intermediate feature amount, a phoneme probability distribution calculation unit for calculating the phoneme probability distribution,
A parameter optimization unit that optimizes an acoustic model parameter that is a parameter of the acoustic model by using the phoneme probability distribution and the correct phoneme number;
Among the acoustic model parameters, the parameter optimization unit includes parameters that are included in the intermediate feature amount calculation unit and characterize the neural network used in the j'th type intermediate feature amount calculation unit corresponding to the type number j'. An acoustic model learning device characterized by optimizing. A voice feature amount extraction unit that extracts a voice feature amount of the recognition voice data from the recognition voice data;
A voice recognition unit that generates a voice recognition result, which is a recognition result of the recognition voice data, from the voice feature amount using the acoustic model learned by the acoustic model learning device according to claim 2.
Speech recognition device including. The number of types for classifying the voice data is J, the number for identifying the type is a type number, the number of phonemes is M, and the number for identifying the phoneme is a phoneme number,
A type in which the intermediate feature amount calculation device determines a type number j′ (where j′ is an integer that satisfies 1≦j′≦J) corresponding to the type of voice data from which the voice feature amount is extracted, from the voice feature amount An identification step,
The intermediate feature amount calculation device is a distribution of the probability p _m from the voice feature amount and the type number j′ that the phoneme to which the voice feature amount corresponds is the phoneme with the phoneme number m (1≦m≦M). Intermediate feature including a phoneme intermediate feature calculation step of calculating a j'th type intermediate feature that is a feature used to calculate a phoneme probability distribution p=(p ₁ ,..., P _M ) as a phoneme intermediate feature How to calculate quantity,
The phoneme intermediate feature amount calculation step, for each integer j that satisfies 1≦j≦J,
A j-type intermediate feature amount calculation step of calculating a j-type intermediate feature amount from the voice feature amount extracted from the voice data of the type number j using a neural network;
Including
The type identification step,
A type intermediate feature amount calculating step of calculating a type intermediate feature amount, which is an intermediate feature amount for type identification, from the voice feature amount;
From the type intermediate feature amount, a type probability distribution calculating step of calculating a type probability distribution,
And a type determining step of determining the type number having the maximum probability as the type number j'from the type probability distribution . An acoustic model learning device is an acoustic model learning method for learning an acoustic model used for speech recognition, from a correct phoneme number that is a phoneme number for identifying a phoneme corresponding to a voice feature and a phoneme corresponding to the voice feature.
The acoustic model learning apparatus uses the intermediate feature amount calculation method according to claim 4, wherein a phoneme corresponding to the voice feature amount is a phoneme with a phoneme number m (1≦m≦M) from the voice feature amount. A j'th type intermediate feature amount (where j'is the above-mentioned speech feature amount) which is a feature amount used for calculating a phoneme probability distribution p=(p ₁ ,..., P _M ) which is a distribution of a certain probability p _m An intermediate feature amount calculation step of calculating a phoneme intermediate feature amount, which is a type number corresponding to the type of the extracted speech data and satisfies 1≦j′≦J),
The acoustic model learning device, from the phoneme intermediate feature amount, a phoneme probability distribution calculation step of calculating the phoneme probability distribution,
The acoustic model learning device, using the phoneme probability distribution and the correct answer phoneme number, a parameter optimization step for optimizing an acoustic model parameter that is a parameter of the acoustic model,
The parameter optimizing step includes, among the acoustic model parameters, parameters that characterize the neural network used in the j'th type intermediate feature amount calculating step corresponding to the type number j'included in the intermediate feature amount calculating step. An acoustic model learning method characterized by optimizing. A voice recognition device, a voice feature amount extraction step of extracting a voice feature amount of the recognition voice data from the recognition voice data,
A voice recognition step in which the voice recognition device generates a voice recognition result, which is a recognition result of the recognition voice data, from the voice feature amount, using the acoustic model learned by the acoustic model learning method according to claim 5. ,
Speech recognition method including. Program for causing a computer to function as an intermediate feature quantity calculation equipment according to claim 1.

Images