JP6588936B2 - Noise suppression apparatus, method thereof, and program - Google Patents

Description

  The present invention relates to a noise suppression device that suppresses a noise component superimposed on an input audio signal, a method thereof, and a program.

  In an application or service involving voice input such as voice recognition, if noise is superimposed on the input voice, the accuracy is adversely affected. By suppressing the noise component superimposed on the input speech (noise suppression), it is possible to prevent subsequent deterioration in accuracy. As a noise suppression method, the effectiveness of a stochastic model-based method that expresses clean speech and noise with a probabilistic model, estimates the noise component superimposed on the input speech, and suppresses it is widely known (Non-Patent Literature). 1).

Fujimoto, "A VOICE ACTIVITY DETECTION BASED ON THE ADAPTIVE INTEGRATION OF MULTIPLE SPEECH FEATURES AND A SIGNAL DECISION SCHEME", Proc. ICASSP '08, pp.4441-4444.

  However, with the conventional technology, there is a case where noise suppression cannot be performed with high accuracy under high noise.

  It is an object of the present invention to provide a noise suppression device, a method thereof, and a program capable of performing noise suppression with high accuracy under high noise or the like as compared with the prior art.

In order to solve the above problem, according to one aspect of the present invention, a noise suppression device includes a feature amount calculation unit that calculates a speech feature amount O _t of speech data to be noise-suppressed, and speech and noise. Speech features using the posterior probabilities b _{j, N_t} (O _t ) of each state of the observed signal model expressed by a Gaussian distribution with K number of mixtures composed of states and states consisting of non-speech and noise Using the noise estimation unit that estimates the noise component contained in the speech data subject to noise suppression from the amount O _t and the noise component estimation value, noise-suppressed speech data obtained by suppressing the noise component is obtained from the speech data subject to noise suppression. The posterior probability b _{j, N_t} (O _t ) is a value obtained by using the posterior probability p _{j, k} of each state and each distribution as the output weight of the Gaussian distribution. DNN corresponds to each distribution in each state of the Gaussian mixture model output layer.

In order to solve the above-described problem, according to another aspect of the present invention, a noise suppression method includes a feature amount calculation step of calculating a speech feature amount O _t of speech data to be noise suppressed, and speech and noise. Using the posterior probabilities b _{j, N_t} (O _t ) of each state of the observed signal model represented by a Gaussian distribution of K number of mixtures consisting of a state consisting of non-voice and noise Using the noise estimation step for estimating the noise component contained in the speech data subject to noise suppression from the feature amount O _t and the noise component estimation value, the noise-suppressed speech data obtained by suppressing the noise component from the speech data subject to noise suppression is obtained. The posterior probability b _{j, N_t} (O _t ) is the value obtained using the posterior probability p _{j, k} of each state and each distribution as the output weight of the Gaussian distribution. , DNN corresponds to each distribution in each state of the Gaussian mixture model output layer. Respond.

  According to the present invention, there is an effect that noise suppression can be performed with high accuracy under high noise or the like, as compared with the prior art.

The functional block diagram of the noise suppression apparatus which concerns on 1st, 2 embodiment. The figure which shows the example of the processing flow of the noise suppression apparatus which concerns on 1st, 2 embodiment.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following explanation, the symbol “^” etc. used in the text should be described immediately above the character immediately after it, but it is described immediately before the character due to restrictions on the text notation. In the formula, these symbols are written in their original positions. Further, the processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

<Points of first embodiment>
Assuming that the feature distribution of speech and non-speech follows a mixed Gaussian distribution, the posterior probability of each Gaussian distribution is learned by deep learning, and the output value of the learned DNN (deep neural network) is used as the weight of each Gaussian distribution. By using it, it is possible to capture voice / non-voice feature quantities with higher accuracy.

<First embodiment>
FIG. 1 is a functional block diagram of a noise suppression apparatus according to the first embodiment, and FIG. 2 shows a processing flow thereof.

  The noise suppression apparatus 100 receives speech data to be noise-suppressed as input, suppresses noise included in the speech data, obtains and outputs noise-suppressed speech data.

  The noise suppression apparatus 100 includes a CPU, a RAM, and a computer that includes a ROM that stores a program for executing the following processing, and is functionally configured as follows.

  The noise suppression apparatus 100 includes a speech model learning unit 101, a feature amount calculation unit 102, a noise estimation unit 103, and a noise suppression unit 104.

<Voice model learning unit 101>
In order to estimate the noise component to be superimposed on the input speech data, the probability density function of speech / non-speech that can be learned in advance is expressed by a mixed Gaussian distribution (GMM).

For example, the speech model learning unit 101 receives learning speech data to which a label that can identify speech / non-speech (hereinafter also referred to as a speech identification label) is given for each short-time frame prior to the noise suppression processing. The The speech model learning unit 101 calculates a speech feature amount _{OH, t} of the speech data for learning. As the audio feature quantity _{OH, t} , an L-dimensional filter bank output value or the like can be used. When the l-dimensional filter bank output value is O _{H, t, l} , O _{H, t} = {O _{H, t, 1} , O _{H, t, 2} ,..., O _{H, t, L} }. Here, H is an index indicating learning data, and t is an index indicating frame time.

The speech model learning unit 101 estimates an optimal parameter value (for example, an average vector and a covariance matrix) of the GMM by using an EM algorithm or the like using the speech feature quantity _{OH, t} and the speech identification label.

Furthermore, in this embodiment, the speech model learning unit 101 uses the constructed GMM to construct a DNN so that the output layer corresponds to each Gaussian distribution in each state of the GMM. DNN matches the number of pre-constructed GMM states J and number of mixes K, and the number of outputs in the output layer is J x K. For example, j = 0, 1,..., J−1, J = 2 (that is, j = 0, 1), j = 0 is a non-interaction, and j = 1 is a label indicating an interaction. DNN is a model that takes speech feature values as input and outputs posterior probabilities of each Gaussian distribution in each state, and learns from speech feature values _{OH, t} and the posterior probability of the constructed GMM.

  The speech model learning unit 101 outputs the constructed GMM and DNN to the noise estimation unit 103 prior to the noise suppression process.

<Feature amount calculation unit 102>
The feature amount calculation unit 102 receives the speech data to be noise-suppressed, calculates the speech feature amount O _t (S102), and outputs it. Note that the type and calculation method of the voice feature amount are the same as those used in the voice model learning unit 101.

<Noise estimation unit 103>
The noise estimation unit 103 receives the GMM and DNN prior to noise suppression processing.

The noise estimation unit 103 receives the speech feature amount O _t as input, estimates a noise component included in the speech data to be noise-suppressed (S103), and outputs it. For example, the noise component at time t is estimated using a known noise estimation method (see Reference 1) based on a probability model.
(Reference 1) Fujimoto, "Noise Robust Voice Activity Detection Based on Switching Kalman Filter", IEICE Trans. On Info. & And Syst., Vol. E91-D, No.3, pp.467-477, March 2008.

The estimated value ^ N _{t, l} of the noise component weighted and averaged using the normalized probability at time t is given in the following form.

Where l = 1,2,…, L, ^ N _t = (^ N _{t, 1} , ^ N _{t, 2} , _… , ^ N _{t, L} ), N _{t, j} = (N _{t, j, 1} , N _{t, j, 2} , _… , N _{t, j, L} ), N _{t, j, l} is the l-dimensional filter bank output value of the estimated noise component N _{t, j} at time t and state j, b _{j, N_t} (O _t ) (where the subscript N_t means N _t ) consists of two states: voice and noise (j = 0), non-voice and noise (j = 1) This is the posterior probability of each state of the observed signal model (O _t ) expressed by a Gaussian distribution of K number of mixtures, and is expressed as follows.

Where μ _{O_t, j, k} , Σ _{O_t, j, k} (where subscript O_t means O _t ) is the state j (j = 0,1) of the observed sound model, and each normal distribution k (k = 1, 2,..., K), mean vector and covariance matrix at time t. These values can be obtained based on the GMM learned by the speech model learning unit 101.

である。 In the known technique (see Reference 1), w _{S_j, k} (where subscript S_j, k means S _{j, k} ) consists of two states of speech and non-speech learned in advance. This is the mixing weight of K number of GMMs. In the present embodiment, each state and the posterior probability p _{j, k} of each state that is the output of the DNN constructed by the speech model learning unit 101 is used as the weight w _{S_j, k} . That means

It is.

<Noise Suppression Unit 104>
The noise suppression unit 104 receives the speech data to be noise-suppressed and the estimated noise component ^ N _{t, l} as input, and uses the estimated noise component ^ N _{t, l} to generate noise from the speech data to be suppressed. Noise-suppressed voice data with suppressed components is obtained (S104) and output. For example, first, the filter gain is obtained using the estimated value ^ N _{t, l} of the noise component. Next, the filter gain is converted into an impulse response. Then, the impulse response is convoluted with the speech data to be noise-suppressed to obtain and output the noise-suppressed speech data.

For example, noise suppression is performed using a method based on Wiener filter (Reference 2)
(Reference 2) Segura, "Model-based compensation of additive noise for continuous speech recognition.experiments using AURORA II database and tasks", Proc. Of Euro Speech '01, Vol.1, pp.221-224.

フィルタゲインを算出後は、Mel-warped DCTなどを用いてインパルス応答に変換し、入力信号波形に畳み込むことで、雑音抑圧音声データを得ることができる。 In general, the filter gain G _{t, l} of the Wiener filter at time t is expressed as follows, _{St, l} uses the average value of the probabilistic model of speech / non-speech learned in advance, and N _{t, l} is noise Calculation is possible by using the estimated value ^ N _{t, l} estimated by the estimation unit 103.

After calculating the filter gain, noise-suppressed voice data can be obtained by converting it into an impulse response using Mel-warped DCT or the like and convolving it with the input signal waveform.

<Effect>
With the above configuration, it is possible to perform noise suppression with high accuracy under high noise or the like as compared with the conventional technology.

<Modification>
In the present embodiment, the speech model learning unit 101 is provided in the noise suppression device 100, but may be configured as a separate device.

<Second embodiment>
A description will be given centering on differences from the first embodiment.

  The noise suppression apparatus 100 includes a speech model learning unit 201, a feature amount calculation unit 102, a noise estimation unit 203, and a noise suppression unit 104. The processing contents of the speech model learning unit 201 and the noise estimation unit 203 are different from those in the first embodiment.

<Voice model learning unit 201>
The DNN in the present embodiment is constructed as learning data by using the feature amount of the learning speech data and the feature amount of the speech data composed of the learning noise to be superimposed on the learning speech data.

First, the speech model learning unit 201 constructs a GMM as in the first embodiment. Furthermore, in this embodiment, noise is intentionally superimposed on the learning speech data, and the feature amount of the noise component (hereinafter also referred to as learning noise) (for example, an L-dimensional filter bank output as with speech) is input. A DNN is constructed by giving it as feature data as learning data. That is, the GMM learns using the feature amount of the L-dimensional learning speech data, and the DNN learns using the 2L-dimensional feature amount of the feature amount of the learning speech data + the feature amount of the learning noise. DNN is learning audio feature O _H, and _t and the feature amount of the learning noise as an input, the model deli to output a posteriori probability for each Gaussian distribution at each state, the voice feature amount O _H, and _t Learning from noise features and the posterior probabilities of the constructed GMM.

  The speech model learning unit 201 outputs the constructed GMM and DNN to the noise estimation unit 203 prior to noise suppression processing.

<Noise estimation unit 203>
The noise estimation unit 203 receives GMM and DNN prior to noise suppression processing.

により求める。 The noise estimation unit 203 receives the speech feature amount O _t as input, estimates a noise component included in the speech data to be noise-suppressed (S203), and outputs it. In the present embodiment, each state that is an output of the DNN constructed by the speech model learning unit 201 using the feature value of the noise component estimate ^ N _{tn, l} and the speech feature value O _{t at} one or more previous times Calculate the posterior probabilities p _{j, k} in each distribution and use them as the mixture weights w _{S_j, k} . n is any integer of 1 or more. The estimate ^ N _{t, l} is

Ask for.

Note that the posterior probability p _{j, k} is required when obtaining the estimated value ^ N _{t, l} . Here, in the calculation of the posterior probability p _{j, k} , for example, the calculation is performed using the estimated value ^ N _{t−1, l} at the previous time t−1. In addition, at time t = 0 when the estimated value ^ N _{t-1, l} cannot be obtained, the estimated value ^ N _{t-1, l is set} to 0, and the speech feature O _{t is changed} to the estimated value ^ N _{t-1, It} is better to substitute _l .

<Effect>
By setting it as such a structure, the effect similar to 1st embodiment can be acquired. Furthermore, by learning not only speech feature values in advance but also noise feature amounts, it is possible to calculate posterior probabilities for speech / non-speech robustly even in noisy environments, and to accurately suppress noise. Can do.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by an electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (5)

A feature amount calculation unit that calculates a speech feature amount O _t of the speech data to be noise-suppressed;
A posteriori probability b _{j, N_t} (O _t ) of each state of the observed signal model expressed by a Gaussian distribution of K number of mixtures consisting of a state consisting of speech and noise and a state consisting of non-speech and noise A noise estimation unit for estimating a noise component included in the speech data to be noise-suppressed from the speech feature amount O _t ,
A noise suppression unit that obtains noise-suppressed speech data in which the noise component is suppressed from the speech data to be noise-suppressed using the estimated value of the noise component;
The posterior probabilities b _{j, N_t} (O _t ) are values obtained by using the posterior probabilities p _{j, k} of the respective states and distributions as outputs of the DNN as the output weights of the Gaussian distribution, and the DNN is an output layer Corresponds to each distribution in each state of the Gaussian mixture model, and the DNN receives the speech feature O _t as an input ,
Noise suppression device. The noise suppression device of claim 1,
The DNN is constructed as learning data, the feature amount of learning speech data and the feature amount of speech data composed of learning noise superimposed on the learning speech data,
The noise estimation unit uses the feature value of the estimated value of the noise component at one or more previous times and the speech feature value O _t, and the posterior probability in each state and each distribution that is the output of the DNN p _{j, k} is calculated,
Noise suppression device. A feature amount calculating step for calculating a speech feature amount O _t of the speech data subject to noise suppression;
A posteriori probability b _{j, N_t} (O _t ) of each state of the observed signal model expressed by a Gaussian distribution of K number of mixtures consisting of a state consisting of speech and noise and a state consisting of non-speech and noise using a noise estimation step of estimating a noise component included in the speech data noise suppression target from speech features O _t,
A noise suppression step for obtaining noise-suppressed speech data in which the noise component is suppressed from the speech data subject to noise suppression using the estimated value of the noise component,
The posterior probabilities b _{j, N_t} (O _t ) are values obtained by using the posterior probabilities p _{j, k} of the respective states and distributions as outputs of the DNN as the output weights of the Gaussian distribution, and the DNN is an output layer Corresponds to each distribution in each state of the Gaussian mixture model, and the DNN receives the speech feature O _t as an input ,
Noise suppression method. The noise suppression method according to claim 3,
The DNN is constructed as learning data, the feature amount of learning speech data and the feature amount of speech data composed of learning noise superimposed on the learning speech data,
The noise estimation step uses the feature value of the estimated value of the noise component at one or more previous times and the speech feature value O _t, and the posterior probability in each state and each distribution that is the output of the DNN p _{j, k} is calculated,
Noise suppression method.   A program for causing a computer to function as the noise suppression device according to claim 1.

Images