JP6139430B2 - Signal processing apparatus, method and program - Google Patents

Description

  The present invention relates to a technique for processing a signal such as an audio signal or an acoustic signal.

  When an acoustic signal is collected in an environment with noise or reverberation, a signal in which acoustic distortion (noise or reverberation) is superimposed on the original signal is observed. When the acoustic signal is speech, the clarity of speech is greatly reduced due to the effect of superimposed acoustic distortion. As a result, it becomes difficult to extract the nature of the original speech signal, and for example, the recognition rate of the speech recognition system decreases. In order to prevent this decrease in the recognition rate, it is necessary to devise a method for removing the superimposed acoustic distortion.

  Therefore, a conventional signal processing apparatus described below has been proposed. In addition to voice recognition, this signal processing device can be used for, for example, a hearing aid, a TV conference system, a machine control interface, a music information processing system for searching for music, and recording music.

[Signal processing equipment]
FIG. 1 shows a functional configuration example of a conventional signal processing apparatus, and its operation will be briefly described. The signal processing apparatus includes a Fourier transform unit 101, a feature value generation unit 102, a matching unit 103, a voice enhancement filtering unit 104, and a case model storage unit 105.

  Voice including noise / reverberation is input to the Fourier transform unit 101 as an input signal. The input signal is windowed by a short Hamming window of about 30 ms, for example, and the windowed input signal is converted into an amplitude spectrum through a discrete Fourier transform (step S1, FIG. 2). An amplitude spectrum is amplitude data of a frequency spectrum. The amplitude spectrum is provided to the feature quantity generation unit 102 and the speech enhancement filtering unit 104.

  The feature amount generation unit 102 converts all of the amplitude spectrum output from the Fourier transform unit 101 into, for example, a mel cepstrum feature amount (step S2, FIG. 2). In general, the mel cepstrum widely used is about 10 to 20th order, but in order to accurately represent the case data, a mel cepstrum having a high order (for example, about 30 to 100th order) is used. Note that feature quantities other than the mel cepstrum may be used. The generated feature amount is provided to the matching unit 103.

The case model storage unit 105 stores clean speech data corresponding to a case, and a case model M that is a series (segment) of indexes of a Gaussian mixture distribution that gives the maximum likelihood with respect to a feature amount for each frame. Has been. The clean sound data corresponding to the case is, for example, the amplitude spectrum of the clean sound corresponding to the case. An example of segments included in the case model M is shown in FIG. Each cell corresponds to the i-th time frame. The number in each cell represents the index mi of the Gaussian distribution in the Gaussian mixture distribution g giving the maximum likelihood. The example model uses a large amount of clean speech obtained from a speech corpus and noise / reverberation data (noise signal waveform and room impulse response) obtained in any environment to simulate generation of observation signals in various environments. Then, using the simulation observation signal converted into the feature amount region, it is generated in advance by the case model generation device and stored in the case model storage unit 105 in advance. Details of the case model generation apparatus will be described later.

  The matching unit 103 matches the feature quantity of the input signal with the case example of the feature quantity included in the case model storage unit 105, and searches for a segment in the case model closest to the input signal (step S3, FIG. 2). ). Information about the segment in the case model closest to the input signal found by the search is provided to the speech enhancement filtering unit 104. Details of the matching unit 103 will be described later.

  The speech enhancement filtering unit 104 creates a filter for speech enhancement using the amplitude spectrum of clean speech corresponding to the segment in the case model closest to the input signal searched by the matching unit 103, and the created filter is To filter the input signal (step S4, FIG. 2). As the amplitude spectrum of the clean speech corresponding to the segment in the case model closest to the input signal, the amplitude spectrum read from the case model storage unit 105 is used. For details of the voice enhancement filtering unit 104, see, for example, Non-Patent Document 1 and Patent Document 1.

  According to this signal processing apparatus, it has been reported that it is possible to remove noise that has been difficult in the past and has a very large time variation. The noise having a very large time change is a noise such as an alarm sound of an alarm clock with respect to the background noise.

[Case model generator]
Here, a case model generation apparatus that generates a case model stored in the case model storage unit 105 will be described. FIG. 4 shows a functional configuration example of the case model generation apparatus. The example model generation apparatus includes a Fourier transform unit 201, a feature value generation unit 202, a Gaussian mixture model learning unit 203, and a maximum likelihood Gaussian distribution calculation unit 204.

  The function of each part of the case model generation apparatus is realized by, for example, a predetermined program being read into a computer including a ROM, a RAM, a CPU, and the like, and the CPU executing the program.

The input to the case model generator is speech data of various noise / reverberation environments. It is assumed that the sound data of various noise / reverberation environments includes the sound data of clean environments. The following processing is performed for each of the audio data of various noise / reverberation environments, because the Fourier transform unit 201 and the feature amount generation unit 202 are the same as the Fourier transform unit 101 and the feature amount generation unit 102 of FIG. 1, respectively. The duplicated explanation is omitted.

Gaussian mixture model learning unit 203, a feature amount x _i the learning data in each short time frame t obtained by the feature amount generating unit 202 to obtain the Gaussian mixture model g by a conventional maximum likelihood estimation. The Gaussian mixture model g is expressed by the following equation.

g (x _i | m) represents the m-th Gaussian distribution having mean μ _m and variance Σ _m . In many cases, g (x _i | m) is a multidimensional Gaussian distribution, and the number of dimensions is the same as the number of dimensions of the feature quantity x _i . When g (x _i | m) is a multidimensional Gaussian distribution, each of the mean μ _m and the variance Σ _m is a vector. Here, even if g (x _i | m) is a multidimensional Gaussian distribution, g (x _i | m) is simply expressed as a Gaussian distribution for simplification of the description. w (m) represents the mixing weight for the mth Gaussian distribution. Q represents the number of mixtures. For Q, for example, a fairly large value such as 4096 or 8192 is set.

Maximum likelihood Gaussian distribution calculation unit 204, the index m _i of the Gaussian distribution in the Gaussian mixture distribution g which gives the maximum likelihood for each time frame i calculated, the time sequence of the index m _i as a case model M Ask. Case model M, using the set and Gaussian mixture model g of the index m _i of the Gaussian distribution is expressed as shown in the following equation.

Here, m _i is the index of the Gaussian distribution that gives the maximum likelihood for the feature amount x _i of i-th frame, Gaussian g in Gaussian mixture m | represents the (x _i m) ing. I represents the total number of frames of learning data. For example, assuming 1 hour of learning data, I = 3.5 × 10 ⁵ . The generated case model M is stored in the case model storage unit 105 (FIG. 1). This case model is generated for each learning data of various noise / reverberation environments.

  If the environment is clean, the amplitude spectrum data output from the Fourier transform unit 201 is also stored in the case model storage unit 105 (FIG. 1).

[Specific Processing of Matching Unit 103]
Here, the processing in the matching unit 103 will be described in detail. For simplicity, consider only one example model M of a noise / reverberation environment. For simplicity, it is assumed that time expansion and contraction is not considered when matching the feature amount series of the input signal and the learning data segment. Matching unit 103 uses the feature quantity y _t and case model M of the input signal, searching the segment closest training data to the feature amount sequence of the input signal, nearest clean clean speech included in the input signal training data segment ^M _{t u} is believed to give a speech _sequence: output _{u + .tau.max.}

Assume that the input signal is composed of T time frames, and the feature quantity sequence of the input signal is y = {y _t : t = 1, 2,..., T}. Also, let _{yt: t + τ be} a sequence from the time frame t to t + τ of the feature quantity of the input signal. Then, M _{u: u + τ} = {g, m _i : i = u, u + 1,..., U + τ} is a Gaussian distribution sequence corresponding to continuous time frames from u-th to u + τ-th in the learning data.

Feature amount sequence y _t of the input _signal: definition and of the distance between a segment in the _{t + tau} training data, feature amount sequence y _t of the input _signal: a method of searching for _{t + tau} and closest training data, Euclidean distance, etc. You can think of several other ways. Here, it is assumed that the learning data segment closest to the feature quantity sequence of the input signal has the longest length among learning data segments that closely match the feature quantity series of the input signal. In other words, the closest training data segments M ^{t u} the feature amount sequence of the input _{signal: u + tau} can be determined by maximizing a posterior probability shown in the following equation.

Here, p (M _{u: u + τ} | y _{t: t + τ} ) represents the posterior probability, and when y _{t: t + τ} and M _{u: u + τ} are relatively well matched, τ is The longer it is, the higher the posterior probability is. The proof of this feature is described in detail in Non-Patent Document 1. By taking a measure of searching for a longer segment, it becomes difficult to be affected by noise that exists locally at a certain time, and it can be expected that relatively robust matching is performed with respect to noise.

The numerator term p (y _{t: t + τ} | M _{u: u + τ} ) in equation (2) is the likelihood of y _{t: t + τ} for the training data segment corresponding to M _{u: u + τ.} . The likelihood is calculated by the following equation.

For simplicity, it is assumed that adjacent frames are independent. The first term of the denominator of Equation (2) is the sum of p (y _{t: t + τ} | M _{u ′: u ′ + τ} ) starting from any time frame u ′ in the learning data. It is. The second term of the denominator of Equation (2) is the likelihood of yt _{: t + τ} for the Gaussian mixture model g, and is calculated by the following equation.

Here, the procedure of the segment search process in the matching unit 103 will be described more specifically. First, the maximum segment length is limited to (τ _lim +1) frames. For example, if the maximum length of the segment is limited to 30 frames, τ _lim = 29. Under this restriction, first, τ = 0, that is, segment length = 1, and a segment with segment length = 1 that gives the maximum posterior probability is found according to the equation (2). Next, assuming τ = 1, that is, segment length = 2, a segment with segment length = 2 that gives the maximum posterior probability is found according to equation (2). This process is repeated until τ = τ _lim , and finally, a segment that gives the maximum posterior probability is found from the segment candidates of different lengths that have been found. The length of the segment giving the maximum posterior probability is τ _max .

  The segment search process in the matching unit 103 can be performed on a case model M that can be expressed by a linear memory for I frames as shown in FIG.

J. Ming and R. Srinivasan, and D. Crooke, “A Corpus-Based Approach to Speech Enhancement From Nonstationary Noise,” IEEE Trans. On Acoustics, Speech and Signal Processing, 19 (4), pp. 822-836, 2011 .

JP 2013-37174 A

  In the conventional signal processing apparatus, the matching unit 103 compares segments having different lengths when searching for the segment closest to the input signal. However, it is not possible to compare segments of different lengths. For this reason, in the conventional signal processing apparatus, the segment search with high accuracy is not always performed.

  An object of this invention is to provide the signal processing apparatus, method, and program which can perform a segment search with higher precision than before.

  A signal processing apparatus according to an aspect of the present invention stores a segment that is a series of Gaussian distribution indexes in a Gaussian mixture distribution that gives a maximum likelihood to a feature amount of each frame of a predetermined signal. A case model storage unit, and a matching unit that searches for a segment that gives the maximum posterior probability for the feature quantity sequence of the input signal using the segments stored in the case model storage unit as candidates, and includes an input signal The first half is divided into two, the first half is the first half signal and the second half is the second half signal. The posterior probability in the matching section is evaluated based on the length of the first half signal corresponding to the first half signal. The likelihood is expressed using the likelihood and the likelihood evaluated based on the model with the Gaussian mixture distribution for the second half signal.

  A segment search with high accuracy can be performed.

The block diagram for demonstrating the example of a signal processing apparatus. The flowchart for demonstrating the example of a signal processing method. The figure for demonstrating the example of a segment. The figure for demonstrating the example of an example model production | generation apparatus. The figure for demonstrating the segment evaluation by Formula (7).

  Hereinafter, embodiments of a signal processing apparatus and method will be described with reference to the drawings.

  Similar to the conventional signal processing apparatus, the signal processing apparatus according to this embodiment includes a Fourier transform unit 101, a feature amount generation unit 102, a matching unit 103, a speech enhancement filtering unit 104, as illustrated in FIG. The case model storage unit 105 is provided.

  Hereinafter, the matching unit 103, which is a part different from the conventional one, will be mainly described. The Fourier transform unit 101, the feature amount generation unit 102, and the speech enhancement filtering unit 104 of the signal processing device according to the first embodiment are respectively the Fourier transform unit 101, the feature amount generation unit 102, and the conventional signal processing device. Since it is the same as that of the voice emphasis filtering unit 104, duplicate description is omitted.

  In the signal processing apparatus according to this embodiment, the matching unit 103 searches for a segment closest to the input signal by fairly evaluating segments having different segment lengths under a common length called a frame.

In the matching unit 103 of this embodiment, instead of the equation (3), the likelihood of the feature quantity sequence yt _{: t + τ} of the input signal of the frame having a predetermined length is calculated by using the case model M and the Gaussian mixture model g. Calculate using both. That is, y _{t: t + τ} is divided into y _{t: t + ν} and y _{t + ν + 1: t + τ} (0 ≦ ν ≦ τ), and the former is evaluated by M and the latter is evaluated by g. Specifically, the likelihood of the feature quantity sequence yt _{: t + τ} of the input signal is calculated as follows.

Here, p (y _{t: t + ν} | M _{u: u + ν} ) is the likelihood of y _{t: t + ν} of the feature quantity sequence of the input signal when the case model M _{u: u + ν} is given. Represents degrees. p (y _{t + ν + 1: t + τ} | φ _{u + ν + 1: u + τ} ) represents the likelihood of the feature quantity sequence y _{t: t + ν} of the input signal when the mixed model φ _{u + ν + 1: u + τ} is given. φ _{u + ν + 1: u + τ} is a Gaussian mixture distribution corresponding to the frame u + ν + 1 to the frame u + τ. p (y _{t: t + ν} | M _{u: u + ν} , φ _{u + ν + 1: u + τ} ) is a feature quantity sequence of the input signal when the case model M _{u: u + ν} and the mixed model φ _{u + ν + 1: u + τ} are given. y _t: represents the likelihood of _{t + ν} .

y _{t: t + ν} is a feature amount sequence of the input signal having a length corresponding to the segment M _{u: u + ν} of the case model in the feature amount sequence y _{t: t + τ} of the input signal. In other words, yt _{: t + ν} is a feature quantity sequence of the input signal corresponding to the frame t to the frame t + ν. y _{t + ν + 1: t + τ} is the feature amount sequence of the input signal in the portion of the feature amount sequence y _{t: t + τ} of the input signal that exceeds the length of the segment M _{u: u + ν} of the case model. In other words, yt _{+ ν + 1: t + τ} is a feature quantity sequence of the input signal corresponding to the frame t + ν + 1 to the frame t + τ.

In other words, Equation (5) is obtained by using the input signal to be evaluated as an input signal having a predetermined length (in this case, τ + 1), and the portion that can be evaluated based on the case model in the feature quantity series of the input signal to be evaluated is the case model. Degree p (y _{t: t + ν} | M _{u: u + ν} ) and cannot be evaluated by the segment M _{u: u + ν} of the case model (the portion exceeding the segment length of the case model) For the feature quantity sequence y _{t + ν + 1: t + τ} of the input signal, this means that the likelihood p (y _{t + ν + 1: t + τ} | φ _{u + ν + 1: u + τ} ) is evaluated based on the mixed model g.

In other words, the likelihood that the matching unit 103 calculates based on Equation (4) using the first half when the input signal is divided into two as the first half signal and the second half as the second half signal is that for the first half signal. The likelihood p (y _{t: t + ν} | M _{u: u + ν} ) evaluated based on the segment of the length corresponding to the first half signal and the second half signal based on the above-described model based on the Gaussian mixture distribution It can be said that the likelihood p (y _{t + ν + 1: t + τ} | φ _{u + ν + 1: u + τ} ) is an integrated likelihood.

  The likelihood based on the mixed model g corresponds to a likelihood smoothed over the entire model. For the part that cannot be evaluated by the case model, the average likelihood is substituted to try to evaluate the input signal fairly with the same frame length.

The y _t: using the likelihood of the _{t + tau,} the matching unit 103 y _{t: t +} best fits segment _{τ ^M t u:} finding according the following equation (6) (7) _{u + .nu.max.} t, τ, u, ν, u ′, ν ′ are integers.

Here, the denominator of equation (7), 'and, y _t: any division point of _{t + τ ν'} u any starting point of the learning data _{for, p (y t: t +} τ | M u ': u' _{+ ν ′} , φu _{′ + ν ′ + 1: u ′ + τ} ).

The posterior probability p (M _{u: u + ν} , φ _{u + ν + 1: u + τ} | y _{t: t + τ} ) defined by the equation (7) is as shown in the above equation (4) and the above equation (5): Likelihood p (y _t) evaluated based on a segment having a length corresponding to the first half signal, with the first half of the input signal divided into two as the first half signal and the second half as the second half signal. _{: t + ν} | M _{u: u + ν} ) and likelihood p (y _{t + ν + 1: t + τ} | φ _{u + ν + 1: u + τ} ) evaluated based on the model of the Gaussian mixture distribution for the latter half signal. Is done.

The maximum length of the segment is limited to (τ _lim +1) frames as in the conventional method. For example, if the maximum length of the segment is limited to 30 frames, τ _lim = 29. FIG. 5 shows the segment evaluation according to the equation (7) under this restriction. As is apparent from this figure, according to this embodiment, it can be seen that the segments of any segment length are evaluated fairly under a common length of (τ _lim +1) frames. From another viewpoint, according to this embodiment, the optimum segment length (ν _max ) and the segment start point (u) are searched simultaneously.

Hereinafter, in the case where the posterior probability of Equation (7) according to the present invention is relatively good in agreement with _{yt : t + τ} and _{Mu: u + τ} , similarly to the posterior probability of Equation (2) by the conventional method It proves that the longer τ is, the higher posterior probability is given. Therefore, when y _{t: t + τ} is divided into y _{t: t + ν} and y _{t + ν + 1: t + τ} and the former is evaluated by M and the latter is evaluated by g (equation (4)), y _{t: t +} a _{τ y t: t + ν-} 1 and y _{t + [nu:} the former is divided into _{t + tau} in the case of evaluating the latter in g in M, compares the magnitude of the posterior probability.

  As is clear from equation (7), the denominator is equal in both cases, so the ratio in both cases is equal to the following likelihood ratio from equation (4).

Here, it is assumed that _{y t + [nu} is good agreement in _{m u + ν.} In this case, the denominator of Equation (8) can be approximated as w (m _{u + ν} ) g (y _{t + ν} | _{mu + ν} ). Thus, equation (8) is equal to 1 / w (m _{u + ν} ). Since w (m _{u + ν} ) is 1 or less, Expression (8) becomes 1 or more. Thus, it can be seen that when yt _{: t + τ} and _{Mu: u + τ} match relatively well, the longer τ is, the higher the posterior probability calculated by equation (7) is. .

[Modifications, etc.]
Note that the present invention can be extended as described in Non-Patent Document 1 when considering a plurality of case models of noise / reverberation environments and considering time expansion and contraction at the time of matching.

  The processes described in the above signal processing apparatus and method are not only executed in chronological order according to the order of description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the process. .

  Further, when each unit in the signal processing device is realized by a computer, the processing contents of the functions that each unit of the signal processing device should have are described by a program. And each part is implement | achieved on a computer by running this program with a computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

101 Fourier Transform Unit 102 Feature Quantity Generation Unit 103 Matching Unit 104 Speech Enhancement Filtering Unit 105 Case Model Storage Unit

Claims (4)

A case model storage unit storing a segment that is a series of indices of a Gaussian distribution in a Gaussian mixture distribution that gives a maximum likelihood to a feature amount of each frame of a predetermined signal;
A matching unit that searches the segment stored in the case model storage unit as a candidate and searches for a segment that gives the maximum posterior probability to the feature amount series of the input signal,
When the input signal is divided into two, the first half is the first half signal and the second half is the second half signal.
The posterior probabilities in the matching unit are the likelihood evaluated based on the segment of the length corresponding to the first half signal for the first half signal and the likelihood evaluated based on the model based on the Gaussian mixture distribution for the second half signal. Expressed in degrees,
Signal processing device. The signal processing apparatus according to claim 1,
t, τ, u, ν, u ′, ν ′ are integers, the feature quantity of the input signal corresponding to frame t + τ from frame t is yt _{: t + τ,} and the frame u stored in the case model storage unit , The segment corresponding to frame u + ν is M _{u: u + ν} , the Gaussian mixture distribution corresponding to frame u + ν + 1 to frame u + τ is φ _{u + ν + 1: u + τ,} and y _t when M _{u: u + ν} and φ _{u + ν + 1: u + τ} are given. _: The posterior probability of _{t + τ} is p (y _{t: t + τ} | M _{u: u + ν} , φ _{u + ν + 1: u + τ} ),
The posterior probability is p ( _{Mu: u + ν} , φu _{+ ν + 1: u + τ} | y _{t: t + τ} ) defined below.

Signal processing device.   In the case model storage unit, a segment that is a series of Gaussian distribution indexes in the Gaussian mixture distribution that gives the maximum likelihood to the feature amount of each frame of a predetermined signal is stored.
  A matching step for searching for a segment that gives the maximum posterior probability for the feature amount series of the input signal, with the segment stored in the case model storage unit as a candidate,
  When the input signal is divided into two, the first half is the first half signal and the second half is the second half signal.
  The posterior probability in the matching step is the likelihood evaluated based on the segment of the length corresponding to the first half signal for the first half signal and the likelihood evaluated based on the model based on the Gaussian mixture distribution for the second half signal. Expressed in degrees,
  Signal processing method.   The program for functioning a computer as each part of the signal processing apparatus of Claim 1 or 2.

Images