JP4960933B2 - Acoustic signal enhancement apparatus and method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve sound source separation, even when a reverberation period is longer than a frame width. <P>SOLUTION: The sound signal enhancement device includes: a band splitting section; a parameter estimation section; a reverberation removing section; a sound source separation section; and a band combining section. The band splitting section splits an observation sound signal of a time domain into observation signals for each of the predetermined number of frequency bands. The parameter estimation section generates: an estimation value of a regression matrix, by which an observation signal of each of frequency bands is input and a reverberation component until fixed frame before is estimated; and an estimation value of a separation matrix for estimating a sound source signal. The reverberation removing section calculates a reverberation component by performing convolution arithmetic on the regression matrix to the observation signal for each frequency band, and generates a reverberation removing signal by subtracting the calculated reverberation component from the observation signal for each frequency band. The sound source separation section generates an estimation value of the sound source signal by multiplying the separation matrix with the reverberation removing signal. The band combining section synthesizes an estimation value of the sound source signal and converts it into an enhancement sound signal in the time domain. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to an acoustic signal enhancement apparatus and method, program, and recording medium that can be used in, for example, a voice recognition system, a video conference system, and the like.

A general configuration of an acoustic signal enhancement device based on a conventional sound source separation technique will be described with reference to FIG. In the following description, a sound signal emitted from each sound source is called a sound source signal, and a sound signal observed by a microphone is called an observation signal. Unless otherwise specified, these represent narrowband signals for each frequency band. The signal in the time domain is represented by adding the word “acoustic” such as an observed acoustic signal. The acoustic signal enhancement device 600 includes a band dividing unit 60, a parameter estimating unit 62, a sound source separating unit 64, and a band synthesizing unit 66. The band dividing unit 60 divides a sampled and quantized time-domain observation acoustic signal input from a microphone, an audio file, or the like into observation signals y _{t, p} ^(m) for each frequency band. m, t, and p indicate indexes of the microphone, time, and frequency band, respectively. The observation signals from all the microphones are expressed in a vector format as shown in Equation (1).

Here, M represents the number of microphones, and τ represents non-conjugate transposition. In the conventional sound source separation technique, it is assumed that the observation signal vector y _{t, p} is generated by Equation (2).

Here, H is a conjugate transpose, -H is a conjugate transpose of an inverse matrix, st _{, p} are vectors composed of true sound source signals, and W _p is a true separation matrix. The sound source signal vector is defined by Equation (3), where s _{t, p} ^(m) is the mth sound source signal.

The parameter estimation unit 62 estimates the separation matrix W _p of each frequency band from the observed signal vectors {y _{t, p} } _{0 ≦ t ≦ T−1, 0 ≦ p ≦ P−1 over} all time and frequency bands. Calculate the value W _p ^. For each frequency band, the sound source separation unit 64 applies the estimated value W _p ^ of each frequency band separation matrix to the observed acoustic signal vector yt _{, p} of each frequency band as shown in Equation (4). Calculate the estimated values s _{t, p} ^ of the sound source signal vectors s _{t, p in} each frequency band.

The band synthesizing unit 66 synthesizes the estimated values of the sound source signal vectors to calculate a time domain enhanced acoustic signal. The band dividing unit 60 divides the observation acoustic signal into observation signals y _{t, p} ^(m) for each frequency band using short-time Fourier transform or polyphase filter bank analysis. The band synthesizing unit 66 performs overlap addition synthesis when the band dividing unit 60 uses short-time Fourier transform, and executes polyphase filter bank synthesis when the band dividing unit 60 uses polyphase filter bank analysis. To do. Hereinafter, a case where the band dividing unit 60 performs the short-time Fourier transform will be described as an example, but other time frequency analysis methods may be used for the band dividing unit 60.

For example, in the conventional sound source separation technique disclosed in Non-Patent Document 1, the parameter estimation unit 62 statistically independent of each sound source signal s _{t, p} ⁽¹⁾ ,..., S _{t, p} ^(m) from each other. Assuming that there is an estimated value W _p of the separation matrix W _p of each frequency band so that the independence between s _{t, p} ⁽¹⁾ ，,..., S _{t, p} ^(m)に な_る is maximized. Calculate ^.
Shoji Makino, Akiko Araki, Ryo Mukai, Hiroshi Sawada "Blind Sound Source Separation with Convolution Mixing" System / Control / Information, Vol.48, No.10, pp.401-408,2004

  The conventional acoustic signal emphasizing apparatus has a problem that the reverberation component of the interference signal is not removed when the reverberation time exceeds the frame width of the Fourier transform for a short time. In order to remove the reverberation component of the interference signal, a method of increasing the frame width so that the reverberation component is included in the frame can be considered. However, it is known that even if the frame width is simply increased, it becomes difficult to estimate each separation matrix because the assumption of independence between sound source signals does not hold because the number of divisions in the band dividing unit 60 increases. . This phenomenon is described, for example, in the references `` Araki, S., Mukai, R., Makino, S., Nishikawa, T and Saruwatari, H .: The fundamental limitation of frequency domain blind source separation for convolutivemixtures of speech, IEEE Trans. , Audio Process., Vol. 11, No. 2, pp. 109-1166, 2003 ".

  The present invention has been made in view of such problems, and an acoustic signal emphasizing apparatus and method, program, and recording medium capable of sound source separation even when the reverberation time exceeds the frame width of Fourier transform for a short time. The purpose is to provide.

The acoustic signal enhancing apparatus of the present invention includes a band dividing unit, a parameter estimating unit, a dereverberation unit, a sound source separating unit, and a band synthesizing unit. The band dividing unit divides the observation acoustic signal in the time domain into observation signals for a predetermined number of frequency bands. Parameter estimating unit estimates the regression matrix for estimating the reverberation component up to the previous predetermined number of frames of the observation signals of each frequency band as an input, and a separation matrix for separating the source signals. The dereverberation unit calculates a reverberation component by convolving the regression matrix with the observation signal for each frequency band. Then, the dereverberation signal is generated by subtracting the obtained reverberation component from the observation signal for each frequency band. The sound source separation unit generates an estimated value of the sound source signal by multiplying the dereverberation signal for each frequency band by the separation matrix. The band synthesis unit synthesizes the estimated values of the sound source signals to generate a time domain enhanced acoustic signal . Then, the parameter estimation unit uses the estimated value of the power spectrum density of the sound source signal, the estimated value of the separation matrix, and the observed signal to determine whether the regression matrix depends on both the regression matrix and the separation matrix. The degree function is estimated to be maximized, and the separation matrix is obtained as a separation matrix that maximizes the likelihood function or a separation matrix obtained by using a sound source separation technique.

  The acoustic signal enhancing apparatus according to the present invention performs sound source separation by multiplying the dereverberation signal after the dereverberation is removed by the dereverberation unit by the separation matrix. Therefore, sound source separation can be performed even when the reverberation time exceeds the frame width. In addition, since the reverberation component is suppressed in the enhanced acoustic signal, a higher quality enhanced acoustic signal can be obtained.

Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.
[Basic idea of the present invention]
Prior to the description of the embodiments, the basic idea of the acoustic signal enhancement method of the present invention will be described. In the present invention, it is assumed that the observation signal yt _{, p} is generated not by the equation (2) but by the equation (5).

That is, the observation signal y _{t, p} is modeled as being generated from the unknown parameters G _{k, p} and W _p and the sound source signal _{st, p} . In this model, the sound source signal s _{t, p} is mixed by the mixing matrix W _p ^-H to become x _{t, p} , and further, by an autoregressive system of M input and M output having parameters G _{k, p} in a k-th order regression matrix. This means that the reverberation is superimposed and becomes the observation signal yt _{, p} . The parameter estimation unit according to the present invention estimates G _{k, p} and W _p over the entire frequency band p (0 ≦ p ≦ P−1) from the observation signals yt _{, p} . K _p corresponds to the length of reverberation in the p-th frequency band, and is referred to as the number of dereverberation frames.

When the estimated value of the regression matrix of each frequency band estimated by the parameter estimation unit is set to G _{k, p} ^ and the estimated value of the separation matrix is set to W _p ^, the present invention is expressed by the following equations (7) and (8). A sound source signal is estimated as shown.

In the present invention, as shown in Equation (7), the reverberation component is obtained by convolving the observation signal y _{t, p} with the estimated value G _{k, p} ^ of the regression matrix, and the reverberation component is subtracted from the observation signal. The removal signal x _{t, p} ^ is obtained. Then, sound source separation is performed by multiplying the dereverberation signal x _{t, p} ^ by the estimated value W _p ^ of the separation matrix. Therefore, sound source separation can be performed even when the reverberation time exceeds the frame width.

  FIG. 1 shows a functional configuration example of an acoustic signal emphasizing apparatus 100 according to the present invention. The operation flow is shown in FIG. The acoustic signal enhancing apparatus 100 includes a band dividing unit 60, a parameter estimating unit 12, an enhanced signal calculating unit 14, and a band synthesizing unit 66. The band dividing unit 60 and the band synthesizing unit 66 are the same as the conventional acoustic signal enhancing apparatus 600. The acoustic signal emphasizing apparatus 100 is realized by reading a predetermined program into a computer configured with, for example, a ROM, a RAM, a CPU, and the like, and executing the program by the CPU.

The band dividing unit 60 divides the observation acoustic signal in the time domain into observation signals yt _{and p} for a predetermined number of frequency bands (step S60). The parameter estimator 12 receives the observed signals yt _{, p} as input and estimates the regression matrix G _{k, p} ^ for estimating reverberation components up to a predetermined number of frames and the separation matrix for estimating the sound source signal. Estimated value W _p ^ is generated (step S12). Here, “predetermined number of frames” means the number of dereverberation frames K _p described above, and this value may be set from the outside, or set beforehand as a fixed value in the parameter estimation unit 12. You can keep it.

The estimated values of the regression matrix G _{k, p} ^ and the separation matrix W _p ^ are obtained as values of G _{k, p} and W _p that maximize the log-likelihood function shown in Equation (9).

式（１１）の_ｓΘは、以後に説明する音源信号の全極パラメータの集合である。また、式（１２）の_ｗΘと式（１３）の_ｇΘは、それぞれ分離行列と回帰行列の集合である。また、式（９）に現れる_ｓΛ_ｔについては後述する。 However, Θ is a set of all unknown parameters and is defined as follows.

In Equation (11), _s Θ is a set of all-pole parameters of the sound source signal described below. In addition, _w Θ in Expression (12) and _g Θ in Expression (13) are a set of a separation matrix and a regression matrix, respectively. Further, _s Λ _t appearing in Equation (9) will be described later.

The process of deriving Equation (9) will be described. First, a probability density function of the estimated sound source signal _{St, p} ^(m) is obtained. In the present invention, it is assumed that the short-time power spectral density of each sound source signal is given by a Q-order all-pole function. That is, when ω∈ [−π, π] is an angular frequency and the power spectral density of the m-th sound source signal in the t-th frame is written as _s λ _t ^(m) (ω), the power spectral density _s λ _{t of the} sound source signal. ^(M) (ω) is given by equation (14).

Here, a _{t, k} ^(m) is a linear prediction coefficient, and _s v _t ^(m) is a prediction residual power. Further, at _{, 1} ^(m) ,..., At _{, Q} ^(m) and _s v _t ^(m) are collectively referred to as an all-pole parameter. Further, the covariance matrix of the sound source signal vector s _{t, p} is expressed as in Expression (16) using the M-dimensional diagonal matrix _s Λ _t (ω). That is, it is assumed that there is no correlation between a plurality of sound source signals.

Since the sound source signal S _{t, p} ^(m) follows a complex normal distribution with an average of 0 and a variance _s λ _t ^(m) (2πp / P), the probability density function of the sound source signal S _{t, p} ^(m) 17).

ここで、任意のマイクロホン番号ｍについて、（ｔ₁，ｐ₁）＝（ｔ₂，ｐ₂）でなければ、Ｓ_ｔ1，ｐ1 ^（ｍ）とＳ_ｔ2，ｐ2 ^（ｍ）は統計的に独立である。また、ｍ₁≠ｍ₂、すなわちマイクロホン番号が異なるならばＳ_ｔ1，ｐ1 ^（ｍ1）とＳ_ｔ2，ｐ2 ^（ｍ2）は常に統計的に独立である。 N {x; μ, Σ} represents a probability density function of a random variable x according to a complex normal distribution of mean μ and covariance matrix Σ. For example, when x is a D-dimensional vector, N {x; μ, Σ} is specifically given by Expression (18).

Here, for any microphone number m, if (t ₁ , p ₁ ) = (t ₂ , p ₂ ), S _{t1, p1} ^(m) and S _{t2, p2} ^(m) are statistically independent. is there. If m ₁ ≠ m ₂ , that is, if the microphone numbers are different, S _{t1, p1} ^(m1) and S _{t2, p2} ^(m2) are always statistically independent.

From Equation (6) and Equation (17) _, the probability density function of x _{t, p} can be written as Equation (19).

Therefore, it can be seen from the equation (5) that the conditional probability density function of the observation signals yt _{and p} given the past observation signals is given by the equation (20).

Therefore, the probability density function for all observed signals y can be expressed by equation (21).

Taking the logarithm of both sides of Equation (21) leads to a log likelihood function, Equation (9). Since the log-likelihood function of Equation (9) also depends on the all-pole parameter _s Θ, the parameter estimator 12 uses not only the separation matrix _w Θ and the regression matrix _g Θ, but also all the parameters Θ = { _s Θ, _w Estimate Θ, _g Θ}. That is, Θ ^ = { _s Θ ^, _w Θ ^, _g Θ ^} that maximizes the expression (9) is obtained as an estimated value of the unknown parameter Θ. Various methods can be employed for obtaining the estimated value Θ ^ of the unknown parameter that maximizes Equation (9). For example, an arbitrary optimization method such as a steepest descent method or a quasi-Newton method is used. Since these optimization methods are well known, a description thereof will be omitted. The parameter estimation unit 12 supplies the separation matrix estimation value _w Θ ^ and the regression matrix estimation value _g Θ ^ among the unknown parameters Θ ^ to the enhancement signal calculation unit 14 (step S12).

The dereverberation unit 141 constituting the enhancement signal calculation unit 14 performs a convolution operation on the estimated values G _{k, p} ^ of the regression matrix to the observation signals yt _{, p} divided for each frequency band output by the band division unit 60. A reverberation component is obtained, and the reverberation component xt _{, p} is generated by subtracting the reverberation component from the observation signal yt _{, p} (formula (7)) (step S141). The sound source separation unit 142 multiplies the dereverberation signal x _{t, p} by the estimated value W _p ^ of the separation matrix (formula (8)) to generate an estimated value of the sound source signal (step S142). The band synthesizing unit 66 synthesizes the estimated values of the sound source signals to generate a time domain enhanced acoustic signal (step S66). The enhanced acoustic signal generated in this manner is a sound source separated sound source even when the reverberation time exceeds the frame width because the reverberation removal signal after removing the reverberation component from the observation signal is the sound source separated. In addition, since the reverberation component is suppressed in the enhanced acoustic signal, a higher quality enhanced acoustic signal can be obtained.

Although the predetermined number of frames for removing reverberation (the number of reverberation removed frames K _p ) has been described as one fixed value, this value may be varied according to the center frequency of the frequency band. It is well known that the reverberation time varies depending on the frequency band. In the field of room acoustics, since the reverberation time of a frequency band of 500 Hz or less is long, a method of setting a predetermined number of frames large for the frequency band and setting a small number of predetermined frames for other frequency bands is conceivable. The parameter estimation unit 12 includes a predetermined number of frames variable unit 120, and the predetermined number of frames variable unit 120 can efficiently remove the reverberation by changing and setting the predetermined number of frames according to the frequency band. become. That is, the effect of reducing the amount of calculation can be obtained.

Next, a second embodiment in which the unknown parameter Θ is estimated by a method that is not a conventional general optimization algorithm such as the steepest descent method will be described. FIG. 3 shows a functional configuration example of the parameter estimation unit 30 of the second embodiment. The operation flow is shown in FIG. The parameter estimation unit 30 uses the unknown parameter Θ
The separation matrix estimate _w Θ ^ and the regression matrix estimate _g Θ ^ are generated using an efficient algorithm specialized in the estimation of.

  The parameter estimation unit 30 includes an observation signal recording unit 301, a parameter estimation value initialization unit 302, an all-pole parameter estimation value update unit 303, a sound source signal power spectrum estimation value update unit 304, and a regression matrix estimation value update unit 305. And a separation matrix estimated value update unit 306 and a convergence determination unit 307. The observation signal recording unit 301 records the observation signal divided into a predetermined number of frequency bands by the band dividing unit 60. The observation signal recording unit 301 records all the observation signals in the observation section and records the recorded observation signals as an all-pole parameter estimated value update unit 303, a regression matrix estimated value update unit 305, a separation matrix estimated value update unit 306, Are output in chronological order.

Parameter estimation value initialization unit 302 sets the estimated value _w theta ^ initial value of the estimated value _g theta ^ a separation matrix of regression matrix (step S302). Also, the index i of the number of repetitions is set to 0. The initial value _g Θ ^ ⁽⁰⁾ of the estimated value of the regression matrix is set to a zero matrix. The estimated value _w Θ ^ ⁽⁰⁾ of the separation matrix is set to a unit matrix. These mean that a setting in which neither reverberation nor a mixture of sound source signals exists is set as an initial value.

The all-pole parameter estimated value updating unit 303 includes an observed signal, an initial value _g Θ ^ ^{(0) of the} regression matrix and an initial value _w Θ ^ ^{(0) of the} separation matrix or an updated estimated value _g Θ ^ ^{( i)} and the initial value _w Θ ^ ⁽ⁱ⁾ of the separation matrix are input, and the all-pole parameter estimated value _s Θ ^ ^{(i + 1)} is updated according to the equation (22) (step S303).

The update rule of all pole parameters shown in Expression (22) will be described. Now, the temporary estimate _g Θ ^ ^{(i) of} the regression matrix and the temporary estimate _w Θ ^ ⁽ⁱ⁾ of the separation matrix are known. Using the estimated values, the estimated values of the sound source signal vectors _{st, p} are calculated according to equations (7) and (8).
The estimated value of each sound source signal obtained in this way is written as _{St, p} ^ ^{(m) (i + 1)} . updated estimation values a ^ _{t, 1} ^{(m) (i + 1)} ,..., a _{t, Q} ^{(m) (i + 1)} , _s v _t ^ in the t-th frame of the m-th sound source signal ^{(M) (i + 1)} is obtained by applying linear prediction analysis to the estimated value S _{t, p} ^ ^{(m) (i + 1)} of each sound source signal.

By applying linear prediction analysis to _{St, p} ^ ^(m) over all microphone numbers m (1 ≦ m ≦ M) and frame numbers t (0 ≦ t ≦ T−1), Equation (23) The updated all-pole parameter _s Θ ^ ^{(i + 1) shown} can be obtained.

更新された音源信号のパワースペクトル密度の推定値_ｓΛ_ｔ＾^{（ｉ＋１）}（ω）は、回帰行列推定値更新部３０５に入力される。 The updated all-pole parameter _s Θ ^ ^{(i + 1)} is input to the sound source signal power spectrum estimation value updating unit 304. The sound source signal power spectrum estimated value update unit 304 receives the all-pole parameter _s Θ ^ ^{(i + 1)} as an input and updates the estimated value _s Λ _{t t} ^{(i + 1)} (ω) of the power spectrum density of the sound source signal (step S304). The estimated value _s Λ _t ^{（(i + 1)} (ω) of the power spectrum density of the sound source signal is obtained from the input all-pole parameter _s Θ ^{（(i + 1) using} Equation (14), Equation (15), and Equation (16). Updated using. That is, _s Λ _t ^ ^{(i + 1)} (ω) is calculated by the equation (24).

The updated estimated value _s Λ _{t t} ^{(i + 1)} (ω) of the power spectrum density of the sound source signal is input to the regression matrix estimated value update unit 305.

The regression matrix estimated value update unit 305 includes an estimated value _s Λ _t ^ ^{(i + 1)} (ω) of the power spectrum density of the sound source signal, an initial value _w Θ ^ ^{(0) of} the separation matrix, or an updated estimated value of the separation matrix. _{With w} Θ ^ ⁽ⁱ⁾ and the observation signal as inputs, the regression matrix estimate _g Θ ^ ^{(i + 1)} is updated according to equation (27) (step S305).

右下の添え字は行列の大きさを表す。ここで、ｇ_ｋ，ｐ ^（ｍ）はＧ_ｋ，ｐのｍ番目の列を表すものとする。以降、ｇ_ｐを回帰行列の成分ベクトルと称する。成分ベクトルｇ_ｐの全周波数帯域に渡る集合{ｇ_ｐ}_{０≦ｐ≦Ｐ-1}は回帰行列_ｇΘに一致する。 The update rule of the regression matrix shown in Expression (27) will be described. First, each component of the regression matrix in the p-th frequency band is collected into a single vector as shown in Expression (28).

The subscript at the lower right indicates the size of the matrix. Here, g _{k, p} ^(m) represents the m-th column of G _{k, p} . Hereinafter referred to as a component vector of the regression matrix g _p. The set {g _p } _{0 ≦ p ≦ P−1 over} the entire frequency band of the component vector g _p matches the regression matrix _g Θ.

Next, an observation signal matrix Y _{t−1, p} one frame before in the p-th frequency band is defined as shown in Expression (30).

Using these values, the estimated value g _p ^ ^{(i + 1)} after updating the component vector of the regression matrix can be calculated by the equation (32).

By calculating equation (30) over all frequency band numbers p (0 ≦ p ≦ P−1), the regression matrix estimate _g Θ ^ ^{(i + 1)} can be updated. _G Θ ^ ^{(i + 1)} thus obtained is the solution of the equation (25). The updated regression matrix estimated value _g Θ ^ ^{(i + 1)} is input to the separation matrix estimated value update unit 306.
The separation matrix estimated value update unit 306 receives the estimated value _g Θ ^ ^{(i + 1)} of the regression matrix, the estimated value of the power spectrum density of the sound source signal, and the observation signal as inputs, and the estimated value _w Θ ^ ^{(i + 1)} of the separation matrix. Is updated according to equation (4) (step S306).

式（３３）に示す分離行列の更新則について説明する。式（３４）は例えば準ニュートン法を用いて解くことが出来る。マイクロホンの数が２本（Ｍ＝２、ステレオ録音）の場合、より効率的な閉形式のアルゴリズムが存在する。ここでは、そのアルゴリズムを用いて分離行列の推定値_ｗΘ＾^{（ｉ＋1）}を更新する例を説明する。

The update rule for the separation matrix shown in Expression (33) will be described. Equation (34) can be solved using, for example, the quasi-Newton method. If the number of microphones is two (M = 2, stereo recording), there is a more efficient closed-form algorithm. Here, an example will be described in which the estimated value _w Θ ^ ^{(i + 1)} of the separation matrix is updated using the algorithm.

First, the covariance matrix R _p of the dereverberation signal x _{t, p} ^{（(i + 1)} obtained by Expression (7) is defined for each microphone as Expression (34) and Expression (35).

In this case, the estimated value _w Θ ^ ^{(i + 1)} of the separation matrix can be obtained by the following algorithm.

First, the inverse square root V _p of the matrix R _p ⁽¹ ) that satisfies Expression (36) is calculated.

Here, I ₂ represents a two-dimensional unit matrix. Next, a unitary matrix E _p satisfying Expression (37) and d _p ⁽¹⁾ and d _p ⁽²⁾ determined accompanying the unit matrix are calculated.

式（３６）を全ての周波数帯域番号ｐ（０≦ｐ≦Ｐ−１）に渡って計算することで、分離行列の推定値_ｗΘ＾^{（ｉ＋1）}を得る。こうして求めた_ｗΘ＾^{（ｉ＋1）}が式（３１）の解である。 Specifically, E _p is obtained by eigenvalue decomposition of V _p ^H R _p ⁽²⁾ V _p . _{Then, V p, E p, W} p (i + 1) using the ^{d p (2)} ^ a is calculated by equation (38).

By calculating Equation (36) over all frequency band numbers p (0 ≦ p ≦ P−1), an estimated value _w Θ ^ ^{(i + 1)} of the separation matrix is obtained. The calculated _w Θ ^ ^{(i + 1)} is the solution of equation (31).

Further, instead of the algorithm described above, the estimated value _w Θ ^ ^{(i + 1)} of the separation matrix may be obtained using a known sound source separation technique. As well-known sound source separation techniques, FastICA, Bel-Seinovsky algorithm, Chikkoki-Umbehauen algorithm, JADE, etc. can be used. The sound source separation technology presented here is, for example, the reference documents “A. Hyvarinen, J. Karhunen, E. Oja, Imoto Nemoto (translation), Maki Kawakatsu (translation)“ A New World of Detailed Independent Component Analysis Signal Analysis ”Tokyo Denki University Publishers, 2005 ”.

The convergence determination unit 307 determines whether or not the regression matrix estimated value _g Θ ^ ^{(i + 1)} and the separation matrix estimated value _w Θ ^ ^{(i + 1)} updated as described above have converged (step S3). S307). As a method for determining whether or not it has converged, for example, if the number of iterations i has reached a predetermined number, it may be judged that it has converged, or a log likelihood function (equation (9) obtained by iteration). If the increment of the value of)) is smaller than a predetermined threshold value, it may be determined that the value has converged. The operations of step S303 to step S306 are repeated until convergence, and the estimated regression matrix estimated value _g Θ ^ ^{(i + 1)} and the separation matrix estimated value _w Θ ^ ^{(i + 1)} are output to the enhancement signal calculation unit 14. Is done. At the same time, both parameters may be recorded in the parameter estimated value recording unit 308.

〔Experimental result〕
An experiment was conducted for the purpose of confirming the effect of the acoustic signal enhancement method of the present invention. The experimental conditions will be described. As the sound source signal, the voices of one male and one female speaker were extracted from the ASJ-JNAS database. This sound source signal was reproduced from different positions in a room with a reverberation time of about 0.6 seconds, and recorded with two microphones. The sampling frequency during recording was 8 kHz. The acoustic signal enhancing device has the configuration described in the second embodiment. Polyband filter bank analysis was used for the band division unit. The number of band divisions was 256, and the thinning rate was 128.

The number of poles of the acoustic signal was Q = 12. Number dereverberation frames K _p is, K _{p =} 40 in the frequency band below 1500 Hz, and set to a small value extent that the frequency becomes higher at higher frequency bands. Convergence determination is assumed to have converged with 3 repetitions. The waveform of the acoustic signal indicating the result is shown in FIG. In FIG. 5, the vertical axis represents amplitude, and the horizontal axis represents time (seconds). FIG. 5 (a) is an observed acoustic signal recorded by one microphone, FIG. 5 (b) is a female speaker's voice emphasized by a conventional sound source separation technique, and FIG. 5 (c) is emphasized by a conventional sound source separation technique. 5 (d) is a voice of a female speaker emphasized in the present invention, FIG. 5 (e) is a voice of a male speaker emphasized in the present invention, and FIG. 5 (f) is a female talk. FIG. 5G shows a sound source signal of a male speaker.

  Comparing the enhanced speech by the conventional sound source separation technology with the enhanced speech by the present invention, the acoustic signal enhancement method of the present invention suppresses the reverberation component and achieves better sound source separation for both men and women. Can be seen. For example, a male speaker has not spoken for about 1 second at the beginning. However, the prior art cannot remove the interfering voice of the female speaker as shown in the first 1 second of FIG. On the other hand, the female voice in the same section of FIG. 5 (e) showing the emphasized voice of the male speaker according to the present invention is sufficiently suppressed. As described above, the acoustic signal emphasizing method of the present invention can remove the interfering voice as compared with the prior art. When the average improvement ratio of the S / N ratio in the experiment shown in FIG. 5 is compared, the present invention is 4.9 dB compared to 2.1 dB of the prior art. Thus, according to the acoustic signal emphasizing device of the present invention, a high quality enhanced acoustic signal can be obtained.

The acoustic signal emphasizing device and method based on the technical idea of the present invention are not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention. The processes described in the above-described apparatus and method are not only executed in time series according to the order described, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the process. .
Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape, etc., and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (ReWritable), etc. can be used as magneto-optical recording media, MO (Magneto Optical disc) can be used, and flash memory can be used as semiconductor memory.

The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.
Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

The figure which shows the function structural example of the acoustic signal emphasis apparatus 100 of this invention. The figure which shows the operation | movement flow of the acoustic signal emphasis apparatus 100. The figure which shows the function structural example of the parameter estimation part 30. FIG. The figure which shows the operation | movement flow of the parameter estimation part 30. FIG. It is a figure which shows the result of the sound source separation experiment using the acoustic signal emphasis apparatus 100, (a) is a figure which shows an observed acoustic signal, (b) is a figure which shows the voice of the female speaker emphasized with the conventional sound source separation technique. (C) is the figure which shows the voice of the male speaker emphasized by the conventional sound source separation technique, (d) is the figure which shows the voice of the female speaker emphasized by this invention, (e) is the man who emphasized by this invention The figure which shows the voice of a speaker, (f) is a figure which shows the sound source signal of a female speaker, (g) is a figure which shows the sound source signal of a male speaker. The figure which shows the function structural example of the conventional acoustic signal enhancement apparatus 600. FIG.

Claims (8)

A band dividing unit for dividing the observation acoustic signal in the time domain into observation signals for a predetermined number of frequency bands;
A parameter estimation unit for estimating a regression matrix for estimating the reverberation component up to the previous predetermined number of frames of the observed signal as an input, and a separation matrix for separating the sound source signal,
A reverberation removing unit that generates a reverberation removal signal by subtracting the reverberation component from the observation signal by calculating the reverberation component by convolving the regression matrix with the observation signal;
A sound source separation unit for multiplying the dereverberation signal by the separation matrix to generate an estimated value of the sound source signal;
A band synthesizing unit that synthesizes the estimated values of the sound source signal to generate a time-domain emphasized acoustic signal;
An acoustic signal enhancement device comprising :
The parameter estimation unit
Using the estimated value of the power spectrum density of the sound source signal, the estimated value of the separation matrix, and the observed signal to maximize the likelihood function that depends on both the regression matrix and the separation matrix. Estimate
The separation matrix, an acoustic signal enhancement device according to claim Rukoto determined as the separation matrix obtained with the separation matrix or source separation techniques to maximize the likelihood function. In the acoustic signal emphasizing device according to claim 1,
The parameter estimation unit
An observation signal recording unit for recording the observation signal;
A parameter estimation value initialization unit that sets initial values of the regression matrix estimation value and the separation matrix estimation value;
An all-pole parameter estimated value update unit that updates the all-pole parameter estimated value with the initial value or the estimated value of the regression matrix and the estimated value of the separation matrix and the observed signal as inputs;
A sound source signal power spectrum estimated value update unit that updates the estimated value of the power spectrum density of the sound source signal using the all-pole parameter estimated value as an input;
A regression matrix estimated value update unit that updates the estimated value of the regression matrix by inputting the estimated value of the power spectrum density of the sound source signal, the estimated value of the separation matrix, and the observed signal;
A separation matrix estimated value update unit that updates the estimated value of the separation matrix by inputting the estimated value of the regression matrix, the estimated value of the power spectrum density of the sound source signal, and the observed signal;
It is determined whether or not the estimated values of the regression matrix and the separation matrix have converged, and the regression matrix estimate and the separation matrix estimate when it is determined to have converged are separated from the regression matrix and the separation matrix. A convergence determination unit that outputs as a matrix ;
An acoustic signal enhancing apparatus comprising: In the acoustic signal emphasizing device according to claim 1 or 2 ,
The acoustic parameter emphasizing apparatus, wherein the parameter estimation unit includes a predetermined frame number varying unit that varies the predetermined frame number according to a frequency band of the observation signal. A band dividing process in which the band dividing unit divides the observation acoustic signal in the time domain into observation signals for a predetermined number of frequency bands;
Parameter estimation unit comprises: a parameter estimation step of estimating a regression matrix for estimating the reverberation component up to the previous predetermined number of frames of the observed signal as an input, and a separation matrix for separating the sound source signal,
A dereverberation unit, which convolves the regression matrix with the observed signal to obtain a reverberation component, subtracts the reverberation component from the observed signal, and generates a dereverberation signal;
A sound source separation unit that multiplies the dereverberation signal by the separation matrix to generate an estimated value of the sound source signal; and
A band synthesizing method including a band synthesizing process for synthesizing the estimated value of the sound source signal to generate a time domain enhanced acoustic signal ,
The parameter estimation process is as follows:
Using the estimated value of the power spectrum density of the sound source signal, the estimated value of the separation matrix, and the observed signal to maximize the likelihood function that depends on both the regression matrix and the separation matrix. Estimate
An acoustic signal emphasizing method, wherein the separation matrix is a process of obtaining the separation matrix as a separation matrix that maximizes the likelihood function or a separation matrix obtained by using a sound source separation technique . The acoustic signal enhancement method according to claim 4 ,
The parameter estimation process is as follows:
A parameter estimation value initialization unit that sets an estimation value of the regression matrix and an initial value of the estimation value of the separation matrix;
An all-pole parameter estimated value update unit that updates the all-pole parameter estimated value by using the initial value or the estimated value of the regression matrix, the estimated value of the separation matrix, and the observed signal as inputs. When,
A sound source signal power spectrum estimated value update unit updates the estimated value of the power spectrum density of the sound source signal with the all-pole parameter estimated value as an input, and
A regression matrix estimation value updating unit, wherein the regression matrix estimation value update unit updates the estimation value of the regression matrix by inputting the estimated value of the power spectrum density of the sound source signal, the estimation value of the separation matrix, and the observation signal;
A separation matrix estimated value update unit, which receives the estimated value of the regression matrix, the estimated value of the power spectrum density of the sound source signal, and the observation signal as input, and updates the estimated value of the separated matrix;
A convergence determination unit determines whether or not the estimated value of the regression matrix and the estimated value of the separation matrix have converged, and determines the estimated value of the regression matrix and the estimated value of the separation matrix when determined to have converged. A convergence determination step for outputting the regression matrix and the separation matrix , and
An acoustic signal emphasizing method comprising: The acoustic signal enhancement method according to any one of claims 4 to 5 ,
The acoustic signal enhancement method, wherein the parameter estimation step includes a predetermined frame number variable step of changing the predetermined frame number according to a frequency band of the observation signal. A device program for causing a computer to function as the acoustic signal enhancing device according to any one of claims 1 to 3 . A computer-readable recording medium on which any of the apparatus programs according to claim 7 is recorded.

Images