JP6448567B2 - Acoustic signal analyzing apparatus, acoustic signal analyzing method, and program - Google Patents

Description

  The present invention relates to an acoustic signal analysis device, an acoustic signal analysis method, and a program, and in particular, an acoustic signal analysis device, method, and method for performing dereverberation and sound source separation using acoustic signals acquired by a plurality of microphones. Regarding the program.

  A framework that processes multi-channel signals acquired by a plurality of microphones and performs sound source separation using the spatial information of the sound source as a clue is called microphone array signal processing.

  In recent years, with regard to microphone array signal processing, research on ad hoc microphone arrays using multi-channel recording by various recording devices such as voice recorders, notebook computers, smartphones, and video cameras has been actively conducted. In the case of using an ad hoc microphone array, a microphone array system can be easily and inexpensively constructed as compared with the case of using a conventional microphone array that requires special devices and wiring.

  Many of the conventional microphone arrays that have been commercialized have their microphones concentrated in a small scale, so the time difference of the audio signal between the recording channels is a clue for sound source separation, while the ad hoc microphone When an array is used, it becomes easier to place microphones in a wider range compared to conventional microphone arrays. Therefore, in addition to the time difference between recording channels, the intensity ratio of the audio signal is also used for sound source separation. It becomes a clue.

  In general, when reverberation and noise are superimposed on a speech signal and the process of obtaining the observation signal is regarded as a forward problem, the problem of separating and extracting only the target speech from the observation signal collected by the ad hoc microphone array can be regarded as an inverse problem. it can. If the information of the noise or the indoor transmission system is unknown, and there is an underdetermined condition that the number of sound sources is larger than the number of microphones, there are an infinite number of solutions to this inverse problem. Assumptions are needed.

  On the other hand, in recent years, an approach using multi-channel extension of non-negative matrix factorization (NMF) has attracted attention as an example of a sound source separation method in an underdetermined condition (Non-Patent Documents 1 and 2). .

  NMF is the decomposition of a non-negative matrix into the product of two non-negative matrices (base matrix and coefficient matrix), and applying NMF with the spectrogram regarded as a non-negative matrix means that the spectrogram is a low-rank non-negative value. This corresponds to approximating with a matrix, and means that the spectrum at each time is to be explained by a non-negative combination of spectrum templates corresponding to the number of columns of the base matrix. NMF multi-channel extension is an example of a multi-channel sound source separation method that assumes this structure in the power spectrogram of each sound source. In addition, an example of NMF multi-channel expansion under an overdetermined condition that the number of microphones is larger than the number of sound sources has been proposed (Non-Patent Document 3).

A. Ozerov and C. F_evotte, “Multichannel nonnegative matrix factorization in convolutive combination for audio source separation,” IEEE Transactions on Audio, Speech and Language Processing, vol.18, no. 3, pp. 550-563, 2010. A. Ozerov, C. F_evotte, R. Blouet and JL Durrieu, “Multichannel nonnegative tensor factorization with structured constraints for user-guided audio source separation,” in Proc. 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2011), pp. 257-260, 2011. Hirokazu Kameoka, Takuya Yoshioka, Mariko Hamamura, Jonathan Le Roux, Kunio Kashino, “Statistical model of speech signals based on composite autoregressive system with application to blind source separation,” in Proc. 9th International Conference on Latent Variable Analysis and Signal Separation (LVA / ICA 2010), LNCS 6365, pp. 245-253, Sep. 2010. T. Yoshioka, Tomohiro Nakatani, Masato Miyoshi, and Hiroshi G. Okuno, “Blind separation and dereverberation of speech mixture by joint optimization,” IEEE Transactions on Audio, Speech and Language Processing, vol. 19, no. 1, pp. 69 -84, 2011.

  In the conventional approach described above, constraints such as time invariance are placed on the indoor transmission system, and the inverse problem is formulated under those conditions. However, the ad hoc microphone array framework has the advantage that an array system can be constructed easily. On the other hand, since the position of each microphone is not fixed, there is a vulnerability that the relative positional relationship between the sound source and the microphone easily changes during recording. Thus, when the relative positional relationship between the sound source and the microphone changes during recording, the above assumption for the indoor transmission system is not satisfied, and the algorithm designed under the assumption cannot exhibit high performance with respect to sound source separation. There was a problem.

  The present invention has been made in view of the above circumstances, and even in a time-varying reverberation environment in which the relative positional relationship between the sound source and the microphone changes, each sound source is obtained from observation signals in which the sound of a plurality of sound sources is superimposed. An object of the present invention is to provide an acoustic signal analyzing apparatus, method, and program capable of accurately separating signals.

In order to achieve the above object, an acoustic signal analyzing apparatus according to the present invention receives time series data of acoustic signals collected by J microphones j as input, and the observed signal time frequency of each frequency k at each time l. A time-frequency analysis unit that outputs a component y _{j, k, l,} and an amplitude component A _{j, i,} of a time-varying steering vector that represents a transmission characteristic that is collected with a delay of time n from the sound source i to the microphone j _{. k, n} , at each time τ corresponding to a spectrum segment obtained by concatenating spectra of a plurality of frames, corresponding to each spectrum m and non-negative basis spectrum W _{i, k, m, τ at} each frequency k, corresponding to the spectrum segment An initial value setting unit for setting an initial value for each non-negative base onset H _{i, m, τ} of each base m and each frequency k at each time τ, and all of (j, k, l) The observed signal time frequency component y _j in the combination _{, k, l} and non-negative base spectra W _{i, k, m, τ of} each basis m and each frequency k at each time τ corresponding to the spectrum segment, and each time τ corresponding to the spectrum segment. The microphone is calculated based on the non-negative base onset H _{i, m, τ of} each base m and each frequency k, and each amplitude source A _{j, i, k, n} at each time n. j so that the distance from the power spectrogram model X _{j, k, l} is reduced, the base spectrum W _{i, k, m, τ} , the base onset H _{i, m, τ,} and the amplitude component A _{j , i, k, n,} and an end determination unit that repeatedly updates the parameter update unit until a predetermined end condition is satisfied.

An acoustic signal analysis method according to the present invention is an acoustic signal analysis method in an acoustic signal analysis apparatus including a time-frequency analysis unit, an initial value setting unit, a parameter update unit, and an end determination unit, wherein the time-frequency analysis Input the time-series data of the acoustic signals collected by each of the J microphones j, and output the observed signal time frequency components y _{j, k, l} at each frequency k at each time l to set the initial value Is a spectral element obtained by concatenating the amplitude components A _{j, i, k, n of} the time-varying steering vector representing the transmission characteristics collected from the sound source i to the microphone j with a delay of time _n , and the spectra of a plurality of frames. The non-negative base spectrum W _{i, k, m,} τ of each base m and each frequency k at each time τ corresponding to the piece, and each base m and each frequency k at each time τ corresponding to the spectrum fragment Nonnegative base of Onset H _{i, m,} set the initial value to each of _tau, the parameter updating unit, and the (j, k, l) for all the combinations of the observation signal time-frequency component y _{j, k, l,} Non-negative base spectra W _{i, k, m, τ of} each basis m and each frequency k at each time τ corresponding to the spectrum segment, each basis m and each base m at each time τ corresponding to the spectrum segment The power spectrogram model X of the microphone j calculated based on the non-negative base onset H _{i, m, τ of} each frequency k, each sound source i, and the amplitude components A _{j, i, k, n} at each time n. _The base spectrum W _{i, k, m, τ} , the base onset H _{i, m, τ,} and the amplitude component A _{j, i, k, n} so that the distance from _{j, k, l} becomes small. And the update by the parameter update unit is repeated until the end determination unit satisfies a predetermined end condition. Do.

  The program according to the present invention is a program for causing a computer to function as each part of the acoustic signal analysis apparatus.

As described above, according to the acoustic signal analyzing apparatus, method, and program of the present invention, each base m and each of the observation signal time frequency components y _{j, k, l} and each time τ corresponding to the spectrum segment are Non-negative base spectrum W _{i, k, m, τ for} each frequency k, non-negative base onset H _{i, m, τ} , for each base m and each frequency k at each time τ corresponding to the spectral segment, The distance from the power spectrogram model X _{j, k, l} of the microphone j calculated based on the amplitude components A _{j, i, k,} n of the time-varying steering vector at each sound source i and each time n is reduced. By repeatedly updating the base spectrum W _{i, k, m, τ} , the base onset H _{i, m, τ} and the amplitude component A _{j, i, k, n} , the sound of multiple sound sources is superimposed. That each sound source signal can be accurately separated from the observed signal Obtained.

It is the schematic which shows the structure of the acoustic signal analyzer which concerns on embodiment of this invention. It is a flowchart which shows the content of the acoustic signal analysis process routine in the acoustic signal analyzer which concerns on embodiment of this invention. It is a figure which shows the environment of the room which implements the evaluation experiment of an acoustic signal analyzer. It is a graph which shows an example of the change of Source to Distortion Ratio (SDR) with respect to the reflection coefficient of a wall in the evaluation experiment of an acoustic signal analyzer. It is a graph which shows an example of the change of SDR with respect to the disturbance to a transmission system in the evaluation experiment of an acoustic signal analyzer.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Principle of the invention)

  First, the proposed model of the present invention will be described.

(Convolution mixing model in the time-frequency domain)

  If the transmission system from the sound source to the microphone array is linear time-invariant and the reverberation component can be assumed to be within the window length of the time-frequency analysis, the signal obtained by the microphone array is described by instantaneous mixing of the sound sources. be able to.

  On the other hand, under the situation where reverberation components exceeding the window length cannot be ignored, the observation signal observed by the microphone is, for example, as shown in Non-Patent Document 4, using a convolutional mixture model in the time-frequency domain (1 )

Here, the variables i and k represent the sound source and frequency indexes, respectively, and l and n represent the time frame indexes. y ^ _{k, l} ∈ C ^J is an observed signal vector observed by the microphone array, j represents the index of the microphone, and J represents the number of microphones. a ^ _{i, k, n} is a steering vector from each sound source to the microphone, and corresponds to an echo component that arrives with a delay of n frames. Moreover, s _{i, k, l} is a complex spectrogram in the time-frequency domain of each sound source. Hereinafter, a variable representing a vector, a matrix, or a random variable is appended with “^”.

Assuming that the position of the sound source or microphone changes with time, the steering vector a ^ _{i, k, n} depends on the time l, and the mixing process of equation (1) is time-varying steering like equation (2). It is expressed using the vector a ^ _{i, k, n, l} .

Here, assuming that the complex spectrogram of each sound source i follows a complex Gaussian distribution, that is, s _{i, k, l to} N _C (0, P _{i, k, l} ), the observed signal vector y ^ _k at the microphone. _{, l} follow the distribution shown in equation (3). P _{i, k, l} represents the power spectrogram of the sound source i at time l, and a ^ ^H _{i, k, n, l} represents the Hermitian transpose of the time-varying steering vector a ^ _{i, k, n, l.} Represent.

Here, equation (4) shows an expression in which the time-varying steering vector a _{i, k, n, l} is decomposed into absolute value and declination elements.

Now, for minor changes in the acoustic signal analysis environment, such as the slight movement of a sound source or microphone, a special assumption is made that the time-varying steering vector a ^ _{i, k, n, l} is time-invariant and the phase component is time-varying. Such a mixing process is called "half-time deformation". That is, | a ^ _{j, i, k, n, l} | does not depend on time l. Therefore, if | a ^ _{j, i, k, n, l} | = A _{j, i, k, n} , Equation (4) can be expressed as Equation (5).

  Substituting equation (5) into equation (3) yields equation (6).

(Non-negative tensor double superposition model)

  In an ad hoc microphone array, a slight shift in the sampling frequency of the audio signal due to the asynchronous array elements and a slight change in the position of the sound source or microphone are likely to occur compared to a normal microphone array. It is necessary to handle this as a mixing process.

One solution when dealing with the mixing process of a half-time varying system is to estimate the time variation of the time-varying steering vector a ^ _{i, k, n, l} online and compensate for it, and then perform a known array signal processing. Is to apply.

On the other hand, a method of treating the phase component of the time-varying steering vector a ^ _{i, k, n, l} as a random variable that varies stochastically can be considered, but here, the time-varying steering vector a ^ _{i in} accordance with the latter concept. _{, k, n, l} are set to the following two conditions.

(Condition 1) φ _{j, i, k, n, l} and φ _{j ′, i, k, n, l ′} (j ≠ j ^′ or l ≠ l ^′ ) are independent of each other.

(Condition 2) φ _{j, i, k, n, l} follows a uniform distribution in the interval [0, 2π).

When marginalization is performed on the phase components φ _{j, i, k, n, l} under (Condition 1) and (Condition 2), E [ψ ^ _{i, k, n, l} ψ ^ ^H _{i, k, n , l} ] is a unit matrix, so equation (6) is expressed as equation (7).

Next, the power spectrogram P _{i, k, l} of the sound source i will be examined.

Assuming that the maximum value of the frequency index k is K and the maximum value of the time frame index l is L, in the multi-channel extension of NMF, the power spectrogram vector P ^ _i = (P _i) represented by K rows and L columns. _{, k, l} ) _{K × L} is represented by the product of two non-negative matrices. This is based on the assumption that the power spectrogram P _{i, k, l} of the sound source i at time l is represented by a non-negative combination of a limited number of spectral templates.

  However, it is considered that speech has a large feature not only in the spectrum in a single time frame l but also in its dynamics, that is, a local time change pattern. Therefore, rather than considering the spectrum in each time frame l as an element constituting speech, it is better to characterize the speech by considering the concatenation of the spectra over a plurality of time frames l as the element unit constituting the speech. It can be considered a favorable expression.

Therefore, in the present embodiment, the power spectrogram P _{i, k, l} is modeled by a mixed model that convolves a spectrogram segment template in which spectra over a plurality of time frames l are connected and an activation sequence. Specifically, the concept of known nonnegative matrix factor deconvolution (NMFD) is applied to modeling of the power spectrogram P _{i, k, l} . In this case, the power spectrogram P _{i, k, l} of the sound source i at time l is expressed by equation (8).

Here, W _{i, k, m, τ} represents the base spectrum of the power spectrogram P _{i, k, l} of the sound source i, and H _{i, m, l} represents the base onset. A variable m represents a base index, and a variable τ represents a time frame index. In the case of τ = {0} ({x} indicates that “x” is included in the elements of the set), it matches the mixed model using NMF.

(Parameter estimation)

Next, parameter estimation using maximum likelihood estimation will be described. First, the negative value of the log-likelihood function of equation (7) is defined as _CML as equation (9). Y ^ is a random variable corresponding to the observed signal y _{j, k, l} , P ^ is a random variable corresponding to the power spectrogram P _{i, k, l} , and A ^ is a time-varying steering vector a ^ _{i. , k, n, l} are random variables corresponding to absolute values. The observation signal y _{j, k, l} is an example of the observation signal time frequency component.

If _CML is specifically calculated using equation (9), equation (10) is obtained.

Here, the symbol = ^c indicates that terms other than the constant term are equal.

That is, the parameter estimation using the maximum likelihood estimation is performed by using the microphone j when the expression (8) is applied as the observed power spectrogram Y _{j, k, l} of the microphone j and the power spectrogram P _{i, k, l} of the sound source i. Reduced to the Itakura Saito distance minimization problem with the power spectrogram model X _{j, k, l,} and the objective function (11) is expressed as A _{i, j, k, n} ≧ 0, W _{i, k, m, tau} ≧ 0, and H _{i, m,} a problem of optimization under the constraint of _tau ≧ 0.

  Therefore, the expression (11) is optimized based on the principle of a known auxiliary function method. Here, an optimization algorithm is derived using β divergence, which is a more general divergence criterion, including the Itakura Saito distance.

  Although a detailed description of the optimization algorithm derivation process is omitted, the auxiliary function is designed by using Jensen's inequality and tangent inequality for the objective function expressed by equation (11). A multiplicative update formula shown in formula (15) can be obtained.

  However, ρ (β) is a value set according to the value of β, and is defined by equation (16).

(System configuration)

  Next, the acoustic signal analysis apparatus that analyzes the observation signals of a plurality of sound sources on which disturbance and reverberation components on the transmission system obtained by an ad hoc microphone array are superimposed and separated into each of the sound source signals of the plurality of sound sources will be described. The embodiment of the present invention will be described by taking the case of applying the invention as an example.

  As shown in FIG. 1, the acoustic signal analysis apparatus according to the embodiment of the present invention is a computer that includes a CPU, a RAM, and a ROM that stores a program for executing an acoustic signal analysis processing routine described later. It is configured and functionally configured as follows.

  The acoustic signal analysis device 100 includes an input unit 10, a calculation unit 20, a storage unit 30, and an output unit 40.

The input unit 10 inputs time series data of an observation signal y _j [l] including a plurality of sound sources i on which disturbance and reverberation components are superimposed on the transmission system. The storage unit 30 stores time series data of the observation signal y _j [l] input by the input unit 10. In addition, the storage unit 30 stores the result of each process to be described later, and stores the initial value of each parameter used in this process routine.

  The calculation unit 20 includes a time frequency analysis unit 21, an initial setting unit 22, a parameter update unit 23, an end determination unit 24, and a signal conversion unit 25.

The time frequency analysis unit 21 calculates the observed power spectrogram Y _{j, k, l} of the microphone j, for example, using the observed observation signal y _j [l] as the time series signal of the microphone j as an input. The calculated observation power spectrogram Y _{j, k, l} of the microphone j is stored in the storage unit 30. More specifically, the time-frequency analysis unit 21 performs time-frequency analysis using, for example, short-time Fourier transform (STFT) using time-series data of an observation signal observed by the microphone j as an input. Thus, the observed power spectrogram Y _{j, k, l} of the microphone j is calculated.

The initial setting unit 22 sets initial values of parameters A _{j, i, k, n} , W _{i, k, m, τ} and H _{i, m, τ} used in the processing described later. In addition, what is necessary is just to set the initial value of each parameter to an appropriate value, for example using a random number. In this case, the initial values of the parameters A _{j, i, k, n} , W _{i, k, m, τ} and H _{i, m, τ} are set to be non-negative values.

The parameter update unit 23 performs the above-described ((i, k, l)) based on W _{i, k, m, τ} and H _{i, m, τ} stored in the storage unit 30 for each combination of (i, k, l). The power spectrogram P _{i, k, l} of the sound source i is calculated according to the equation (8) and stored in the storage unit 30.

In addition, the parameter update unit 23 calculates, based on P _{i, k, l} and A _{j, i, k, n} stored in the storage unit 30 for each combination of (j, k, l). According to the above equation (12), the power spectrogram model X _{j, k, l} of the microphone j is calculated and stored in the storage unit 30.

Further, the parameter updating unit 23 stores Y _{j, k, l} , X _{j, k, l} , A _j, A stored in the storage unit 30 for each of all combinations of (j, i, k, n) _{. Based on i, k, n} and P _{i, k, l} , the amplitude component A _{j, i, k of the} time-varying steering vector is determined according to equation (13) so as to reduce the objective function of equation (11). _{, n} are updated and stored in the storage unit 30. At this time, the parameter updating unit 23 uses the updated A _{j, i, k, n} to update the power spectrogram model X _{j, k, l} of the microphone j according to the above equation (12), and stores it in the storage unit 30. Store.

In addition, the parameter update unit 23 stores Y _{j, k, l} , X _{j, k, l} , A _j, A stored in the storage unit 30 for each of all combinations of (i, k, m, τ) _{. Based on i, k, n} , W _{i, k, m, τ} and H _{i, m, τ} , the power spectrogram of the sound source i according to the equation (14) so as to reduce the objective function of (11). The base spectrum W _{i, k, m, τ} of P _{i, k, l} is updated and stored in the storage unit 30. At this time, the parameter updating unit 23 uses the updated W _{i, k, m, τ} and the power spectrogram P _{i, k, l of the} sound source i and the microphone j according to the above equations (8) and (12). The power spectrogram model X _{j, k, l} is updated and stored in the storage unit 30.

Further, the parameter updating unit 23 performs the processing for each combination of (i, m, τ), Y _{j, k, l} , X _{j, k, l} , A _{j, i,} stored in the storage unit 30 _{. Based on k, n} , H _{i, m, τ} and W _{i, k, m, τ} , the power spectrogram P _{i of the} sound source i according to the equation (15) so as to reduce the objective function of (11). _{, k, l} base onsets H _{i, m, τ} are updated and stored in the storage unit 30. At this time, the parameter updating unit 23 uses the updated H _{i, m, τ} and the power spectrogram P _{i, k, l of the} sound source i and the power spectrogram of the microphone j according to the above equations (8) and (12). The model X _{j, k, l} is updated and stored in the storage unit 30.

  The end determination unit 24 determines whether or not a predetermined end condition is satisfied. If the end condition is not satisfied, each process of the parameter update unit 23 is repeated. When it is determined that the end condition is satisfied, the end determination unit 24 proceeds to processing by the signal conversion unit 25.

Signal converting unit 25, the power spectrogram P _i of the sound source i in the storage unit 30 is _{stored, k,} basal spectrum W _i of _{l, k, m, tau,} and the power spectrogram P _i of the sound source _{i, k, l} of Based on the base onset H _{i, m, τ} , a sound source signal of the sound source i is generated and output to the output unit 40 for each of the plurality of sound sources i. The output unit 40 outputs each sound source signal of the plurality of sound sources i.

  As an end condition, the difference between the value of the objective function (11) with the number of repetitions L-1 and the value of the objective function (11) with the number of repetitions L is less than a predetermined threshold. What has become smaller can be used. Alternatively, the termination condition may be that the number of repetitions has reached a predetermined upper limit number.

(Operation of acoustic signal analyzer)

Next, the operation of the acoustic signal analysis apparatus 100 according to the present embodiment will be described. The time-series data of the observation signal y _j [l] obtained by an ad hoc microphone array composed of J microphones j and including a plurality of sound sources i on which disturbances and reverberation components to the transmission system are superimposed is an acoustic signal analyzer. 100 and stored in the storage unit 30. Then, in the acoustic signal analysis apparatus 100, an acoustic signal analysis processing routine shown in FIG.

First, in step S100, from the storage unit 30, reads the observed signal y _j [l], with respect to the observed signal y _j [l], performs time-frequency analysis using short-time Fourier transform, the observation of the microphone j The power spectrogram Y _{j, k, l} is calculated and the obtained observation power spectrogram Y _{j, k, l} of the microphone j is stored in the storage unit 30.

In step S102, initial values of A _{j, i, k, n} , W _{i, k, m, τ} and H _{i, m, τ} are set using random numbers and stored in the storage unit 30. To do.

Next, in step S104, the power spectrogram P _{i, k, l of the} sound source i according to the above equation (8) based on W _{i, k, m, τ} and H _{i, m, τ} set in step S102. Is calculated for each (i, k, l) combination and stored in the storage unit 30. Further, in step S104, the power spectrogram of the microphone j according to the above equation (12) based on A _{j, i, k, n} set in step S102 and P _{i, k, l} calculated in this step. The model X _{j, k, l} is calculated for each (j, k, l) combination and stored in the storage unit 30.

In step S106, Y _j calculated in step _{S100, k, l} and, set A _j in step _{S102, i, k, n} and, P _i calculated in step _{S104, k, l} and X _j, Based on _{k, l} , the amplitude component A _{j, i, k, n} of the time-varying steering vector is updated for each (j, i, k, n) combination according to the above equation (12), and the storage unit 30 To store.

In step S108, based on A _{j, i, k, n} updated in step S106 and P _{i, k, l} calculated in step S104, the power spectrogram model X _{j, k, l} are updated for each (j, k, l) combination and stored in the storage unit 30.

In step S110, Y _{j, k, l} calculated in step S100, X _{j, k, l} updated in step S108, A _{j, i, k, n} updated in step S106, and step S102 _Is based on H _{i, m, τ} and W _{i, k, m, τ set} in step (14) _, the base spectrum W _{i, k, m,} of the power spectrogram P _{i, k, l} of the sound source i according to the equation (14) _{. τ} is updated for each (i, k, m, τ) combination and stored in the storage unit 30.

In step S112, based on W _{i, k, m, τ} updated in step S110 and H _{i, m, τ} set in step S102, the power spectrogram P _{i, k, l} is updated for each (i, k, l) combination and stored in the storage unit 30. Further, based on P _{i, k, l} updated in this step and A _{j, i, k, n} updated in step S106, the power spectrogram model X _{j, k of the} microphone j according to the equation (12). _{, l} are updated for each (j, k, l) combination and stored in the storage unit 30.

In step S114, Y _{j, k, l} calculated in step S100, H _{i, m, τ} set in step S102, X _{j, k, l} updated in step S112, and update in step S106 On the basis of the power spectrogram P _{i, k, l} of the sound source i according to the equation (15), based on _{the generated} A _{j, i, k, n} and W _{i, k, m, τ} updated in step S110 The set H _{i, m, τ} is updated for each (i, m, τ) combination and stored in the storage unit 30.

In step S116, based on W _{i, k, m, τ} updated in step S110 and H _{i, m, τ} updated in step S114, the power spectrogram P _{i, k, l} is updated for each (i, k, l) combination and stored in the storage unit 30. Further, based on P _{i, k, l} updated in this step and A _{j, i, k, n} updated in step S106, the power spectrogram model X _{j, k of the} microphone j according to the equation (12). _{, l} are updated for each (j, k, l) combination and stored in the storage unit 30.

In the next step S118, the value of the objective function is calculated according to the equation (11) based on Y _{j, k, l} calculated in step S100 and X _{j, k, l} updated in step S116. Store in the storage unit 30. Then, the value of the objective function calculated in the previous step S118 is read from the storage unit 30, and the difference between the value of the objective function calculated in the current step S118 and the value of the objective function calculated in the previous step S108 is calculated in advance. It is determined whether or not the threshold value is smaller than a predetermined threshold value stored in the storage unit 30. If the difference is equal to or larger than the predetermined threshold value, it is determined that the end condition is not satisfied, and the above step is performed. It returns to S106 and repeats the process of said step S106-step S118.

On the other hand, if the difference is less than the predetermined threshold value, it is determined that the end condition is satisfied, and the basis of the power spectrogram P _{i, k, l} of the sound source i stored in the storage unit 30 is determined in step S120. Based on the spectrum W _{i, k, m, τ} and the base spectrogram H _{i, m, τ} of the power spectrogram P _{i, k, l} of the sound source i, for each of the plurality of sound sources i, the sound source signal of the sound source i Is output from the output unit 40, and the acoustic signal analysis processing routine is terminated.

(Implementation results)

  Next, for the purpose of showing the effectiveness of the method according to the present embodiment, a sound source separation experiment was performed under indeterminate conditions under two reverberant environments.

  First, in order to confirm the degree of robustness in a reverberant environment, a sound source separation experiment was performed in an environment with different reverberant strengths. Next, in order to confirm the degree of robustness against disturbance to the transmission system, a sound source separation experiment was performed when the position of the microphone changed during acquisition of the observation signal.

  As an inferior decision condition, three sound sources and two microphones were installed in the room, and an impulse response was generated using a mirror image method from the shape of the room, the position of the sound source, and the position of the microphone.

  FIG. 3 shows the shape of a room (hereinafter simply referred to as “room”) in which the sound source separation experiment is performed, the position of each sound source, and the position of each microphone. As shown in FIG. 3, the room has a rectangular size of 5 m × 10 m, “X” indicated by S1 to S3 indicates the position of the sound source, and “●” indicated by M1 and M2 Each microphone position is shown.

  As the sound source used in the sound source separation experiment, 15 utterances of 3 speakers in the ATR speech dialogue database were used. Of the sound sources 1 to 3 of the three speakers, the speakers of the sound sources 1 and 2 are women, and the speaker of the sound source 3 is a man.

  The intensity of reverberation was changed by adjusting the reflection coefficient of the room wall. Specifically, when the reflection coefficient of the room wall is 0.5, the reverberation time is 60 ms, and when the reflection coefficient of the room wall is 0.8, the reverberation time is 210 ms.

  In the sound source separation experiment, a comparison was made between a method using a known multichannel extensions of non-negative matrix factorization (MNMF) and a case using the proposed method according to the present embodiment. In the case of MNMF, since an instantaneous mixing model is assumed, it is expected that the performance will be degraded if reverberation components exist outside the STFT frame.

  Further, out of 15 prepared utterances of each speaker, one utterance was used as a separation signal, and the remaining 14 utterances were used as teacher data for prior learning. Note that NMFD and MNMF use NMF for the prior learning of the proposed method according to the present embodiment to learn the base spectrum, and each of the sound sources 1 to 3 has 40 and 20 bases, respectively. Learned. Further, the generalized KL divergence was used as the distance measure, and the source-to-distortion ratio (SDR) was used as the evaluation index. At this time, the STFT frame length was set to 32 ms, and the shift length was set to 16 ms.

  FIG. 4 is a graph showing an example of a change in the performance of each comparison method with respect to a change in the reflection coefficient of the room wall. The horizontal axis of the graph shown in FIG. 4 represents the reflection coefficient of the wall of the room, the vertical axis represents SDR, the graph 50 uses MNMF, and the graph 51 uses the proposed method according to the present embodiment. The graph of is shown. A value obtained by averaging the SDRs of the sound sources in the microphones is represented as SDR with respect to the reflection coefficient of the corresponding room wall.

  As shown in FIG. 4, the MNMF exhibits better performance than the proposed method according to the present embodiment in a range where the instantaneous mixing model with a low reflection coefficient can be considered to be established. However, as the reflection coefficient increases, the proposed method according to the present embodiment employs a mixed model that convolves the spectrogram segment template and the activation sequence, so that the reverberation component is accurately estimated, There is a tendency to show better performance than MNMF.

  Next, in order to confirm the degree of robustness against disturbance to the transmission system, a sound source separation experiment in the case where the position of the microphone changes during acquisition of the observation signal will be described.

  In this experiment, the observation signals used in the sound source separation experiment are connected by connecting the observation signals acquired by the microphones M1 and M2 shown in FIG. Was generated. In this case, a larger disturbance is generated in the phase component than in the amplitude component of the transmission system. The reflection coefficient of the room wall in this sound source separation experiment was set to 0.8.

  FIG. 5 shows a graph illustrating an example of a change in performance of each comparison method with respect to the deviation amount Δx of the observation signal acquisition position. The horizontal axis of the graph shown in FIG. 5 represents the deviation Δx of the observation signal acquisition position, the vertical axis represents SDR, and when the graph 52 uses MNMF, the graph 53 uses the proposed method according to the present embodiment. The graph when there is.

  As shown in FIG. 5, the graph 52 showing the performance in the MNMF shows that the value of SDR drops significantly as Δx increases, that is, as the disturbance increases. Specifically, the SDR when Δx = 0.2 is reduced to a value of about 86% with respect to the SDR when Δx = 0.

  On the other hand, in the case of the proposed method, the SDR when Δx = 0.2 is reduced only to a value of about 93% with respect to the SDR when Δx = 0. Therefore, the proposed method according to the present embodiment has less deterioration in sound source separation performance against disturbance to the transmission system and higher degree of robustness against disturbance to the transmission system than when using MNMF. be able to.

  As described above, in the proposed method according to the present invention, a spectrogram in which a half-time deformation model is set for a slight change in an acoustic signal analysis environment such as a minute movement of a sound source or a microphone, and spectra over a plurality of time frames are connected. By applying a mixed model that convolves the template of the fragment and the activation sequence, even in a time-varying reverberation environment where the relative positional relationship between the sound source and the microphone changes, it can The sound source signal of each sound source can be accurately separated.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the above-described acoustic signal analysis apparatus has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Calculation part 21 Time frequency analysis part 22 Initial setting part 23 Parameter update part 24 End determination part 25 Signal conversion part 30 Storage part 40 Output part 100 Acoustic signal analysis apparatus

Claims (5)

A time-frequency analysis unit that outputs time-series data of acoustic signals collected by J microphones j and outputs observation signal time-frequency components y _{j, k, l} of each frequency k at each time l;
Corresponds to time-varying steering vector amplitude components A _{j, i, k, n} representing the transmission characteristics collected from the sound source i to the microphone j with a delay of time _n , and a spectrum segment connecting multiple frames of spectrum. Non-negative base spectra W _{i, k, m,} τ of each base m and each frequency k at each time τ, and non-negative values of each base m and each frequency k at each time τ corresponding to the spectrum fragment An initial value setting unit for setting an initial value for each of the base onsets H _{i, m, τ} of
The non-negative values of each base m and each frequency k at each time τ corresponding to the observed signal time frequency component y _{j, k, l} and the spectrum segment in all combinations of (j, k, l). Base spectrum W _{i, k, m, τ} , non-negative base onset H _{i, m, τ} , each sound source i, and each time at each base m and each frequency k at each time τ corresponding to the spectrum fragment The base spectrum W _{i, k, m} so that the distance from the power spectrogram model X _{j, k, l} of the microphone j calculated based on the amplitude component A _{j, i, k, n} of _n becomes small. _{, τ} , the base onset H _{i, m, τ,} and the amplitude component A _{j, i, k, n} are updated,
An end determination unit that repeatedly performs update by the parameter update unit until a predetermined end condition is satisfied;
An acoustic signal analyzing apparatus including: The acoustic signal analyzer according to claim 1, wherein the Itakura Saito distance is used as a measure of the distance between the observed signal time frequency component y _{j, k, l} and the power spectrogram model X _{j, k, l} of the microphone j. An acoustic signal analysis method in an acoustic signal analysis device including a time-frequency analysis unit, an initial value setting unit, a parameter update unit, and an end determination unit,
The time frequency analysis unit inputs time series data of acoustic signals collected by each of the J microphones j, and outputs observed signal time frequency components y _{j, k, l} of each frequency k at each time l,
The initial value setting unit calculates the amplitude components A _{j, i, k, n of} the time-varying steering vector representing the transmission characteristics collected from the sound source i to the microphone j with a delay of time _n , and the spectrum of a plurality of frames. Non-negative base spectra W _{i, k, m, τ of} each base m and each frequency k at each time τ corresponding to the connected spectral segment, and each base m at each time τ corresponding to the spectral segment And an initial value for each of the non-negative base onsets H _{i, m, τ} of each frequency k,
The parameter updating unit includes each base m and each base at each time τ corresponding to the observed signal time frequency component y _{j, k, l} and the spectrum segment in all combinations of (j, k, l). A non-negative base spectrum W _{i, k, m, τ of} frequency k, each base m and non-negative base onset H _{i, m, τ} , at each time τ corresponding to the spectrum segment, The base spectrum so that the distance from the power spectrogram model X _{j, k, l} of the microphone j calculated based on the amplitude components A _{j, i, k, n} at each sound source i and each time n becomes small. Update W _{i, k, m, τ} , the basis onset H _{i, m, τ} and the amplitude component A _{j, i, k, n} ,
The acoustic signal analysis method in which the update by the parameter update unit is repeatedly performed until the end determination unit satisfies a predetermined end condition. The acoustic signal analysis method according to claim 3, wherein the Itakura Saito distance is used as a measure of the distance between the observed signal time frequency component y _{j, k, l} and the power spectrogram model X _{j, k, l} of the microphone j.   The program for functioning a computer as each part of the acoustic signal analyzer of Claim 1 or Claim 2.