JP7776016B2 - Signal processing device, signal processing method, and program - Google Patents

Description

The present invention relates to a technology for high-quality estimation of audio signals contained in signals recorded using a microphone.

When recording a speech signal using a microphone in a noisy, reverberant environment, the quality of the speech signal contained in the recorded signal is low because the microphone contains not only the desired speech components but also unwanted components such as noise, reverberation, and interfering sounds. Therefore, signal source extraction techniques have been actively researched to estimate the speech signals contained in the recorded signal with high quality. A known method for signal source extraction using multiple sensors is the convolutional beamformer (CBF; see Non-Patent Document 1). The Minimum-Variance Distortionless Response (MVDR) criterion has been used to optimize the CBF (see Non-Patent Document 1).

T. Nakatani, C. Boeddeker, K. Kinoshita, R. Ikeshita, M. Delcroix and R. Haeb-Umbach, "Jointly Optimal Denoising, Dereverberation, and Source Separation", IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 2267-2282, 2020.

However, when designing CBF based on the MVDR standard, the spatial information (spatial covariance matrix) of the target sound source to be extracted is compressed into a steering vector to design the CBF, which poses the problem that it is not possible to use all of the spatial information possessed by the target sound source.

The present invention aims to provide a signal processing device, signal processing method, and program that can use all of the spatial information of the target sound source by introducing the MaxSNR criterion instead of the MVDR criterion.

In order to solve the above problem, according to one aspect of the present invention, a signal processing device includes a second spatial covariance matrix estimation unit that estimates the spatial covariance matrix of a non-target sound source using an estimate of the spatiotemporal covariance matrix of the non-target sound source; a dereverberation filter estimation unit that estimates a dereverberation filter using the estimate of the spatiotemporal covariance matrix of the non-target sound source; a beamformer estimation unit that estimates a convolution beamformer using the estimate of the spatial covariance matrix of the observed signal or the target sound source, the estimate of the spatial covariance matrix of the non-target sound source, and the estimated dereverberation filter; and a sound source extraction unit that performs beamforming processing using the observed signal and the estimated convolution beamformer to estimate a sound source signal.

According to the present invention, by introducing the MaxSNR criterion, it is possible to use all of the spatial information of the target sound source.

FIG. 1 is a functional block diagram of a signal processing device according to a first embodiment. FIG. 2 is a diagram showing an example of a processing flow of the signal processing device according to the first embodiment. FIG. 10 is a functional block diagram of a signal processing device according to a second embodiment. FIG. 10 is a diagram showing an example of a processing flow of a signal processing device according to a second embodiment. FIG. 10 is a functional block diagram of a signal processing device according to a third embodiment. FIG. 11 is a diagram showing an example of a processing flow of a signal processing device according to a third embodiment. FIG. 1 is a diagram showing an example of the configuration of a computer to which the present technique is applied.

An embodiment of the present invention will be described below. In the drawings used in the following description, components having the same function and steps performing the same processing are denoted by the same reference numerals, and duplicate explanations will be omitted. In the following description, symbols such as "^" and " ^- " used in the text should normally be written directly above the character immediately following them, but due to limitations in text notation, they are written immediately before the character in question. In formulas, these symbols are written in their original positions. Furthermore, unless otherwise specified, processing performed on each element of a vector or matrix is assumed to apply to all elements of that vector or matrix.

<Sound source extraction problem>
The problem addressed in this embodiment is a sound source extraction problem, which involves estimating a sound source signal s _f,t or a spatial image s _f,t ^image =a _f s f,t obtained by removing reverberation from the sound source signal s _f,t from a signal x f _{,t observed} by a microphone. Here, a _f represents the acoustic transfer function of the sound source. The sound source signal is a signal based on a sound emitted by a sound source (target sound source) to be recorded by the microphone. In this embodiment, the target sound source is a speaker (hereinafter also referred to as the "target speaker"), the target sound is the speech uttered by the target speaker (hereinafter also referred to as the "target speech"), and the target signal is a signal corresponding to the target speech. However, the target sound source is not limited to these, and may be any sound source such as a sound source such as a musical instrument or a playback device, and the target sound is not limited to speech but may also be a sound other than speech. Sound sources other than the target sound source are also referred to as non-target sound sources.

<Key Points of the First Embodiment>
The steering vectors used by the MVDR CBF correspond to the principal components of the spatial covariance matrix V _S , and the MVDR CBF cannot use all of the spatial information contained in the spatial covariance matrix V _S. In this embodiment, the MaxSNR criterion is introduced as a new criterion for designing the CBF. When designing the CBF using the MaxSNR criterion, there is an advantage in that the spatial information (spatial covariance matrix V _S ) of the target sound source can be fully utilized.

First, the MaxSNR-based CBF will be explained. Let M be any integer equal to or greater than 2 representing the number of microphones, L+1 be the number of taps of the CBF, S ₊ be a set consisting of all non-negative definite matrices, let matrix A ^B be a square matrix with B rows and B columns, let matrix A ^B×C be a matrix with B rows and C columns, let ^R _N ∈ ^{S M+ML} ₊ be the space-time covariance matrix of the non-target sound source, let V _S ∈ ^{S M} ₊ be the spatial covariance matrix of the target sound source, and let O _A×B be a zero matrix with A rows and B columns,

Then, MaxSNR CBF ^w is defined as follows:

Note that when L=0, MaxSNR CBF becomes MaxSNR beamformer.

Furthermore, the MaxSNR CBF ^w of this embodiment has the feature that it can be decomposed into the product of the dereverberation filter ^G and the MaxSNR beamformer w for the instantaneous mixture, as shown in the following equation.

where the subscript opt denotes the optimal solution, and C is the set of all complex numbers. In other words, the MaxSNR CBF has the advantage of being able to jointly optimize the dereverberation filter ^G and the MaxSNR beamformer w.

To explain that equation (2) can be decomposed as equation (3), ^w and ^R _N are written as follows:



where S ₊₊ is the set of all positive definite matrices.

Here, the optimal solution of MaxSNR CBF ^w _is


can be obtained as follows, where


where I _M is an M-row, M-column identity matrix, and A ^H denotes the Hermitian transpose of A.

Note that equation (7) can be solved as the optimal eigenvector of the generalized eigenvalue decomposition.

V _S w _opt = λ _max V _N w _opt
where λ _max is the maximum eigenvalue.

In equation (8), ̂G is a multi-channel linear prediction (MCLP)-based dereverberation filter used in dereverberation. Also, V _N in equation (9) is the Schur complement of ̂R _N , which can be regarded as the spatial covariance matrix of the non-target sound source after reverberation.

First Embodiment
FIG. 1 is a functional block diagram of a signal processing device according to a first embodiment, and FIG. 2 shows the processing flow thereof.

The signal processing device 100 includes a first spatial covariance matrix estimation unit 110, a space-time covariance matrix estimation unit 120, a second spatial covariance matrix estimation unit 140, a dereverberation filter estimation unit 130, a beamformer estimation unit 150, a sound source extraction unit 160, and a spatial image estimation unit 170.

The signal processing device 100 receives an observation signal x _f,t observed by a microphone as input, estimates a sound source signal s _f,t or a spatial image s _{f ,} t ^image =a _f s _{f,t (with reverberation removed) of the sound source signal s f,t} , and outputs the signal. The observation signal may be, for example, an acoustic signal observed by a microphone array consisting of multiple microphones. The microphone output signal may be input as is, or an output signal stored in some kind of storage device may be read and used as the input, or the microphone output signal may be input after undergoing some processing. Here, f (f = 1, ..., F) indicates frequency, and t (t = 1, ..., T) indicates a frame number. The observation signal x _f,t and the sound source signal s _f,t are frequency-domain signals. Alternatively, a time-domain observation signal may be input, converted into a frequency-domain observation signal x _f,t in a frequency-domain transformer (not shown), and the estimated value of the sound source signal s _f,t may be converted into a time-domain sound source signal in a time-domain transformer (not shown) and output. The frequency domain transformation and the time domain transformation may be performed by any method, for example, a Fourier transform, an inverse Fourier transform, or the like.

The signal processing device 100 is a special device configured by loading a special program into a publicly available or dedicated computer having, for example, a central processing unit (CPU) and a main memory (RAM). The signal processing device 100 executes various processes under the control of the central processing unit. Data input to the signal processing device 100 and data obtained from each process are stored in the main memory, for example. The data stored in the main memory is read by the central processing unit as needed and used for other processes. At least a portion of each processing unit in the signal processing device 100 may be configured using hardware such as an integrated circuit. Each memory unit in the signal processing device 100 may be configured using, for example, a main memory such as RAM (Random Access Memory) or middleware such as a relational database or key-value store. However, each memory unit does not necessarily have to be internal to the signal processing device 100; it may be configured using an auxiliary memory device consisting of a hard disk, optical disk, or semiconductor memory element such as flash memory, and be configured external to the signal processing device 100.

Each part is explained below.

<First spatial covariance matrix estimation unit 110>
The first spatial covariance matrix estimation unit 110 estimates the spatial covariance matrix of the target sound source (S110) and outputs an estimated value V _S ∈ ^{S M} ₊ . Various methods can be used as a method for estimating the spatial covariance matrix of the target sound source. For example, the first spatial covariance matrix estimation unit 110 receives an observed signal x _f,t as input, estimates a section including a sound emitted by the target sound source from the observed signal x _f,t (hereinafter also referred to as a target signal), and estimates the spatial covariance matrix of the target sound source using the estimated target signal. Furthermore, if the direction of the target sound source is known, the spatial covariance matrix of the target sound source may be approximated in advance by experiment or simulation, and the approximate value may be used as the estimated value V _S ∈ ^{S M} ₊ .
<Spatio-temporal covariance matrix estimation unit 120>
The space-time covariance matrix estimation unit 120 estimates the space-time covariance matrix of the non-target sound source (S120) and outputs the estimated value ^R _N ∈S ^M+ML ₊ . Various methods can be used to estimate the space-time covariance matrix of the non-target sound source. For example, the space-time covariance matrix estimation unit 120 receives the observed signal x _f,t as input, estimates a section from the observed signal x _f,t that does not include sound emitted by the target sound source (hereinafter also referred to as the non-target signal), and estimates the space-time covariance matrix of the non-target sound source using the estimated non-target signal.
<Dereverberation Filter Estimation Unit 130>
The dereverberation filter estimation unit 130 receives the estimated value ^R _N of the spatiotemporal covariance matrix, estimates a dereverberation filter from the block matrices ^−P _N , ^−R _N included in the estimated value ^R _N (S130), and outputs the estimated dereverberation filter ^G. For example, the dereverberation filter is estimated by equation (8).

In addition,


That is, R _N is a block matrix consisting of elements from row 1, _column 1 to M, column M of the estimated value ^R _N , ^- P _N is a block matrix consisting of elements from row (M+1) to column (M+ML) of the estimated value ^R _N , ( ^- P _N ) ^H is a block matrix consisting of elements from row 1, column (M+1) to column M of the estimated value ^R _N , and ^- R _N is a block matrix consisting of elements from row (M+1) to column (M+ML) of the estimated value ^R N.

<Second spatial covariance matrix estimation unit 140>
The second spatial covariance matrix estimation unit 140 receives the estimated value ^R _N of the space-time covariance matrix as input, estimates the spatial covariance matrix of the non-target sound source from the block matrices R _N , ^−P _N , ^−R _N included in the estimated value ^R _N (S140), and outputs an estimated value V _N ∈S ^M+ML ₊ . For example, the spatial covariance matrix of the non-target sound source is estimated by equation (9).

is.

The second spatial covariance matrix estimator 140 may receive the estimated value ^R _N of the spatiotemporal covariance matrix and the dereverberation filter ^G estimated by the dereverberation filter estimator 130 as input, and estimate the spatial covariance matrix of the non-target sound source from the estimated value ^R _N and the dereverberation filter ^G using equation (9).

<Beamformer Estimation Unit 150>
The beamformer estimation unit 150 receives as input the estimated value V _S of the spatial covariance matrix of the target sound source, the estimated value V _N of the spatial covariance matrix of the non-target sound sources, and the estimated dereverberation filter ^G. The beamformer estimation unit 150 obtains a MaxSNR beamformer w opt for the instantaneous mixture using equation (7) from the estimated value V _S of the spatial covariance matrix of the target sound source and the estimated value V _N of the spatial covariance matrix of the non- _target sound sources.

Note that equation (7) can be solved as the optimal eigenvector of the generalized eigenvalue decomposition.

V _S w _opt = λ _max V _N w _opt
where λ _max is the maximum eigenvalue.

The beamformer estimation unit 150 estimates a convolution beamformer from the MaxSNR beamformer w _opt for the instantaneous mixture and the estimated dereverberation filter ^G using equation (3) (S150), and outputs the estimated convolution beamformer ^w.

<Sound source extraction unit 160>
The sound source extraction unit 160 receives the observed signal x _f,t and the estimated convolution beamformer ^w as input, performs beamforming processing using the following equation, estimates the sound source signal (S160), and outputs the estimated value y _f,t .

y _f,t =^w _f ^H ^x _f,t ∈ C
^w ∈ C ^M+ML
^w=[^w ₁ | … | ^w _F ]
^x _f,t =[x _f,t ^T |x _f,tD-1 ^T |…|x _f,tDL ^T ] ^T ∈ C ^M+ML
A ^H denotes the Hermitian transpose of A, A ^T denotes the transpose of A, Y=(y _t ) _t=1 ^T is an estimate of the source signal S, and D is the prediction delay.

<Spatial image estimation unit 170>
Although the scale of the convolution beamformer ^w _f for each frequency bin f is indefinite, it can be restored by estimating a vector u _f that approximates the spatial image s _f,t ^image using the following equation.

s _f,t ^image =a _f s _f,t ≒u _f y _f,t =(u _f w _f ^H )(^G _f ^H ^x _f,t )∈C ^M
^G=[^G ₁ | … | ^G _F ], where the vectors u and _f are required to satisfy the following conditions:

(i) w _f ^H u _f =1 (no distortion constraint)
(ii) u _f ∝V _N,f w _f (ideally, a _f ∝V _N,f w _f holds)
V _N =[V _N,1 | ... | V _N,F ]. Due to the two constraints, the vector u _f is uniquely determined as follows:

The spatial image estimation unit 170 receives as input the estimated value V _N of the spatial covariance matrix of the non-target sound source, the estimated value y _f,t , and the MaxSNR beamformer w _opt for the instantaneous mixture, calculates a vector u _f from the estimated value V _N and the MaxSNR beamformer w _opt for the instantaneous mixture according to equation (11), approximates a spatial image s _f,t ^image from the estimated value y _f,t and the vector u _f according to the following equation, and outputs an approximate value u _f y _f,t .

s _f,t ^image ≒u _f y _f,t
<Effects>
With the above configuration, by introducing the MaxSNR criterion, it is possible to use all the spatial information of the target sound source.

<Key Points of the Second Embodiment>
The MVDR CBF (a method for estimating CBF based on the MVDR standard) requires a steering vector of a target sound source to be estimated separately in advance, which causes problems such as the sound source extraction performance of the MVDR CBF being highly dependent on the steering vector estimation performance and poor usability. This embodiment solves these problems.

To estimate the MaxSNR CBF, it is necessary to calculate in advance the estimated value V _S of the spatial covariance matrix of the target sound source and the estimated value ^R _N of the spatiotemporal covariance matrix of the non-target sound source, as shown in equations (1) and (2).


In this embodiment, a Blind MaxSNR CBF will be described, which eliminates the need to calculate these two estimated values in advance. Note that "Blind" here means that no prior knowledge is required.

The Blind MaxSNR CBF of this embodiment is a method of estimating MaxSNR CBF by repeatedly performing calculations similar to the MaxSNR CBF given by equation (2) or equation (7).

The Blind MaxSNR CBF of this embodiment is defined as the following local optimum solution (Equations (20a) and (20b)) using an arbitrary super-Gaussian function φ:R _≧0 →R and the Schur complement matrix _VX of the following matrix ^R _X.



θ=(^w _f ) _f=1 ^F is a variable, y _f,t =(^w _f ) ^H ^x _f,t is an estimate of the source signal, y _t =[y _1,t | … |y _F,t ] ^T ∈C ^F , for vector A, ||A|| ₂ =√(A ^H A) is the Euclidean norm, and C on the right-hand side of equation (20b) is a constant that is adaptively and heuristically determined for each iteration of the algorithm to maximize or minimize the function.

More specifically, the MaxSNR CBF is optimized without prior knowledge by iterative optimization that alternately repeats a process of obtaining the spatiotemporal covariance matrix ^R _{Z ,f,} which is interpreted as an estimate of the spatiotemporal covariance matrix ^R N,f of the non-target sound source based on the following equations (21) and (22), and a process of estimating MaxSNR CBF ^w based on the following equations (23) to (26).

y _t ^k =[…|y _f,t ^k |…] ^T , y _f,t ^k =(^w _f ^k ) ^H ^x _f,t (22)




Here, k is an index indicating the number of repetitions.

In addition, in each iteration of the iterative optimization, the scale of MaxSNR CBF ^w _f is adjusted for each frequency f=1, . . . , F based on the following equation (27).

w _f ←(u _f,m ) ^* w _f =(e _m ^T u _f ) ^* w _f (27)
where m (1≦m≦M) is the index of the reference microphone, * indicates the complex conjugate, u _f is expressed by equation (11) (where V _Z,f is used instead of V _N,f ), and u _f,m =e _m ^T u _f ∈C is the m-th element of u _f .

Second Embodiment
The following description will focus on the differences from the first embodiment.

Figure 3 shows a functional block diagram of the signal processing device related to the first embodiment, and Figure 4 shows the processing flow.

The signal processing device 200 includes an initialization unit 201, a first spatial covariance matrix estimation unit 210, a space-time covariance matrix estimation unit 220, a second spatial covariance matrix estimation unit 240, a dereverberation filter estimation unit 230, a beamformer estimation unit 250, a sound source extraction unit 160, and a determination unit 280.

The signal processing device 200 receives an observation signal x _f,t observed by a microphone and an index m of a reference microphone as input, estimates a sound source signal s _f,t , and outputs it. Note that f indicates frequency, t indicates a frame number, and the observation signal x _f,t and the sound source signal s _f,t are frequency domain signals. However, a time domain observation signal may be input, converted into a frequency domain observation signal x _f,t in a frequency domain conversion unit (not shown), and the sound source signal s _f,t may be converted into a time domain sound source signal in a time domain conversion unit (not shown) and output. The frequency domain conversion and the time domain conversion may be performed by any method, and for example, a Fourier transform, an inverse Fourier transform, etc. may be used.

<Initialization unit 201>
The initialization unit 201 receives the index m of the reference microphone as input, sets the initial value ^w ⁰ =[^w ₁ ⁰ , . . . , ^w _F ⁰ ] of the convolution beamformer ^w to be estimated using the following equation (S201), and outputs it.

where e _m is the unit vector corresponding to the reference microphone.
<First spatial covariance matrix estimation unit 210>
The first spatial covariance matrix estimator 210 receives the observed signal x _f,t as input, estimates the spatial covariance matrix of the observed signal x _f,t using equations (28) to (30) (S210), and outputs the estimated value V _X.

^x _f,t =[x _f,t ^T |x _f,tD-1 ^T |…|x _f,tDL ^T ] ^T ∈ C ^M+ML

R _X,f is a block matrix consisting of elements from row 1 _, column 1 to M rows and column M of the estimate ^R _X,f , ^- P _X,f is a block matrix consisting of elements from row (M+1) to column (M+ML) of the estimate ^R _X,f , ( ^- P _X,f ) ^H is a block matrix consisting of elements from row 1, column (M+1) to column M of the estimate ^R _{X ,f} , ^- R X,f is a block matrix consisting of elements from row (M+1) to column (M+ML) of the estimate ^R X,f.

V _X =[V _X,1 ,...,V _X,f ,...,V _X,F ]
<Spatio-temporal covariance matrix estimation unit 220>
The space-time covariance matrix estimation unit 220 receives as input the convolution beamformer ^w ^k or its initial value ^w ⁰ estimated in the previous iteration and the observed signal x _f,t , and calculates and outputs the space-time covariance matrix ^R _Z =[^R _Z,1 ,...,^R _Z,f ,...,^R _Z,F ] interpreted as the estimated value ^R _N,f of the space-time covariance matrix of the non-target sound source based on the following equations (21) and (22) (S220).

y _t ^k =[…|y _f,t ^k |…] ^T , y _f,t ^k =(^w _f ^k ) ^H ^x _f,t (22)
When calculating the spatial-temporal covariance matrix ^R _Z,f for the first time, in other words, before estimating the convolution beamformer ^w by the beamformer estimation unit 250 described later, the output value of the initialization unit 201 is used as the initial value of the convolution beamformer ^w ⁰ = [^w ₁ ⁰ , ..., ^w _F ⁰ ].

<Dereverberation Filter Estimation Unit 230>
The dereverberation filter estimation unit 230 receives the spatiotemporal covariance matrix ^R _Z,f as input, estimates a dereverberation filter from ^−P _Z,f , ^−R _{Z, f included in the estimated value ^R Z,} f (S230), and outputs the estimated dereverberation filter ^G. For example, the dereverberation filter is estimated using equation (25).

In addition,


That is, R _Z,f is a block matrix consisting of elements from row 1, column 1 to column M of rows and M of the space-time covariance matrix ^R _Z,f , ^- P _Z,f is a block matrix consisting of elements from row (M+1) to column (M+ML) of rows and M of the space-time covariance matrix ^R Z _{, f} , ( ^- P _Z,f ) ^H is a block matrix consisting of elements from row 1, column (M+1) to column M of rows and (M+ML) of the space-time covariance matrix ^R Z _, f, ^- R Z _,f is a block matrix consisting of elements from row (M+1) to column (M+ML) of rows and (M+ML) of the space-time covariance matrix ^R Z,f.

<Second spatial covariance matrix estimation unit 240>
The second spatial covariance matrix estimation unit 240 receives the spatiotemporal covariance matrix ^R _Z,f as input, estimates the spatial covariance matrix of the non-target sound source from R _{Z ,f} , ^- P _Z,f , ^- R _{Z,f included in the spatiotemporal covariance matrix ^R Z,f} (S240), and outputs an estimated value V _Z,f ∈ ^{S M+ML} ₊ . For example, the spatial covariance matrix of the non-target sound source is estimated by equation (31).

is.

The second spatial covariance matrix estimator 240 may receive the spatiotemporal covariance matrix ^R _Z,f and the dereverberation filter ^G estimated by the dereverberation filter estimator 230 as input, and estimate the spatial covariance matrix of the non-target sound source from the spatiotemporal covariance matrix ^R _Z,f and the dereverberation filter ^G using equation (31).

<Beamformer Estimation Unit 250>
The beamformer estimation unit 250 receives as input an estimate V _X =[V _X,1 , ...,V _X,F ] of the spatial covariance matrix of the observed signal x _f,t , an estimate V _Z =[V _Z,1 , ...,V _Z,F ] of the spatial covariance matrix of the non-target sound source, and an estimated dereverberation filter ^G=[^G _,1 , ...,^G _F ]. The beamformer estimation unit 250 obtains w _f ^k+1 from the estimate V _X of the spatial covariance matrix of the observed signal x _f,t and the estimate V _Z of the spatial covariance matrix of the non-target sound source using equation (24).

The beamformer estimation unit 250 estimates a convolution beamformer from the MaxSNR beamformer w _f ^k+1 for the instantaneous mixture and the estimated dereverberation filter ^G according to equation (23) (S250).

The beamformer estimation unit 250 obtains a vector u f from the estimated value V _Z =[V _Z,1 , . . . , V _Z,F ] of the spatial covariance matrix of the non-target sound source and the convolution beamformer ^w ^k+1 _by the following equation.

Furthermore, the beamformer estimation unit 250 uses the m-th element u _f,m of the vector u _f to adjust the scale of MaxSNR CBF ^w _f ^k+1 for each frequency f = 1, ..., F based on the following equation (29), and outputs the scaled convolution beamformer ^w ^k+1 .

^w ^k+1 ←(u _f,m ) ^* ^w ^k+1 =(e _m ^T u _f ) ^* ^w ^k+1 (29)
<Sound source extraction unit 160>
The sound source extraction unit 160 receives the observed signal x _f,t and the estimated convolution beamformer ^w ^k+1 as input, performs beamforming processing using the following equation, estimates the sound source signal (S160), and outputs the estimated value y _f,t .

y _f,t =(^w _f ^k+1 ) ^H ^x _f,t ∈ C
^w ^k+1 ∈C ^M+ML
^w ^k+1 =[^w ₁ ^k+1 | … | ^w _F ^k+1 ]
<Determination unit 280>
The determination unit 280 determines whether the convergence condition is satisfied (S280). If the convergence condition is satisfied (YES in S280), the determination unit 280 outputs the estimated value y _f,t at that time as the output of the signal processing device and ends the processing. If the convergence condition is not satisfied (NO in S280), the determination unit 280 sends a control signal to each unit to repeat S220 to S160, thereby controlling the processing of each unit. Note that the estimated value y _f,t output by the sound source extraction unit 160 can be used in the space-time covariance matrix estimation unit 220, and the calculation of equation (22) can be omitted. Note that the convergence condition can be, for example, whether learning has been repeated a certain number of times (for example, several times), whether the difference between the convolution beamformer ^w ^k+1 before and after estimation is equal to or less than a predetermined threshold, etc.

<Effects>
With this configuration, it is possible to obtain the same effects as in the first embodiment. Furthermore, the Blind MaxSNR CBF of this embodiment is an ultra-fast method that can estimate the MaxSNR CBF with high accuracy in just a few iterations.

In this embodiment, an estimated value y _{f, t of the sound source signal s f} ,t is output. However, a spatial image estimation unit 170 may be provided, and an approximate value u f _{y f,t} of the spatial image s _f,t ^image may be calculated and output using the estimated value _{y f,t} at the point when the convergence condition is satisfied.

<Key Points of the Third Embodiment>
In this embodiment, as a by-product of the Blind MaxSNR CBF of the second embodiment, "Iteratively Reweighted MaxSNR CBF (IR-MaxSNR CBF)" is realized, which is a method for estimating MaxSNR CBF with high accuracy under the circumstances where the spatial covariance matrix V _S of the target sound source is known (= estimated in advance) and the spatiotemporal covariance matrix ^R _N of the unwanted sound is unknown (= not estimated in advance).

When the spatial covariance matrix V _S of the target sound source can be estimated with high accuracy, the MaxSNR CBF can be estimated with higher accuracy than the Blind MaxSNR CBF of the second embodiment by using that information.

Third Embodiment
The following description will focus on the differences from the second embodiment.

Figure 5 shows a functional block diagram of a signal processing device related to the third embodiment, and Figure 6 shows the processing flow.

The signal processing device 300 includes an initialization unit 201, a first spatial covariance matrix estimation unit 110, a space-time covariance matrix estimation unit 220, a second spatial covariance matrix estimation unit 240, a dereverberation filter estimation unit 230, a beamformer estimation unit 250, a sound source extraction unit 160, and a determination unit 280.

This embodiment differs from the second embodiment in that it includes a first spatial covariance matrix estimator 110 instead of the first spatial covariance matrix estimator 210. Note that the first spatial covariance matrix estimator 110 is as described in the first embodiment. Also, the beamformer estimator 250 differs from the second embodiment in that it uses an estimated value V _S of the spatial covariance matrix of the target sound source instead of an estimated value V _X of the spatial covariance matrix of the observed signal x _f,t . Other processing is the same as in the second embodiment.

<Other Modifications>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capabilities of the devices that execute the processes or as needed. Other modifications are possible within the scope of the present invention.

<Program and recording medium>
The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 7, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

The program describing this processing content can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic recording devices, optical disks, magneto-optical recording media, and semiconductor memory.

This program may be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs or CD-ROMs on which the program is recorded. Furthermore, this program may be distributed by storing it in a storage device of a server computer and transferring it from the server computer to other computers via a network.

A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, each time a program is transferred from a server computer to the computer, the computer may execute the process in accordance with the received program. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function simply by issuing execution instructions and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that does not directly instruct the computer but has properties that define computer processing).

In addition, in this form, the device is configured by executing a specified program on a computer, but at least some of the processing content may also be realized in hardware.

Claims (6)

a second spatial covariance matrix estimation unit that estimates a spatial covariance matrix of the non-target sound source using an estimated value of the space-time covariance matrix of the non-target sound source;
a dereverberation filter estimator that estimates a dereverberation filter using the estimated value of the spatiotemporal covariance matrix of the non-target sound source;
a beamformer estimation unit that estimates a convolution beamformer using an estimated value of a spatial covariance matrix of an observed signal or a target sound source, an estimated value of a spatial covariance matrix of the non-target sound source, and the estimated dereverberation filter;
a sound source extraction unit that performs beamforming processing using the observed signal and the estimated convolution beamformer to estimate a sound source signal;
Signal processing device. 2. The signal processing device of claim 1,
a first spatial covariance matrix estimation unit that estimates a section including a sound emitted from a target sound source (hereinafter also referred to as a target signal) from the observed signal and estimates a spatial covariance matrix of the target sound source using the estimated target signal;
a space-time covariance matrix estimation unit that estimates a section that does not include a sound emitted by a target sound source (hereinafter also referred to as a non-target signal) from the observed signal, and estimates a space-time covariance matrix of the non-target sound source using the estimated non-target signal,
the beamformer estimation unit estimates a convolution beamformer using an estimate of a spatial covariance matrix of the target sound source, an estimate of a spatial covariance matrix of the non-target sound source, and the estimated dereverberation filter.
Signal processing device. 2. The signal processing device of claim 1,
a first spatial covariance matrix estimator that estimates a spatial covariance matrix of the observed signal using the observed signal;
a space-time covariance matrix estimator that estimates a space-time covariance matrix of the non-target sound source by using the observed signal and the estimated convolution beamformer,
the beamformer estimation unit estimates a convolution beamformer using an estimate of a spatial covariance matrix of the observed signals, an estimate of a spatial covariance matrix of the non-target sound source, and the estimated dereverberation filter;
repeating the processes in the spatiotemporal covariance matrix estimator, the second spatial covariance matrix estimator, the dereverberation filter estimator, the beamformer estimator, and the sound source extractor until a convergence condition is satisfied;
Signal processing device. 2. The signal processing device of claim 1,
a first spatial covariance matrix estimation unit that estimates a section including a sound emitted from a target sound source (hereinafter also referred to as a target signal) from the observed signal and estimates a spatial covariance matrix of the target sound source using the estimated target signal;
a space-time covariance matrix estimator that estimates a space-time covariance matrix of the non-target sound source by using the observed signal and the estimated convolution beamformer,
the beamformer estimation unit estimates a convolution beamformer using an estimate of a spatial covariance matrix of the target sound source, an estimate of a spatial covariance matrix of the non-target sound source, and the estimated dereverberation filter;
repeating the processes in the spatiotemporal covariance matrix estimator, the second spatial covariance matrix estimator, the dereverberation filter estimator, the beamformer estimator, and the sound source extractor until a convergence condition is satisfied;
Signal processing device. a second spatial covariance matrix estimation step in which the computer estimates a spatial covariance matrix of the non-target sound source using an estimate of the space-time covariance matrix of the non-target sound source;
a dereverberation filter estimation step in which a computer estimates a dereverberation filter using the estimated spatiotemporal covariance matrix of the non-target sound source;
a beamformer estimation step in which a computer estimates a convolution beamformer using an estimated value of a spatial covariance matrix of an observed signal or a target sound source, an estimated value of a spatial covariance matrix of the non-target sound source, and the estimated dereverberation filter;
a sound source extraction step of performing beamforming processing by a computer using the observed signal and the estimated convolution beamformer to estimate a sound source signal,
Signal processing methods. A program for causing a computer to function as a signal processing device according to any one of claims 1 to 4.