JP5698166B2 - Sound source distance estimation apparatus, direct ratio estimation apparatus, noise removal apparatus, method thereof, and program - Google Patents

Description

  INDUSTRIAL APPLICABILITY The present invention is applicable to, for example, a hands-free method for operating a device by voice input, and is used for a sound source distance measuring device, a noise removing device, and the like that estimate a distance from a microphone array to a sound source using a microphone array. The present invention relates to a direct ratio estimation apparatus, a method thereof, and a program.

  In the prior art shown in Patent Document 1, the received sound signal of the microphone array is converted to the frequency domain in order to obtain the direct ratio, and the power of each of the direct sound and the indirect sound is calculated using the spatial correlation matrix obtained from the signal. (See, for example, paragraphs [0025] to [0040] of Example 1).

JP 2011-53062 A

  In the method disclosed in Patent Literature 1, since direct sound and indirect sound coming from the same direction cannot be distinguished, all sounds coming from the direct sound direction are determined to be direct sounds. As a result, there is a problem that the direct sound power is overestimated (or the indirect sound power is underestimated), and the finally obtained direct ratio becomes larger than the true value.

  The present invention has been made in view of such problems, and distinguishes reverberant sounds coming from the direction of the direct sound, and estimates the direct sound power and the reverberant sound power. Provided is a direct ratio estimating device capable of obtaining a direct-to-reverberation energy ratio (DRR) close to a value, and a sound source distance estimating device and noise elimination using the direct ratio estimating device It is an object to provide apparatuses, methods and programs thereof.

  The direct ratio estimation apparatus of the present invention includes a received sound power estimation unit, a direct sound direction power estimation unit, a reverberation sound direction power estimation unit, a subtraction unit, and a direct ratio calculation unit. The reception sound power estimation unit obtains a power estimation value of the frequency domain signal using a frequency domain signal obtained by converting a reception signal received by a plurality of microphones included in the microphone array into a frequency domain. The direct sound direction power estimator is a power estimation value of a direct sound direction signal obtained by performing a process of mainly passing a signal component arriving from a direct sound source direction to a frequency domain signal, or a received sound signal. Thus, the power estimation value of the direct sound direction signal obtained by converting the signal subjected to the processing of mainly passing the signal component arriving from the direct sound source direction into the frequency domain is obtained. The reverberant sound direction power estimation unit passes signal components that mainly come from other than the direct sound source direction in the same directivity shape as the processing that mainly passes signal components that come from the direct sound source direction of the direct sound direction power estimation unit. The power estimation value of the reverberant sound direction signal obtained by performing the processing, or the signal obtained by performing the processing of passing the signal component mainly coming from outside the direct sound source direction to the received sound signal is converted into the frequency domain. To obtain the estimated power value of the reverberation direction signal. The subtracting unit outputs a direct sound power estimated value obtained by subtracting the power estimated value of the reverberant sound direction signal from the power estimated value of the direct sound direction signal. The direct ratio calculation unit uses the power estimation value of the frequency domain signal and the power estimation value of the reverberation sound direction signal, and indicates the ratio of the direct sound power estimation value to the power estimation value of the reverberation sound direction signal. Get.

  The sound source distance estimation apparatus of the present invention estimates the distance to the sound source using the direct ratio estimation value output from the direct ratio estimation apparatus. Moreover, the noise removal apparatus of this invention removes sounds other than a desired sound source using the above-described direct ratio estimation value.

  The direct ratio estimation apparatus of the present invention is a new apparatus that obtains the direct ratio by paying attention to its isotropy due to the strong diffusibility of reverberant sound. The direct ratio estimation method distinguishes between direct sound and reverberant sound from signals coming from the direct sound direction by using two or more beamformers with the same directivity shape realized by the microphone array, and sets the power of each. It can be estimated correctly. Therefore, the direct ratio estimation apparatus of the present invention can improve the estimation accuracy of the direct ratio.

  The sound source distance estimation apparatus of the present invention can accurately estimate the distance to the sound source based on the direct ratio estimation value obtained by the proposed direct ratio estimation method. Further, the noise removal apparatus of the present invention emphasizes or suppresses only the sound source component determined to be within a certain distance range by filtering the received sound according to the above-described direct ratio estimation value. can do. As a result, only the sound source at a specific distance can be accurately picked up by the microphone array.

The figure which shows an example of the scene using the sound source distance estimation apparatus. The figure which shows the propagation path of the sound indoors. The figure which shows the relationship between direct ratio and the distance between microphones. The figure which shows the principle corresponding to an Example notionally. It is a figure which shows the unit vector showing a sound source direction, (a) is a figure which shows the example of the three-dimensional unit vector r, (b) is the two-dimensional unit vector u. It is a figure which shows the two beam formers which have the same directivity shape and the main beam is directed in different directions, (a) is a beam former in which the beam is directed to the sound source direction, (b) is directed to the sound source direction. Shows the beamformer. The figure which shows the function structural example of the direct ratio estimation apparatus 100 of Example 1. FIG. The figure which shows the operation | movement flow of the direct ratio estimation apparatus 100. The figure which shows the function structural example of the direct ratio calculation part 44. The figure which shows the function structural example of direct ratio calculation part 44 '. Schematically illustrates an example of a directional shape of each reverberation directivity forming section 4431 ₁ ~4431 _N. The figure which shows the function structural example of direct ratio calculation part 44 '' The figure which shows the function structural example of the sound source distance estimation apparatus 400 of Example 2. FIG. FIG. 10 is a diagram illustrating a functional configuration example of a noise removal device 700 according to a third embodiment. The figure which shows the operation | movement flow of the noise removal apparatus 700. The figure which shows the function structural example of the process target signal production | generation part 72. FIG. The figure which shows the experimental condition of an effect confirmation experiment. The figure which shows the simulation result of direct ratio estimation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated. In the following description, the symbols “記号”, “^”, etc. used in the text should be written immediately above the original character, but immediately before the character due to restrictions on the text notation. It describes. In the formula, these symbols are written in their original positions.

Prior to the description of the embodiments, the principle corresponding to each embodiment will be described.
〔principle〕
FIG. 1 illustrates a scene where the sound source distance estimation apparatus 400 according to the second embodiment is used. A microphone array 11 and a speaker 12 are present in a room 10 having reverberation characteristics. The microphone array 11 and the speaker 12 are arranged at a distance.

  In this situation, we want to estimate the distance d between the speaker 12 and the microphone array 11. Therefore, the present invention estimates the distance between the sound source and the microphone array using the direct ratio estimation value.

  The direct ratio is the ratio of direct sound and indirect sound (also called reverberation sound) included in the received sound (for example, power ratio, power spectrum ratio, power spectrum density ratio, amplitude monotonically increasing function value) Ratio). A value representing the ratio of the power estimate value of the direct sound to the power estimate value of the indirect sound is referred to as “direct ratio estimate value”. For example, the value obtained by dividing the power estimate value of the direct sound by the power estimate value of the indirect sound may be used as the direct ratio estimate value, or the value obtained by dividing the power estimate value of the indirect sound by the power estimate value of the direct sound It is good also as a ratio estimated value, and it is good also considering one of those function values as a direct ratio estimated value. The power estimated value means a value that increases as the power increases. Examples of power estimates are power, power spectrum, power spectrum density, monotonically increasing function values of amplitude, their estimates, etc.

  FIG. 2 shows a sound propagation path from the sound source 21 to the microphone 22 when a microphone is placed indoors and a sound is recorded. The direct sound is a sound wave indicated by a thick solid line that directly reaches from the sound source 21 to the microphone. One reverberant sound is a sound wave indicated by a broken line that reaches the microphone 22 after the sound emitted from the sound source 21 is reflected by a wall, floor, ceiling, or the like.

  FIG. 3 shows the relationship between the direct ratio and the distance between the microphones. The horizontal axis in FIG. 3 is the distance from the microphone to the sound source, and the vertical axis is the direct ratio. In general, the indirect sound has a certain magnitude that does not depend on the distance from the microphone. In contrast to the indirect sound, the direct sound exhibits a characteristic that monotonously decreases as the distance from the microphone increases. The direct ratio obtained by dividing the direct sound by the indirect sound has a characteristic that decreases monotonously as the distance increases, as in the case of the direct sound. From this direct ratio, the distance between the microphone array and the sound source can be estimated from the received sound received by the microphone array 11. The direct ratio estimation apparatus of the present invention outputs a direct ratio estimated value. In addition, the noise removal apparatus of the present invention removes noise from the received sound signal in accordance with the direct ratio estimation value output from the direct ratio estimation apparatus.

FIG. 4 conceptually shows the idea of this principle. In general, when reverberation is sufficient, diffusivity can be assumed for reverberant sound, and it is known that reverberant sound can be modeled as sound arriving at the same magnitude from all directions when viewed from a microphone. Applying any beamformer BF1 output signal of the small microphone array 11, may be received sound reverberation direction power 23 in a predetermined directional shape D _1. Three arrows reverberation direction power 23, the magnitude of reverberation obtained by directional shape D ₁ are schematically represented.

Assuming that the position of the sound source 21 is already known, the direct sound power 25 that directly arrives at the small microphone array 11 from the sound source 21 has the directivity shape D ₀ of the beamformer BF ₀ as D _1, and its directivity. By setting the direction to the direction of the sound source 21, it is possible to receive the direct sound direction power 26 including the reverberation sound direction power having the same magnitude as the reverberation sound direction power 23.

  The direct sound power 25 can be obtained by subtracting the reverberant sound direction power 23 from the direct sound direction power 26 including the same reverberation component as the reverberant sound direction power 23. Next, this principle will be explained theoretically.

<Model of arrival of isotropic reverberation>
In the proposed method, a model considering the isotropic nature of reverberant sound is introduced. Here, an example in which the power spectral density or its estimated value is used as the power estimated value will be described, but this does not limit the present invention.

When the received sound signal of the m-th microphone of the microphone array composed of M (M ≧ 2) microphones is converted into the frequency domain by short-time Fourier transform or the like, the following frequency domain signal X ^(m) (ω, t) Is obtained.

Where ω is a frequency, H _D ^(m) (ω) is a direct sound transfer function from the sound source to the m-th microphone, and H _R ^(m) (ω) is a sound source to the m-th microphone. It is a transfer function of indirect sound, and S (ω, t) is a signal obtained by converting the sound of the sound source into the frequency domain. t is a time frame index.

  Here, it is assumed that the direct sound is coherent while the indirect sound is a diffuse sound because its main component is reverberation. In other words, when focusing on each direction of arrival, direct sound arrives only from the direction of the sound source, while indirect sound has the property of arriving with uniform power from all directions (hereinafter referred to as “isotropic”). . In the proposed method, the direct ratio is obtained by estimating the direct sound power and the indirect sound power by paying attention to the difference in these arrival characteristics.

As a prerequisite, the direct sound arrival direction (hereinafter referred to as “direct sound source direction”) is known, and direct sound and indirect sound coming from any direction can be regarded as plane waves. Uncorrelated with each other. At this time, the transfer functions H _D ^(m) (ω) and H _R ^(m) (ω) of the direct sound and the indirect sound from the sound source to the m-th microphone can be expressed as follows.

Here, H _Dref (ω) is a direct sound component of the transfer function from the sound source to the reference point (referred to as “reference point”) of the microphone array, and H _{Rref, θ} (ω) is an indirect sound in the direction θ viewed from the reference point. It is an ingredient. The reference point may exist inside the microphone array or may exist outside the microphone array. The inside of the microphone array is, for example, surrounded by a straight line passing through a plurality of microphones constituting the microphone array, a plane surrounded by a line segment passing through the plurality of microphones, or a surface passing through the plurality of microphones. It means the inside of a solid. The outside of the microphone array means a position other than the inside of the microphone array. For example, the distance between each of the plurality of microphones constituting the microphone array and the reference point is shorter than the distance between each microphone and the direct sound source. An example of the reference point is the center point of the microphone array, or the position of any one microphone. At this time, the propagation delay τ _θ ^(m) of the sound arriving from the direction θ from the reference point to the m-th microphone is expressed by the following equation.

Here the position p _m of the m-th microphone is expressed by Equation (5), the unit vector u representing the direct sound source direction as shown in FIG. 5 (a) and 5 (b) represented by the formula (6) . c is the propagation speed of the sound wave.

Θ _D is the direct sound source direction as seen from the reference point, j is an imaginary unit, and e is a natural logarithm (Equations (2) and (3)). Further, the integration with respect to θ is performed in the range of 0 ≦ θ <2π (the same applies to the following integration).

That is, the transfer functions H _D ^(m) (ω) and H _R ^(m) (ω) of the direct sound and the indirect sound are respectively transferred from the sound source to the reference point and from the reference point to the mth microphone. It can be expressed by being decomposed into phase difference components due to the propagation delay. Therefore, a microphone array input vector whose elements are frequency domain signals X ^(m) (ω, t) (mε {1,..., M}) ^→ x (ω, t) = [X ⁽¹⁾ (ω, t ,..., X ^(M) (ω, t)] ^T is expressed by the following equation. T represents transposition.

However, S _D (ω, t) = H _Dref (ω) S (ω, t), S _{R, θ} (ω, t) = H _{Rref, θ} (ω) S (ω, t). ^→ a _θ (ω) is a steering vector in the θ direction represented by Expression (8). Each element of the array manifold vector depends on the propagation delay τ _θ ^(m) . When direct sound and indirect sound can be regarded as plane waves, the propagation delay τ _θ ^(m) depends on the relative position and direction θ of each microphone relative to the reference point of the microphone array. For details of the array manifold vector, refer to Reference Document 1 “Taita Asano,“ Sound Array Signal Processing-Sound Source Localization / Tracking and Separation (Sound Technology Series edited by the Acoustical Society of Japan) ”, Corona Co., Ltd. Company, February 25, 2011, ISBN978-4-339-01116-6 ”, Chapter 1 (p1 to 26).

When an arbitrary beamformer (BF) is applied to the microphone array input, the output power spectral density (PSD) is changed to the output power spectral density (PSD) of each of direct sound and indirect sound shown in Equation (9). The sum of (BF) multiplied by the power gain | _Dθ (ω) | ² .

_{However, P D (ω) = E} [| S D (ω, t) | 2] t, P R, θ (ω) = E [| S R (ω, t) | 2] t, → w (ω ) Is a filter coefficient of the beamformer (BF), and R (ω) is an input of a microphone array having R _ij (ω) = E [X _i (ω, t) X _j ^* (ω, t)] _t in the ij component. It is a signal space correlation matrix. E [•] represents an expected value calculation.

<Direct ratio estimation using multiple beamformers>
In the sound field that can be assumed that the indirect sound isotropically arrives in Equation (9), the reverberant power P _R , _θ (ω) can be replaced with a constant ￣P _R (ω) that does not depend on the direction θ. The power spectral density can be expressed by equation (10).

Here, assuming that there are two beamformers BF0 and BF1 having the same directivity shape and having the main beam directed in different directions as shown in FIG. 6, the second term ∫ _θ | D on the right side of Expression (10). _θ (ω) | ² dθ are equal, and the output of each beamformer changes only with the first term on the right side, that is, the beamformer's power gain for direct sound.

Therefore, the output power spectral density P ₁ beamformer BF1 with its null (low point directivity sensitivity) from the output power spectral density P ₀ beamformer BF0 with its beam to the sound source direction _(omega) in the sound source direction _(omega ) Is subtracted, the direct sound power 25 can be obtained.

  Based on the above principle, reverberant sound coming directly from the sound source direction can be distinguished, and as a result, the accuracy of the direct ratio can be improved.

FIG. 7 illustrates a functional configuration example of the direct ratio estimation apparatus 100 according to the first embodiment. The operation flow is shown in FIG. The direct ratio estimation apparatus 100 includes a microphone array 41, a plurality of frequency domain converters 42 _{1 to} 42 _M, and a direct ratio calculator 44. Each functional configuration unit excluding the microphone array 41 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

Microphone array 41 is a plurality of microphones m _1, consisting of ... m _M. A plurality of frequency domain transform section 42 _1, ..., 42 _M, a plurality of microphones m _1, ... m _M received sound has been received sound signal x _m (n) are inputted, respectively, each received sound signal in the frequency domain It converts into a signal (step S42). The frequency domain converters 42 ₁ ,..., 42 _M sample the received sound signal x _m (n), for example, at a sampling frequency of 16 kHz and convert it into a digital signal, for example, 256 samples as one frame. In Step S42, discrete Fourier transform is performed to output a frequency component X _m (ω, t) (step S42). ω is a frequency, and t is a frame number. An A / D converter that converts the received sound signal x _m (n) into a digital signal is omitted.

Chokkan ratio calculation unit 44, a plurality of frequency domain transform section 42 _1, ..., 42 _m signal X _m (ω, t) in the frequency domain to output Chokkan ratio estimate DRR of the received sound signals as input (omega , T) is calculated (step S44).

  FIG. 9 shows a functional configuration example of the direct ratio calculation unit 44. The direct ratio calculation unit 44 includes a received sound power estimation unit 441, a direct sound direction power estimation unit 442, a reverberation sound direction power estimation unit 443, a subtraction unit 444, and a direct ratio calculation unit 445.

The reception sound power estimation unit 441 converts frequency reception signals received by a plurality of microphones included in the microphone array 41 into frequency domain signals X ₁ (ω, t),..., X _M Using (ω, t), a power estimation value of the frequency domain signal corresponding to the received sound signal is generated and output. This power estimated value is a power estimated value of the frequency domain signal X _m (ω, t) corresponding to any one microphone m (mε {1,..., M}) as shown in the equation (12). Alternatively, as in Expression (13), the power estimation values of the frequency domain signals X ₁ (ω, t),..., X _M (ω, t) may be weighted and averaged. In the first embodiment, the power spectral density P _{X, L} (ω) is obtained as the power estimation value of the frequency domain signal corresponding to the received sound signal.

However, L is the number of frames, and α _m is a non-negative weight to the microphone m set so as to satisfy the equation (14). E [•] represents an expected value calculation.

The direct sound direction power estimation unit 442 is obtained by performing a process of passing only the signal component coming from the direct sound source direction with respect to the frequency domain signals X ₁ (ω, t),..., X _M (ω, t). Obtained by converting the power estimation value P _DD (ω) of the direct sound direction signal or the signal obtained by performing the process of passing only the signal component coming from the direct sound source direction to the received sound signal into the frequency domain. A power estimate value P _DD (ω) of the direct sound direction signal is obtained. The power P _DD (ω) of the direct sound direction signal is the same as P ₀ (ω) in the above equation (11).

  The direct sound direction power estimation unit 442 includes a directivity forming unit 4421 and a power estimation unit 4422. The directivity forming unit 4421 forms directivity so that a directional beam is directed in a predetermined direction, and outputs a signal that has passed the directivity. The directivity of the directivity forming unit 4421 is set so that the main beam having directivity faces the direct sound direction. As a method of directivity formation, for example, the delayed sum beam described in Reference Document 1 (Taro Asano, “Sound Array Signal Processing-Sound Source Localization / Tracking and Separation” Corona, pp. 70-79)). A method such as forming can be used.

When the output of the directivity forming unit 4421 is expressed as Y _BF (ω, t), the power estimation value P _DD (ω) of the direct sound direction signal output from the power estimation unit 4422 is obtained by Expression (15).

Further, the output power spectral density of the power estimation value P _DD (ω) of the direct sound direction signal is expressed by Expression (16).

Here, | D _{0, θ} (ω) | ² corresponds to the power gain in the θ direction of the beam former BF ₀ described with reference to FIG.

  The reverberant sound direction power estimation unit 443 has the same directivity shape as the process of mainly passing the signal component arriving from other than the direct sound source direction through the signal component arriving from the direct sound source direction of the direct sound direction power estimation unit 442. Converts the estimated power of the reverberant sound direction signal obtained by passing the signal, or the signal that has been processed to pass the signal component that mainly arrives from outside the sound source direction to the received sound signal into the frequency domain. Thus, the power estimate value of the reverberation direction signal is obtained.

  Ideally, the reverberant sound direction power estimation unit 443 includes a reverberation directivity forming unit 4431 and a reverberation power estimation unit 4432. The directivity of the reverberation directivity forming unit 4431 is set so that the directivity main beam avoids the direct sound direction. The directivity shape is set to be the same as that of the directivity forming portion 4421. The directivity shapes of the reverberation directivity forming unit 4431 and the directivity forming unit 4421 are desirably set to be the same as much as possible. The setting of the directivity shape can be easily realized by the prior art.

The reverberation power estimation unit 4432 receives the reverberant sound received so as to avoid the direct sound direction, and outputs the power estimate value P _RD (ω) of the reverberation sound direction signal (Equation 17). Since the power estimation value P _RD (ω) of the reverberant sound direction signal is received so as to avoid the direct sound direction, by setting | D _{1, θD} | ² << _1, the direct sound component | D _{0, θ} (ω) | ² P _D (ω) is sufficiently small.

Here, | D _{1, θ} (ω) | ² corresponds to the power gain in the θ direction of the beam former BF1 described in FIG.

The subtracting unit 444 uses the power estimation value P _RD (ω) of the reverberation sound direction signal output from the reverberation power estimation unit 4432 from the power estimation value P _DD (ω) of the direct sound direction signal output from the direct sound direction power estimation unit 442. ) Is subtracted from the direct sound power estimated value ^ P _D (ω) (formula (18)).

The denominator of the equation (18) normalizes the direct sound power estimated value ^ P _D (ω) by the difference in power gain between the beamformers (BF) of the directivity forming unit 4421 and the reverberant directivity forming unit 4431. It is a term of.

The direct ratio calculation unit 445 uses the power spectral density P _{X, L} (ω) and the direct sound power estimated value ^ P _D (ω) output from the received sound power estimating part 441, and uses the direct sound power estimated value ^ P _D. A direct ratio estimated value DRR (ω), which is the ratio of the power of (ω) and the power estimated value of the reverberant sound direction signal, is obtained (formula (19)).

  Further, when the reception sound power output from the reception sound power estimation unit 441 is expressed by the equation (12) corresponding to any one microphone m (mε {1,..., M}), the direct ratio Can also be estimated by equation (20).

  Furthermore, the direct ratio can also be estimated by the equations (21) and (22) as the direct ratio independent of the frequency. In addition, since it is the value calculated | required for every frame number L, it describes with DRR ((omega)), However, The value calculated | required for every frequency for every frame is described with DRR ((omega), t).

  The direct ratio estimation method described above is a new method that pays attention to the fact that reverberant sound is isotropically arrives at the microphone array since it is a highly diffuse signal. By using two beamformers with the same directional shape realized by a microphone array, a direct sound component and an indirect sound component are correctly separated by obtaining a signal including direct sound and reverberation sound and a signal including only reverberation sound. As a result, the estimation accuracy of the direct ratio can be improved.

  Expressions (19), (20), and expressions (21), (22) may be a direct ratio estimated value DRR not expressed in decibels as follows.

[Modification 1]
FIG. 10 shows a functional configuration example of a direct ratio calculation unit 44 ′ in which the functional configuration of the reverberation sound direction power estimation unit 443 of the direct ratio calculation unit 44 is modified. Chokkan ratio calculating unit 44 'obtains the reverberation direction power _{P RD} (omega), a plurality (two or more) orientation of the reverberation sound direction power _{P RD1} an _{(omega) to P RDN} (omega) On average It is what I did.

The reverberation sound direction power estimation unit 443 ′ of the direct ratio calculation unit 44 ′ includes two or more reverberation directivity forming units 4431 _{1 to} 4431 _N , two or more reverberation power estimation units 4432 _{1 to} 4432 _N , and reverberation. It differs from the direct ratio calculation unit 44 in that a directional power calculation unit 4433 is provided. The direction of the main beam of the beam former of the reverberation directivity forming unit 4431 ₁ is, for example, the direction θ ₁ from the reference point. Direction of the main beam beamformer reverberation directivity forming section 4431 ₂ is the direction theta _1, the direction of the main beam beamformer reverberation directivity forming section 4431 _N is the direction theta _N.

FIG. 11 schematically shows the directivity shapes of the reverberation directivity forming units 4431 _{1 to} 4431 _N. The directivity shapes of the reverberation directivity forming units 4431 _{1 to} 4431 _N are different only in the direction θ of the main beam and have the same shape. The reverberant power estimation values P _RD1 (ω) in the respective directivity directions from the signals that have passed through the directivities of the respective reverberation directivity forming units 4431 _{1 to} 4431 _N by the reverberation power estimation units 4432 _{1 to} 4432 _N connected thereto. ~ _PRDN (ω) is determined.

The reverberation direction power calculation unit 4433 calculates the reverberation sound direction power P _RD (ω) by performing a weighted average (Expression 23) on the plurality of power estimation values P _RD1 (ω) to P _RDN (ω).

However, β _n is a non-negative weighting factor, and is set in advance so as to satisfy Expression (24). Since the reverberant sound direction power P _RD (ω) obtained in this way is a value obtained by averaging the reverberant sound direction powers in a plurality of directions, the accuracy can be improved. As a result, the accuracy of the direct ratio estimation value DRR (ω) can be improved.

[Modification 2]
FIG. 12 shows a functional configuration example of the direct ratio calculation unit 44 ″ obtained by changing the functional configuration of the reverberation sound direction power estimation unit 443 of the direct ratio calculation unit 44. The direct ratio calculation unit 44 ″ is a directivity generator. The direction of the main beam of the beam former of the unit 4421 and the reverberation directivity forming unit 4431 can be automatically set.

The direct ratio calculation unit 44 ″ is different from the direct ratio calculation unit 44 in that it includes a sound source direction estimation unit 446 and a beam former generation unit 447. The sound source direction estimation unit 446 is included in the microphone array 41. The direction of the sound source is estimated by using as input the frequency domain signals X ₁ (ω, t),..., X _M (ω, t) obtained by converting the received signals received by a plurality of microphones into the frequency domain. The direction of the sound source can be obtained by a conventional technique from the phase difference of the frequency domain signals X ₁ (ω, t),..., X _M (ω, t), for example. The estimation of the direction of the sound source is described in, for example, Chapter 7.2 of Reference Document 2 “Oga, Yamazaki, Kanada,“ Sound System and Digital Signal Processing ”published by the Institute of Electronics, Information and Communication Engineers”.

The beamformer generation unit 447 receives a sound source direction signal, generates a beamformer BF0 having a main beam in the sound source direction, and a beamformer BF1 in which the main beam is set so as to avoid the sound source direction. The former BF0 is output to the direct sound direction power estimating unit 442, and the beam former BF1 is output to the reverberant sound direction power estimating unit 443. The directivity forming unit 4421 of the direct sound direction power estimating unit 442 applies the beamformer BF0 and outputs the output signal Y _BF (ω, t) described above. The reverberant sound direction power estimation unit 443 outputs the reverberant sound direction power P _RD (ω) by applying the beam former BF1.

  In this way, the direct ratio calculation unit 44 ″ can automatically set the directivity shapes of the direct sound direction power estimation unit 442 and the reverberant sound direction power estimation unit 443.

  The operation of the direct ratio calculation units 44, 44 ', and 44' 'has been described above with an example of operating in the frequency domain. However, the technical idea of the present invention including the modification is applied to the operation in the time domain as it is. It is also possible to apply the idea of the direct ratio calculation unit 44 ″ to the direct ratio calculation unit 44 ′.

  FIG. 13 illustrates a functional configuration example of the acoustic distance estimation apparatus 400 according to the second embodiment. The acoustic distance estimation apparatus 400 includes a direct ratio estimation apparatus 100, a distance-direct ratio database (hereinafter referred to as a distance-direct ratio DB) 45, and a distance determination unit 46. The direct ratio estimation apparatus 100 is the same as that described in the first embodiment. Each functional configuration unit excluding the microphone array 41 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

Information regarding the relationship between the distance and the direct ratio is recorded in advance in the distance-direct ratio DB 45. Information relating to the relationship between the distance and the direct ratio is obtained by linearly interpolating a set (d ₁ , DRR ₁ ), (d ₂ , DRR ₂ ),. Or a functional expression d = f (DRR) indicating the relationship between the distance and the direct ratio of the function obtained from the above, the approximate function obtained from the set of (d ₁ , DRR ₁ ), (d ₂ , DRR ₂ ),. ). The function f (DRR) is described in, for example, the reference “M. Tohyama et. Al.” The Nature and Technology of Acoustic Space, “Academic Press, 1995.”.

  The distance determination unit 46 refers to the relationship between the direct ratio estimated value DRR input from the direct ratio estimation apparatus 100 and the distance and direct ratio recorded in the distance-direct ratio DB 45, A sound source distance estimated value ^ d corresponding to the ratio DRR is output.

When the pair (d ₁ , DRR ₁ ), (d ₂ , DRR ₂ ),..., Which associates the distance with the direct ratio is stored in the distance-direct ratio DB 45, the following three steps are performed. The sound source distance estimated value ^ d is obtained and output.

First step: Of the DRR ₁ , DRR ₂ ,... Stored in the distance-direct ratio DB 45, two direct ratios DRR _m adjacent to the direct ratio estimated value DRR obtained by the direct ratio estimating device 100 Obtain DRR _n .

Second step: Chokkan ratio DRR _m and DRR distance the distance d _m and d _n corresponding to each of the _n - obtained from Chokkan ratio DB 44.

Third step: the distance d _m and d _n from the sound source distance estimate ^ d as shown in Equation (25) obtained by linear interpolation.

Incidentally, from the distance d _m and d _n when estimating the sound source distance estimate ^ d, it may also be used DRR (omega) of a specific frequency (or frequency band). In that case, Expression (25) becomes the following expression.

  In addition, the distance determination unit 46 determines the sound source from the direct ratio estimated value DRR input from the direct ratio estimation apparatus 100 when the functional equation d = f (DRR) is stored in the distance-direct ratio DB 45. Calculate and output the estimated distance ^ d.

  FIG. 14 shows a functional configuration example of the noise removal apparatus 700 of the present invention. The operation flow is shown in FIG. The noise removal apparatus 700 includes the direct ratio estimation apparatus 100 described in the first embodiment, a processing target signal generation unit 72, a target signal adjustment unit 73, and an inverse frequency domain conversion unit 74.

The processing target signal generation unit 72 receives the frequency domain signals X _m (ω, t) output from the plurality of frequency domain conversion units 42 _{1 to} 42 _M in the direct ratio estimation apparatus 100 as input, and the processing target signal X (ω , T) is output (step S72). The processing target signal Y (ω, t) is a signal obtained by synthesizing the frequency domain signal X _m (ω, t) by, for example, an adding means (not shown). Before the addition, the signal X _m (ω, t) in each frequency domain may be multiplied by a weight.

FIG. 16 shows a more specific functional configuration example of the processing target signal generation unit 72. The processing target signal generation unit 72 includes a plurality of weight multiplication units 721 _{1 to} 721 _M and an addition unit 722. The plurality of weight multiplying means 721 _{1 to} 721 _M are frequency components X ₁ (ω, t),..., X _M (ω) of a plurality of received signals x _m (n) received by M microphones. , T) is multiplied by a weighting factor w _m (ω).

Weight multiplier unit 721 ₁ to weights used in ～721 _M, for example, when the M microphones of omnidirectional w _m = 1 / all frequency components by the M X ₁ (omega, t), .., X _M (ω, t) is averaged to stabilize the processing target signal Y (ω, t). Also, when M microphones have directivity, use only a specific microphone signal by setting w ₁ = 1, w _m = 0 (m = {2,..., M}). Can do. For example, if the filter coefficient of the weight beamforming is used by using a method described in Reference Document 2 “Oga, Yamazaki, Kanada,“ Sound System and Digital Signal Processing ”published by the Institute of Electronics, Information and Communication Engineers” An arbitrary directivity can be formed by the microphone array.

The adding means 722 adds all the frequency components X ₁ (ω, t),..., X _M (ω, t) multiplied by the weights, and outputs a processing target signal Y (ω, t).

  The target signal adjustment unit 73 receives the direct ratio estimated value DRR (ω) output from the direct ratio estimation apparatus 100 and the processing target signal X (ω, t) output from the processing target signal generation unit 72 as inputs. A post-processing signal Y (ω, t) in which the amplitude of the target signal X (ω, t) is adjusted is generated (step S73). The inverse frequency domain transform unit 74 transforms the processed signal Y (ω, t) into a time domain signal y (n) (step S74).

  The target signal adjustment unit 73 includes, for example, a distance calculation unit 731, a filter formation unit 732, and a multiplication unit 733. The distance calculation means 721 incorporates a function formula d = f (DRR) indicating the relationship between the distance between the microphone array 41 and the sound source and the direct ratio estimate value DRR, and the input direct ratio estimate value DRR. A sound source distance estimated value ^ d corresponding to is calculated (distance calculation step S731).

Filter formation section 732, as shown in equation (27), set as the sound source distance estimate ^ d is two sizes to emphasize the temporal frequency components take values between different threshold d _f and d _n Then, a filter that emphasizes only the sound source in the band-like region within the two distance sections is formed.

  Here, t and ω of G (ω, t) are received sound power estimation unit 441, direct sound direction power estimation unit 442, and reverberation sound direction power estimation unit 443 in the processing of the direct ratio estimation apparatus 100 described above. Thus, the same G (ω, t) is multiplied to all the frequencies included in the averaged L frames and the averaged frequency. Further, in the equation (22), the value of G (ω, t) is not necessarily 1 and 0, and may be a sufficiently different value such as 0.9 and 0.1, for example.

  The multiplier 733 multiplies the processing target signal X (ω, t) by the filter G (ω, t) to generate a post-processing signal Y (ω, t). Therefore, the processed signal Y (ω, t) is obtained by enhancing or suppressing the sound of the sound source located within the two distance sections, that is, in a specific distance range from the microphone array 41. The post-process signal Y (ω, t) is converted into a time domain signal y (n) by the inverse frequency domain converter 74.

In addition, this invention is not limited to the above-mentioned embodiment. For example, the second embodiment may be executed using DRR (ω) obtained in units of frequency ω as a direct ratio estimation value. A specific example of DRR (ω) is a direct ratio estimated value of Expressions (19) and (20) or Expressions (21) and (22). In this case, ^ d (ω) is obtained by substituting the direct ratio estimated value DRR (ω) into the relational expression d (ω) = f (DRR (ω)) prepared for each frequency ω in advance, A filter is formed as in 28). However, d _f (ω) and d _n (ω) are threshold _values prepared in advance and having different sizes.

The functional configuration included in the sound source distance measurement device, the direct ratio estimation device, or the noise removal device may be realized by an external device. For example, the sound source distance measuring device 400 and the noise removing device 700 may be connected to an external microphone array without including a microphone array to realize the same function. Similarly, the noise removal apparatus 700 does not include the frequency domain conversion unit and the inverse frequency domain conversion unit, and the same function may be realized by using an external frequency domain conversion unit and an inverse frequency domain conversion unit.
〔Experimental result〕
In order to confirm the effect of the present invention, a simulation experiment using a mirror image method was performed.

FIG. 17 shows the simulation conditions. FIG. 17 is a plan view, assuming a room having a width of 4 m, a depth of 6 m, and a height of 2.7 m. The wall sound absorption coefficient was set to α = 0.05 (reverberation time T ₆₀ = 1.8 seconds). A microphone array in which eight microphones were arranged in a circle was used, and the height of the reference point was 1.5 m. The height of the sound source was also 1.5 m.

FIG. 18 shows a result of comparison between the _actual measured DRR value DRR _actual (□) estimated from the impulse response, the present invention (▽), and the conventional method (◯) under these conditions. The DRR (▽) estimated by the method of the present invention is closer to the actually measured value DRR _actual (□) than the conventional method, and is improved by about 3 dB particularly when the sound source is far away.

  In general, the power of the indirect component is constant regardless of the distance of the sound source, whereas the power of the direct component is inversely proportional to the square of the distance. For this reason, in the case of a distant sound source, the power of the direct component is smaller than that of the indirect component, and even if the error included in the estimated direct component is small, the DRR estimation result is greatly affected. In the method of the present invention, the power of the indirect sound is obtained by suppressing the influence of the signal arriving from the sound source direction as much as possible by controlling the directivity of the microphone array, so that a more accurate estimation is possible, and the DRR can be applied to a sound source farther away. Can be estimated correctly.

  As described above, the new direct ratio estimation method of the present invention is a new method that assumes that the reverberant sound is isotropically arrives at the microphone array because it is a highly diffuse signal. The sound source direction by the beamformer with the same directional shape realized by the microphone array and the main beam direction set directly to the sound source direction, and the beamformer set the main beam direction to avoid the direct sound source direction The direct component and the indirect component coming from can be correctly separated, and as a result, the accuracy of the estimate of the direct ratio can be improved.

  Moreover, since the sound source distance estimation apparatus 400 uses an accurate direct ratio estimation value, the distance to the sound source can be accurately estimated. Moreover, since the noise removal apparatus 700 uses an accurate direct ratio estimation value, only a sound source located at a specific distance can be collected accurately.

  In addition, although the example expressed in decibel as the direct ratio estimated value DRR is shown in the equations (19) to (22), it goes without saying that the direct ratio estimated value may be obtained by the ratio of the power spectral density. The direct ratio estimation value may be obtained by multiplying the DRR value represented by the above formula by any constant, or the constant may be multiplied by the reciprocal of the DRR value represented by the above formula. It is good also as a direct ratio estimated value. The constant may be a monotonically increasing function value. That is, the direct ratio estimated value DRR according to the present invention is not limited to that represented by the above-described equations (19) to (22).

Note that the processes described in the above method and apparatus are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (7)

A received sound power estimation unit that obtains a power estimation value of the frequency domain signal using a frequency domain signal obtained by converting a received sound signal received by a plurality of microphones included in the microphone array into a frequency domain;
The power estimation value of the direct sound direction signal obtained by performing the process of mainly passing the signal component arriving from the direct sound source direction with respect to the frequency domain signal, or the direct sound source direction with respect to the received signal A direct sound direction power estimation unit that obtains a power estimation value of a direct sound direction signal obtained by converting the signal that has been processed to pass through the signal component mainly into the frequency domain;
It is obtained by performing a process of passing signal components mainly coming from other than the direct sound source direction with the same directivity shape as the process of mainly passing the signal components coming from the direct sound source direction of the direct sound direction power estimation unit. It is possible to convert the power estimation value of the reverberant sound direction signal obtained or the signal obtained by performing the process of passing the signal component mainly coming from outside the direct sound source direction to the received sound signal into the frequency domain. A reverberant sound direction power estimation unit for obtaining a power estimate of a reverberant sound direction signal;
A subtracting unit that outputs a direct sound power estimated value obtained by subtracting a power estimated value of the reverberant sound direction signal from a power estimated value of the direct sound direction signal;
Using the power estimation value of the frequency domain signal and the power estimation value of the reverberation sound direction signal, the direct ratio to obtain the direct ratio estimation value representing the ratio of the power estimation value of the direct sound to the power estimation value of the reverberation sound direction signal A ratio calculator;
A distance-direct ratio database that records the relationship between the estimated direct ratio and distance;
A distance determination unit that refers to the distance-direct ratio database with the direct ratio estimate as an input and outputs a sound source distance estimate corresponding to the direct ratio estimate;
A sound source distance estimation apparatus comprising: A received sound power estimation unit that obtains a power estimation value of the frequency domain signal using a frequency domain signal obtained by converting a received sound signal received by a plurality of microphones included in the microphone array into a frequency domain;
The power estimation value of the direct sound direction signal obtained by performing the process of mainly passing the signal component arriving from the direct sound source direction with respect to the frequency domain signal, or the direct sound source direction with respect to the received signal A direct sound direction power estimation unit that obtains a power estimation value of a direct sound direction signal obtained by converting the signal that has been processed to pass through the signal component mainly into the frequency domain;
It is obtained by performing a process of passing signal components mainly coming from other than the direct sound source direction with the same directivity shape as the process of mainly passing the signal components coming from the direct sound source direction of the direct sound direction power estimation unit. It is possible to convert the power estimation value of the reverberant sound direction signal obtained or the signal obtained by performing the process of passing the signal component mainly coming from outside the direct sound source direction to the received sound signal into the frequency domain. A reverberant sound direction power estimation unit for obtaining a power estimate of a reverberant sound direction signal;
A subtracting unit that outputs a direct sound power estimated value obtained by subtracting a power estimated value of the reverberant sound direction signal from a power estimated value of the direct sound direction signal;
Using the power estimation value of the frequency domain signal and the power estimation value of the reverberation sound direction signal, the direct ratio to obtain the direct ratio estimation value representing the ratio of the power estimation value of the direct sound to the power estimation value of the reverberation sound direction signal A ratio calculator;
A direct ratio estimation apparatus comprising: A direct ratio estimation device according to claim 2;
A processing target signal generation unit that generates a processing target signal by using the frequency domain signals output from the plurality of frequency domain conversion units in the direct ratio estimation apparatus;
A target for generating a post-processing signal in which the direct ratio estimation value output by the direct ratio estimation device and the processing target signal are input and the amplitude of the processing target signal is adjusted according to the direct ratio estimation value A signal conditioning unit;
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A received sound power estimation step for obtaining a power estimation value of the frequency domain signal using a frequency domain signal obtained by converting a received sound signal received by a plurality of microphones included in the microphone array into a frequency domain;
The power estimation value of the direct sound direction signal obtained by performing the process of mainly passing the signal component arriving from the direct sound source direction with respect to the frequency domain signal, or the direct sound source direction with respect to the received signal Direct sound direction power estimation step for obtaining a power estimation value of the direct sound direction signal obtained by converting the signal that has been processed to pass through the signal component mainly into the frequency domain;
Obtained by performing processing to pass signal components mainly coming from other than the direct sound source direction in the same directivity shape as processing to mainly pass signal components coming from the direct sound source direction in the direct sound direction power estimation step. It is possible to convert the power estimation value of the reverberant sound direction signal obtained or the signal obtained by performing the process of passing the signal component mainly coming from outside the direct sound source direction to the received sound signal into the frequency domain. A reverberant sound direction power estimating step for obtaining a power estimate of the reverberant sound direction signal;
A subtracting step for outputting a direct sound power estimated value obtained by subtracting a power estimated value of the reverberant sound direction signal from a power estimated value of the direct sound direction signal;
Using the power estimation value of the frequency domain signal and the power estimation value of the reverberation sound direction signal, the direct ratio to obtain the direct ratio estimation value representing the ratio of the power estimation value of the direct sound to the power estimation value of the reverberation sound direction signal A ratio calculating step;
Distance as input the Chokkan ratio estimate - Chokkan ratio Chokkan ratio recorded in the database by reference to the relationship between the estimated value and the distance, the corresponding sound source distance estimation and input Chokkan ratio estimate A distance determination step for outputting a value;
A sound source distance estimation method comprising: A received sound power estimation step for obtaining a power estimation value of the frequency domain signal using a frequency domain signal obtained by converting a received sound signal received by a plurality of microphones included in the microphone array into a frequency domain;
The power estimation value of the direct sound direction signal obtained by performing the process of mainly passing the signal component arriving from the direct sound source direction with respect to the frequency domain signal, or the direct sound source direction with respect to the received signal Direct sound direction power estimation step for obtaining a power estimation value of the direct sound direction signal obtained by converting the signal that has been processed to pass through the signal component mainly into the frequency domain;
Obtained by performing processing to pass signal components mainly coming from other than the direct sound source direction in the same directivity shape as processing to mainly pass signal components coming from the direct sound source direction in the direct sound direction power estimation step. It is possible to convert the power estimation value of the reverberant sound direction signal obtained or the signal obtained by performing the process of passing the signal component mainly coming from outside the direct sound source direction to the received sound signal into the frequency domain. A reverberant sound direction power estimating step for obtaining a power estimate of the reverberant sound direction signal;
A subtracting step for outputting a direct sound power estimated value obtained by subtracting a power estimated value of the reverberant sound direction signal from a power estimated value of the direct sound direction signal;
Using the power estimation value of the frequency domain signal and the power estimation value of the reverberation sound direction signal, the direct ratio to obtain the direct ratio estimation value representing the ratio of the power estimation value of the direct sound to the power estimation value of the reverberation sound direction signal A ratio calculating step;
A direct ratio estimation method comprising: The direct ratio estimation method according to claim 5,
And processing signal generating step of generating a processed signal a signal of a plurality of frequency domain obtained in the previous SL Chokkan ratio in estimating method as an input,
A target for generating a post-processing signal in which the direct ratio estimation value obtained by the direct ratio estimation method and the processing target signal are input and the amplitude of the processing target signal is adjusted according to the direct ratio estimation value A signal conditioning step;
An inverse frequency domain transforming step for transforming the processed signal into a time domain signal;
A noise removal method comprising:   A program for causing a computer to function as the sound source distance measuring device according to claim 1, the direct ratio estimating device according to claim 2, or the noise removing device according to claim 3.

Images