JP5826465B2 - Instantaneous direct ratio estimation device, noise removal device, perspective determination device, sound source distance measurement device, method of each device, and device program - Google Patents

Description

  The present invention can be applied to, for example, a voice call, a hands-free method of operating a device by voice input, and the like, and is used when collecting sound by emphasizing only the sound of a sound source located within a specific distance range from a microphone. The present invention relates to an instantaneous direct ratio estimation device, a noise removal device, a perspective determination device, a sound source distance measurement device, a method of each device, and a device program.

  Conventionally, for the purpose of identifying the distance from a sound source and enhancing or suppressing only the sound from the sound source within a specific distance range, the power of the direct sound and reverberant sound is estimated from the signal received from the microphone, There is an idea of obtaining the ratio (for example, Non-Patent Document 1). The idea of obtaining the direct ratio with a conventional direct ratio estimating device will be described with reference to the drawings.

  FIG. 1 illustrates a scene in which the direct ratio estimation apparatus is used. Assume that a small microphone array 11 is surrounded by, for example, four speakers 12 to 14 for a conference. It is assumed that a television 16, a telephone 17, and a speaker 18 for broadcasting in the hall are arranged in the conference room. In such a scene, the speakers 12 to 14 located within a predetermined distance range (within a circle indicated by a broken line) around the small microphone array 11 without picking up the voice of the in-house broadcast or the sound of the telephone. I want to collect only the utterances.

  In order to distinguish the distance from the microphone array to the sound source, attention is paid to the ratio of direct sound and indirect sound (reverberation sound) included in the received sound (hereinafter referred to as direct ratio). FIG. 2 shows a sound propagation path from the sound source 21 to the microphone 22 when a microphone is placed indoors and a sound is recorded. The direct sound is a sound wave indicated by a thick solid line that directly reaches from the sound source 21 to the microphone. One reverberant sound is a sound wave indicated by a broken line that reaches the microphone 22 after the sound emitted from the sound source 21 is reflected by a wall, floor, ceiling, or the like.

  FIG. 3 shows the relationship between the direct ratio and the distance between the microphones. The horizontal axis in FIG. 3 is the distance from the microphone to the sound source, and the vertical axis is the direct ratio. In general, the indirect sound has a certain magnitude that does not depend on the distance from the microphone. In contrast to the indirect sound, the direct sound exhibits a characteristic that monotonously decreases as the distance from the microphone increases. The direct ratio obtained by dividing the direct sound by the indirect sound has a characteristic that decreases monotonously as the distance increases, as in the case of the direct sound.

  The conventional direct ratio estimation device can estimate the direct ratio from the received sound, and can estimate the distance of the sound source included in the received sound from the microphone array.

Y.Hioka, K.Niwa, S.Sakauchi, K.Furuya, and Y.Haneda. Estimating direct-to-reverberant energy ratio based on spatial correlation model segregating direct sound and reverberation.Proceedings of 2010 IEEE International Conference of Acoustics, Speech and Signal Processing (ICASSP2010), pages 149-152, 2010. Yusuke Hioka, Sumio Hannai, Kenichi Furuya, Yoichi Haneda, “Examination of sound collection by distance based on direct ratio of received signal”, Acoustical Society of Japan 2009 Autumn Meeting, pp.633-634

  In the conventional method, when the sound source is moving and the direct ratio changes from moment to moment, there is a problem that the direct ratio following the change cannot be estimated.

  This invention was made in view of such problems, and even when the sound source moves, an instantaneous direct ratio estimation device that can correctly estimate the direct ratio without observing the signal for a long time, and It is an object of the present invention to provide a noise removal device, a perspective determination device, a sound source distance measurement device, a method of each device, and a device program using the same.

  The instantaneous direct ratio estimation apparatus of the present invention includes a microphone array, a plurality of frequency domain conversion units, a spatial correlation matrix calculation unit, an eigenvalue expansion unit, and an eigenvalue distribution calculation unit. The microphone array is composed of a plurality of microphones provided at overlapping positions by parallel movement of a small microphone array composed of a small number of microphones. The plurality of frequency domain converters each receive a received sound signal received by a microphone array and convert the received sound signal into a frequency domain signal. The spatial correlation matrix calculation unit receives the frequency domain signals output from a plurality of frequency domain transformation units, and uses the frequency domain signals to obtain the small spatial correlation obtained by the sum or average of the spatial correlation matrices obtained for each small microphone array. Calculate the matrix. The eigenvalue expansion unit expands the small space correlation matrix into eigenvalues, decomposes them into eigenvalues and eigenvectors, and outputs the eigenvalues. The eigenvalue distribution calculation unit converts the eigenvalue distribution ratio into a numerical value and outputs a direct ratio equivalent value.

  Further, the noise removal apparatus of the present invention includes the instantaneous direct ratio estimation apparatus of the present invention, and additionally includes a processing target signal generation unit, a target signal adjustment unit, and an inverse frequency domain conversion unit. It has.

  The instantaneous direct ratio estimation apparatus according to the present invention provides a plurality of different positions instead of the method of obtaining an average of spatial correlation matrices at a plurality of times, which has been performed in the prior art, when obtaining a cross-correlation from observation signals of a microphone array. The sum or the average of the spatial correlation matrix obtained from the observation signal of the small microphone array arranged in is obtained. Since the average of the spatial correlation matrix of the small microphone array is obtained at each time, a correct direct ratio equivalent value can be estimated even when the direct ratio changes from moment to moment.

  In addition, from this, the noise removal apparatus of the present invention can extract only the direct sound from which noise is removed even if the sound source moves.

The figure which shows an example of the scene using the conventional direct ratio estimation apparatus. The figure which shows the propagation path of the sound indoors. The figure which shows the relationship between direct ratio and the distance between microphones. The figure which shows an example of the microphone array of this invention. The figure which shows the function structural example of the instantaneous direct ratio estimation apparatus 100 of this invention. The figure which shows the operation | movement flow of the instantaneous direct ratio estimation apparatus 100. The figure which shows the function structural example of the noise removal apparatus 200 of this invention. The figure which shows the operation | movement flow of the noise removal apparatus. The figure which shows the function structural example of the process target signal production | generation part 46. FIG. The figure which shows the function structural example of the perspective determination apparatus 300 of this invention. The figure which shows the function structural example of the direct ratio estimation part 84. FIG. The figure which shows the function structural example of the process target signal generation part 83. FIG. The figure which shows the function structural example of the instantaneous direct ratio estimation apparatus 400 of this invention. The figure which shows the function structural example of the distance determination apparatus 500 of this invention. The figure which shows the function structural example of the sound source distance measuring apparatus 600 of this invention. The figure which shows the function structural example of the noise removal apparatus 700 of this invention. The figure which shows the operation | movement flow of the noise removal apparatus 700. The figure which shows the function structural example of instantaneous direct ratio estimation apparatus 400 'of this invention. It is a figure which shows the other example of the microphone array of this invention, (a) is a figure which shows the example which made the number of a few microphones three, (b) is a figure which shows the example which has arrange | positioned a small number of microphones three-dimensionally. .

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated. In the following description, the symbols “￣”, “^”, etc. used in the text should be written directly above the immediately preceding character, but immediately after the character due to restrictions on the text notation. It describes. In the formula, these symbols are written in their original positions.

  Prior to the description of the embodiments, the idea of the present invention will be described.

[Concept of this invention]
A conventional method for determining the direct ratio is to convert a received signal received by a plurality of microphones into a signal in the frequency domain, and signal X ₁ (ω, l),..., X _M (ω, l) in the frequency domain. ) Is vectorized, the spatial correlation matrix R (ω, l) shown in Equation (1) is calculated using the input signal, the power of the direct sound and the indirect sound is estimated from the spatial correlation matrix, and the direct ratio is calculated. Ask.

  Here, T is a matrix transposition, H is a conjugate transposition, L is the number of frames for which an average is obtained, and M is the number of microphones. As is clear from the equation (1), the direct ratio obtained by the conventional technique is a value calculated based on the average of the spatial correlation matrix of a predetermined number of frames. Therefore, as in the case of a moving sound source, the direct ratio that changes from moment to moment cannot be accurately estimated.

The idea of the present invention is new in that a spatial correlation matrix before time averaging is used. As each component of the spatial correlation matrix, an instantaneous (arbitrary one frame) frequency domain signal is used as shown in Expression (3). However, in formula (3), [] _ij represents a component of the i-th row and j-th column of the matrix.

  In the present invention, as the microphone array, as shown in FIG. 4, for example, another small microphone array is arranged at a position obtained by translating a small microphone array composed of two microphones arranged with an interval d therebetween. In addition, a microphone array 40 including a plurality of microphones is used. Then, the direct ratio is estimated based on the small spatial correlation matrix obtained by the sum or average of the spatial correlation matrices obtained for each small microphone array (40a to 40g). Here, obtaining the sum or average of the spatial correlation matrices obtained for each small microphone array corresponds to the average processing of a plurality of frames of the spatial correlation matrix required in the prior art. That is, in the method according to the present invention, it is not necessary to perform an average process for a plurality of frames, and a spatial correlation matrix can be obtained instantaneously.

FIG. 5 shows a functional configuration example of the instantaneous direct ratio estimation apparatus 100 of the present invention. The operation flow is shown in FIG. The instantaneous direct ratio estimation apparatus 100 includes a microphone array 41, a plurality of frequency domain conversion units 42 _{1 to} 42 _M , a spatial correlation matrix calculation unit 43, a fixed value expansion unit 44, and a fixed value calculation unit 45. It has. The spatial correlation matrix calculation unit 43, the fixed value expansion unit 44, and the fixed value calculation unit 45 constitute a direct ratio estimation unit 50. Each unit of the instantaneous direct ratio estimation apparatus 100 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, a CPU, and the like, and the CPU executing the program.

The microphone array 41 includes a plurality of microphones m _{1 to} m _M provided at positions 41 _b , 41 _c ,..., 41 _M−1 , 41 _M that are overlapped by parallel movement of _a small microphone array 41 _a composed of a small number of microphones. Is done. Each of the plurality of frequency domain transform unit 42 ₁ through 42 _M, the received sound signals received sound by a plurality of microphones m ₁ ~m _M constituting the microphone array 41 is input.

A plurality of frequency domain transform section 42 _1, ..., 42 _M converts a plurality of microphones m _1, ... received sound signals are received sound in m _M x _m (n) to a frequency domain signal (step S42). The frequency domain converters 42 ₁ ,..., 42 _M sample the received sound signal x _m (n), for example, at a sampling frequency of 16 kHz and convert it into a digital signal, for example, 256 samples as one frame. In Step S42, discrete Fourier transform is performed to output a frequency component X _m (ω, l) (step S42). ω is a frequency, and l is a frame number. An A / D converter that converts the received sound signal x _m (n) into a digital signal is omitted.

Spatial correlation matrix calculating section 43, a plurality of frequency domain transform section 42 _1, ..., 42 signal X _{1 in} the frequency domain _M is output (ω, l), ..., X M (ω, l) as input, the A small spatial correlation matrix R ′ (ω, l) obtained from the sum of the spatial correlation matrices obtained for each small microphone array 41 _{a to} 41 _M is calculated using the frequency domain signal (equation (4), step S43).

However, R _ij (ω, l) which is each element of R ′ (ω, l) is a value defined in Expression (3), that is, each element of the matrix shown in Expression (5).

The small space correlation matrix R ′ (ω, l) is input to the eigenvalue expansion unit 44. The eigenvalue expansion unit 44 expands the small space correlation matrix R ′ (ω, l) into eigenvalues and decomposes them into eigenvalues λm (ω, l) and eigenvectors v _m (ω, l) (equation (6)), and eigenvalues λ _m (ω, l) is output to the eigenvalue distribution calculation unit 45 (fixed value expansion step).

Here, v ^H _m (ω, l) is a Hermitian transpose of the eigenvector v _m (ω, l). Further, it is assumed that the eigenvalues λ _m (ω, l) are arranged in ascending order (λ ₁ (ω, l) <λ ₂ (ω, l) <... <Λ _M (ω, l)).

The eigenvalue distribution calculation unit 45 quantifies the distribution ratio of the eigenvalue λ _m (ω, l) and outputs a direct ratio equivalent value E _F (ω, l) (eigenvalue distribution calculation step). For this digitization, for example, the ratio of the maximum eigenvalue λ _M (ω, l) to the whole (the sum of eigenvalues) as shown in Expression (7), the entropy shown in Expression (8), or the like is used.

Equations (4) and (5) calculate a small spatial correlation matrix obtained by the sum or average of the spatial correlation matrices when adjacent microphones are moved as two small microphone arrays as shown in FIG. That is, a small microphone array including two adjacent microphones is moved (41a → 41b →... → 41 _M−1 → 41 _M ) to obtain the sum or average of the spatial correlation matrix. A formula that generalizes M microphones will be described later.

  This instantaneous direct ratio estimation apparatus 100 can be used for a noise removal apparatus. FIG. 7 shows a functional configuration example of the noise removal apparatus 200 including the instantaneous direct ratio estimation apparatus 100. The operation flow is shown in FIG.

The noise removal apparatus 200 includes the above-described instantaneous direct ratio estimation apparatus 100, a processing target signal generation unit 46, a target signal adjustment unit 47, and an inverse frequency domain conversion unit 48. Each functional configuration unit excluding the microphone array 41 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.
Processing signal generating unit 46, a plurality of frequency domain transform section 42 _1, ..., generates a 42 _M signals X _m of frequency domain output (omega, l) synthesizing the processing object signal Y (ω, l) (Step S46).

The direct ratio estimation unit 50 receives the frequency domain signal X _m (ω, l) output from the plurality of frequency domain transform units 42 ₁ ,..., 42 _m as an input and corresponds to the direct ratio corresponding to the direct ratio of the received sound signal. A ratio equivalent value E _F (ω, l) is estimated (step S50).
The target signal adjustment unit 47 receives the processing target signal Y (ω, l) and the direct ratio equivalent value E _F (ω, l) as input, and determines the amplitude of the processing target signal Y (ω, l) according to the values. The adjusted post-processing signal Z (ω, l) is generated (step S47).

The inverse frequency domain transform unit 48 transforms the processed signal Z (ω, l) into a time domain signal z (n) (step S48). The operations from step S41 to step S48 are continued until all sound reception signals x _m (n) are completed.

Here, adjustment according to the value of the direct ratio equivalent value E _F (ω, l) is the threshold processing of E _F (ω, l) or the processed signal Z (ω, l) as the value increases. Includes a process of increasing the amplitude of the signal Z, and a process of decreasing the amplitude of the processed signal Z (ω, l) as the value increases. Details will be described later.

  With the above operation, noise removal is performed by, for example, emphasizing only sounds within a specific distance range and suppressing and collecting sounds outside the range by one microphone array. Hereinafter, the present invention will be described in more detail by showing more specific functional configuration examples of the respective units.

[Processing signal generator]
FIG. 9 shows a more specific functional configuration example of the processing target signal generation unit 46. The processing target signal generation unit 46 includes a plurality of weight multiplication units 461 _{1 to} 461 _M and an addition unit 462. The plurality of weight multiplying means 461 _{1 to} 461 _M are respectively frequency components X ₁ (ω, l),..., X _M (ω) of a plurality of received signals x _m (n) received by M microphones. , L) is multiplied by a weighting factor w _m (ω).

For the weights used in the weight multiplication means 461 _{1 to} 461 _M , for example, when M microphones are omnidirectional, w _m = 1 / M so that all frequency components X ₁ (ω, l), .., X _M (ω, l) is averaged to stabilize the processing target signal Y (ω, l). Also, when M microphones have directivity, use only a specific microphone signal by setting w ₁ = 1, w _m = 0 (m = {2,..., M}). Can do. For example, using a method such as that described in the reference “Oga, Yamazaki, Kanada,“ Sound System and Digital Signal Processing ”published by the Institute of Electronics, Information and Communication Engineers”, using filter coefficients for weighted beamforming, Arbitrary directivity can be formed with a microphone array.

The adding means 432 adds all the frequency components X ₁ (ω, l),..., X _M (ω, l) multiplied by the weights and outputs the processing target signal Y (ω, l).

[Target signal adjustment section]
The target signal adjustment unit 47 can be configured by, for example, a filter coefficient calculation unit 471 and a multiplication unit 472 (FIG. 7). The filter coefficient calculation means 471 calculates and outputs the filter coefficient G (ω, l) with the direct ratio equivalent value E _F (ω, l) as an input. For the calculation of the filter coefficient G (ω, l), for example, a binary filter using a threshold as shown in Expression (9) is used.

The threshold Th can be set to any value between the minimum value and the maximum value of the direct ratio equivalent value E _F (ω, l). The sound quality is improved when the threshold Th is brought close to the minimum value (0). On the contrary, when the threshold value Th is brought close to the maximum value, the noise suppression effect is enhanced, but the distortion of the received sound signal is increased and the sound quality is deteriorated.

  Thus, the threshold Th has a trade-off relationship between the sound quality and the noise suppression. Therefore, the threshold Th is determined empirically in accordance with the purpose of use in consideration of this trade-off relationship.

The filter coefficients G (omega, l) as shown in equation (10) when calculating the, if to emphasize time-frequency band Chokkan ratio equivalent value E _F is less than the threshold value Th _2, the specific distance range Distant sound sources can be emphasized.

  In addition, although the binary filter of 0 or 1 was mentioned as an example of filter coefficient G ((omega), l), filter coefficient G ((omega), l) does not necessarily need to be 0 and 1, for example, 0.1 And a sufficiently different value such as 0.9.

  Further, a real number of 1 or more may be set for the filter coefficient G (ω, l). That is, the processing target signal Y (ω, l) may be amplified. Alternatively, the processing target signal Y (ω, l) may be greatly suppressed by setting the value to 0.1 or less.

The multiplication coefficient 472 multiplies the processing target signal Y (ω, l) by the filter coefficient G (ω, l) obtained in this way, and the processed signal Z (ω, l) = G (ω, l) · Y (ω, l) is generated. Therefore, the post-processing signal Z (ω, l) can be configured only with the processing target signal Y (ω, l) having a large direct ratio equivalent value E _F (ω, l). That is, only the direct sound can be extracted.

  FIG. 10 shows a functional configuration example of the noise removal apparatus 300 of the present invention. The noise removal apparatus 300 differs from the noise removal apparatus 200 described above in the operations of the processing target signal generation unit 83 and the direct ratio estimation unit 84.

FIG. 11 shows a functional configuration example of the direct ratio estimation unit 84. The direct ratio estimator 84 is such that the eigenvalue expansion means 842 outputs the eigenvector v _L (ω, l) corresponding to the maximum eigenvalue λ _M (ω, l). And different. The eigenvector v _L (ω, l) is input to the processing target signal generation unit 83.

FIG. 12 shows a functional configuration example of the processing target signal generation unit 83. The processing target signal generation unit 83 is different from the processing target signal generation unit 46 in that the eigenvector v _L (ω, l) is used as a weight of the weight multiplication units 831 _{1 to} 831 _M.

The number of microphones constituting the microphone array 41 and the number of eigenvectors corresponding to M are used as the weights of the weight multipliers 831 _{1 to} 831 _M , respectively. That is, the m-th component v _{L, m} (ω, l) of the eigenvector v _L (ω, l) is used as the weight w _m (ω) of the m-th microphone.

It is known that the eigenvector v _L (ω, l) corresponding to the maximum eigenvalue λ _M (ω, l) functions as a beamforming weight that enhances the direct sound. Therefore, the noise removal device 300 can improve the noise removal performance as compared with the noise removal device 200.

As the third embodiment, an instantaneous direct ratio estimation apparatus 400 that calculates the true direct ratio E _R (l) will be described. FIG. 13 shows a functional configuration example of the instantaneous direct ratio estimation apparatus 400. In the instantaneous direct ratio estimation device 400, the signal power estimation unit 44 and the direct ratio calculation unit 45 of the direct ratio estimation unit 85 are different from those of the instantaneous direct ratio estimation device 100.

The signal power estimation unit 44 includes components R ′ _{i, j} (ω, l) of the small spatial correlation matrix R ′ (ω, l) output from the spatial correlation matrix calculation unit 43 and a microphone array given in advance. The components of the i-th row and j-th column of each matrix of the matrix R _d (ω) (formula (11)) and matrix R _r (ω) (formula (12)) given by the microphone arrangement and the direction of the sound source; From d _{i, j} (ω) and r _{i, j} (ω), a matrix A (ω) shown in Equation (13) and B (ω, l) shown in Equation (14) are used. Here, the small spatial correlation matrix R ′ (ω, l) is a matrix obtained by the sum or average of the spatial correlation matrices obtained for each small microphone array (formula (4)).

However, R ₁₁ ′ (ω, l), R ₁₂ ′ (ω, l), R ₂₁ ′ (ω, l), R ₂₂ ′ (ω, l), which are the components of B (ω, l), Each element of R ′ (ω, l) shown in Formula (4).

  Equations (4) and (5) above represent the small spatial correlation matrix that is the sum or the average of the spatial correlation matrices obtained from each small microphone array when two adjacent microphones are used as the small microphone array. Is calculated. As a more general expression, when the number of microphones constituting one small microphone array is M ′, Expression (13) is Expression (15), Expression (14) is Expression (16), Expression (4). Can be expressed by equation (17).

Then, a simultaneous equation shown in the equation (18) is set up and solved to solve the vector P (ω, l) composed of the direct sound power P _d (ω, l) and the reverberant power P _r (ω, l). l) (Expression (19)) is obtained, and direct sound power P _d (ω, l) and reverberant power P _r (ω, l) are output.

Note that the matrix R _d (ω) in the case where the arrangement of the microphone array 41 is an arrangement other than a straight line can be expressed in the form shown in the more general expression (20).

Here, D _mn (θ) ￣ represents a distance difference between the m-th microphone and the n-th microphone when viewed from the direction of the angle θ °. Further, the derivation of the solution of the simultaneous equations of Expression (18) is performed by, for example, converting the pseudo inverse matrix A ⁺ (ω) (Expression (21)) of A (ω) to B (ω, l as shown in Expression (22). ) From the left side.

The direct ratio calculation unit 45 calculates the direct ratio E _R (l) from the direct sound power P _d (ω, l) and the reverberation sound power P _r (ω, l) according to the equation (23) and outputs it. .

  In the method of the third embodiment, since the direct ratio is directly obtained, it is possible to accurately estimate the direct ratio.

As the fourth embodiment of the present invention, the perspective of the sound source is determined using the direct ratio equivalent value E _F (ω, l) described in the first embodiment or the direct ratio E _R (l) described in the third embodiment. The perspective determination device 120 will be described. FIG. 14 shows a functional configuration example of the perspective determination device 500. The perspective determination device 500 includes a direct ratio estimation device 100 and a perspective determination unit 121. The microphone array 41, the plurality of frequency domain conversion units 42 _{1 to} 42 _M, and the direct ratio estimation unit 44 are the same as those of the noise removal apparatus 300. The perspective determination device 500 is also realized by a predetermined program being read into a computer constituted by, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The perspective determination device 500 determines whether a sound source of a sound received at a certain time is far or near when sound sources at a plurality of different distances sound at different times. The perspective determination unit 121 included in the perspective determination device 500 includes a frequency averaging unit 1210, an accumulation unit 1211, and a determination unit 1212.

The frequency averaging means 1210 receives the direct ratio equivalent value E _F (ω, l) as an input and averages the direct ratio equivalent value E _F (ω, l) in the frequency direction to obtain a frequency average direct ratio equivalent value E _l. ￣ is output (formula (24)).

When the direct ratio E _R (l) is input to the frequency averaging means 1210, it is output as it is without any particular processing (broken line passing through the frequency averaging means 1210 in FIG. 14).

Here, K is the total number of frequency bins of the Fourier transform performed by the frequency domain transform units 42 _{1 to} 42 _M. In the following description, an example using the frequency average direct ratio equivalent value E _l ￣ will be described. Replacing the frequency average direct ratio equivalent value E _lに with the direct ratio E _R (l) does not change the operation of the perspective determination device 500 .

The accumulating unit 1211 accumulates the frequency average direct ratio equivalent value E _l分 for the past L time frames and outputs the comparison target direct ratio equivalent value E ^. In comparison Chokkan ratio equivalent value E ^, for example stored frequency mean Chokkan ratio equivalent value E _l ¯ of the average value ^{_{E ^ = 1 / LΣ l L}} E l ¯ and the minimum and the average value of the maximum value E ^ = 1/2 (maxE _l ￣ + minE _l ￣) or the like is used.

The determination unit 1212 compares the frequency average direct ratio equivalent value E _l ￣ with the comparison target direct ratio equivalent value E ^, and if E _l ￣> E ^, the distance is close to the perspective determination result Y _l. For example, 1 is output, and when E _l ￣ <E ^, for example, 0 indicating that the distance is long is output to the perspective determination result Y _l . The distance determination result Y _l is nearest received sound signals of the past L time period is either a sound from relatively close sound source or those indicating which sounds from a relatively distant sound source.

The distance determination result by using a Y _l, the received sound signal inputted sequentially, it is possible to isolate the distance between the between the microphone and the sound source. That is, sounds from a plurality of sound sources can be selected according to the distance from the microphone.

  FIG. 15 shows a functional configuration example of the sound source distance measuring apparatus 600 of the present invention. The sound source distance measuring device 600 includes an instantaneous direct ratio estimation device 100, a distance-direct ratio database (hereinafter referred to as a distance-direct ratio DB) 55, and a distance determination unit 56. Each functional configuration unit excluding the microphone array 41 of the sound source distance measuring apparatus 600 is realized by reading a predetermined program into a computer including, for example, a ROM, a RAM, and a CPU and executing the program by the CPU. Is.

The instantaneous direct ratio estimation apparatus 100 estimates the direct ratio equivalent value E _F or the direct ratio E _R of the received sound signal by the operation described in the first and third embodiments. The distance-direct ratio DB 56 records the relationship between the direct ratio equivalent value E _F or the direct ratio E _R and the distance between the microphone array and the sound source. The distance determination unit 55 receives the direct ratio equivalent value or direct ratio and inputs the direct-to-direct ratio DB 56 and estimates the sound source distance estimated value d ^ corresponding to the direct ratio equivalent value or direct ratio. .

  Some received signals have components concentrated in a specific frequency band. When the direct ratio E of such a sound reception signal is calculated by the direct ratio calculation unit 45 (FIG. 13), the estimation accuracy of the direct ratio E deteriorates.

  Therefore, as shown in the equation (25), by using the direct ratio calculation unit 45 ′ (FIG. 13) that calculates the direct ratio E in a specific frequency region Ω, the accuracy of the direct ratio is improved. I can do it.

Here, the frequency region Ω is determined, for example, by selecting a frequency band in which signal components are concentrated. For example, among the outputs X _m (ω, l) of the frequency domain converter 42 _m connected to an arbitrary m-th microphone, the absolute value of X _m (ω, l) is preliminarily set as shown in Expression (26). It is determined by selecting the frequency ω having a value larger than the set threshold value P _th or by selecting the frequency ω from the largest absolute value of X _m (ω, l) to the Kth.

Here, P _th is, for example _{| X m (ω, l)} | of an average value of all the frequency used.

  FIG. 16 shows a functional configuration example of the noise removal apparatus 700 of the present invention. The operation flow is shown in FIG. The noise removal apparatus 700 includes the instantaneous direct ratio estimation apparatus 400 described in the third embodiment, a processing target signal generation unit 72, a target signal adjustment unit 73, and an inverse frequency domain conversion unit 74.

The processing target signal generation unit 72 receives the frequency domain signal X _m (ω, l) output from the plurality of frequency domain conversion units 421 to 42 _M in the instantaneous direct ratio estimation device 400 as an input. ω
, L) is output (step S72). The processing target signal X (ω, l) is a signal obtained by synthesizing the frequency domain signal X _m (ω, l) with, for example, an adding means (not shown). Before the addition, the signal X _m (ω, l) in each frequency domain may be multiplied by a weight.

The target signal adjustment unit 73 receives the direct ratio E _R ( l ) output from the instantaneous direct ratio estimation device 400 and the processing target signal X (ω, l) output from the processing target signal generation unit 72 as inputs. A post-processing signal Y (ω, l) in which the amplitude of the processing target signal X (ω, l) is adjusted is generated (step S73). The inverse frequency domain transform unit 74 transforms the processed signal Y (ω, l) into a time domain signal y (n) (step S74).

  The target signal adjustment unit 73 includes, for example, a distance calculation unit 731, a filter formation unit 732, and a multiplication unit 733. The distance calculation means 731 incorporates a function formula d = f (E) indicating the relationship between the distance between the microphone array 41 and the sound source and the direct ratio E, and the sound source corresponding to the input direct ratio E An estimated distance d ^ is calculated (distance calculation step S731).

Filter formation section 732, as shown in equation (27), set as the sound source distance estimate d ^ are two size to emphasize the temporal frequency components take values between different threshold d _f and d _n Then, a filter that emphasizes only the sound source in the band-like region within the two distance sections is formed.

  Here, l and ω of G (ω, l) are included in the frequency Ω (formula (25)) obtained by averaging in the direct ratio calculation unit 45 ′ in the processing of the direct ratio estimation unit 85 described above. The same G (ω, l) is multiplied for all frequencies that are transmitted. In the equation (27), the value of G (ω, l) is not necessarily 1 and 0, and may be a sufficiently different value such as 0.9 and 0.1, for example.

  The multiplying unit 733 multiplies the processing target signal X (ω, l) by the filter G (ω, l) to generate a processed signal Y (ω, l). Therefore, the post-processing signal Y (ω, l) is obtained by enhancing or suppressing the sound of the sound source located in a specific distance range from the microphone array 41 in two distance sections. The post-process signal Y (ω, l) is converted into a time domain signal y (n) by the inverse frequency domain converter 74.

  In the above-described embodiment, electric noise superimposed on each microphone is not taken into consideration. Therefore, when the noise level is high, the estimation accuracy of the direct ratio may be deteriorated. FIG. 18 shows a functional configuration example of the instantaneous direct ratio estimation apparatus 400 ′ that can prevent the deterioration of the direct ratio estimation accuracy due to noise. The instantaneous direct ratio estimation apparatus 400 ′ is different only in that the signal power estimation unit 44 in the instantaneous direct ratio estimation apparatus 400 is formed of a signal power estimation unit 44 ′.

The signal power estimator 44 'includes each component R' _{i, j} (ω, l) of the small spatial correlation matrix R '(ω, l) output from the spatial correlation matrix calculator 43 and a microphone array given in advance. , The matrix R _d (ω) (formula (11)) given by the direction of the sound source, the matrix R _r (ω) (formula (12)), and the matrix R _n (ω) (formula (28)) The point of using the matrix A (ω) shown in Expression (29), which is composed of each component, d _{i, j} (ω), r _{i, j} (ω), and n _{i, j} (ω). , Different from the signal power estimation unit 44. However, the matrix R _n (ω) is a unit matrix having a size equal to the number of microphones of the small microphone array.

Then, the simultaneous equation shown in the equation (30) is established by using the equation (14) for B (ω, l), and by solving this, the direct sound power P _d (ω, l) and the reverberant power P _r ( A vector P (ω, l) (formula (31)) composed of ω, l) and noise power P _n (ω, l) is obtained, and direct sound power P _d (ω, l) and reverberant power P _r (ω, l) is output. Similarly to the equation (23), the direct ratio is obtained from the direct sound power P _d (ω, l) excluding the noise power and the reverberant power P _r (ω, l), so that the electric noise superimposed on each microphone is obtained. It is possible to accurately estimate the direct ratio without being affected by the above.

  It should be noted that Equation (29) and Equation (14) described above calculate a small spatial correlation matrix obtained by summing or averaging the spatial correlation matrices obtained by making adjacent microphones into two small arrays as shown in FIG. Is. When the number of microphones in the small microphone array is M ′, Expression (29) can be expressed by Expression (32), and Expression (14) can be expressed by Expression (16). Equation (28) is a unit matrix of size M ′ × M ′.

  It should be noted that this instantaneous direct ratio estimation apparatus 400 ′ can be replaced with the respective instantaneous direct ratio estimation apparatuses 100 of the noise elimination apparatuses 200, 300, 700, the perspective determination apparatus 500, and the sound source distance estimation apparatus 600 that have already been described. Is possible. Each device in the case of replacement can estimate an accurate direct ratio even when the sound source moves in a noisy environment.

  In addition, although the number of microphones constituting a small microphone array including a small number of microphones has been described with two examples, the number of microphones may be any number, and the arrangement of the microphones is not limited to a linear array arranged linearly at equal intervals. The number of microphones may be three as shown in FIG. Three black circles indicate small microphone arrays, and the overlapping positions are indicated by ◯. Further, as shown in FIG. 19B, the microphones may be arranged three-dimensionally. In short, any microphone array may be used as long as a plurality of microphones are provided so that another small microphone array is arranged at a position overlapping by parallel movement of a small microphone array composed of a small number of microphones arranged in a regular rule.

Note that the processes described in the above method and apparatus are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (19)

A microphone array composed of a plurality of microphones provided at overlapping positions by translation of a small microphone array composed of a small number of microphones;
A plurality of frequency domain conversion units that receive the received sound signals received by the microphone array and convert the received sound signals into frequency domain signals;
A space for calculating a small spatial correlation matrix obtained by summing or averaging the spatial correlation matrices obtained for each of the small microphone arrays using the frequency domain signals output from the plurality of frequency domain transform units as inputs. A correlation matrix calculation unit;
An eigenvalue expansion unit that expands the small space correlation matrix into eigenvalues, decomposes them into eigenvalues and eigenvectors, and outputs the eigenvalues;
An instantaneous direct ratio estimation apparatus comprising: an eigenvalue distribution calculation unit that digitizes a distribution ratio of the eigenvalues and outputs a direct ratio equivalent value. A microphone array composed of a plurality of microphones provided at overlapping positions by translation of a small microphone array composed of a small number of microphones;
A plurality of frequency domain conversion units that receive the received sound signals received by the microphone array and convert the received sound signals into frequency domain signals;
A space for calculating a small spatial correlation matrix obtained by summing or averaging the spatial correlation matrices obtained for each of the small microphone arrays using the frequency domain signals output from the plurality of frequency domain transform units as inputs. A correlation matrix calculation unit;
A vector composed of direct sound power, reverberant sound power and noise power is obtained from the microphone arrangement information given in advance and the spatial correlation matrix, and the direct sound power and reverberation among the vector elements are obtained. A signal power estimator that outputs the power of the sound;
A direct ratio calculation unit for calculating a direct ratio obtained by dividing the power of the direct sound by the power of the reverberant sound;
An instantaneous direct ratio estimation apparatus comprising: A noise removal device including the instantaneous direct ratio estimation device according to claim 1,
A processing target signal generation unit that generates a processing target signal by combining frequency domain signals output from the plurality of frequency domain conversion units;
A target signal adjusting unit that generates a post-processing signal in which the amplitude of the processing target signal is largely adjusted as the direct ratio equivalent value is large;
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A noise removal device including the instantaneous direct ratio estimation device according to claim 1,
A processing target signal generation unit that generates a processing target signal by combining frequency domain signals output from the plurality of frequency domain conversion units;
A target signal adjustment unit that generates a post-processing signal in which the amplitude of the processing target signal is largely adjusted as the direct ratio equivalent value is small;
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A perspective determination device including the instantaneous direct ratio estimation device according to claim 1 and including a perspective determination unit,
The perspective determination unit
A frequency averaging means for averaging the direct ratio equivalent value in the frequency direction and outputting a frequency average direct ratio equivalent value;
Accumulating means for accumulating the frequency average direct ratio equivalent value for a predetermined time in the past and outputting a comparison target direct ratio equivalent value;
A determination means for comparing the frequency average direct ratio equivalent value and the comparison target direct ratio equivalent value to output a perspective determination result;
A perspective determination device comprising: A sound source distance measuring device including the instantaneous direct ratio estimating device according to claim 1,
A distance-direct ratio database that records the relationship between the direct ratio equivalent value and distance;
A distance determination unit that estimates the sound source distance estimated value corresponding to the direct ratio equivalent value by referring to the distance-direct ratio database with the direct ratio equivalent value as an input;
A sound source distance measuring device comprising: A noise removal device including the instantaneous direct ratio estimation device according to claim 2,
A processing target signal generation unit that generates a processing target signal by combining frequency domain signals output from the plurality of frequency domain conversion units;
A target signal adjustment unit that generates a post-processing signal in which the amplitude of the processing target signal is adjusted to be larger as the direct ratio is larger;
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A noise removal device including the instantaneous direct ratio estimation device according to claim 2,
A processing target signal generation unit that generates a processing target signal by combining frequency domain signals output from the plurality of frequency domain conversion units;
A target signal adjustment unit that generates a post-processing signal in which the amplitude of the processing target signal is adjusted to be larger as the direct ratio is smaller;
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A perspective determination device including the instantaneous direct ratio estimation device according to claim 2 and including a perspective determination unit,
The perspective determination unit
Accumulating means for accumulating the direct ratio for a predetermined time in the past and outputting the comparative direct ratio;
A determination means for comparing the direct ratio and the comparison target direct ratio and outputting a perspective determination result;
A perspective determination device comprising: A sound source distance measuring device including the instantaneous direct ratio estimating device according to claim 2,
A distance-direct ratio database that records the relationship between the direct ratio and distance;
A distance determination unit that estimates the sound source distance estimation value corresponding to the direct ratio by referring to the distance-direct ratio database using the direct ratio as an input;
A sound source distance measuring device comprising: The sound source distance measuring device according to claim 10,
The sound source distance measuring device, wherein each of the direct sound power, the reverberant power, and the noise power is an added value in a specific frequency region. A noise removal device including the instantaneous direct ratio estimation device according to claim 1,
A processing target signal generation unit that outputs a processing target signal with the frequency domain signals output by the plurality of frequency domain conversion units;
Generates a post-processing signal that emphasizes or suppresses the sound of a sound source located within a specific distance range from the microphone array, with the direct ratio equivalent value output from the instantaneous direct ratio estimation device and the processing target signal as inputs. A target signal adjustment unit to perform,
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A noise removal device including the instantaneous direct ratio estimation device according to claim 2,
A processing target signal generation unit that outputs a processing target signal with the frequency domain signals output by the plurality of frequency domain conversion units;
Target for generating a post-processing signal that emphasizes or suppresses the sound of a sound source located in a specific distance range from the microphone array, with the direct ratio output from the instantaneous direct ratio estimation device and the processing target signal as inputs. A signal conditioning unit;
An inverse frequency domain transform unit for transforming the processed signal into a time domain signal;
A noise removal apparatus comprising: A plurality of frequency domain converters convert a received sound signal received by a microphone array composed of a plurality of microphones provided at overlapping positions by parallel movement of a small microphone array consisting of a small number of microphones into a frequency domain signal. Frequency domain transformation process,
The spatial correlation matrix calculation unit receives the frequency domain signals output from the plurality of frequency domain transform units, and uses the frequency domain signals to obtain the small or the average of the spatial correlation matrices obtained for each small microphone array. A spatial correlation matrix calculation process for calculating a spatial correlation matrix;
An eigenvalue expansion unit eigenvalue-expanding the small spatial correlation matrix to decompose it into eigenvalues and eigenvectors, and outputting the eigenvalues;
An eigenvalue distribution calculation unit, wherein the eigenvalue distribution calculation unit quantifies the distribution ratio of the eigenvalues and outputs a direct ratio equivalent value. A plurality of frequency domain converters convert a received sound signal received by a microphone array composed of a plurality of microphones provided at overlapping positions by parallel movement of a small microphone array consisting of a small number of microphones into a frequency domain signal. Frequency domain transformation process,
The spatial correlation matrix calculation unit receives the frequency domain signals output from the plurality of frequency domain transform units, and uses the frequency domain signals to obtain the small or the average of the spatial correlation matrices obtained for each small microphone array. A spatial correlation matrix calculation process for calculating a spatial correlation matrix;
The signal power estimation unit obtains a vector composed of the direct sound power and the reverberant power from the microphone arrangement information given in advance and the spatial correlation matrix, and the direct sound among the elements of the vector is obtained. A signal power estimation process for outputting power and the power of reverberation,
The direct ratio calculation unit calculates the direct ratio obtained by dividing the power of the direct sound by the power of the reverberant sound; and
An instantaneous direct ratio estimation method including Device program for causing a computer to function as the instantaneous Chokkan ratio estimation equipment according to claim 1 or 2.   An apparatus program for causing a computer to function as the noise removing apparatus according to claim 3, claim 4, claim 7, claim 8, claim 12, or claim 13.   An apparatus program for causing a computer to function as the perspective determination apparatus according to claim 5.   An apparatus program for causing a computer to function as the sound source distance measuring apparatus according to claim 6, claim 10, or claim 11.

Images