JP5285665B2 - Reflected sound information estimation apparatus, reflected sound information estimation method, program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology which estimates a relative difference in the arrival time between a standard reflection sound and other reflection sounds. <P>SOLUTION: When Q is an preset integer no less than 2, a complex amplitude (arrival amplitude) of each of Q reflection sounds (the reflection sound is shown by a function (transfer characteristic function) simulating a transfer characteristic between an optional position in space and multiple microphones for every frequency included in a prescribed frequency bandwidth multiplied by the complex amplitude) is used. Concerning each reflection sound (target reflection sound) other than a standard reflection sound, the arithmetic average of the frequency in relation to a deflection angle of the arrival amplitude of the target reflection sound in relation to the arrival amplitude of the standard reflection sound being divided by the frequency is defined as a relative difference in the arrival time between the standard reflection sound and the target reflection sound. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to information relating to reflected sound, and more particularly to a technique for estimating a relative arrival time difference of another reflected sound with respect to a reference reflected sound.

  A system for exchanging voice information such as telephone calls and voice conferences is generally called a voice communication system. In a voice communication system, it is very important to obtain information about the reflected sound (arrival amplitude, arrival direction, etc.). In a reverberant environment such as a conference room, the collected sound signal collected through the microphone is reflected not only from the direct sound coming from the sound source such as the speaker but also from the floor, wall or ceiling. Reflected sound is mixed. Therefore, when a speaker's utterance is recorded in such a reverberant environment, the reflected sound is mixed with a delay from the direct sound, making it difficult to hear. If arrival information of each reflected sound can be estimated from the collected sound signal and the reflected sound can be removed, it is possible to recover a sound that is easy to hear. Here, Non-Patent Document 1 is given as a conventional study for estimating reflected sound information.

  A functional configuration for realizing the technology disclosed in Non-Patent Document 1 is shown in FIG. The processing procedure in this technique is as follows.

1. The sound source signal radiated from the impulse sound source 100 is picked up using 4ch microphones 110-1, 110-2, 110-3, 110-4. The AD converter 120 converts the collected analog signal into a digital signal x ^→ (t) = [x ₁ (t), x ₂ (t), x ₃ (t), x ₄ (t)] ^T . Here, [•] ^T represents transposition. t represents a discrete time index. Assume that four microphones are arranged at the apexes of a regular tetrahedron.

2. The impulse response calculation unit 130 receives the digital signal x ^→ (t) = [x ₁ (t), x ₂ (t), x ₃ (t), x ₄ (t)] ^T as an input, and the impulse response of each microphone. h ^→ (t) = [h ₁ (t), h ₂ (t), h ₃ (t), h ₄ (t)] ^T is calculated. The impulse response calculation method includes a TSP method, an M-sequence method, and the like, and any method may be used to calculate the impulse response.

3. The virtual sound source calculation unit 140 receives 4ch impulse response h ^→ (t) = [h ₁ (t), h ₂ (t), h ₃ (t), h ₄ (t)] ^T as input, and generates virtual sound source information. v ^→ = [v ^→ ₁ ,…, v ^→ _D ] ^T is output. D represents the number of virtual sound sources. A virtual sound source is a sound source that is virtually present to represent the arrival amplitude, arrival direction, and arrival time of each reflected sound. The virtual sound source will be described with reference to FIG. FIG. 2 shows a path for receiving a sound source signal reflected by the right wall with a microphone. The sound source signal reflected from the right wall (reflected sound) is equivalent to the signal coming directly from the position written as “virtual sound source” (however, the effects of attenuation and distance attenuation due to reflection on the wall are not receive).

Details of this prior art will be described. When the impulse response is measured at four adjacent sound receiving points (microphone positions), there is a slight difference in the arrival time of the reflected sound. By using the cross-correlation of the short section of the impulse response and associating the reflected sound with each microphone, as shown in FIG. 3, the arrival time t _1n at each sound receiving point regarding the nth reflected wave, t _2n , t _3n , t _4n (1 ≦ n ≦ D) are obtained. Each virtual sound source information v _n ^→ = [X _n , Y _n , Z _n , S _n ] ^T is obtained when the length of the side of the regular tetrahedral microphone array is d and the speed of sound is c. Here, X _n , Y _n , and Z _n represent the position of the n-th virtual sound source (see equations (1) to (3)), which has information corresponding to the arrival direction and arrival time of each reflected sound. . Further, S _n represents the strength of the n-th virtual source, determined by the average of the amplitudes of the an n-th reflected sound associated with impulse 4ch.

Yoshio Yamazaki et al., “Spatial information obtained from impulse responses of four adjacent points”, Proc. Of the Acoustical Society of Japan, 1981, pp.759-760.

  According to the prior art, in order to estimate the “arrival amplitude”, “arrival direction”, and “arrival time” of the reflected sound called virtual sound source information, it is necessary to prepare an impulse response in advance. However, since it is necessary to observe using a special signal in order to prepare an impulse response, the condition that impulse responses at all positions are prepared in advance is not realistic.

  Therefore, an object of the present invention is to provide a technique for estimating a relative arrival time difference of reflected sound information, particularly, a reflected sound with respect to a reference reflected sound, without using a special signal.

In the storage unit, P is a predetermined integer of 2 or more, p is an integer of 1 to P, and transmission for each frequency between the p-th position and each position where M microphones are arranged. Functions that simulate characteristics (hereinafter referred to as transfer characteristic functions) are stored as templates for each p, and M sound pickup signals obtained by collecting sound source signals with M microphones are converted into frequency domains, respectively. Input signals (hereinafter referred to as observation signals) and a template, Q as a predetermined integer of 2 or more, and observation signals from Q directions (hereinafter referred to as arrival directions) with respect to M microphones. Assume that the incoming reflected sound is superimposed, and based on the template, estimate the complex amplitude (hereinafter referred to as arrival amplitude) of each of the Q reflected sounds, and other than the reference reflected sound (hereinafter referred to as reference reflected sound) For the reflected sound (hereinafter referred to as the target reflected sound), the arithmetic average of the frequency obtained by dividing the deviation angle of the arrival amplitude of the target reflected sound with respect to the reference reflected sound by the frequency is the target reflected sound with respect to the reference reflected sound. Relative arrival time difference.

  According to the present invention, the arithmetic mean of the frequency relative to the deviation of the arrival amplitude of the target reflected sound relative to the arrival amplitude of the reference reflected sound divided by the frequency is obtained as a relative arrival time of the target reflected sound with respect to the reference reflected sound. Because of the difference, it is possible to estimate reflected sound information (relative arrival time difference of other reflected sounds with respect to the reference reflected sound) without using a special signal as a sound source signal to obtain an impulse response. is there. When the reflected sound information is obtained, it can be applied to uses such as estimation of a sound source distance and sound enhancement (collection of far-field sound and sound collection by distance) that could not be realized by the conventional sound information processing technology.

The figure which shows the function structure of the reflected sound information estimation technique in a prior art. The figure for demonstrating a virtual sound source. The figure for demonstrating matching of the reflected sound in a prior art. The figure which shows the function structure of the reflected sound information estimation apparatus which concerns on 1st Embodiment. The figure which shows the process sequence of the reflected sound information estimation method which concerns on 1st Embodiment. The figure which shows the structural example of a two-dimensional microphone array. p-th point _{[x p, y p, z} p] diagram for explaining the transfer characteristic between the m-th receiving point _{[u m, v m, w} m]. The figure for demonstrating that the sound pressure distribution in a certain plane observed using the microphone array shown in FIG. 6 is obtained by superimposing the direct sound, the reflected sound 1, and the reflected sound 2, for example. The figure for demonstrating the principle of this invention. The figure which shows the positional relationship of the area coordinate for template information preparation and a microphone array in an experiment example. The figure which shows an experimental result. The figure for demonstrating the effectiveness of this invention by an experiment example. The figure which shows the process sequence of the reflected sound information estimation method which concerns on 2nd Embodiment. (A) The figure for demonstrating that only the information regarding an estimated arrival direction should be extracted ideally. (B) The figure for demonstrating that the information regarding directions other than an estimated arrival direction will actually be mixed. The figure for demonstrating reducing the influence of directions other than an estimated arrival direction by summarizing the power of a residual signal over all the frequencies. The figure which shows the sound pressure distribution at the time of using a two-dimensional matrix microphone array of a practical use level, and its decomposition | disassembly.

<< First Embodiment >>
In the present invention, the “arrival amplitude” and “ At least one of the “arrival directions” is estimated. The functional configuration and processing flow of the first embodiment are shown in FIGS.

The sound source signal radiated from the sound source 200 is collected using the Mch microphones 210-1,..., 210-M (step S1). A value greater than 4 is desirable for M. The AD converter 220 converts the collected analog signal into a digital signal xx ^→ (t) = [xx ₁ (t),..., Xx _M (t)] ^T (step S2). Here, [•] ^T represents transposition. t represents a discrete time index.

It is desirable to arrange the M microphones at equal intervals in two dimensions or three dimensions. This is to uniquely determine the correspondence between the arrival direction of the reflected sound and the template (which will be described later, which simulates the transfer characteristic of the reflected sound). In principle, the present invention can be implemented even if the microphones are arranged one-dimensionally or not at regular intervals, but there is a one-to-one relationship between the transmission characteristics of the reflected sound and the arrival directions of the reflected sound. Therefore, it is desirable to arrange them at equal intervals in two dimensions or three dimensions. An example when microphones are arranged at equal intervals on a two-dimensional plane is shown in FIG. The microphone interval d is preferably set so as to satisfy the spatial sampling theorem. When the spatial sampling theorem is satisfied, the microphone interval d is a numerical value that satisfies Equation (4). c is the speed of sound, and f is the frequency to be analyzed. For example, when analyzing a frequency of 4 kHz, it is preferable to set the microphone interval to about 4 cm.

Frame division unit 230, a digital signal xx ^→ AD conversion unit 200 is output _{(t) = [xx 1 (} t), ..., xx M (t)] as input ^T, a digital signal consisting of a plurality of samples for each channel The signal x ^→ (k) = [x ₁ (k),..., X _M (k)] ^T divided into groups (frames) is output (step S3). k is an index representing a frame number. The frame division is a process of buffering and outputting W points for each digital signal xx _i (t) (1 ≦ i ≦ M) of each channel. W depends on the sampling frequency, but in the case of 16 kHz sampling, around 512 points are appropriate.

The frequency domain converter 240 receives the digital signal x ^→ (k) of each frame as an input, and the frequency domain signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k )] Converted to ^T and output (step S4). This signal X ^→ (ω, k) is called an observation signal. Here, ω indicates a discrete frequency index (there is a relationship of ω = 2πf between the frequency f and the angular frequency ω, and therefore the frequency index ω may be identified with the angular frequency ω. Hereinafter, with respect to ω, “frequency index” is also simply referred to as “frequency”), and k indicates a frame index. One method for transforming into the frequency domain is discrete Fourier transform, but other methods may be used as long as the transform is performed into the frequency domain. The observation signal X ^→ (ω, k) in the frequency domain is output for each frequency ω and frame k.

The template generation unit 250 includes template information S ^→ (ω) = [S ₁ ^→ (ω),..., Which is a set of P templates _Sp ^→ (ω) (however, for convenience of calculation, expressed in vector). , S _P ^→ (ω)] ( ^∀ ω∈Ω; Ω is a set of frequency indices ω) is generated for each frequency ω (step Sp). This process is usually performed prior to each process of steps S1-S4. P represents the total number of templates, and is set in advance to an integer value of 2 or more. The larger the total number of templates P, the more accurately the reflected sound information is estimated. However, since the amount of calculation increases, for example, P = 1000 is preferable. This processing is performed in advance before observing the signal with the microphone. Further, unless the position of the microphone (for example, the distance d between the microphones) is changed or the total number P of templates is not changed, it is usually unnecessary to recreate the template every time. The “template” referred to here is a simulation of a transfer characteristic (acoustic propagation characteristic) corresponding to the arrival direction of the reflected sound. The p-th (1 ≦ p ≦ P) template S _p ^→ (ω) = [S _p1 (ω),..., S _pM (ω)] ^T (ω∈Ω) is a predetermined p-th point [ x _p , y _p , z _p ] and M sound receiving points (where the sound receiving point is the position where the microphone is placed, and the m th (1 ≦ m ≦ M) sound receiving point is represented by [u _m , v _m , w _m ]) (see FIG. 7). An example of a calculation formula for each element S _pm (ω) of the p-th template _Sp ^→ (ω) is shown in Formula (5). The symbol i represents an imaginary unit.

Direction information θ _p ^→ (ω) is associated with the p-th template S _p ^→ (ω). The direction information θ _p ^→ (ω) is a three-dimensional orthogonal coordinate system that serves as a reference for the position coordinates of the p th point [x _p , y _p , z _p ] and the sound receiving point [u _m , v _m , w _m ]. The direction of viewing the p-th point [x _p , y _p , z _p ] from the origin of, for example, two declinations in a spherical coordinate system (having a common origin with the origin of the 3D Cartesian coordinate system) ( _Polar angle θ _{p, pol} and azimuth angle θ _{p, azi} ). That is, θ _p ^→ (ω) = [θ _{p, pol} (ω), θ _{p, azi} (ω)]. If the p-th template S _p ^→ (ω) is associated with the p-th point [x _p , y _p , z _p ], the direction information θ _p ^→ (ω) is the position [x _p , y _p , since can be calculated from z _p], p-th template S _p ^→ (ω) be the direction information θ _p ^→ (ω) is associated with is not a requirement. Since the three-dimensional orthogonal coordinate system and the spherical coordinate system can be converted to each other (coordinate conversion), the right side of Equation (5) is not the position [x, y, z] but the direction information θ _p ^→ (ω). = [Θ _{p, pol} (ω), θ _{p, azi} (ω)], for example, can also be expressed as shown in equation (5a). Here, d is a microphone interval, the microphone array is a two-dimensional microphone array of Φ rows and Ξ columns (Φ × Ξ = M), and the position of the mth microphone is φ rows and ξ columns (1 ≦ φ ≦ Φ, 1 ≦ ξ ≦ Ξ).

Further, when the template corresponds to the direction as in the first embodiment, the positions of the P points [x _p , y _p , z _p ] (1 ≦ p ≦ P) are positions having different directions from each other. For example, assuming that each point [x _p , y _p , z _p ] is at an equal distance sufficiently away from the origin, different P points on the sphere centered on the origin may be used. The reason why each point [x _p , y _p , z _p ] is located sufficiently away from the origin is that the signal radiated from the sound source or virtual sound source is transmitted spherically but sufficiently separated from the sound source or virtual sound source This is because a direct sound or reflected sound can be simulated as a plane wave in a local region at the position (origin). However, this does not mean that the template information includes a template corresponding to a position in the same direction. It is assumed that the microphone array is disposed in the vicinity (local region) of the origin of the coordinate system.

The template storage unit 260 stores the template information S ^→ (ω) output from the template generation unit 250 and plays a role of providing the template information S ^→ (ω) to the reflected sound information estimation unit 270 at the time of analysis.

The reflected sound information estimation unit 270 receives a frequency domain observation signal X ^→ (ω, k) and template information S ^→ (ω) as an input, and a set of Q reflected sound information components rs _q ^→ (ω, k) ( However, the reflected sound information rs ^→ (ω, k) = [rs ₁ ^→ (ω, k),..., Rs _Q ^→ (ω, k)] ^T is calculated for each frame. k is output for each frequency ω (step S5). Here, Q represents the total number of reflected sounds to be estimated, and is set in advance to an integer value of 1 or more. The q-th (1 ≦ q ≦ Q) reflected sound information component rs _q ^→ (ω, k) is expressed as rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] RsA _q (ω, k) is the arrival amplitude of the qth reflected sound, and rsB _q (ω, k) is the arrival direction of the qth reflected sound.

  The principle of estimating the reflected sound information will be described. An example of the sound pressure distribution in a certain plane observed using a two-dimensional microphone array as shown in FIG. 6 is shown as a shading diagram at the left end of FIG. Regarding the view of the sound pressure distribution shown as a shading diagram, the black portion indicates that the sound pressure is low and the white portion indicates that the sound pressure is high. The observed sound pressure distribution includes not only the sound pressure distribution of the direct sound but also the sound pressure distribution of the reflected sound. When the direct sound and the reflected sound come sufficiently far away, each sound pressure distribution on the two-dimensional plane has a striped pattern as shown in the three shades on the right side of FIG. Striped “shading” corresponds to the arrival amplitude of direct sound or reflected sound, and “rotation / period” corresponds to the direction of arrival of direct sound or reflected sound. In the example of FIG. 8, it is shown that the sound pressure distribution of the observation signal is configured by superimposing the sound pressure distributions of the direct sound, the reflected sound 1 and the reflected sound 2 having different arrival amplitudes and directions. When considered in the frequency domain, the direct sound and each reflected sound are represented by a complex sine wave whose frequency changes according to the direction of arrival, and the observation signal is a superposition of the direct sound and multiple complex sine waves corresponding to each reflected sound. Represented as: Incidentally, the problem to be solved by the present invention is to estimate the arrival amplitude and / or the arrival direction of the reflected sound using only the observation signal. To solve this problem, from the sound pressure distribution depicted at the left end of FIG. 8, the direct sound of the three shades on the right side of FIG. 8 and the “tone” and / or “rotation / period” of the striped pattern corresponding to each reflected sound Corresponds to the estimation.

With reference to FIG. 9, an outline of a method for estimating the reflected sound information rs ^→ (ω, k) will be described. The reflected sound 0 with the strongest power included in the observation signal observed on a two-dimensional plane (corresponding to q = 1 and having the strongest power, this reflected sound 0 is usually understood as a “direct sound”. to) estimates to obtain a residual signal E ₂ from the observed signal by subtracting the reflected sound 0. Next, the reflected sound 1 having the strongest power (corresponding to q = 2) included in the residual signal E ₂ is estimated, and the reflected sound 1 is subtracted from the residual signal E ₂ to obtain a new residual sound. obtaining a difference signal E _3. Next, the reflected sound 2 having the strongest power (corresponding to q = Q = 3) included in the residual signal E ₃ is estimated. Here, the case of Q = 3 has been described, but generally, the qth strongest power included in the _qth residual signal E _q (where the first residual signal is an observed signal) The reflected sound information components (rs ₁ ^→ (ω, k),..., Are sequentially executed by subtracting the reflected sound q-1 (where reflected sound 0 is a direct sound) until q = Q. rs _Q ^→ (ω, k)). The first reflected sound information component rs ₁ ^→ (ω, k) corresponds to the reflected sound 0 (direct sound), the second reflected sound information component rs ₂ ^→ (ω, k) corresponds to the reflected sound 1, The third reflected sound information component rs ₃ ^→ (ω, k) corresponds to the reflected sound 2..., The Qth reflected sound information component rs _Q ^→ (ω, k) corresponds to the reflected sound Q-1. To do. Q depends on the application using the calculation power and the reflected sound information, but is preferably set to about 30.

Note that the sound pressure distributions in FIGS. 8 and 9 are shown as high-resolution shading diagrams, but in order to show the sound pressure distribution as such a high-resolution shading diagram, an extremely large number of microphones are required, Not practical. On the other hand, even when, for example, 100 microphones are used as a 10 × 10 two-dimensional matrix microphone array as a practical level two-dimensional matrix microphone array, the sound pressure shown as a rough (low resolution) gray scale (see FIG. 16). Only a distribution can be obtained. Thus, from a practical point of view, it is required to accurately estimate the arrival amplitude and direction of the reflected sound in a situation where only a low-resolution sound pressure distribution can be obtained. In the present invention, a plane wave arriving from an arbitrary position is specifically expressed in order to improve spatial resolution (formulation), and reflected sound with low power can be estimated under the influence of reflected sound with high power. In order to prevent disappearance, the already estimated reflected sound is removed from the observation signal to estimate the next reflected sound (decomposition). The formulation is as described in the template information, and the decomposition is as described in the outline of the estimation method of the reflected sound information rs ^→ (ω, k).

The method for estimating the reflected sound information rs ^→ (ω, k) described above will be described in detail. Prior to explanation, symbols are defined. The q-th residual signal is represented by E _q ^→ (ω, k) = [E _q1 (ω, k),..., E _qM (ω, k)] ^T , and the estimated q-th reflected sound (q = 1 In this case, direct sound is represented as A _q (ω, k, g (q, k)) S _{g (q, k)} ^→ (ω). Here, g (q, k) represents a template index that can represent the q-th reflected sound in the frame k with the highest accuracy. The coefficient A _q (ω, k, g (q, k)) constituting the reflected sound is determined by the template S _{g (q, k)} ^→ (ω ) And the reflected sound. The above-described method for subtracting the reflected sound from the residual signal in ascending order of q can be expressed by equation (6). However, 1 ≦ q ≦ Q, and E ₁ ^→ (ω, k) = X ^→ (ω, k).

At this time, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by equations (7) and (8). The direction information θ _{g (q, k)} ^→ (ω, k) is direction information corresponding to the template S _{g (q, k)} ^→ (ω).

Next, a technique for estimating the _qth reflected sound A _q (ω, k, g (q, k)) S _{g (q, k)} ^→ (ω) will be described. The qth reflected sound A _q (ω, k, g (q, k)) S _{g (q, k)} ^→ (ω) is the power of the q + 1th residual signal E _{q + 1} ^→ (ω, k) (E _{q + 1} ^→ (ω, k)) It is estimated that ^H E _{q + 1} ^→ (ω, k) is minimized. The symbol H represents conjugate transposition. There are various estimation methods, but one of them will be described. The reflected sound is composed of two elements, A _q (ω, k, g (q, k)) and S _{g (q, k)} ^→ (ω). Is required. <Process 1> and <Process 2> to be described later are performed for each q in ascending order of q.

<Process 1>
The symbol Λ is a set obtained by excluding the set of indexes determined by Expression (10) described later from the entire set {1,..., P,. That is, Λ = {1, ..., p, ..., P}-{g (1, k), ..., g (q-1, k)}. However, when <Process 1> is performed for the first time, Λ = {1,..., P,.
The p-th template S _p ^→ (ω) is the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) of the residual signal E _{q + 1} ^→ (ω, k) The coefficient A _q (ω, k, p) when it is assumed that the template is the optimum template for minimization is obtained by Expression (9) based on the least square method. Note that at this stage, q on the left side of Equation (9) has no meaning.

<Process 2>
When the number (concentration) of elements of the set Λ is | Λ |, the | Λ | coefficients A _q (ω, k, p) (p∈Λ) obtained based on the equation (9) are used to generate a template. _{G (q, k)} representing the index of S _{g (q, k)} ^→ (ω) is the power of the residual signal E _{q + 1} ^→ (ω, k) (E _{q + 1} ^→ (ω, k)) ^As an index that minimizes ^H E _{q + 1} ^→ (ω, k), it is obtained by Expression (10).

The coefficient A _q (ω, k, g (q, k)) constituting the q-th reflected sound is obtained by substituting g (q, k) obtained in Equation (10) for p in Equation (9). can get. Since the coefficient A _q (ω, k, g (q, k)) is calculated at the time of the calculation of the expression (10), it is obtained at the time of the expression (10) when there is a margin in the memory. The coefficient A _q (ω, k, g (q, k)) is stored in the memory, and after g (q, k) is obtained by Expression (10), p = g (q, k) is obtained from the memory. The coefficient A _q (ω, k, g (q, k)) corresponding to

Through the above process, Q reflected sound information components rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] (q = 1,..., Q) are obtained. .

The arrival time difference estimation unit 280 receives the reflected sound information rs ^→ (ω, k) = [rs ₁ ^→ (ω, k),..., Rs _Q ^→ (ω, k)] ^T as an input (more precisely, complex Amplitude {rsA ₁ (ω, k), ..., rsA _Q (ω, k)} is sufficient), and the relative arrival time difference rsD _q of each reflected sound with respect to the reference reflected sound (reference reflected sound) (k) arrival time difference information rsD which is a set of (1 ≦ q ≦ Q) ^→ (k) = [rsD ₁ (k),..., rsD _Q (k)] ^T Is output for each frame k (step S6). Specifically, rsD _q (k) (1 ≦ q ≦ Q) is obtained by performing ((Process 1)) and ((Process 2)) described later for each q (1 ≦ q ≦ Q). . Here, q to be processed is particularly referred to as qc.

((Process 1))
An index N of the reference reflected sound is determined from the estimated Q reflected sounds. When ((Process 1)) is performed for each q (1 ≦ q ≦ Q), if N = qc, it is obvious that the arrival time difference of the qc-th reflected sound with respect to the reference reflected sound is zero. N is preferably selected from an index difference set {1,..., Q} − {qc} that identifies the reflected sound. There are various methods for determining the reference reflected sound. For example, it is preferable to select a reflected sound having a high power among the reflected sounds corresponding to the difference set {1,..., Q} − {qc}. At this time, the influence of the frequency can be canceled by calculating the sum of the magnitudes of the arrival amplitudes for the frequency index ω included in the frequency band to be analyzed. In this case, N is determined by equation (10a). A set of indexes ω of frequencies included in the frequency band to be analyzed is Ω. For example, when a voice signal is handled, a set of indexes corresponding to the 1.0 to 3.0 kHz band may be Ω.

((Process 2))
N-th reflected sound of the incoming amplitude rsA _N (ω, k) and qc th relative arrival time differences using the phase difference between the incoming amplitude RSA _qc reflected sound (ω, k) of (qc ≠ N) RSD _qc (k) is calculated. In general, since the arrival amplitude rsA _q (ω, k) is a complex amplitude, rsA _q (ω, k) = λ _q (ω, k) using amplitude λ _q (ω, k) and time τ _q (ω, k) k) exp [iωτ _q (ω, k)]. Therefore, the phase difference of the qth (q ≠ N) reflected sound with respect to the Nth reflected sound is equal to the qth (q ≠ N) reflected sound with respect to the arrival amplitude rsA _N (ω, k) of the Nth reflected sound. It is given as the argument arg (rsA _q (ω, k) / rsA _N (ω, k)) of the arrival amplitude rsA _q (ω, k). The time difference between the Nth reflected sound and the qth (q ≠ N) reflected sound by dividing the argument arg (rsA _q (ω, k) / rsA _N (ω, k)) by the frequency index ω. λ _q (ω, k) −λ _N (ω, k) can be obtained. That is, λ _q (ω, k) −λ _N (ω, k) = arg (rsA _q (ω, k) / rsA _N (ω, k)) / ω. In order to distinguish whether the time λ _q (ω, k) relative to the time λ _N (ω, k) is “early” or “late”, a negative sign is introduced. That is, if -arg (rsA _q (ω, k) / rsA _N (ω, k)) / ω <0, the time λ _q (ω, k) with respect to the time λ _N (ω, k) is “early”. If -arg (rsA _q (ω, k) / rsA _N (ω, k)) / ω> 0, the time λ _q (ω, k) with respect to the time λ _N (ω, k) is “slow” Represents that. _{-arg (rsA q (ω, k} ) / rsA N (ω, k)) / ω = 0, indicating that the time difference is not. At this time, the influence of the frequency is canceled by calculating the sum of −arg (rsA _q (ω, k) / rsA _N (ω, k)) / ω with respect to the frequency index ω included in the frequency band to be analyzed. be able to. That is, the relative arrival time difference rsD _qc (k) is given by equation (10b). Let | Ω | be the number of elements in the set Ω (concentration).

  Note that the index of the reference reflected sound is determined in ((Process 1)). However, if the reference reflected sound is common to each q, it is sufficient that ((Process 1)) is performed once. In this case, ((Processing 1a)) is first performed, and then (Processing 2) described above is performed for each element qc of the difference set {1,..., Q} − {N}.

((Process 1a))
An index N of the reference reflected sound is determined from the estimated Q reflected sounds. There are various methods for determining the reference reflected sound. For example, it is preferable to select a reflected sound (direct sound) having a high power among the Q reflected sounds. At this time, the influence of the frequency can be canceled by calculating the sum of the magnitudes of the arrival amplitudes for the frequency index ω included in the frequency band to be analyzed. In this case, N is determined by equation (10c).

Note that the sound source distance dis _q (ω, k) from the microphone array to the sound source or the virtual sound source is an amount obtained by multiplying the relative arrival time difference by a constant, and is given by the equation (10d). c is the speed of sound.

Below, reflected sound needed to determine the relative arrival time difference information ^{rs → (ω, k) =} [rs 1 → (ω, k), ..., rs Q → (ω, k)] some to seek a ^T The embodiment will be described. In any embodiment, the relative arrival time difference rsD _q (k) (1 ≦ q ≦ Q) can be obtained by performing the process of step S6 of the first embodiment.

<< Second Embodiment >>
In the first embodiment, the reflected sound information rs ^→ (ω, k) is obtained using the template information S ^→ (ω). However, the template information S ^→ (ω) is a set of P templates _Sp ^→ (ω). ) Is not necessarily required in advance. An embodiment in which template information S ^→ (ω) is not obtained in advance will be described as a second embodiment.

  In the second embodiment, the processes in steps S1 to S4 in the first embodiment are performed. However, the process in step Sp in the first embodiment is not necessary, and is further replaced with the process in step S5 in the first embodiment. Step S5a is then performed (see FIG. 13). Therefore, the duplicated description of the same items as those in the first embodiment will be omitted, and items different from those in the first embodiment will be described.

Define the symbol. q q residual signal is represented by E _q ^→ (ω, k) = [E _q1 (ω, k), ..., E _qM (ω, k)] ^T , qth reflected sound (if q = 1, direct A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)). R _q ^→ (ω, θ _q ^→ (ω, k)) = (R ₁ (ω, θ _q ^→ (ω, k)), ..., R _M (ω, θ _q ^→ (ω, k))] ^T is a function that simulates the transfer characteristics for each frequency between an arbitrary position [x, y, z] in space and each microphone (hereinafter referred to as the transfer characteristic function). Any function that simulates the transfer characteristics may be used. Usually, each transfer characteristic R _m (ω, θ _q ^→ (ω, k)) constituting the transfer characteristic function and the calculation formula for each element S _pm (ω) of the template are the same. In this case, between the position [x, y, z] in the direction represented by the direction information θ _q ^→ (ω, k) and the m th sound receiving point [u _m , v _m , w _m ] The transfer characteristic R _m (ω, θ _q ^→ (ω, k)) for each frequency is expressed by equation (11). Note that the position [x, y, z] in the direction represented by the direction information θ _q ^→ (ω, k) may be, for example, a position on a spherical surface sufficiently away from the coordinate system origin. The reason why the position [x, y, z] is sufficiently distant from the origin is as described above. Specifically, the position [x, y, z] is a sound source in the local region where the microphone array is arranged. Or it is preferably an arbitrary position in a space at a distance that can simulate a direct sound or reflected sound from a virtual sound source as a plane wave. Since the three-dimensional orthogonal coordinate system and the spherical coordinate system can be converted to each other (coordinate conversion), the right side of Equation (11) is not the position [x, y, z] but the direction information θ _q ^→ (ω, Using k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)], for example, it can also be expressed as in equation (11a). Here, d is a microphone interval, the microphone array is a two-dimensional microphone array of Φ rows and Ξ columns (Φ × Ξ = M), and the position of the mth microphone is φ rows and ξ columns (1 ≦ φ ≦ Φ, 1 ≦ ξ ≦ Ξ).

A _q (ω, k) constituting the reflected sound is a template R _q ^→ (ω, θ _q ^→ (ω, k)) such as the phase of the sound source 200 itself, reflection on the wall, attenuation due to distance, and reflected sound. This corresponds to the arrival amplitude. The above-described method for subtracting the reflected sound from the residual signal in ascending order of q can be expressed by equation (12). However, 1 ≦ q ≦ Q, and E ₁ ^→ (ω, k) = X ^→ (ω, k).

Next, a method for optimizing the reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) will be described.
The q-th optimized reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) is expressed by the q + 1-th residual signal E _{q +} represented by Expression (12). ₁ ^→ (ω, k) power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) is determined according to a standard that minimizes the power. Specifically, when it is noted that the transfer characteristic function R _q ^→ (ω, θ _q ^→ (ω, k)) is determined by the direction information θ _q ^→ (ω, k), the q-th reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) parameter A _q (ω, k), θ _q ^→ (ω, k) optimum value A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k) is obtained by equation (13). Note that the symbol H represents conjugate transposition.

At this time, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by the equations (14) and (15).

Various specific calculation methods of Expression (13) are conceivable, but an example is shown here. The direction information θ _q ^→ (ω, k) is determined by an arrival direction estimation method such as a beam former method. The beam former method is a method in which a directional beam is spatially scanned and a direction in which power is increased is searched for from the obtained power spectrum. Here, it is assumed that P arrival directions can be estimated by the beamformer method.

  In practice, the power spectrum obtained by the beamformer method may not be steep with respect to the direction of arrival. In such a case, for example, the range of the direction corresponding to the power spectrum showing the spectral intensity equal to or higher than the predetermined spectral intensity. The arrival direction may be determined at predetermined intervals. As a specific example, assuming that a power spectrum showing a spectral intensity equal to or higher than a predetermined spectral intensity in a polar angle range of 5 ° and an azimuth angle of 10 ° to 20 ° is obtained, the direction of arrival is determined every predetermined interval of 2 °. (Polar angle 5 °, azimuth angle 10 °), (polar angle 5 °, azimuth angle 12 °), (polar angle 5 °, azimuth angle 14 °), (polar angle 5 °, azimuth angle 16 °), (Polar angle 5 °, azimuth angle 18 °), (polar angle 5 °, azimuth angle 20 °) may be the arrival direction.

  Even if the power spectrum shows a steep peak in a certain direction, the direction of arrival may be determined within a predetermined range of the direction instead of simply determining that direction as one of the directions of arrival. As a specific example, assuming that a power spectrum showing a steep peak at a polar angle of 30 ° and an azimuth angle of 50 ° is obtained, the direction of arrival is within a predetermined range (polar angle ± 4 °, azimuth angle ± 4 °, interval 2 °). (Polar angle 26 °, azimuth angle 46 °), (polar angle 28 °, azimuth angle 46 °), (polar angle 30 °, azimuth angle 46 °), (polar angle 32 °, azimuth angle 46 ° ), (Polar angle 34 °, azimuth angle 46 °), (polar angle 26 °, azimuth angle 48 °), (polar angle 28 °, azimuth angle 48 °), (polar angle 30 °, azimuth angle 48 °), (Polar angle 32 °, azimuth angle 48 °), (polar angle 34 °, azimuth angle 48 °), (polar angle 26 °, azimuth angle 50 °), (polar angle 28 °, azimuth angle 50 °), (polar Angle 30 °, azimuth angle 50 °), polar angle 32 °, azimuth angle 50 °, polar angle 34 °, azimuth angle 50 °, polar angle 26 °, azimuth angle 52 °, and polar angle 28 °, azimuth angle 52 °), (polar angle 30 °, azimuth 52 °), (polar angle 32 °, azimuth angle 52 °), (polar angle 34 °, azimuth angle 52 °), (polar angle 26 °, azimuth angle 54 °), (polar angle 28 °, azimuth angle 54 ° ), (Polar angle 30 °, azimuth angle 54 °), (polar angle 32 °, azimuth angle 54 °), and (polar angle 34 °, azimuth angle 54 °). Note that P is a fixed value in the first embodiment, but in the second embodiment, P is a value that depends on an estimation result obtained by an arrival direction estimation method such as a beamformer method.

A template is generated for P arrival directions obtained by the beamformer method. A calculation formula for each element of the template is, for example, Formula (6). The processing described in the first embodiment may be performed using the P templates (template information S ^→ (ω)).

<< Third Embodiment >>
Unlike the second embodiment, the third embodiment uses the template information to optimize the reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)). An example of a specific calculation method of Expression (13) will be shown. The optimization method described below is applied to each q in ascending order of q.

§1 Initial value setting of direction information First, initial value θ _{ini, q} ^→ (ω, k) of direction information θ _q ^→ (ω, k) is determined using template information S ^→ (ω). For this purpose, a template corresponding to the direction considered to be closest to the direction of arrival to be estimated is determined, and the direction information corresponding to the determined template is determined as the initial value θ of the direction information θ _q ^→ (ω, k). _{ini, q} ^→ (ω, k).

Therefore, in order to determine the template as described above from the template information, the reflected sound is expressed as A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) for convenience. I will decide. Here, g (ω, q) represents an index of a template that can represent the q-th reflected sound most accurately in the template information. The coefficient A _q (ω, k, g (ω, q)) constituting the reflected sound is determined by the template S _{g (ω, q)} ^→ (ω ) And the reflected sound. In this case, the q + 1-th residual signal E _{q + 1} ^→ (ω, k) is expressed as in Expression (16). However, E ₁ ^→ (ω, k) = X ^→ (ω, k).

The reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) is expressed as q + 1th residual signal E _{q + 1} ^→ (ω, k) based on the equation (16). ) Power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) is estimated according to a standard that minimizes. There are various estimation methods, but one of them will be described. The reflected sound consists of two elements, A _q (ω, k, g (ω, q)) and S _{g (ω, q)} ^→ (ω). Is required. The following = process 1 = and = process 2 = are performed for each q in ascending order of q.

= Process 1 =
The symbol Λ is a set obtained by excluding the set of indexes determined by the equation (18) described later from the entire set {1,..., P,. That is, Λ = {1, ..., p, ..., P}-{g (ω, 1), ..., g (ω, q-1)}. However, when <Process 1> is performed for the first time, Λ = {1,..., P,.
The p-th template S _p ^→ (ω) is the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) of the residual signal E _{q + 1} ^→ (ω, k) The coefficient A _q (ω, k, p) when it is assumed that the template is the optimum template for minimization is obtained by Expression (17) based on the least square method. Note that at this stage, q on the left side of Equation (17) has no meaning.

= Process 2 =
When the number (concentration) of elements of the set Λ is | Λ |, a template is obtained using | Λ | coefficients A _q (ω, k, p) (p∈Λ) obtained based on the equation (17). _{G (ω, q)} representing the index of S _{g (ω, q)} ^→ (ω) is the power of the residual signal E _{q + 1} ^→ (ω, k) (E _{q + 1} ^→ (ω, k)) ^As an index that minimizes ^H E _{q + 1} ^→ (ω, k), it is obtained by Expression (18).

Therefore, the initial value θ _{ini, q} ^→ (ω, k) of the direction information θ _q ^→ (ω, k) is a template S _{g (ω,} k) having g (ω, q) obtained by the equation (18) as an index _{. q)} ^→ Direction information corresponding to (ω) θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] As given. That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)]. Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

§2 Optimization of reflected sound Next, with the initial value θ _{ini, q} ^→ (ω, k) of the direction information θ _q ^→ (ω, k) as the starting point, the q + 1th residual represented by equation (12) In order to minimize the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) of the signal E _{q + 1} ^→ (ω, k), the reflected sound A _q (ω, k) k) Optimize R _q ^→ (ω, θ _q ^→ (ω, k)). The reflected sound is composed of two elements: coefficient A _q (ω, k) and R _q ^→ (ω, θ _q ^→ (ω, k)), so it is necessary to optimize for the two elements It becomes. There are various optimization methods, but one of them (gradient method) will be described. In the illustrated method, correction of the direction information θ _q ^→ (ω, k) and correction of the coefficient A _q (ω, k) are alternately repeated a predetermined number of times (δ times), whereby the reflected sound A _q (ω , k) R _q ^→ (ω, θ _q ^→ (ω, k)) is optimized. For example, δ is about 50, but may be 1.

§2.1 Correction of direction information The correction of direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] is updated by equation (19). Is done by. When the process of §2.1 is performed for the first time, the direction information θ _q ^→ (ω, k) on the right side of the equation (19) is the initial value θ _{ini, q} ^→ (ω, k) obtained by the process of §1. When the processing of §2.1 is not the first time, the direction information θ _q ^→ (ω, k) on the right side of Equation (19) is the direction information obtained by the processing of §2.1 immediately before. Further, when the processing of §2.1 is performed for the first time, the coefficient A _q (ω, k) used for calculating the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) Is A _q (ω, k, p) obtained in equation (17), and power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} if the processing of §2.1 is not the first time ^{→ The} coefficient A _q (ω, k) used in the calculation of (ω, k) is the coefficient A _q (ω, k) obtained in the immediately preceding §2.2 process (described later). The step widths α ₁ and α ₂ are small positive constants, which are determined in consideration of the convergence speed and the like, and are each about 0.1, for example.

§2.2 Correction of coefficient The coefficient A _q (ω, k) is corrected by obtaining a new coefficient A _q (ω, k) according to the equation (20) based on the least square method. R _q ^→ (ω, θ _q ^→ (ω, k)) used in the equation (20) is obtained from the direction information θ _q ^→ (ω, k) obtained by the processing of §2.1 and the equation (6). .

The coefficient A _q (ω, k) and direction information θ _q ^→ (ω, k) obtained when δ iterations are completed are A _{q, opt} (ω, k) and θ _{q, opt} ^→ ( ω, k), and the q-th reflected sound information component rs _q ^→ (ω, k). That is, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by the equations (21) and (22).

Through the above process, Q reflected sound information components rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] (q = 1,..., Q) are obtained. . When δ = 1 is set, only the direction of arrival can be obtained as reflected sound information by not correcting the coefficient.

<< 4th Embodiment >>
Next, unlike the first, second, and third embodiments, Q reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) are collectively optimized. A method will be described.

Q optimized reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) are residual signals obtained by removing Q reflected sounds from the observed signal. E ^→ (ω, k) power of ^{(E → (ω, k)} ) H E → (ω, k) is determined according to the criteria to minimize the (formula (23a) refer). Specifically, if it is noted that the transfer characteristic function R _q ^→ (ω, θ _q ^→ (ω, k)) is determined by the direction information θ _q ^→ (ω, k), Q reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) (1 ≦ q ≦ Q) Each parameter A _q (ω, k), θ _q ^→ (ω, k) (1 The optimum values A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k) (1 ≦ q ≦ Q) of ≦ q ≦ Q are obtained by the equation (23b). Note that the symbol H represents conjugate transposition. In equation (23b), {(A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k))} _{q∈ {1,..., Q}} is {(A _{1, opt} (ω , k), θ _{1, opt} ^→ (ω, k)),…, (A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k)),…, (A _{Q, opt} (ω , k), θ _{Q, opt} ^→ (ω, k))}, {(A _q (ω, k), θ _q ^→ (ω, k))} _{q∈ {1,.} {(A ₁ (ω, k), θ ₁ ^→ (ω, k)), ..., (A _q (ω, k), θ _q ^→ (ω, k)), ..., (A _Q (ω, k ), Θ _Q ^→ (ω, k))}.

At this time, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by equations (24) and (25).

  Various specific calculation methods of the equation (23b) are conceivable, but an example is shown here.

§1.1 Initial value setting of direction information First, initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) are set. As a method for determining Q initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q), in this embodiment, initial values θ _{ini of} Q direction information θ _q ^→ (ω, k). _{, q} ^→ (ω, k) (1 ≦ q ≦ Q) will be described with reference to the observation signal X ^→ (ω, k) and the template information S ^→ (ω). According to this method, Q templates corresponding to Q directions considered to be closest to each of Q estimated directions of arrival are determined, and direction information corresponding to the determined Q templates is determined. The initial value θ _{ini, q} ^→ (ω, k) of Q pieces of direction information θ _q ^→ (ω, k) may be set to 1 ≦ q ≦ Q. In this case, there is a relationship of Q <P between P and Q.

Therefore, in order to determine the template as described above from the template information, for convenience, the q-th reflected sound is expressed as A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω ). Here, g (ω, q) represents an index of a template that can represent the q-th reflected sound most accurately in the template information. The coefficient A _q (ω, k, g (ω, q)) constituting the reflected sound is determined by the template S _{g (ω, q)} ^→ (ω ) And the reflected sound. In this case, the remaining signal obtained by removing the q-th reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) from the observed signal X ^→ (ω, k). The difference signal E _q ^→ (ω, k) is expressed as in Expression (26).

The q-th reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) is obtained from the residual signal E _q ^→ (ω, k) based on the equation (26). Power (E _q ^→ (ω, k)) Estimated according to a standard that minimizes ^H E _q ^→ (ω, k). There are various estimation methods, but one of them will be described.

The reflected sound consists of two elements, A _q (ω, k, g (ω, q)) and S _{g (ω, q)} ^→ (ω). Is required. First, the p-th template S _p ^→ (ω) minimizes the power (E _q ^→ (ω, k)) ^H E _q ^→ (ω, k) of the residual signal E _q ^→ (ω, k). The coefficient A _q (ω, k, p) in the case where it is assumed that the template is an optimal template is obtained from the equation (27) based on the least square method. Note that at this stage, q on the left side of Equation (27) has no meaning.

Next, among the P coefficients A _q (ω, k, p) (1 ≦ p ≦ P) obtained based on the equation (27), the Q coefficients A _q are ordered in descending order of their absolute values. The index q (1 ≦ q ≦ Q) of (ω, k, p) is determined (see formula (28)). The symbol Λ is a set obtained by subtracting the set of indexes determined by the equation (28) from the entire set {1,..., P,..., P} of the index p, and Λ = {1,. , P}-{g (ω, 1),..., G (ω, q-1)}.

Accordingly, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of the Q pieces of direction information θ _q ^→ (ω, k) is obtained by Q pieces of g (ω , q) (S ≦ _G ≦ ω ≦ q ≦ Q) (1 ≦ q ≦ Q) ^→ (ω) (1 ≦ q ≦ Q) direction information θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

§1.2 coefficients A _q (ω, k) initial value setting Next, Q-number of coefficients A _q (ω, k) initial value A _{ini of, q (ω,} k) is set to. There are various methods for determining the Q initial values A _{ini, q} (ω, k) (1 ≦ q ≦ Q). Here, as an example, the Q initial values A _{ini on the} power minimization basis are considered. _{, q} (ω, k) (1 ≦ q ≦ Q) will be described. First, A _q (ω, k, p) = 0 (1 ≦ p ≦ P). The initial value A _{ini, q} (ω, k) (1 ≦ q ≦ Q) is the power (E ^→ (ω, k)) ^H E ^→ (ω, k) of the residual signal E ^→ (ω, k). ) Based on the least square method so as to minimize. In Equation (29), F _q ^→ (ω, k) is given by Equation (30). In equation (30), Υ = {1, ..., q-1, q + 1, ..., Q}, and F _q ^→ (ω, k) is the residual obtained by removing the q-th reflected sound from the observed signal Signal. Note that Q initial values θ _{ini, q} ^→ (ω, k) determined in §1.1 are used as Q pieces of direction information θ _q ^→ (ω, k). R _q ^→ (ω, θ _q ^→ (ω, k)) used in Expression (29) is obtained from the initial value θ _{ini, q} ^→ (ω, k) of the direction information and Expression (6).

§2 Optimization of reflected sound Next, starting from the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k), the equation (23a ) Q reflected sounds A _q (ω to minimize the power (E ^→ (ω, k)) ^H E ^→ (ω, k) of the residual signal E ^→ (ω, k) , k) R _q ^→ (ω, θ _q ^→ (ω, k)) (1 ≦ q ≦ Q) is optimized collectively. Each reflected sound is composed of two elements: coefficient A _q (ω, k) and R _q ^→ (ω, θ _q ^→ (ω, k)). Necessary. There are various optimization methods, but one of them (gradient method) will be described. In the illustrated method, the correction of the direction information θ _q ^→ (ω, k) and the correction of the coefficient A _q (ω, k) are alternately repeated a predetermined number of times (δ times), thereby Q reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) (1 ≦ q ≦ Q) is optimized. For example, δ is set to a value of about 100, but may be 1.

§2.1 Correction of direction information Q pieces of direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] (1 ≦ q ≦ Q) Is corrected by updating according to the equation (31). When the process of §2.1 is performed for each q (1 ≦ q ≦ Q) for the first time, the direction information θ _q ^→ (ω, k) on the right side of the equation (31) is the initial value obtained by the process of §1.1 If the value θ _{ini, q} ^→ (ω, k) and the processing of §2.1 is not the first time, the direction information θ _q ^→ (ω, k) on the right side of Equation (31) is The direction information obtained by the processing is used. Further, when the processing of §2.1 is performed for the first time, the coefficient A _q (ω, k) (q∈Υ) used for calculating the power (F _q ^→ (ω, k)) ^H F _q ^→ (ω, k) ) Using A _{ini, q} (ω, k) (1 ≦ q ≦ Q) obtained by the equation (29), the power (F _q ^→ (ω, k) )) The coefficient A _q (ω (k, k) (q∈Υ) used in the calculation of ^H F _q ^→ (ω, k) is the coefficient A _q ( ω, k) (1 ≦ q ≦ Q) is used. The step widths α ₁ and α ₂ are small positive constants, which are determined in consideration of the convergence speed and the like, and are each about 0.1, for example.

§2.2 Correction of coefficients The correction of the Q coefficients A _q (ω, k) (1 ≦ q ≦ Q) is based on the least square method, and a new coefficient A _q (ω, k) is obtained according to equation (32). This is performed by obtaining (1 ≦ q ≦ Q). R _q ^→ (ω, θ _q ^→ (ω, k)) used in the equation (32) is obtained from the direction information θ _q ^→ (ω, k) obtained by the processing of §2.1 and the equation (6). . When the processing of §2.2 is performed for the first time as the coefficient A _q (ω, k) (q∈Υ) used in the calculation of F _q ^→ (ω, k) in Equation (32), When the initial value A _{ini, q} (ω, k) obtained in the processing is used and the processing in §2.2 is not the first time, the coefficient A _q (ω, k) obtained in the processing in the previous §2.2 is used. ) (1 ≦ q ≦ Q) is used.

Q combinations (A _q (ω, k), θ _q ^→ ) of coefficient A _q (ω, k) and direction information θ _q ^→ (ω, k) obtained at the end of δ iterations (ω, k)) (1 ≦ q ≦ Q) is {(A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k))} _{q∈ {1,..., Q}} Q reflected sound information components rs _q ^→ (ω, k) (1 ≦ q ≦ Q). That is, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by Expression (33) and Expression (34).

Through the above process, Q reflected sound information components rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] (q = 1,..., Q) are obtained. . When δ = 1 is set, only the direction of arrival can be obtained as reflected sound information by not correcting the coefficient.

<< 5th Embodiment >>
The fifth embodiment is different from the fourth embodiment in “§1.1 Initial value setting of direction information”. Therefore, the duplicated description of the same items as those in the fourth embodiment will be omitted, and items different from those in the fourth embodiment will be described.

§1.1 Initial value setting of direction information First, initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) are set. As a method for determining Q initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q), in this embodiment, initial values θ _{ini of} Q direction information θ _q ^→ (ω, k). _{, q} ^→ (ω, k) (1 ≦ q ≦ Q) will be described using the observation signal X ^→ (ω, k) and the template information S ^→ (ω). According to this method, Q templates corresponding to Q directions considered to be closest to each of Q estimated directions of arrival are determined, and direction information corresponding to the determined Q templates is determined. The initial value θ _{ini, q} ^→ (ω, k) of Q pieces of direction information θ _q ^→ (ω, k) may be set to 1 ≦ q ≦ Q. In this case, there is a relationship of Q <P between P and Q.

Therefore, in order to determine the template as described above from the template information, for convenience, the q-th reflected sound is expressed as A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω ). Here, g (ω, q) represents an index of a template that can represent the q-th reflected sound most accurately in the template information. The coefficient A _q (ω, k, g (ω, q)) constituting the reflected sound is determined by the template S _{g (ω, q)} ^→ (ω ) And the reflected sound. In this case, the residual signal E _{q + 1} ^→ (ω, k) obtained by removing the first to q-th reflected sounds from the observed signal is expressed as in Expression (35). However, 1 ≦ q ≦ Q, and E ₁ ^→ (ω, k) = X ^→ (ω, k).

The q-th reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) is the residual signal E _{q + 1} ^→ (ω, k) based on the equation (35). ) Power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) is estimated according to a standard that minimizes. There are various estimation methods, but one of them will be described. The reflected sound consists of two elements, A _q (ω, k, g (ω, q)) and S _{g (ω, q)} ^→ (ω). Is required. <Process 1> and <Process 2> to be described later are performed for each q in ascending order of q.

<Process 1>
The symbol Λ is a set obtained by excluding the set of indexes determined by the equation (37) described later from the entire set {1,..., P,. That is, Λ = {1, ..., p, ..., P}-{g (ω, 1), ..., g (ω, q-1)}. However, when <Process 1> is performed for the first time, Λ = {1,..., P,.
The p-th (p∈Λ) template S _p ^→ (ω) is the power (E _{q + 1} ^→ (ω, k)) ^{H of the} residual signal E _{q + 1} ^→ (ω, k) based on the equation (35). The coefficient A _q (ω, k, p) when assuming that E _{q + 1} ^→ (ω, k) is an optimal template is obtained from Equation (36) based on the method of least squares. It is done. Note that at this stage, q on the left side of Equation (36) has no meaning.

<Process 2>
If that was obtained based on the equation (36) | | number of elements in the set lambda (concentration) | lambda lambda | among the pieces of coefficients _{A q (ω, k, p} ) (p∈Λ), its The index q (1 ≦ q ≦ Q) of the coefficient A _q (ω, k, p) having the maximum absolute value is determined (see Expression (37)).

Therefore, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of the Q pieces of direction information θ _q ^→ (ω, k) is expressed by Q pieces of g (ω , q) (S ≦ _G ≦ ω ≦ q ≦ Q) (1 ≦ q ≦ Q) ^→ (ω) (1 ≦ q ≦ Q) direction information θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

In the fifth embodiment, a generalized harmonic analysis determination method is used to determine the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k). Using. According to the generalized harmonic analysis determination method, the initial value calculation amount increases compared to the beam search determination method, but it is highly possible to set an initial value close to the reflected sound (correct answer) to be estimated. In addition to improving the estimation accuracy, the number of iterations can be reduced.

<< 6th Embodiment >>
In the sixth embodiment, “§1.1 Initial value setting of direction information” is different from the fourth embodiment. Therefore, the duplicated description of the same items as those in the fourth embodiment will be omitted, and items different from those in the fourth embodiment will be described. The concept of the sixth embodiment is to prevent the above-described bias of the initial values while being simpler than the fourth embodiment.

§1.1 Initial value setting of direction information First, a plurality of directions are determined so that there is no bias for either one of θ _pol (ω) and θ _azi (ω) constituting the direction information. In this example, β (β ≧ 2) directions {θ _{1, pol} (ω),..., Θ _{β, pol} (ω)} are determined so that there is no deviation with respect to the polar angle θ _pol (ω). Usually, the polar angle satisfies 0 ° ≦ θ _pol (ω) ≦ 180 °. For example, by setting the direction at equal intervals of 10 °, {θ _{1, pol} (ω),…, θ _{β, pol} (ω) } = {0,10,20, ..., 180} (β = 19). Corresponds to the direction information (position) having one of the elements of the set {θ _{1, pol} (ω),…, θ _{β, pol} (ω)} as the polar angle θ _pol (ω) among the templates included in the template information Let Ψ be the set of template indexes to be performed. At this time, it is preferable that the number (concentration) | Ψ | of the elements of the set Ψ satisfies Q ≦ | Ψ | <P. The set Ψ is a true subset of the set of direction information (position) corresponding to the template included in the template information.

The p-th template S _p ^→ (ω) minimizes the power (E _q ^→ (ω, k)) ^H E _q ^→ (ω, k) of the residual signal E _q ^→ (ω, k). The coefficient A _q (ω, k, p) when it is assumed to be an optimal template is obtained from the equation (38a) based on the least square method. However, the index p of the template used on the right side of the equation (38a) is pεΨ. Note that at this stage, q on the left side of Equation (38a) has no meaning.

Next, out of | Ψ | coefficients A _q (ω, k, p) (1 ≦ p ≦ | Ψ |) obtained based on the equation (38a) under the condition of p∈Ψ, The index q (1 ≦ q ≦ Q) of the Q coefficients A _q (ω, k, p) is determined in order from the largest value (see equation (38b)). The symbol Γ is a set obtained by subtracting the set of indexes determined by the equation (38b) from the set Ψ, and Γ = Ψ− {g (ω, 1),..., G (ω, q−1)}. .

Accordingly, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of the Q pieces of direction information θ _q ^→ (ω, k) is expressed by Q pieces of g (ω , q) (S ≦ _G ≦ ω ≦ q ≦ Q) (1 ≦ q ≦ Q) ^→ (ω) (1 ≦ q ≦ Q) direction information θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

In addition to these embodiments, an embodiment in which the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) is set at random is allowed. The

<Modification>
In the above-described embodiment, the reflected sound information rs ^→ (ω, k) is estimated using the observation signal X ^→ (ω, k) for each frequency. However, when the reflected sound information is estimated for each frequency, it is uniquely estimated. Information regarding directions other than the direction of the virtual sound source (estimated arrival direction) may be included, and as a result, errors may occur in the reflected sound information. For example, it is desirable that only information related to the estimated arrival direction can be extracted as shown in FIG. 14 (a), but in reality, information related to directions other than the estimated arrival direction is mixed as shown in FIG. 14 (b). There can be.

  Therefore, in the modification, the estimation error of the reflected sound information is reduced by calculating the power collectively over all frequencies. That is, as shown in FIG. 15, the influence of directions other than the estimated arrival direction can be reduced as much as possible by integrating the power of the residual signal over all frequencies. In general, the power at each frequency varies in directions other than the estimated direction of arrival. Therefore, by integrating the power of the residual signal over all frequencies, the power in the other direction compared to the power in the estimated direction of arrival. The relative influence of power can be reduced. Note that in FIG. 15, the power of the vertical axis indicates a relative value, so the scales of the graphs are not the same.

The processing in this modification is as follows. A set of frequency indexes ω included in the frequency band to be analyzed is Ω. For example, if an audio signal is handled, the set of indexes corresponding to the 1.0 to 3.0 kHz band may be Ω. Then, an index g (q, k) of the template S _{g (q, k)} ^→ (ω) is obtained by Expression (39) instead of Expression (10).

The index g (ω, q) of the template S _{g (ω, q)} ^→ (ω) is obtained by equation (40) instead of equation (18). Further, the correction of the direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] is performed by the equation (41) instead of the equation (19). Done by renewal.

The index g (ω, q) of the template S _{g (ω, q)} ^→ (ω) is obtained by the equation (42) instead of the equations (28) and (37). Similarly, it calculates | requires by Formula (43) instead of Formula (38). Further, the correction of the direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] is performed by the equation (44) instead of the equation (31). Done by renewal.

  The experimental result which estimated the relative arrival time difference using this invention is shown. In a rectangular parallelepiped room, 100 microphones were two-dimensionally arranged in 10 rows and 10 columns at equal intervals, and an environment in which the microphones were arranged near a wall was simulated. The microphone interval d is 4 cm. Detailed experimental conditions are shown in FIG. As the coordinates of the template, as shown in FIG. 10, they were arranged at equal intervals on the hemisphere surrounding the microphone array. FIG. 12 shows the experimental results when the 1.0-3.0 kHz band is analyzed. A direct sound was selected as the reference reflected sound. In order to avoid complication of the figure, only the reflected sound 1-3 is illustrated here. For reflected sound 1, the arrival time difference from the direct sound is 1.4 msec on average in frequency, for reflected sound 2, the arrival time difference from the direct sound is 2.5 msec on average in frequency, and for reflected sound 3 from the direct sound The arrival time difference was 4.7 msec on average in frequency. Since the change in the phase with respect to the distance is small in the low frequency band, the error in the estimated value of the arrival time difference tends to increase. Therefore, it is preferable to obtain the arrival time difference by taking an average over the entire frequency band.

<Application example>
The reflected sound information is very important voice information for human life. For example, a visually impaired person grasps the environment by reflecting a sound source signal generated by tapping on a wall or ceiling and observing with an ear. Further, even in everyday conversation, there is a difference in the ease of conversation between talking in a room where moderate reflection occurs and talking in an environment with relatively few reflected sounds. Hereinafter, service examples using reflected sound information estimated according to the present invention will be described.
The first is an example in which the present invention is incorporated in a conference system. Since the amplitude of the reflected sound changes according to the direction of the directional sound source, if the reflected sound information is known, it is possible to estimate in which direction the sound source is directed. If a sound source direction estimation device is incorporated in the conference system, it can be applied to presenting who spoke.
The second is a system that allows users to view video and audio at any position. Sound far away is difficult to pick up because the power of the sound source coming directly is small. If the reflected sound information is known, not only the direct sound but also the reflected sound can be picked up and collected, so that it is possible to enhance the sound in the distance. In the field of audio processing, it is possible to emphasize and collect sound sources by direction, but it is very difficult to emphasize and collect sounds by distance. If the reflected sound information is known, a physical feature amount corresponding to the distance can be obtained, so that sound can be collected for each distance. If far-field sounds can be picked up or picked up by direction and distance, a sound field corresponding to the position selected by the viewer can be generated in a pseudo manner.

  In a voice communication system, estimating reflected sound information leads to obtaining information on a sound field that could not be obtained only by direct sound. If the reflected sound information is known, it will lead to far-field sound collection and sound collection by distance, which could not be done with conventional speech enhancement technology, or information on the sound field that could not be estimated with conventional sound collection technology (for example, The direction of the sound source can be estimated. Such estimation of sound field information leads to the development of a speech processing apparatus that could not be realized by the conventional technology. The prior art related to the estimation of reflected sound information required observation of a special signal in order to obtain an impulse response, but the present invention has an advantage that reflected sound information can be obtained with a general observation signal such as an audio signal. have.

<Example of hardware configuration of reflected sound information estimation device>
The reflected sound information estimation apparatus according to the above-described embodiments may include an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a CPU (Central Processing Unit) [cache memory, or the like. ] RAM (Random Access Memory) or ROM (Read Only Memory) and external storage device as a hard disk, and data exchange between these input unit, output unit, CPU, RAM, ROM, and external storage device It has a bus that can be connected. Further, if necessary, the reflected sound information estimation device may be provided with a device (drive) that can read and write a storage medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the reflected sound information estimation device stores a program for estimating reflected sound information and data necessary for processing of the program [not limited to the external storage device, for example, a program is read-only. You may memorize | store in ROM which is a memory | storage device. ]. Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device. Hereinafter, a storage device that stores data, addresses of storage areas, and the like is simply referred to as a “storage unit”.

  The storage unit of the reflected sound information estimation device has a program for performing AD conversion on an analog signal, a program for performing frame division processing, and a program for converting a digital signal for each frame into an observation signal in the frequency domain A program for generating template information and a program for estimating reflected sound information using frequency domain observation signals and template information are stored.

  In the reflected sound information estimation apparatus, each program stored in the storage unit and data necessary for processing each program are read into the RAM as necessary, and are interpreted and processed by the CPU. As a result, the CPU implements predetermined functions (AD conversion unit, frame division unit, frequency domain conversion unit, template generation unit, reflected sound information estimation unit, arrival time difference estimation unit), thereby realizing estimation of reflected sound information. Is done.

<Supplementary note>
The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  When the processing functions in the hardware entity (reflected sound information estimation apparatus) described in the above embodiment are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (21)

A function that simulates transfer characteristics for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer of 1 to P. (Hereinafter referred to as transfer characteristic function) as a template for each p,
A signal (hereinafter referred to as an observation signal) obtained by collecting sound source signals with M microphones and converted into the frequency domain, and the template, are input.
Integer Q defined small 2 or more pre than P, and q is 1 or more Q less each integer, the observed signal to the Q direction with respect to the M microphones (hereinafter, arrival of direction) coming respectively from The Q reflection sounds are superimposed, and a template that can represent the qth reflection sound with the highest accuracy is obtained from the P templates, and the direction corresponding to the template is determined from the Q directions. a q-th direction, and the Q template obtained, based on the direction corresponding to the template, to estimate the Q-number of reflections each complex amplitude, the the Q-number of complex amplitudes obtained Q A reflected sound information estimation unit for the arrival amplitude of each reflected sound;
For reflected sound (hereinafter referred to as target reflected sound) other than the reference reflected sound (hereinafter referred to as reference reflected sound), the deviation of the arrival amplitude of the target reflected sound relative to the arrival amplitude of the reference reflected sound divided by the frequency An arrival time difference estimator that sets the arithmetic mean of the frequencies to the relative arrival time difference of the target reflected sound relative to the reference reflected sound;
Reflected sound information estimation apparatus having The reflected sound information estimation apparatus according to claim 1,
The reflected sound information estimation unit
(1) The p-th complex amplitude is set so that the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the p-th template by the p-th complex amplitude from the observed signal is minimized. Determine the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the determined p-th complex amplitude by the p-th template from the observed signal for each p, Determine Q templates that gave the least power,
(2) Based on the determined Q templates and the direction corresponding to each of the Q templates, the complex amplitude of each of the Q reflected sounds is estimated, and the obtained Q complex amplitudes Is a reflected sound information estimation device, wherein each of the Q reflected sounds is an arrival amplitude . In the reflected sound information estimation device according to claim 2,
The reflected sound information estimation unit
A reflected sound information estimation apparatus, wherein the complex amplitude multiplied by the template determined in (1) is estimated as the arrival amplitude of the p-th reflected sound. In the reflected sound information estimation device according to claim 2,
The reflected sound information estimation unit
Observation of a product obtained by multiplying a template for each frequency between an arbitrary position in the space and each of the microphones by a complex amplitude in the vicinity of the direction D determined by the position corresponding to the template determined in (1) above. The arrival direction of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E obtained by subtracting from the signal is minimized, and the complex multiplied by the template corresponding to the arrival direction is estimated. An apparatus for estimating reflected sound information, wherein an amplitude is estimated as an arrival amplitude of the p-th reflected sound. The reflected sound information estimation apparatus according to claim 1,
The reflected sound information estimation unit
First, p is an integer of 1 to P, and for each p, the pth obtained by subtracting the pth template multiplied by the pth complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal. The p-th virtual amplitude is determined so that the power of the residual signal is minimized, and Q templates in which the top Q virtual amplitudes are multiplied with respect to the magnitude of the obtained P virtual amplitudes are determined. Let Q arrival directions corresponding to be Q initial directions,
A residual signal obtained by subtracting the qth virtual amplitude from the transfer characteristic function determined corresponding to the qth initial direction, where q is an integer between 1 and Q, from the observed signal (hereinafter collectively) The q-th virtual amplitude is determined so that the power of the residual signal after removal) is minimized, and these Q virtual amplitudes are defined as Q initial amplitudes.
Next, the arrival direction correction process for correcting the arrival direction of the Q reflected sounds and the complex amplitude correction process for correcting the complex amplitude of the Q reflected sounds are alternately repeated a predetermined number of times, thereby performing Q The direction of arrival and complex amplitude corresponding to the reflected sound of
In the arrival direction correction process, q is an integer between 1 and Q, and in the first arrival direction correction process, the transfer characteristic function determined corresponding to the qth initial direction is multiplied by the qth initial amplitude. In the second and subsequent arrival direction correction processes, the q th complex amplitude obtained by the immediately preceding complex amplitude correction process is transferred to the transfer characteristic function determined corresponding to the q th arrival direction obtained by the immediately preceding arrival direction correction process. The qth direction of arrival is corrected so that the power of the signal obtained by multiplying by the residual signal after batch removal is reduced,
In the complex amplitude correction process, q is an integer of 1 or more and Q or less, and in the first complex amplitude correction process, the transfer characteristic function determined in correspondence with the qth arrival direction obtained in the immediately preceding arrival direction correction process is q In the second and subsequent complex amplitude correction processes, the complex amplitude correction process immediately before is added to the transfer characteristic function determined corresponding to the qth arrival direction obtained in the previous arrival direction correction process. The q-th complex amplitude is corrected so that the power of the signal obtained by multiplying the q-th reflected sound obtained in step 1 multiplied by the complex amplitude is added to the residual signal after collective removal , and obtained. An apparatus for estimating reflected sound information, wherein the complex amplitude of the Q reflected sounds is estimated as the arrival amplitude of each of the Q reflected sounds. The reflected sound information estimation apparatus according to claim 1,
The reflected sound information estimation unit
First, q is set to an integer of 1 to Q, and for each q in ascending order, a first process for obtaining a complex amplitude and a second process for obtaining an index p for identifying a template are performed, whereby Q initial directions are obtained. In the first process, p represents each index included in the set obtained by excluding the template index determined in the second process from the set of indexes of all templates. , The pth template multiplied by the pth complex amplitude (hereinafter referred to as virtual amplitude) from the qth minimum residual signal (however, the first minimum residual signal is the observed signal). The p-th virtual amplitude is determined so that the power of the q + 1-th residual signal obtained by subtraction is minimized, and in the second process, the maximum virtual amplitude among the virtual amplitudes obtained in the first process is determined. Identify the index of the width is multiplied template, the arrival direction that corresponds to the template by the q-th initial direction, to determine the Q-number of initial direction,
A residual signal obtained by subtracting the qth virtual amplitude from the transfer characteristic function determined corresponding to the qth initial direction, where q is an integer between 1 and Q, from the observed signal (hereinafter collectively) The q-th virtual amplitude is determined so that the power of the residual signal after removal) is minimized, and these Q virtual amplitudes are defined as Q initial amplitudes.
Next, the arrival direction correction process for correcting the arrival direction of the Q reflected sounds and the complex amplitude correction process for correcting the complex amplitude of the Q reflected sounds are alternately repeated a predetermined number of times, thereby performing Q The direction of arrival and complex amplitude corresponding to the reflected sound of
In the arrival direction correction process, q is an integer between 1 and Q, and in the first arrival direction correction process, the transfer characteristic function determined corresponding to the qth initial direction is multiplied by the qth initial amplitude. In the second and subsequent arrival direction correction processes, the q th complex amplitude obtained by the immediately preceding complex amplitude correction process is transferred to the transfer characteristic function determined corresponding to the q th arrival direction obtained by the immediately preceding arrival direction correction process. The qth direction of arrival is corrected so that the power of the signal obtained by multiplying by the residual signal after batch removal is reduced,
In the complex amplitude correction process, q is an integer of 1 or more and Q or less, and in the first complex amplitude correction process, the transfer characteristic function determined in correspondence with the qth arrival direction obtained in the immediately preceding arrival direction correction process is q In the second and subsequent complex amplitude correction processes, the complex amplitude correction process immediately before is added to the transfer characteristic function determined corresponding to the qth arrival direction obtained in the previous arrival direction correction process. The q-th complex amplitude is corrected so that the power of the signal obtained by multiplying the q-th reflected sound obtained in step 1 multiplied by the complex amplitude is added to the residual signal after collective removal , and obtained. An apparatus for estimating reflected sound information, wherein the complex amplitude of the Q reflected sounds is estimated as the arrival amplitude of each of the Q reflected sounds. The reflected sound information estimation apparatus according to claim 1,
The reflected sound information estimation unit
Determine a true subset Ψ composed of directions of arrival selected so that the directions of arrival corresponding to each of the templates are not biased and sparse, and set the concentration of the true subset Ψ to | Ψ |, Q ≦ | Ψ |, X represents each index p that identifies the template corresponding to the direction of arrival included in the true subset Ψ,
For each x, the x-th residual signal obtained by subtracting the x-th template multiplied by the x-th complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal is minimized so that the power of the x-th residual signal is minimized. Of the obtained virtual amplitudes, Q arrival directions corresponding to Q templates each multiplied by the top Q virtual amplitudes are defined as Q initial directions. ,
A residual signal obtained by subtracting the qth virtual amplitude from the transfer characteristic function determined corresponding to the qth initial direction, where q is an integer between 1 and Q, from the observed signal (hereinafter collectively) The q-th virtual amplitude is determined so that the power of the residual signal after removal) is minimized, and these Q virtual amplitudes are defined as Q initial amplitudes.
Next, the arrival direction correction process for correcting the arrival direction of the Q reflected sounds and the complex amplitude correction process for correcting the complex amplitude of the Q reflected sounds are alternately repeated a predetermined number of times, thereby performing Q The direction of arrival and complex amplitude corresponding to the reflected sound of
In the arrival direction correction process, q is an integer between 1 and Q, and in the first arrival direction correction process, the transfer characteristic function determined corresponding to the qth initial direction is multiplied by the qth initial amplitude. In the second and subsequent arrival direction correction processes, the q th complex amplitude obtained by the immediately preceding complex amplitude correction process is transferred to the transfer characteristic function determined corresponding to the q th arrival direction obtained by the immediately preceding arrival direction correction process. The qth direction of arrival is corrected so that the power of the signal obtained by multiplying by the residual signal after batch removal is reduced,
In the complex amplitude correction process, q is an integer of 1 or more and Q or less, and in the first complex amplitude correction process, the transfer characteristic function determined in correspondence with the qth arrival direction obtained in the immediately preceding arrival direction correction process is q In the second and subsequent complex amplitude correction processes, the complex amplitude correction process immediately before is added to the transfer characteristic function determined corresponding to the qth arrival direction obtained in the previous arrival direction correction process. The q-th complex amplitude is corrected so that the power of the signal obtained by multiplying the q-th reflected sound obtained in step 1 multiplied by the complex amplitude is added to the residual signal after collective removal , and obtained. An apparatus for estimating reflected sound information, wherein the complex amplitude of the Q reflected sounds is estimated as the arrival amplitude of each of the Q reflected sounds. In the reflected sound information estimation device according to any one of claims 2 to 7,
The reflected sound information estimation apparatus according to claim 1, wherein the power of the residual signal is a power obtained by adding over all the frequencies. In the reflected sound information estimation device according to any one of claims 1 to 8,
The arrival time difference estimator is
The reflected sound information estimation device, wherein the reference reflected sound common to the target reflected sounds is determined from the Q reflected sounds. In the reflected sound information estimation device according to any one of claims 1 to 8,
The arrival time difference estimator is
The reflected sound information estimation apparatus, wherein the reference reflected sound is determined from Q reflected sounds for each target reflected sound. In the storage unit, P is a predetermined integer of 2 or more, p is an integer of 1 to P, and transmission for each frequency between the p-th position and each position where M microphones are arranged. A function that simulates characteristics (hereinafter referred to as transfer characteristic function) is stored as a template for each p,
A signal (hereinafter referred to as an observation signal) obtained by collecting sound source signals with M microphones and converted into the frequency domain, and the template, are input.
The reflected sound information estimation unit sets Q to 2 or more predetermined integers smaller than P , q to 1 or more and Q or less, and sets the observed signal to Q directions (hereinafter, Q reflections coming from each of the incoming directions) are superimposed, and a template that can represent the qth reflected sound with the highest accuracy is obtained from the P templates, and the direction corresponding to the template is determined as a q-th direction of the Q direction, and the Q template obtained, based on the direction corresponding to the template, to estimate the Q-number of reflections each complex amplitude, resulting Q Reflected sound information estimation process in which the complex amplitude is the arrival amplitude of each of the Q reflected sounds ;
The arrival time difference estimator determines the deviation of the arrival amplitude of the target reflected sound with respect to the arrival amplitude of the reference reflected sound with respect to the reflected sound other than the reference reflected sound (hereinafter referred to as the reference reflected sound). An arrival time difference estimation process in which the arithmetic mean of the frequency relative to the angle divided by the frequency is the relative arrival time difference of the target reflected sound with respect to the reference reflected sound;
A method for estimating reflected sound information. In the reflected sound information estimation method according to claim 11,
In the reflected sound information estimation process,
(1) The p-th complex amplitude is set so that the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the p-th template by the p-th complex amplitude from the observed signal is minimized. Determine the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the determined p-th complex amplitude by the p-th template from the observed signal for each p, Determine Q templates that gave the least power,
(2) Based on the determined Q templates and the direction corresponding to each of the Q templates, the complex amplitude of each of the Q reflected sounds is estimated, and the obtained Q complex amplitudes Is a reflected sound information estimation method, wherein each of the Q reflected sounds is an arrival amplitude . The reflected sound information estimation method according to claim 12,
In the reflected sound information estimation process,
A reflected sound information estimation method, wherein the complex amplitude multiplied by the template determined in (1) is estimated as the arrival amplitude of the p-th reflected sound. The reflected sound information estimation method according to claim 12,
In the reflected sound information estimation process,
Observation of a product obtained by multiplying a template for each frequency between an arbitrary position in the space and each of the microphones by a complex amplitude in the vicinity of the direction D determined by the position corresponding to the template determined in (1) above. The arrival direction of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E obtained by subtracting from the signal is minimized, and the complex multiplied by the template corresponding to the arrival direction is estimated. A reflected sound information estimation method, wherein an amplitude is estimated as an arrival amplitude of the p-th reflected sound. In the reflected sound information estimation method according to claim 11,
In the reflected sound information estimation process,
First, p is an integer of 1 to P, and for each p, the pth obtained by subtracting the pth template multiplied by the pth complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal. The p-th virtual amplitude is determined so that the power of the residual signal is minimized, and Q templates in which the top Q virtual amplitudes are multiplied with respect to the magnitude of the obtained P virtual amplitudes are determined. Let Q arrival directions corresponding to be Q initial directions,
A residual signal obtained by subtracting the qth virtual amplitude from the transfer characteristic function determined corresponding to the qth initial direction, where q is an integer between 1 and Q, from the observed signal (hereinafter collectively) The q-th virtual amplitude is determined so that the power of the residual signal after removal) is minimized, and these Q virtual amplitudes are defined as Q initial amplitudes.
Next, the arrival direction correction process for correcting the arrival direction of the Q reflected sounds and the complex amplitude correction process for correcting the complex amplitude of the Q reflected sounds are alternately repeated a predetermined number of times, thereby performing Q The direction of arrival and complex amplitude corresponding to the reflected sound of
In the arrival direction correction process, q is an integer between 1 and Q, and in the first arrival direction correction process, the transfer characteristic function determined corresponding to the qth initial direction is multiplied by the qth initial amplitude. In the second and subsequent arrival direction correction processes, the q th complex amplitude obtained by the immediately preceding complex amplitude correction process is transferred to the transfer characteristic function determined corresponding to the q th arrival direction obtained by the immediately preceding arrival direction correction process. The qth direction of arrival is corrected so that the power of the signal obtained by multiplying by the residual signal after batch removal is reduced,
In the complex amplitude correction process, q is an integer of 1 or more and Q or less, and in the first complex amplitude correction process, the transfer characteristic function determined in correspondence with the qth arrival direction obtained in the immediately preceding arrival direction correction process is q In the second and subsequent complex amplitude correction processes, the complex amplitude correction process immediately before is added to the transfer characteristic function determined corresponding to the qth arrival direction obtained in the previous arrival direction correction process. The q-th complex amplitude is corrected so that the power of the signal obtained by multiplying the q-th reflected sound obtained in step 1 multiplied by the complex amplitude is added to the residual signal after collective removal , and obtained. A reflected sound information estimation method, wherein the complex amplitude of the Q reflected sounds is estimated as the arrival amplitude of each of the Q reflected sounds. In the reflected sound information estimation method according to claim 11,
In the reflected sound information estimation process,
First, q is set to an integer between 1 and Q, and for each q in ascending order of q, a first process for obtaining a complex amplitude and a second process for obtaining an index for identifying a template are performed. In the first process, p represents each index included in the set obtained by excluding the template index determined in the second process from the set of all template indexes, and for each p, The pth template multiplied by the pth complex amplitude (hereinafter referred to as virtual amplitude) is subtracted from the qth minimum residual signal (however, the first minimum residual signal is referred to as the observed signal). The p-th virtual amplitude is determined so that the power of the q + 1-th residual signal obtained in this way is minimized. In the second process, the largest virtual amplitude among the virtual amplitudes obtained in the first process is determined. Identify the index of was multiplied template, the arrival direction that corresponds to the template by the q-th initial direction, to determine the Q-number of initial direction,
A residual signal obtained by subtracting the qth virtual amplitude from the transfer characteristic function determined corresponding to the qth initial direction, where q is an integer between 1 and Q, from the observed signal (hereinafter collectively) The q-th virtual amplitude is determined so that the power of the residual signal after removal) is minimized, and these Q virtual amplitudes are defined as Q initial amplitudes.
Next, the arrival direction correction process for correcting the arrival direction of the Q reflected sounds and the complex amplitude correction process for correcting the complex amplitude of the Q reflected sounds are alternately repeated a predetermined number of times, thereby performing Q The direction of arrival and complex amplitude corresponding to the reflected sound of
In the arrival direction correction process, q is an integer between 1 and Q, and in the first arrival direction correction process, the transfer characteristic function determined corresponding to the qth initial direction is multiplied by the qth initial amplitude. In the second and subsequent arrival direction correction processes, the q th complex amplitude obtained by the immediately preceding complex amplitude correction process is transferred to the transfer characteristic function determined corresponding to the q th arrival direction obtained by the immediately preceding arrival direction correction process. The qth direction of arrival is corrected so that the power of the signal obtained by multiplying by the residual signal after batch removal is reduced,
In the complex amplitude correction process, q is an integer of 1 or more and Q or less, and in the first complex amplitude correction process, the transfer characteristic function determined in correspondence with the qth arrival direction obtained in the immediately preceding arrival direction correction process is q In the second and subsequent complex amplitude correction processes, the complex amplitude correction process immediately before is added to the transfer characteristic function determined corresponding to the qth arrival direction obtained in the previous arrival direction correction process. The q-th complex amplitude is corrected so that the power of the signal obtained by multiplying the q-th reflected sound obtained in step 1 multiplied by the complex amplitude is added to the residual signal after collective removal , and obtained. A reflected sound information estimation method, wherein the complex amplitude of the Q reflected sounds is estimated as the arrival amplitude of each of the Q reflected sounds. In the reflected sound information estimation method according to claim 11,
In the reflected sound information estimation process,
Determine a true subset Ψ composed of directions of arrival selected so that the directions of arrival corresponding to each of the templates are not biased and sparse, and set the concentration of the true subset Ψ to | Ψ |, Q ≦ | Ψ |, X represents each index identifying a template corresponding to the direction of arrival included in the true subset Ψ,
For each x, the x-th residual signal obtained by subtracting the x-th template multiplied by the x-th complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal is minimized so that the power of the x-th residual signal is minimized. Of the obtained virtual amplitudes, Q arrival directions corresponding to Q templates each multiplied by the top Q virtual amplitudes are defined as Q initial directions. ,
A residual signal obtained by subtracting the qth virtual amplitude from the transfer characteristic function determined corresponding to the qth initial direction, where q is an integer between 1 and Q, from the observed signal (hereinafter collectively) The q-th virtual amplitude is determined so that the power of the residual signal after removal) is minimized, and these Q virtual amplitudes are defined as Q initial amplitudes.
Next, the arrival direction correction process for correcting the arrival direction of the Q reflected sounds and the complex amplitude correction process for correcting the complex amplitude of the Q reflected sounds are alternately repeated a predetermined number of times, thereby performing Q The direction of arrival and complex amplitude corresponding to the reflected sound of
In the arrival direction correction process, q is an integer between 1 and Q, and in the first arrival direction correction process, the transfer characteristic function determined corresponding to the qth initial direction is multiplied by the qth initial amplitude. In the second and subsequent arrival direction correction processes, the q th complex amplitude obtained by the immediately preceding complex amplitude correction process is transferred to the transfer characteristic function determined corresponding to the q th arrival direction obtained by the immediately preceding arrival direction correction process. The qth direction of arrival is corrected so that the power of the signal obtained by multiplying by the residual signal after batch removal is reduced,
In the complex amplitude correction process, q is an integer of 1 or more and Q or less, and in the first complex amplitude correction process, the transfer characteristic function determined in correspondence with the qth arrival direction obtained in the immediately preceding arrival direction correction process is q In the second and subsequent complex amplitude correction processes, the complex amplitude correction process immediately before is added to the transfer characteristic function determined corresponding to the qth arrival direction obtained in the previous arrival direction correction process. The q-th complex amplitude is corrected so that the power of the signal obtained by multiplying the q-th reflected sound obtained in step 1 multiplied by the complex amplitude is added to the residual signal after collective removal , and obtained. A reflected sound information estimation method, wherein the complex amplitude of the Q reflected sounds is estimated as the arrival amplitude of each of the Q reflected sounds. In the reflection sound information estimating method according to claims 1 2 in any one of claims 17,
The reflected sound information estimation method, wherein the power of the residual signal is a power obtained by adding all of the frequencies. The reflected sound information estimation method according to any one of claims 11 to 18,
In the above arrival time difference estimation process,
A method for estimating reflected sound information, wherein the reference reflected sound common to each of the target reflected sounds is determined from the Q reflected sounds. The reflected sound information estimation method according to any one of claims 11 to 18,
In the above arrival time difference estimation process,
A reflected sound information estimation method, wherein the reference reflected sound is determined from Q reflected sounds for each target reflected sound.       A program for causing a computer to execute processing of the reflected sound information estimation method according to any one of claims 11 to 20.