JP5486568B2 - Audio spot reproduction processing method, apparatus, and program - Google Patents

Description

  The present invention relates to a signal processing technique (audio spot reproduction processing technique) for reproducing audio in a narrow area (spot) including a desired position.

  As a situation of sound reproduction using a speaker, there is a situation where it is desired to reproduce sound at a sufficient volume in a specific direction. For example, when playing audio explaining an exhibit in a limited area in front of an exhibit in an exhibition hall, or in a limited area such as the edge of a station platform or near a staircase Is played. Such signal processing technology (narrow-directed speech reproduction processing technology) for reproducing sound in a narrow range including a desired direction (target direction) as viewed from the speaker has been researched and developed conventionally. The relationship (sound pressure distribution) between the surroundings of the speaker and the sound pressure of the sound emitted from the speaker is called directivity. The sharper the directivity in a certain direction, the more the sound is transmitted to a narrow range including that direction. It is possible to reproduce and suppress the sound pressure of the sound in a range other than the range. Here, three conventional techniques related to the narrow-directional sound reproduction processing technique are illustrated. In this specification, “speech” is not limited to a voice uttered by a person, but refers to a general “sound” such as a musical sound or an environmental noise as well as a voice of a person or an animal.

[1] Narrow-directional audio reproduction processing technology using physical characteristics Representative examples of this category include horn speakers and parabolic speakers. The horn speaker is, for example, a speaker in which a saddle-shaped horn whose cross-sectional area gradually increases toward the opening end in front of the speaker. The longer the horn length, the sharper the directivity of the horn speaker. A parabolic speaker has a configuration in which a speaker is arranged at the focal point of a parabolic plate (paraboloid), and radiates sound from the speaker toward the parabolic plate, thereby connecting the top of the parabolic plate and the focal point of the parabolic plate. Sound is transmitted in the direction of.

[2] Narrow-directional sound reproduction processing technology using ultrasonic waves A typical example of this category is a parametric speaker (see, for example, Patent Document 1). The parametric speaker radiates a modulated wave obtained by amplitude-modulating an ultrasonic wave with a sound source signal with a high sound pressure, using an ultrasonic wave having a strong straightness as a carrier wave. A distortion component is generated by the non-linear characteristic of air in the process in which the modulated wave propagates in the air, and sound in an audible band appears due to the distortion component and human auditory characteristics.

[3] Narrow-directional audio reproduction processing technology using signal processing A typical example of this category is a phased speaker array (see, for example, Non-Patent Document 1). A phased speaker array is a speaker array composed of a plurality of speakers, and signals obtained by performing signal processing to superimpose a sound source signal by applying a filter containing information on time difference or level difference to each speaker. And the sound is spatially radiated, and as a result, the sound is reproduced in the target direction.

  By the way, if it is possible to perform spot reproduction that conveys sound not only to a certain direction but also to a listener who is in a specific narrow area, the reproduced sound will be transmitted to people other than the listener in the case of communication using voice amplification. In addition to noise, the privacy protection that communication contents are difficult to leak to the surroundings is further enhanced.

  As a signal processing technique (audio spot reproduction processing technique) for reproducing audio in such a narrow area (spot) including a desired position, when the sound pressure on the boundary of a specific area is controlled, the outside of the area There is a conventional technique for realizing spot reproduction by controlling the sound pressure on the boundary to be zero based on the boundary sound field control theory that the sound field of the sound field can also be controlled (see Patent Document 2).

JP 2010-258938 A JP 2006-74442 A

Yoichi Haneda, Akitoshi Kataoka, “Real-space performance of small speaker array based on multipoint control using free space transfer function”, Acoustical Society of Japan Spring Proceedings, pp.631-632, 2008.

On the other hand, the conventional technology has one or more of the following problems (1) to (5), on the other hand.
(1) Sound cannot be reproduced in the target direction unless the speaker itself is directed in the target direction. That is, when the target direction can be changed, if it does not depend on the human physical activity, a drive control means for changing the direction of the horn speaker or the parabolic speaker itself is required.
(2) Since the directivity relates to the width of the spot, it is desired to realize the narrow directivity if audio is to be reproduced in a spot as narrow as possible. However, it is difficult to realize narrow directivity with a prospective angle of about 5 ° to 10 ° (sharp directivity of about ± 5 ° to ± 10 ° with respect to the target direction).
(3) In order to realize narrow directivity, it is necessary to increase the number of speakers and increase the array size (total length of the array). This is disadvantageous in terms of cost and wiring complexity.
(4) In order to form a sound field for spot reproduction, it is necessary to arrange speakers in various places. This is disadvantageous from the viewpoints of space restriction and cost for installing the speaker array.
(5) It is difficult to reproduce the sound with a high SN ratio in the target direction so as not to be buried in the sound in a direction other than the target direction.

  In view of such a current situation, the present invention reproduces sound with a sufficient signal-to-noise ratio, and can reproduce sound in any direction without requiring physical movement of the speaker, but at a desired position. An object of the present invention is to provide an audio spot reproduction processing technology capable of reproducing audio in a narrow area (spot) including the above.

Transmission characteristics of audio from each speaker to each position (where i is the direction for identifying each position and g is the distance) included in one or more positions assumed as audio spot playback positions Using a _{i, g} , a filter for each frequency is obtained for a position to be reproduced by sound [filter design processing]. M is an integer greater than or equal to 2, and M speakers constitute a speaker array. The sound radiated from the speaker array and having the listening position in the traveling direction of the reflected sound reflected by the reflector is a dual sound, and each transfer characteristic a _{i, g} is expressed by the direction i and the distance g. It is expressed as the sum of the transfer characteristics of a direct sound to a fixed position and the transfer characteristics of one or more dual sounds. The filter converts the frequency domain signal S obtained by converting the sound source signal into the frequency domain for each frequency into an M channel frequency domain signal X. The filter obtained by the filter design process is applied to the frequency domain signal S for each frequency to obtain an M channel frequency domain signal X [filter application process]. The M channel time domain signal x obtained by converting the M channel frequency domain signal X into the time domain is normally reproduced by a speaker array.

Each transfer characteristic a _{i, g} is, as a specific example, a sum of a direct sound steering vector and one or more dual sound steering vectors in which a time difference of the sound due to reflection and the direct sound of the reflected sound is corrected. Alternatively, it may be obtained by actual measurement in an actual environment.

  In the filter design process, a filter may be obtained for each frequency so that the power of the sound to a position other than the position where the sound is to be reproduced is minimized. Or you may obtain | require a filter for every frequency so that the SN ratio of the audio | voice in the position used as the audio | voice reproduction | regeneration object may become the maximum. Alternatively, for each frequency, the sound power to other than one or a plurality of positions assumed as the spot reproduction position is minimized with the filter coefficient for one speaker among M speakers fixed at a constant value. You may ask for a filter.

Alternatively, in the filter design process, the sound reproduction is performed under the conditions of (1) passing the entire band of the sound to a position to be reproduced and (2) suppressing the entire band of the sound to one or more suppression points. A filter may be obtained for each frequency so that the power of the sound to the position other than the target position and each suppression point is minimized. Alternatively, a filter may be obtained for each frequency by normalizing the transfer characteristics a _{s, h} at the positions i = s, g = h that are to be reproduced. Or you may obtain | require a filter for every frequency using the spatial correlation matrix represented by the transfer characteristic _{ai, g} corresponding to each position other than the position used as the object of audio | voice reproduction | regeneration. Alternatively, a filter may be applied for each frequency so that the power of the sound to a position other than the target position for sound reproduction is minimized under the condition that the deterioration amount of the sound at the position for sound playback is a predetermined amount or less. You may ask for it. Or you may obtain | require a filter for every frequency using the spatial correlation matrix represented by the frequency domain signal obtained by converting the signal obtained by observing with a microphone array into a frequency domain. Alternatively, a filter may be obtained for each frequency using a spatial correlation matrix represented by transfer characteristics a _{i, g} corresponding to each position included in one or a plurality of positions assumed as spot reproduction positions.

According to the present invention, not only the direct sound in the sound reproduction target direction but also the reflected sound is used, so that the reproduction can be performed with a sufficiently large SN ratio in the direction and the sound reproduction in the direction by signal processing. Therefore, the sound can be reproduced in any direction without requiring physical movement of the speaker. Further, as will be described in detail in the item “Principle”, which will be described later, each transfer characteristic a _{i, g} is divided into a direct sound transfer characteristic and a transmission of one or more dual sounds to a position determined by the direction i and the distance g. By expressing with the sum of the characteristics, it is possible to design a filter that enhances the degree of coherence suppression that determines the directionality of the direction of sound playback when designing a filter based on general filter design criteria. . That is, it has a sharper directivity than the conventional one with respect to the direction of audio reproduction. Furthermore, as will be described in detail in the item “Principle” described later, by using reflected sound, transmission to a listening position in the same direction but a different distance when viewed from the speaker array is transmitted corresponding to a different listening position. A significant difference will occur between the characteristics. By reproducing the difference between the transfer characteristics by the beam forming method, the sound can be spot-reproduced according to the distance from the speaker array.

(A) A diagram schematically showing that narrow directivity cannot be sufficiently realized when only direct sound is considered, and (b) schematically showing that narrow directivity can be sufficiently realized when direct sound and reflected sound are considered. FIG. The figure which shows the direction dependence of the coherence in the case where it is based on the case of a prior art, and the principle of this invention. FIG. 3 is a diagram illustrating a functional configuration of the audio spot reproduction processing device according to the first embodiment. FIG. 3 is a diagram illustrating a processing procedure of an audio spot reproduction processing method according to the first embodiment. The figure which shows the structure of a 1st Example. (A) The figure which shows a mode that a sound reaches directly to two listening positions A and B from a speaker array. (B) A diagram showing a state in which sound directly reaches the two listening positions A and B from the speaker array and a state in which dual sound reaches the two virtual listening positions A (ξ) and B (ξ) from the speaker array by the reflector. . (A) Directivity (two-dimensional region) of the minimum dispersion beam former when no reflector is installed. (B) Directivity (two-dimensional region) of the minimum dispersion beamformer when a reflector is installed. The figure which shows the implementation structural example of this invention. (A) Top view. (B) Front view. (C) Side view. (A) The side view which shows another implementation structural example of this invention. (B) The side view which shows another implementation structural example of this invention. The figure which shows the usage pattern in the implementation structural example shown in FIG.9 (b). The figure which shows the implementation structural example of this invention. (A) Top view. (B) Front view. (C) Side view. The side view which shows the implementation structural example of this invention. The figure which shows the function structure of the audio | voice spot reproduction | regeneration processing apparatus which concerns on Embodiment 2. FIG. The figure which shows the process sequence of the audio | voice spot reproduction | regeneration processing method which concerns on Embodiment 2. FIG.

"principle"
The principle of the present invention will be described. The present invention is based on the essence of speaker array technology that can reproduce sound in an arbitrary direction based on signal processing, and on reproducing sound at a high S / N ratio by actively using reflected sound. On the other hand, it is characterized by combining signal processing technologies that enable sharp directivity. In particular, in view of the fact that the transmission characteristics between a speaker array and a listening position that are substantially the same in the direction viewed from the speaker array but differing in distance are quite similar, by using a reflector, different listening positions can be obtained. It can be said that one of the salient features of the present invention is to increase the difference in transfer characteristics between the speaker array and the speaker array. By reproducing the transfer characteristic difference by signal processing, an audio spot reproduction processing technique capable of spot reproduction of audio according to the distance from the speaker array is realized.

Since signal processing in the frequency domain will be mainly described, symbols are defined prior to description. The index of the discrete frequency is ω (there is a relationship of ω = 2πf between the frequency f and the angular frequency ω, so the index ω of the discrete frequency may be identified with the angular frequency ω. The frequency index "is also simply referred to as" frequency "), and the frame number index is k. The frequency domain representation of the k-th frame of the sound source signal of one channel is S (ω, k), the position θ _{s when} viewed from the center of the speaker array and the position separated by the distance D _{h when} viewed from the center of the speaker array (listening position ( A filter designed as a spot reproduction position) that converts a frequency domain signal S (ω, k) of a sound source signal into an M channel frequency domain signal at a frequency ω is W ^→ (ω, θ _s , D _h ), an M channel frequency domain signal (hereinafter referred to as a reproduction signal) obtained by applying the filter W ^→ (ω, θ _s , D _h ) to the frequency domain signal S (ω, k) of the sound source signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)]. M is an integer of 2 or more. Hereinafter, the distance D _h is fixed for a while.

The “center of the speaker array” can be arbitrarily determined. In general, the geometric center of the arrangement of the M speakers is set as the “center of the speaker array”. For example, a linear speaker array (M speakers are In the case of a speaker array arranged in a straight line), the midpoint of the speakers at both ends is set as the “center of the speaker array”. For example, a flat speaker array arranged in a square matrix of m × m (m ² = M) If there is, the position where the diagonal lines of the four corners of the speaker intersect is defined as the “center of the speaker array”.

Under such circumstances, the reproduction signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)] of the k-th frame is given by equation (1). H represents Hermitian transpose. Note that the reproduction signal X of the k-th frame ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)] is converted into an M channel time domain signal. Each time domain signal is reproduced by a speaker corresponding to each channel (details will be described later). The number of speakers is M.

There are various design methods for the filter W ^→ (ω, θ _s , D _h ), but here, a case based on the minimum variance distortion response method (MVDR method) will be described. In the minimum variance distortion-free response method, the filter W ^→ (ω, θ _s , D _h ) is moved in a direction other than the direction θ _s using the spatial correlation matrix Q (ω) under the constraint condition of Equation (3). The power of the voice (hereinafter, “sound in a direction other than the direction θ _s ” is also referred to as “leakage voice”) is designed to be a minimum at the frequency ω (see Expression (2), where the distance D _h is Note that the spatial correlation matrix Q (ω) is specified as Q (ω, D _h ) because it is fixed. a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ), ..., a _M (ω, θ _s , D _h )] ^T is viewed from the center of the speaker array ( Suppose that there is a listening position at a position in the direction θ _s , distance D _h ) (hereinafter referred to as “position (θ _s , D _h )” unless otherwise specified), the listening position and the M speakers It is a transfer characteristic at the frequency ω. T represents transposition. In other words, a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ), ..., a _M (ω, θ _s , D _h )] ^T is included in the speaker array Is a transfer characteristic at a frequency ω of sound from each speaker to a position (θ _s , D _h ). The spatial correlation matrix Q (ω) is a collected sound signal obtained by observing with a microphone array composed of M microphones (preferably a microphone array in which the speakers included in the speaker array are replaced with microphones). Can be expressed using a frequency domain signal obtained by converting the signal to the frequency domain, but can also be expressed using transfer characteristics. Hereinafter, the case where the spatial correlation matrix Q (ω) is expressed using transfer characteristics for a while will be described.

It is known that the filter W ^→ (ω, θ _s , D _h ), which is the optimum solution of the equation (2), is given by the equation (4).
(Reference 1) by Simon Haykin, translated by Hiroshi Suzuki et al., "Adaptive Filter Theory", First Edition, Science and Technology Publishing Co., Ltd., 2001. pp.66-73,248-255

As can be seen from the fact that the inverse matrix of the spatial correlation matrix Q (ω, D _h ) is included in equation (4), the structure of the spatial correlation matrix Q (ω, D _h ) is important for realizing sharp directivity. It can be seen that it is. It can also be seen from equation (2) that the power of leaked speech depends on the structure of the spatial correlation matrix Q (ω, D _h ).

A set to which the index p in the traveling direction (propagation direction) of the leaked speech belongs is denoted as {1, 2,..., P-1}. It is assumed that the index s in the direction θ _s does not belong to the set {1, 2,..., P-1}. At this time, the spatial correlation matrix Q (ω, D _h ) is given by equation (5a). From the viewpoint of making a filter that realizes narrow directivity, P is preferably a somewhat large value, but it is assumed that P is an integer that satisfies P ≦ M. Here, from the viewpoint of easily explaining the principle of the invention, the direction θ _s is described as a specific direction (therefore, directions other than the direction θ _s are set as directions of “leakage sound”). As will be apparent from the embodiments described later, in practice, the direction θ _s is an arbitrary direction that can be a target of audio reproduction, and therefore, a plurality of directions are generally assumed as directions that can be the direction θ _s. The From this point of view, the distinction between the direction θ _s and the direction of the leaked voice is almost subjective, and there are P directions that can be assumed as the voice traveling directions without distinction between the reproduced voice and the leaked voice. Different directions are determined in advance, and one of the P directions selected is a direction corresponding to a position (listening position) to be reproduced, and the other directions are directions of leaked sound. It is more accurate to understand that there is. Therefore, assuming that the union of the set {1, 2,..., P-1} and the set {s} is Φ, the spatial correlation matrix Q (ω, D _h ) has a distance from the center of the speaker array of D _h And the transmission characteristics of the sound from each speaker to each position corresponding to each direction θ _φ included in a plurality of directions assumed as the traveling direction of the sound a ^→ (ω, θ _φ , D _h ) = [a ₁ (ω, θ _φ , D _h ),..., A _M (ω, θ _φ , D _h )] ^T (φ∈Φ), which is expressed by equation (5b). Note that | Φ | = P. | Φ | represents the number of elements of the set Φ.

Here, transfer characteristics of the sound direction ^{_{θ s a → (ω, θ}} s, D h) and, the direction p∈ {1,2, ..., P- 1} transfer characteristic of the speech to a ^→ (omega, θ _p , D _h ) = [a ₁ (ω, θ _p , D _h ),..., a _M (ω, θ _p , D _h )] Assume that ^T are orthogonal to each other. That is, it is assumed that there are P orthogonal basis sets that satisfy the condition expressed by Equation (6). The symbol ⊥ represents orthogonality. If A ^→ ⊥B ^→ , the inner product value of vector A ^→ and vector B ^→ is zero. Here, P ≦ M is satisfied. In addition, when the condition represented by Expression (11) is relaxed and it can be assumed that there are P basis sets that can be regarded as approximately orthogonal basis sets, P is about M, or is somewhat larger than M It is preferably a value.

At this time, the spatial correlation matrix Q (ω, D _h ) can be expanded as shown in Equation (7). Equation (7) is a matrix V (ω, D _h ) = [a ^→ (ω, θ _s , D _h ), a ^→ (ω, θ ₁ , D) composed of P transfer characteristics satisfying orthogonality. _h ), ..., a ^→ (ω, θ _P-1 , D _h )] ^T and the unit matrix Λ (ω, D _h ) mean that the spatial correlation matrix Q (ω, D _h ) can be decomposed. ρ is an eigenvalue and real number of the transfer characteristic a ^→ (ω, θ _φ , D _h ) satisfying the equation (6) by the spatial correlation matrix Q (ω, D _h ).

At this time, the inverse matrix of the spatial correlation matrix Q (ω, D _h ) is given by equation (8).

Substituting equation (8) into equation (2) reveals that the power of leaked speech is minimized. If the power of the leaked voice is minimized, the directivity with respect to the direction θ _s is realized. Therefore, the fact that orthogonality is established between transfer characteristics in different directions is an important condition for realizing directivity in the direction θ _s .

Hereinafter, the reason why it is difficult to realize a sharp directivity with respect to the direction θ _s in the prior art will be considered.

In the prior art, the filter is designed on the assumption that the transfer characteristic is composed only of direct sound. In reality, the sound emitted from the speaker is reflected by walls, ceilings, and the like, so there is a reflected sound. However, the reflected sound is considered to be a factor that deteriorates the directivity, and the presence of the reflected sound is ignored. Assuming that sound propagates as a plane wave from a linear speaker array, the steering vector of only direct sound in the direction θ is expressed as h ^→ _d (ω, θ) = [h _d1 (ω, θ),…, h _dM (ω, θ )] If ^T , conventionally, transfer characteristic a ^→ _conv (ω, θ) = [a ₁ (ω, θ), ..., a _M (ω, θ)] ^{T is changed} to a ^→ _conv (ω, θ) = h ^→ _d (ω, θ) (The steering vector does not depend on the distance D because the sound wave is considered as a plane wave). The steering vector is a complex vector in which the phase response characteristics at the frequency ω of each speaker with respect to the reference point are arranged for sound waves in the direction θ as viewed from the center of the speaker array.

Hereinafter, it is assumed that the sound is radiated as a plane wave from the linear speaker array for a while. Assume that the m-th element h _dm (ω, θ) constituting the direct sound steering vector h ^→ _d (ω, θ) is given by, for example, Expression (9c). u represents the distance between adjacent speakers. j is an imaginary unit. In this case, the reference point is a position half the total length of the linear speaker array (the center of the linear speaker array). The direction θ is defined as an angle formed by the direct sound traveling direction and the arrangement direction of the speakers included in the linear speaker array as seen from the center of the linear speaker array (see FIG. 5). There are various ways to express the steering vector. For example, if the reference point is the position of the speaker at one end of the linear speaker array, the mth element constituting the steering vector h ^→ _d (ω, θ) of the direct sound. h _dm (ω, θ) is given by, for example, equation (9d). In the following description, it is assumed that the m-th element h _dm (ω, θ) constituting the direct sound steering vector h ^→ _d (ω, θ) is given by equation (9c).

The inner product value γ _conv (ω, θ) of the transfer characteristic in the direction θ and the transfer characteristic in the direction θ _s is expressed by Expression (10). Note that θ ≠ θ _s .

Hereinafter, γ _conv (ω, θ) is referred to as coherence. The direction θ in which the coherence γ _conv (ω, θ) becomes 0 is given by the equation (11). q is an arbitrary integer other than 0. In addition, since 0 <θ <π / 2, the range of q is limited for each frequency band.

In Formula (11), the only parameters that can be changed are the parameters (M and u) related to the size of the speaker array. Therefore, when the direction difference (angle difference) | θ−θ _s | is small, the speaker array It is difficult to reduce the coherence γ _conv (ω, θ) without changing the parameters related to the size of. In this case, the power of the leaked voice is not sufficiently reduced, and the directivity having a wide beam width with respect to the direction θ _{s is} obtained as schematically shown in FIG.

On the other hand, according to the present invention, based on such consideration, the filter design for having a sharp directivity with respect to the direction θ _s has a coherence even when the direction difference (angle difference) | θ−θ _s | is small. Unlike the prior art, based on the knowledge that it is important to make it sufficiently small, it is characterized by positively considering reflected sound.

  Here, “dual sound” is defined. (1) The sound emitted from the speaker array, and (2) the sound that satisfies the condition that the listening position exists in the traveling direction of the reflected sound reflected by the reflector, "

Assuming that the sound wave is a plane wave, the sound from each speaker of the speaker array that travels without being reflected in any direction (direct sound) and the reflection in which the dual sound is reflected by the reflector 300 are assumed in a certain direction θ. Two kinds of plane waves with sound will head. Let the number of reflected sounds (or dual sounds) be Ξ. Ξ is a predetermined integer of 1 or more. At this time, the transfer characteristic a ^→ (ω, θ) = [a ₁ (ω, θ),..., A _M (ω, θ)] ^T is the direct sound transfer characteristic from the speaker array to the direction θ and the direct sound The sum of the transfer characteristics of one or more dual sounds corresponding to the sound, specifically, the time difference between the direct sound and the ξ-th (1 ≦ ξ ≦ Ξ) reflected sound is τ _ξ (θ), and α _{When ξ} (1 ≦ ξ ≦ Ξ) is a coefficient for considering the sound attenuation due to reflection, the direct sound steering vector, the sound attenuation due to reflection, and the direct sound of the reflected sound are expressed as in Expression (12a). It can be expressed as the sum of steering vectors of two dual tones with corrected time differences. h ^→ _rξ (ω, θ) = [h _r1ξ (ω, θ),..., h _rMξ (ω, θ)] ^T represents the steering vector of the dual sound corresponding to the direct sound in the direction θ. α _ξ (1 ≦ ξ ≦ Ξ) is usually α _ξ ≦ 1 (1 ≦ ξ ≦ Ξ). For each reflected sound, if the number of times that the sound (dual sound) from the speaker array is reflected by the reflecting object is 1, α _ξ (1 ≦ ξ ≦ Ξ) is the number of objects reflected by the ξth dual sound. You can think of it as representing the reflectance of sound.

Since it is desired that one or more reflected sounds exist for a speaker array composed of M speakers, it is preferable that one or more reflectors exist. From this point of view, assuming that there is a listening position in the target direction, the positional relationship between the listening position, the speaker array, and one or more reflectors is that the sound (dual sound) from the speaker array reflects at least one reflection. It is preferable that each reflector is arranged so as to be reflected by the object and reach the listening position. Each reflector has a two-dimensional shape (for example, a flat plate) or a three-dimensional shape (for example, a parabolic shape). Moreover, it is preferable that the size of each reflector is equal to or larger than the speaker array (about 1 to 2 times). In order to effectively use the reflected sound, the reflectance α _ξ (1 ≦ ξ ≦ Ξ) of each reflector is at least greater than 0, and more specifically, the amplitude of the reflected sound reaching the listening position is the direct sound. It is desirable that the amplitude is, for example, 0.2 times or more. For example, each reflector is a rigid solid. The reflecting object may be a movable object (for example, a reflector) or an immovable object (a floor, a wall, or a ceiling). If an immovable object is set as a reflection object, it is necessary to change the steering vector of the dual sound as the installation position of the speaker array is changed (functions Ψ (θ) and Ψ _ξ (θ) described later are changed). (Refer to the above), and the filter calculation must be redone (reset). Therefore, in order to be robust against environmental changes, it is preferable that each reflector is a follower of a speaker array (in this case, it is assumed that the estimated number of reflected sounds is due to each reflector. become). Here, “speaker array subordinate” refers to “a tangible object that can follow changes in the position and orientation of the speaker array while maintaining the positional relationship (geometric relationship) with respect to the speaker array”. A simple example is a configuration in which each reflector is fixed to a speaker array.

Hereinafter, from the viewpoint of specifically explaining the advantages of the present invention, Ξ = 1, the number of reflections of dual sounds is one, and there is one reflector at a position L meters away from the center of the speaker array. Assume. The reflector is a thick rigid body. In this case, since Ξ = 1, the subscript representing this is omitted, and the expression (12a) can be expressed as the expression (12b).

Dual sound steering vector h ^→ _r (ω, θ) = [h _r1 (ω, θ),…, h _rM (ω, θ)] The m-th element of ^T Similarly (refer to Formula (9c)), it is expressed by Formula (13a). The function Ψ (θ) outputs the traveling direction of the dual sound viewed from the center of the speaker array. When the direct sound steering vector is expressed by equation (9d), the dual sound steering vector h ^→ _r (ω, θ) = [h _r1 (ω, θ),..., H _rM (ω, θ) ] The m-th element of ^T is expressed by Expression (13b). If Ξ ≧ 2, the ξ-th (1 ≦ ξ ≦ Ξ) steering vector h ^→ _rξ (ω, θ) = [h _r1ξ (ω, θ),..., H _rMξ (ω, θ)] ^T The m-th element is expressed by Expression (13c) or Expression (13d). The function Ψ _ξ (θ) outputs the traveling direction of the ξ-th (1 ≦ ξ ≦ Ξ) dual sound.

  Since the position of the reflector can be set as appropriate, the traveling direction of the dual sound can be treated as a variable parameter.

Assuming that a flat reflector is in the vicinity of the speaker array (distance L is not extremely large compared to the size of the speaker array), coherence γ (ω, θ) is expressed by equation (14). Note that θ ≠ θ _s .

From equation (14), it can be seen that the coherence γ (ω, θ) of equation (14) may be smaller than the conventional coherence γ _conv (ω, θ) of equation (11). There are parameters (Ψ (θ) and L) that can be changed depending on how the reflector is placed in the second to fourth items of equation (14), so the first item h ^→ _d ^H (ω, θ) h ^→ _d (ω, θ) may be removed.

For example, for a linear speaker array, when a flat reflector is arranged so that the speaker arrangement direction is the normal of the reflector, Ψ (θ) = π-θ holds for the function Ψ (θ), and Since the equation (15) is established for the time difference τ (θ) between the sound and the reflected sound, the conditions of the equations (16) and (17) are generated for the elements constituting the equation (14). The symbol * is an operator representing a complex conjugate.

_{^{h → d H (ω, θ}} ) the absolute value of h ^→ _r (ω, θ) is _{^{h → d H (ω, θ}} ) h → d (ω, θ) sufficiently smaller than, the formula (14) If the second and third terms are ignored, the coherence γ (ω, θ) can be approximated as shown in equation (18).

If _{^{h → d H (ω, θ}} ) h → d (ω, θ) even as ≠ 0, approximate coherence γ ~ (ω, θ) has a local minimum θ of equation (19). q is an arbitrary positive integer. Further, the range of q is limited for each frequency band.

  That is, the coherence can be suppressed not only in the direction given by Expression (11) but also in the direction given by Expression (19). If the coherence can be suppressed, the power of the leaked voice can be reduced, so that a sharp directivity can be realized as schematically shown in FIG.

FIG. 1 schematically shows the difference in directivity between the case of using the principle of the present invention and the case of using the prior art. In FIG. 2, θ given by Expression (11) and Expression (19) are given. The difference in θ obtained will be specifically shown. ω = 2π × 1000 [rad / s], L = 0.70 [m], θ _s = π / 4 [rad]. FIG. 2 shows the direction dependency of the normalized coherence for comparison between the two. The direction indicated by the symbol ○ is θ given by the equation (11), and the direction indicated by the symbol + is It is (theta) given by Formula (19). As is apparent from FIG. 2, according to the prior art, the coherence becomes zero with respect to θ _s = π / 4 [rad] only in the direction indicated by the symbol ○, but according to the principle of the present invention. And θ _s = π / 4 [rad] and coherence is zero in many directions indicated by the symbol +, and in particular, θ _s = π / 4 [ Since the direction indicated by the symbol + exists in a direction much closer to rad], it can be understood that sharp directivity is realized as compared with the prior art.

So far, it has been assumed that the sound wave travels as a plane wave from the viewpoint of explaining the principle of the present invention in an easy-to-understand manner. However, as is clear from the above description, the main feature of the present invention is that the transfer characteristic a ^→ (ω, θ, D) = [a ₁ (ω, θ, D),..., A _M (ω, θ, D)] ^T , for example, as shown in equation (12a), the direct sound steering vector and the number of dual sounds It is to express with the sum of steering vectors. Therefore, it is understood that sound reproduction with a narrower direction than before is possible even when the sound wave travels as a spherical wave without being limited to the case where the sound wave travels as a plane wave.

The transfer characteristic a ^→ (ω, θ, D) when the sound wave travels as a spherical wave will be described. In this case, there are two types of spherical waves, the sound from each speaker of the speaker array that travels without being reflected (direct sound) and the reflected sound that the dual sound is reflected by the reflector 300 (position ( θ, D). Let the number of reflected sounds (or dual sounds) be Ξ. At this time, transfer characteristic a ^→ (ω, θ, D) = [a ₁ (ω, θ, D),..., A _M (ω, θ, D)] ^T is located at position (θ, D) from the speaker array. The sum of the direct sound transfer characteristics and the transfer characteristics of one or more dual sounds corresponding to the direct sound, specifically, the direct sound and the ξth (1 ≦ ξ ≦ Ξ) reflected sound _Assuming that the time difference is τ _ξ (θ) and α _ξ (1 ≦ ξ ≦ Ξ) is a coefficient for considering the attenuation of sound due to reflection, the direct sound steering vector and the reflection It can be expressed by the sum of steering vectors of two dual sounds in which the time difference between the sound attenuation and the reflected sound with respect to the direct sound is corrected. h ^→ _d (ω, θ, D _h ) = [h _d1 (ω, θ, D _h ),…, h _dM (ω, θ, D _h )] ^T is the position of the sound at position (θ _s , D) Represents the direct sound steering vector. h ^→ _rξ (ω, θ) = [h _r1ξ (ω, θ), ..., h _rMξ (ω, θ)] ^T represents the steering vector of the dual sound corresponding to the direct sound at position (θ, D) . Here, attention is paid to the name “steering vector”. In general, a “steering vector” represents a complex vector that depends on “direction”, also referred to as “direction vector”. From this point of view, as a name of a complex vector that depends on position (θ _s , D), for example, “ It is more accurate to call it an “extended steering vector”. However, in this specification, in order to simplify the expression, a “steering vector” is simply used as a name of a complex vector depending on the position (θ _s , D). α _ξ (1 ≦ ξ ≦ Ξ) is usually α _ξ ≦ 1 (1 ≦ ξ ≦ Ξ). For each reflected sound, if the number of times that the sound (dual sound) from the speaker array is reflected by the reflecting object is 1, α _ξ (1 ≦ ξ ≦ Ξ) is the number of objects reflected by the ξth dual sound. You can think of it as representing the reflectance of sound.

In Expression (20), the m-th element h _dm (ω, θ, D _h ) constituting the direct sound steering vector h ^→ _d (ω, θ, D _h ) is given by, for example, Expression (20a). m is an integer satisfying 1 ≦ m ≦ M. c represents the speed of sound. j is an imaginary unit. In an appropriately set spatial coordinate system, v ^→ _{θ, D} ^(d) represents the position vector of the position (θ, D), and u ^→ _m represents the position vector of the mth speaker. The symbols ‖ and ‖ represent the norm. f (‖v ^→ _{θ, D} ^(d) -u ^→ _m ‖) is a function representing the attenuation of the sound wave distance. For example ^{_{f (‖v → θ, D (}} d) -u → m ||) = 1 / ‖v ^→ _θ, a _D ^(d) -u ^→ _m ‖, in this case, equation (20a) is the formula (20b) It is represented by

In equation (20), the steering vector h ^→ _rξ (ω, θ, D) = [h _r1ξ (ω, θ, D),..., H _rMξ (ω, θ, D)] ^T The element h _rmξ (ω, θ, D) is expressed by the equation (21a) in the same manner as the direct sound steering vector (see the equation (20a)). m is an integer satisfying 1 ≦ m ≦ M. c represents the speed of sound. j is an imaginary unit. In the above spatial coordinate system, v ^→ _{θ, D} ^(ξ) is the position vector where the position (θ, D) is moved to the mirror image object on the reflecting surface of the ξth reflector, u ^→ _m is the mth Represents the position vector of the speaker. The symbols ‖ and ‖ represent the norm. f (‖v ^→ _{θ, D} ^(ξ) -u ^→ _m ‖) is a function that represents the distance attenuation of the sound wave. For example _{^{f (‖v → θ, D (}} ξ) -u → m _{^{||) = 1 / ‖v → θ,}} D (ξ) -u → m is ‖, where, equation (21a) is the formula (21b) It is represented by

The ξ-th time difference τ _ξ (θ, D) and the position vector v ^→ _{θ, D} ^(ξ) are determined when the positional relationship between the position (θ, D), the speaker array, and the ξ-th reflector is determined. It can be calculated theoretically based on the positional relationship.

So far, as one of the key points of audio spot playback, we have explained from the viewpoint of narrow directivity mainly with respect to direction by fixing the distance D, but here, from the speaker array, which is another key point It will be described that the sound pressure of the reproduced sound can be controlled according to the distance. Unlike the prior art, the present invention enables sound reproduction by distance by actively considering reflected sound. Hereinafter, this will be described by taking two listening positions as examples. As shown in FIG. 6 (a), sound is spot-reproduced based on only the direct sound from both listening positions A and B, which are at different distances as viewed from the speaker array but are in the same direction. Difficult to do. This is because the value of the attenuation function that appears in the direct sound steering vector h ^→ _d (ω, θ _[A] , D _[A] ) corresponding to the listening position A (θ _[A] , D _[A] ) Steering vector h of direct sound corresponding to f (‖v ^→ _{θ [A], D [A]} ^(d) -u ^→ _m ‖) and listening position B (θ _[B] , D _[B] ) ^→ _between the value f (‖v ^→ _{θ [B], D [B]} ^(d) -u ^→ _m ‖) of the attenuation function appearing in _d (ω, θ _[B] , D _[B] ) _When attention is paid to _[A] ≒ θ _[B] and D _[A] ≠ D _[B] , there is a difference depending on the distance, but in reality, the magnitude (volume) of the sound source signal and the value of the attenuation function This is because they cannot be distinguished. In other words, if a ^→ _conv (ω, θ, D) = h ^→ _d (ω, θ, D) as in the prior art, the direct sound transfer characteristics are identified with respect to the distance of the listening position in the same direction. This is not sufficient as a force, and it becomes difficult to design a filter that can reproduce sound according to distance, as can be seen with reference to equations (4), (5a), and (5b).

On the other hand, in the present invention, when the reflected sound is actively considered, as shown in FIG. 6B, the virtual listening positions A (ξ) and B (ξ) for the ξ-th dual sound are determined from the standpoint of the speaker array. In other words, the listening positions A and B exist at positions where the reflection surfaces of the ξ-th reflector 300 are moved to the mirror image object. Between the ξth dual sound to the virtual listening position A (ξ) and the ξth dual sound to the virtual listening position B (ξ), the position vector of each virtual listening position A (ξ), B (ξ) v ^→ _{θ [A (ξ)], D [A (ξ)]} ^(ξ) and v ^→ _{θ [B (ξ)], D [B (ξ)]} ^(ξ) , each time difference τ _ξ (θ _{[A ]} , D _[A] ) and τ _ξ (θ _[B] , D _[B] ) are significantly different. Therefore, the transfer characteristics a ^→ (ω, θ _[A] , D _[A] ) and a ^→ corresponding to the position (θ _[A] , D _[A] ) and the position (θ _[B] , D _[B] ) (ω, θ _[B] , D _[B] ) is expressed by Expression (22a) and Expression (22b), and θ _{[A] is determined} by the presence of the second term of each of Expression (22a) and Expression (22b) _. Even when ≈θ _[B] , a significant difference occurs between transfer characteristics corresponding to different positions. By reproducing the difference between the transfer characteristics by the beamforming method, it is possible to reproduce the sound at the listening position according to the distance.

So far, in order to explain the realization of narrow directivity, the distance D _h has been fixed. For this reason, the spatial correlation matrix Q (ω) is expressed by Expression (5a) or Expression (5b). However, the amount of information in the sound field increases by taking into account the cross-correlation of the transfer characteristics of the M channel for each distance D _δ (δ = 1,2, ..., G), and the spatial correlation matrix that gives a more accurate filter Can be configured. This spatial correlation matrix Q (ω) is expressed by equation (5c). The set index phi direction theta _phi belongs Φ (| Φ | = P) , the distance D index [delta] belong set of _{δ Δ (| Δ | = G} ) to.

At this time, if the spatial correlation matrix Q (ω) represented by the equation (5c) is used, the filter W ^→ (ω, θ _s , D _h ) based on the minimum variance distortionless response method is replaced with the equation (4). It is represented by (4a).

By the way, as described above, the main feature of the present invention is that the transfer characteristic a ^→ (ω, θ, D) = [a ₁ (ω, θ, D),..., A _M (ω, θ, D)] ^T is expressed as the sum of the steering vector of the direct sound and the steering vectors of the dual dual sounds. Therefore, since the filter design concept itself is not affected, the filter W ^→ (ω, θ _s , D _h ) can be designed by a method other than the minimum variance no distortion response method.

Other than the above-mentioned minimum variance-free response method, <1> filter design method based on SNR maximization criteria, <2> filter design method based on Power Inversion, <3> one or more suppressions Filter design method based on minimum variance distortion-free response method with point (direction in which leaked speech gain is suppressed), <4> filter design method based on delay-and-sum beam forming method, <5 The filter design method using the maximum likelihood method and the filter design method using the <6> AMNOR (Adaptive Microphone-array for Noise Reduction) method will be described. Refer to Reference 2 for the filter design method based on <1> SNR maximization criteria and <2> filter design method based on power inversion. <3> See Reference 3 for the filter design method based on the minimum variance-distortion response method that has one or more suppression points (direction in which the gain of leaked speech is suppressed) as a constraint. <6> See Reference 4 for the filter design method based on the AMNOR (Adaptive Microphone Array for Noise Reduction) method.
(Reference 2) Nobuyoshi Kikuma, “Adaptive Antenna Technology”, 1st Edition, Ohm Corporation, 2003, pp.35-90
(Reference 3) Taiko Asano, “The Acoustical Society of Japan, Acoustic Techno Series 16 Sound Array Signal Processing-Sound Source Localization, Tracking and Separation”, First Edition, Corona Inc., pp.88-89, 259-261
(Reference 4) Yutaka Kaneda, "Directivity characteristics of adaptive noise suppression microphone array (AMNOR)", Journal of the Acoustical Society of Japan, Vol. 44 No. 1 (1988), pp.23-30

<1> Filter design method based on S / N ratio maximization criteria
In the filter design method based on the S / N ratio maximization criterion, the filter W ^→ (ω, θ _s , D _h ) is determined by the criterion that maximizes the S / N ratio (SNR) at the position (θ _s , D _h ). The spatial correlation matrix of speech to the position (θ _s , D _h ) is R _ss (ω), and the spatial correlation matrix of speech to positions other than the position (θ _s , D _h ) is R _nn (ω). At this time, the SNR is expressed by Expression (23). Note that R _ss (ω) is expressed by Expression (24), and R _nn (ω) is expressed by Expression (25). Transfer characteristic a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ),..., A _M (ω, θ _s , D _h )] ^T is expressed by, for example, Expression (20) (Accurately, θ in Equation (20) is θ _s and D is D _h ). In Equation (25), a set to which the index φ in the direction θ _φ belongs is Φ (| Φ | = P), and a set to which the index δ of the distance D _δ belongs is Δ (| Δ | = G).

The filter W ^→ (ω, θ _s , D _h ) that maximizes the SNR in Expression (23) sets the gradient with respect to the filter W ^→ (ω, θ _s , D _h ) to zero, that is, according to Expression (26). Can be sought.

Thereby, the filter W ^→ (ω, θ _s , D _h ) that maximizes the SNR in Expression (23) is given by Expression (27).

Expression (27) includes an inverse matrix of the spatial correlation matrix R _nn (ω) of speech to positions other than the position (θ _s , D _h ), but the inverse matrix of R _nn (ω) is expressed as ( 1) substituted inverse position (theta _s, and audio to D _h) (2) position (θ _s, D _h) than the whole input space correlation matrix including the speech to the position R _xx (omega) It is known that you may. Note that R _xx (ω) = R _ss (ω) + R _nn (ω). That is, the filter W ^→ (ω, θ _s , D _h ) that maximizes the SNR in Expression (23) may be obtained by Expression (28).

<2> In the filter design method based on the filter design method power inversion based on power inversion, filter criteria to minimize the average output power of the beamformer filter coefficients in a fixed state to a constant value for one speaker W ^→ (ω, θ _s , D _h ) is determined. Here, as an example, it is assumed that the filter coefficient for the first speaker among the M speakers is fixed. In this design method, the filter W ^→ (ω, θ _s , D _h ) is all positions (all assumed as listening positions) using the spatial correlation matrix R _xx (ω) under the constraint condition of the equation (30). It is designed so that the power of the voice to the position (see (29)) is minimized. Transfer characteristic a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ),..., A _M (ω, θ _s , D _h )] ^T is expressed by, for example, Expression (20) (Accurately, θ in Equation (20) is θ _s and D is D _h ).

It is known that the filter W ^→ (ω, θ _s , D _h ), which is the optimal solution of the equation (29), is given by the equation (31) (see Reference 2).

<3> Filter design method based on minimum variance distortionless response method with one or more suppression points as constraint conditions In the above minimum variance distortionless response method, the position (θ _s , D _h ) is constrained to pass through the entire band, and the average output power of the beamformer expressed by Equation (2) is minimized (that is, the voice to a position other than the position (θ _s , D _h ) The filter W ^→ (ω, θ _s , D _h ) was designed based on the criterion under the single constraint condition of finding a filter that minimizes the power of a certain leaked voice. According to this method, the power of leaked voice can be suppressed as a whole, but it is not necessarily a preferable method when it is desired to strongly suppress voice propagation to one or more specific positions. In such a case, a filter that strongly suppresses voice propagation to one or more known specific positions (that is, suppression points) is required. For this reason, in the filter design method described here, (1) the entire band pass of the sound to the position (θ _s , D _h ), and (2) known B pieces (B is a predetermined integer of 1 or more) suppression point _{of) (θ N1, D G1)} , (θ N2, D G2), ..., a (theta _NB, constraints entire band suppression, the voice of the D _GB), the formula (2) A filter that minimizes the average output power of the beamformer (that is, the power of the sound at the position excluding the position (θ _s , D _h ) and each suppression point is minimized) is obtained. As described above, if the set to which the index φ in the direction of the listening position belongs is {1, 2, ..., P}, Nj∈ {1,2, ..., P} (where j∈ {1,2, ..., B}), B ≦ P-1. If the set to which the index δ of the distance to the listening position belongs is {1, 2, ..., G}, Gj∈ {1,2, ..., G} (where j∈ {1,2, ..., B }), B ≦ G-1.

At this time, a ^→ (ω, θ _i , D _g ) = [a ₁ (ω, θ _i , D _g ),..., A _M (ω, θ _i , D _g )] ^{T is changed} to the position (θ _i , D _g ), assuming that there is a listening position, transfer characteristics at the frequency ω between the listening position and the M speakers, in other words, a ^→ (ω, θ _i , D _g ) = [a ₁ (ω, θ _i , D _g ), ..., a _M (ω, θ _i , D _g )] ^T is the sound from each speaker included in the speaker array to the position (θ _i , D _g ). Assuming the transfer characteristic at the frequency ω, the constraint condition is expressed by the equation (32). However, for indexes i and g, (i, g) ∈ {(s, h), (N1, G1), (N2, G2), ..., (NB, GB)}, and transfer characteristics a ^→ (ω , θ _i , D _g ) = [a ₁ (ω, θ _i , D _g ),..., a _M (ω, θ _i , D _g )] ^T is expressed by equation (20) (exactly, In the equation (20), θ is θ _i and D is D _g ). f _{i, g} (ω) represents the pass characteristic at the frequency ω with respect to the position (θ _i , D _g ).

When Expression (32) is expressed in a matrix format, it can be expressed as Expression (33), for example. However, A ^→ (ω, θ _s , D _h ) = [a ^→ (ω, θ _s , D _h ), a ^→ (ω, θ _N1 , D _G1 ), ..., a ^→ (ω, θ _NB , D _GB )].

(1) All-band passage of speech to position (θ _s , D _h ), and (2) known B suppression points (θ _N1 , D _G1 ), (θ _N2 , D _G2 ),. _NB , D _GB ) Considering the constraint that the entire band of the voice is suppressed, ideally, f _{s, h} (ω) = 1.0, f _{i, g} (ω) = 0.0 ((i, g) ∈ {(N1, G1), (N2, G2), ..., (NB, GB)}). This is because the entire band of the sound at the position (θ _s , D _h ) is completely passed, and B known suppression points (θ _N1 , D _G1 ), (θ _N2 , D _G2 ),..., (Θ _NB , D _GB ) represents the full blockage of the entire band of speech. However, in reality, it may be difficult to control full band full passage or full band full blocking. In such a case, the absolute value of f _{s, h} (ω) is a value close to 1.0, f _{i, g} (ω) ((i, g) ∈ {(N1, G1), (N2, G2), ..., (NB, GB)}) may be set to a value close to 0.0. Of course, f _{i, g_i} (ω) and f _{j, g_j} (ω) (i ≠ j, i, j∈ {N1, N2,..., NB}) may be equal or different.

According to the filter design method described here, the filter W ^→ (ω, θ _s , D _h ), which is the optimal solution of the equation (2) under the equation (33) representing the constraint condition, is given by the equation (34). (See Reference 3). However, although the spatial correlation matrix Q (ω) represented by the equation (5c) is used, a spatial correlation matrix represented by the equations (5a) to (5b) may be used.

<4> Filter Design Method by Delay Synthesis Method According to the delay synthesis method, assuming that direct sound and reflected sound propagate through a plane wave, the filter W ^→ (ω, θ _s , D _h ) is given by equation (35). That is, the filter W ^→ (ω, θ _s , D _h ) is obtained by normalizing the transfer characteristic a ^→ (ω, θ _s , D _h ). Transfer characteristic a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ), ..., a _M (ω, θ _s , D _h )] ^T is expressed by equation (20). (Accurately, θ in equation (20) is θ _s , and D is D _h ). According to this design method, the filter accuracy may not always be good, but the calculation amount is small.

<5> Filter design method based on maximum likelihood method In the above-described minimum variance distortion-free response method, the spatial correlation matrix Q (ω, D _h ) does not include the spatial information of the speech at the position (θ _s , D _h ) Thus, the degree of freedom for suppressing the leaked voice is improved, and the power of the leaked voice can be further suppressed. In this case, in the filter design method described here, the spatial correlation matrix Q (ω, D _h ) is represented by the second term on the right side of Equation (5a), that is, Equation (5d). The filter W ^→ (ω, θ _s , D _h ) is given by Equation (4) or Equation (34). At this time, the spatial correlation matrix included in Expression (4) or Expression (34) is a spatial correlation matrix represented by Expression (5d).

Alternatively, the spatial information of the voice at the position (θ _s , D _h ) may not be included in the spatial correlation matrix Q (ω). In this case, in the filter design method described here, the spatial correlation matrix Q (ω) is expressed by Expression (5e). The filter W ^→ (ω, θ _s , D _h ) is given by Equation (4) or Equation (34). At this time, the spatial correlation matrix included in Expression (4) or Expression (34) is a spatial correlation matrix represented by Expression (5e).

<6> Filter design method by AMNOR method
The AMNOR method allows a certain amount of speech degradation amount D in the target direction based on the trade-off relationship between the speech degradation amount D in the target direction and the power of noise remaining in the filter output signal (for example, the degradation amount D). Is kept below a certain threshold value D ^), [a] a signal obtained by applying a transfer characteristic between a sound source and a microphone to a virtual signal in a target direction (hereinafter referred to as a virtual target signal) [b] The filter output signal when the mixed signal with noise (for example, obtained by observation with M microphones in a noise environment where there is no voice in the target direction) is input is the virtual target signal most in terms of the least square error. This is a method for obtaining a filter that reproduces well (that is, the noise power included in the filter output signal is minimized).

The filter design method described here can be considered in the same way as the AMNOR method except that the input and output of the filter are reversed. That is, based on the trade-off relationship between the amount of speech degradation D to the position (θ _s , D _h ) and the power of leaked speech remaining in the filter output signal, the speech degradation to the location (θ _s , D _h ) The filter output signal is minimum when the amount D is allowed to some extent (for example, the deterioration amount D is kept below a certain threshold D ^) and the frequency domain signal S (ω, k) of the sound source signal is input. A filter that best reproduces the frequency domain signal S (ω, k) from the viewpoint of the square error (that is, the power of the leaked voice included in the filter output signal is minimized) is obtained. The filter output signal is a signal (a) obtained by applying the transfer characteristic at the frequency ω of sound from each speaker included in the speaker array to the position (θ _s , D _h ) on the frequency domain signal S (ω, k) ( Hereinafter, it is a mixed signal of [listening signal] and [b] noise (for example, obtained by observation with M microphones in a noise environment).

According to the filter design method described here, as in the AMNOR method, the filter W ^→ (ω, θ _s , D _h ) is given by Expression (36) (see Reference 4). Note that R _ss (ω) is expressed by Expression (24), and R _nn (ω) is expressed by Expression (25). Transfer characteristic a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ), ..., a _M (ω, θ _s , D _h )] ^T is expressed by equation (20). (Accurately, θ in equation (20) is θ _s , and D is D _h ).

P _s is a coefficient for weighting the level of the listening signal, and is referred to as the listening signal level. The listening signal level P _s is a constant independent of frequency. The listening signal level P _s may be determined based on an empirical rule, or an error range in which the difference between the audio degradation amount D to the position (θ _s , D _h ) and the threshold value D ^ is arbitrarily determined. It may be determined to be within. The latter example will be described. At the frequency ω, the frequency response F (ω) of the voice to the position (θ _s , D _h ) of the filter W ^→ (ω, θ _s , D _h ) is expressed by the equation (37). When the deterioration amount D when using the filter W ^→ (ω, θ _s , D _h ) given by the equation (36) is expressed as D (P _s ), the deterioration amount D (P _s ) is expressed by the equation (38). Defined. ω ₀ represents the upper limit of the target frequency ω (usually a high-frequency side adjacent to the discrete frequency ω). The deterioration amount D (P _s ) is a monotone decreasing function of P _s . Therefore, the monotonicity of the D (P _s), by repeating the determination of the deterioration amount D (P _s) while changing the P _s, deterioration amount D (P _s) and the difference between any of the threshold D ^ A listening signal level P _s within a predetermined error range can be obtained.

<Modification>
In the above description, the spatial correlation matrices Q (ω), R _ss (ω), and R _nn (ω) are expressed using transfer characteristics. However, as described above, the spatial correlation matrix Q (ω), R _ss (ω), R _nn (using the frequency domain signal obtained by converting the analog signal obtained by observation with the microphone array into the frequency domain. ω) can also be expressed. The spatial correlation matrix Q (ω) will be described below, but the same applies to R _ss (ω) and R _nn (ω) (Q (ω) can be read as R _ss (ω) or R _nn (ω). Just fine). Note that the spatial correlation matrix R _ss (ω) is the frequency domain of the analog signal obtained by observation with a microphone array (including M microphones) in an environment where only the sound at the position (θ _s , D _h ) exists. And the spatial correlation matrix R _nn (ω) is obtained by observation with a microphone array (including M microphones) in an environment where there is no sound at the position (θ _s , D _h ) (ie, a noise environment). Obtained by the frequency domain representation of the analog signal.

The spatial correlation matrix Q (ω) using the frequency domain signal U ^→ (ω, k) = [U ₁ (ω, k),..., U _M (ω, k)] ^T is expressed by Expression (39). . The operator E [•] is an operator representing a statistical average operation. When a discrete time series of analog signals received by M microphones is regarded as a stochastic process, the operator E [•] is the arithmetic mean (expected value) if it is a so-called steady or secondary stationary. It becomes an operation. In this case, the spatial correlation matrix Q (ω) is, for example, a frequency domain signal U ^→ (ω, ki) (i = 0, 1,..., Ζ of a total of ζ frames currently and past stored in a memory or the like. -1) is used to express the equation (40). When i = 0, that is, the k-th frame is the current frame. The spatial correlation matrix Q (ω) according to the equations (39) to (40) may be recalculated for each frame, may be recalculated at regular or irregular intervals, or It may be calculated before the implementation of the embodiment described later (particularly when R _ss (ω) or R _nn (ω) is used for the filter design, it was obtained before the implementation of the embodiment. It is preferable to calculate the spatial correlation matrix Q (ω) in advance using the frequency domain signal). When recalculating the spatial correlation matrix Q (ω) for each frame, since the spatial correlation matrix Q (ω) depends on the current and past frames, the spatial correlation matrix is explicitly expressed as in equations (39a) and (40a). Let the correlation matrix be Q (ω, k).

If the spatial correlation matrix Q (ω, k) represented by the equations (39a) and (40a) is used, the filter W ^→ (ω, θ _s ) also depends on the current and past frames. W ^→ (ω, θ _s , D _h , k). At this time, any one of Expression (4), Expression (4a), Expression (27), Expression (28), Expression (31), Expression (34), and Expression (36) described in the various filter design methods described above. The filter W ^→ (ω, θ _s , D _h ) expressed by the following expression is expressed by the expression (4m), the expression (4am), the expression (27m), the expression (28m), the expression (31m), and the expression (34m). , Corrected to equation (36m).

Embodiment 1
The functional configuration and processing flow of the first embodiment of the present invention are shown in FIGS. The audio spot reproduction processing apparatus 1 according to the first embodiment includes an AD conversion unit 210, a frame generation unit 220, a frequency domain conversion unit 230, a filter application unit 240, a time domain conversion unit 250, a filter design unit 260, and a storage unit 290. .

[Step S1]
The filter design unit 260 calculates the filter W ^→ (ω, θ _i , D _g ) for each frequency in advance for each discrete position (θ _i , D _g ) that can be an audio spot reproduction target. The total number of discrete directions that can be the target of audio spot reproduction is I (I is a predetermined integer of 1 or more and satisfies I ≦ P), and the total number of discrete distances is G (G is 1 or more). If you predetermined is an ^{integer), W → (ω, θ} 1, D 1), ..., W → (ω, θ i, D 1), ..., W → (ω, θ I, D 1) , W ^→ (ω, θ ₁ , D ₂ ),…, W ^→ (ω, θ _i , D ₂ ),…, W ^→ (ω, θ _I , D ₂ ),…, W ^→ (ω, θ ₁ , D _g ), ..., W ^→ (ω, θ _i , D _g ), ..., W ^→ (ω, θ _I , D _g ), ..., W ^→ (ω, θ ₁ , D _G ), ..., W ^→ (ω, θ _i , D _G ),…, W ^→ (ω, θ _I , D _G ) (1 ≦ i ≦ I, 1 ≦ g ≦ G, ω∈Ω; i and g are integers, Ω is frequency (a set of ω) is calculated in advance.

For this purpose, except for the case described in <Modification> above, transfer characteristics a ^→ (ω, θ _i , D _g ) = [a ₁ (ω, θ _i , D _g ), ..., a _M ( ω, θ _i , D _g )] ^T (1 ≦ i ≦ I, 1 ≦ g ≦ G, ω∈Ω), it is necessary to determine the arrangement of the speakers in the speaker array, for example, reflection. Expression (20) based on environmental information such as the positional relationship of the board, floor, wall, and ceiling with respect to the speaker array, the time difference between the direct sound and the ξ-th (1 ≦ ξ ≦ Ξ) reflected sound, and the reflectance of the sound of the reflector. (Accurately, θ in Equation (20) is θ _i , and D is D _g ). In the case of using the filter design method based on the minimum variance distortionless response method having one or more <3> suppression points as constraint conditions, the transfer characteristic a ^→ (ω, θ _i , D _g ) (1 ≦ i ≦ I, 1 ≤ g ≤ G, ω ∈ Ω), the position index (i, g) is the index (N1, G1), (N2, G2), ... in the direction of at least B suppression points. It is desirable to cross all of (NB, GB). In other words, the B indexes N1, N2,..., NB are set as any different integer from 1 to I, and the B indexes G1, G2,. Are set as different integers.

  The number 反射 of the reflected sounds (or dual sounds) is set to an integer that satisfies 1 ≦ Ξ, but the value of Ξ is not particularly limited and may be set appropriately according to the calculation capability.

  For example, the formula (20a), the formula (20b), the formula (21a), and the formula (21b) can be used for the calculation of the steering vector. In addition, as a transfer characteristic used for the filter design, for example, a transfer characteristic obtained by actual measurement in an actual environment may be used instead of the equation (20).

Then, except for the case described in the above <Modification>, for example, Expression (4), Expression (4a), Expression (27), Expression (4) using the transfer characteristic a ^→ (ω, θ _i , D _g ). 28), Formula (31), Formula (34), Formula (35), and Formula (36), W ^→ (ω, θ _i , D _g ) (1 ≦ i ≦ I, 1 ≦ g ≦ G) Ask for. Except for the case described in the above-described <5> maximum likelihood filter design method, when using equation (4), equation (4a), equation (28), equation (31), or equation (34) The spatial correlation matrix Q (ω) (or R _xx (ω)) can be calculated by equation (5b). According to the above-described <5> maximum likelihood filter design method, when using equation (4), equation (4a), equation (28), equation (31), or equation (34), spatial correlation matrix Q ( ω) (or R _xx (ω)) can be calculated by equation (5d) or equation (5e). When Expression (27) is used, the spatial correlation matrix R _nn (ω) can be calculated by Expression (25). The I × G × | Ω | filters W ^→ (ω, θ _i , D _g ) (1 ≦ i ≦ I, 1 ≦ g ≦ G, ω∈Ω) are stored in the storage unit 290. | Ω | represents the number of elements of the set Ω.

[Step S2]
The sound source 200 outputs a sound source signal ss (t). In this embodiment, it is assumed that the sound source signal ss (t) from the sound source 200 is an analog signal. However, a digital signal can also be used as the sound source signal.

[Step S3]
The AD converter 210 AD converts the sound source signal ss (t) into a digital signal s (t). Here, t represents an index of discrete time. When the digital signal is a sound source signal, it is not necessary to perform the process of step S3, and the sound source signal can be regarded as s (t) that is an output signal of the AD conversion unit 210.

[Step S4]
The frame generation unit 220 receives the digital signal s (t) output from the AD conversion unit 210, stores N samples in a buffer, and outputs a digital signal s (k) in units of frames. k is an index of the frame number. s (k) = [s ((k−1) N + 1),..., s (kN)]. N depends on the sampling frequency, but in the case of 16kHz sampling, around 512 points is reasonable.

[Step S5]
The frequency domain converter 230 converts the digital signal s (k) of each frame into a frequency domain signal S (ω, k) and outputs it. ω is an index of discrete frequency. One method for converting a time domain signal to a frequency domain signal is a fast discrete Fourier transform, but the present invention is not limited to this, and other methods for converting to a frequency domain signal may be used. The frequency domain signal S (ω, k) is output for each frequency ω and for each frame k.

[Step S6]
For each frequency ω∈Ω, the filter application unit 240 filters the frequency W of the frequency domain signal S (ω, k) corresponding to the position (θ _s , D _h ) to be spot-reproduced for each frequency ω∈Ω ^→ (ω, θ _{s 1} , D _h ), and the reproduction signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)] ^T is output (see Expression (41)). . The indices s, h at the position (θ _s , D _h ) are s∈ {1,…, I}, h∈ {1,…, G}, and the filter W ^→ (ω, θ _s , D _h ) is Since it is stored in the storage unit 290, for example, each time the process of step S6, the filter application unit 240 filters W ^→ (ω, θ _s , D _h ) corresponding to the position (θ _s , D _h ) at which spot reproduction is desired. ) From the storage unit 290. If the index s in the direction θ _s does not belong to the set {1, ..., I} or the index h in the distance D _h does not belong to the set {1, ..., G}, that is, the position (θ _s , D _h ) If the filter W ^→ (ω, θ _s , D _h ) corresponding to い な い is not calculated in the process of step S1, the filter W ^→ (ω, θ _s , D) corresponding to the temporary position (θ _s , D _h ) it the _h) may be calculated in the filter design unit 260, or the filter corresponds to the _'distance D _h near and distance D _h' direction theta _s close to the direction ^{_{θ s W → (ω, θ}} s', D h ), W ^→ (ω, θ _s , D _{h ′} ) or W ^→ (ω, θ _{s ′} , D _{h ′} ).

[Step S7]
The time domain conversion unit 250 uses the reproduction signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)] of each frequency ω∈Ω of the kth frame in the time domain. The frame unit time domain signal x ^→ (k) = [x ₁ (k),..., X _M (k)] of the k-th frame is obtained by conversion, and the obtained frame unit time domain signal x ^→ ( k) = [x ₁ (k),..., x _M (k)] are connected in the order of the frame number index, and the time domain in which the sound is emphasized toward the position (θ _s , D _h ) where spot reproduction is desired The signal x ^→ (t) = [x ₁ (t),..., X _M (t)] is output. The method for converting the frequency domain signal into the time domain signal is an inverse transformation corresponding to the transformation method used in the process of step S5, for example, a fast discrete inverse Fourier transform.

[Step S8]
M channel time domain signals x ₁ (t),..., X _M (t) are reproduced by the speakers corresponding to the channel among the M speakers 280-1,. Is done. That is, the time domain signal x _m (t) of the m-th (1 ≦ m ≦ M) channel is reproduced by the m-th speaker 280-m.

There is no limit to the way the M speakers are arranged. An array configuration in which speakers are linearly arranged like a linear speaker array may be used, or an array configuration in which M speakers are two-dimensionally or three-dimensionally arranged. In addition, in order to make the direction that can be set as the sound collection direction wide, the directivity of each speaker is the directivity that can reproduce sound with a certain sound pressure in the direction that can be the target direction θ _s that is the reproduction direction. It is better to have Therefore, a speaker having relatively gentle directivity such as a non-directional speaker or a unidirectional speaker is preferable.

Here, the first embodiment in which the filter W ^→ (ω, θ _i , D _g ) is calculated in advance in the process of step S1 has been described. However, depending on the calculation processing capability of the audio spot reproduction processing device 1, the position is determined. An embodiment in which the filter design unit 260 calculates the filter W ^→ (ω, θ _s , D _h ) for each frequency after (θ _s , D _h ) is determined may be employed.

<< Embodiment 2 >>
The functional configuration and processing flow of the second embodiment of the present invention are shown in FIGS. The audio spot reproduction processing apparatus 2 according to the second embodiment includes an AD conversion unit 210, a frame generation unit 220, a frequency domain conversion unit 230, a filter application unit 240, a time domain conversion unit 250, a filter calculation unit 261, a storage unit 290, an AD. A conversion unit 310, a frame generation unit 320, and a frequency domain conversion unit 330 are included.

[Step S11]
The sound source 200 outputs a sound source signal ss (t). In the second embodiment, it is assumed that the sound source signal ss (t) from the sound source 200 is an analog signal. However, a digital signal can also be used as the sound source signal.

[Step S12]
The AD converter 210 AD converts the sound source signal ss (t) into a digital signal s (t). Here, t represents an index of discrete time. When the digital signal is a sound source signal, it is not necessary to perform the process of step S12, and the sound source signal can be regarded as s (t) that is an output signal of the AD conversion unit 210.

[Step S13]
The frame generation unit 220 receives the digital signal s (t) output from the AD conversion unit 210, stores N samples in a buffer, and outputs a digital signal s (k) in units of frames. k is an index of the frame number. s (k) = [s ((k−1) N + 1),..., s (kN)]. N depends on the sampling frequency, but in the case of 16kHz sampling, around 512 points is reasonable.

[Step S14]
The frequency domain converter 230 converts the digital signal s (k) of each frame into a frequency domain signal S (ω, k) and outputs it. ω is an index of discrete frequency. One method for converting a time domain signal to a frequency domain signal is a fast discrete Fourier transform, but the present invention is not limited to this, and other methods for converting to a frequency domain signal may be used. The frequency domain signal S (ω, k) is output for each frequency ω and for each frame k.

[Step S15]
The filter calculation unit 261 uses a filter W ^→ (ω, θ _s , D _h , k) for each frequency corresponding to the position (θ _s , D _h ) (ω∈Ω; Ω is used in the current k-th frame. A set of frequencies ω).

For this purpose, the transfer characteristic a ^→ (ω, θ _s , D _h ) = [a ₁ (ω, θ _s , D _h ), ..., a _M (ω, θ _s , D _h )] ^T (ω∈ Ω) must be prepared. This is because the speaker arrangement in the speaker array, the positional relationship of reflectors such as reflectors, floors, walls, and ceilings with respect to the speaker array, direct sound and ξth (1 ≦ ξ ≦ Based on environmental information such as the time difference from the reflected sound of (ii) and the reflectance of the sound of the reflector, it can be specifically calculated by the equation (20) (exactly, θ in the equation (20) is θ _s , D is is obtained by the D _h). In the case of using the filter design method based on the minimum variance distortionless response method having one or more <3> suppression points as a constraint, the transfer characteristic a ^→ (ω, θ _Nj , D _Gj ) (1 ≦ j ≦ B, ω∈Ω) must also be obtained. These are the arrangement of speakers in the speaker array, the positional relationship of reflectors such as reflectors, floors, walls, and ceilings with respect to the speakerlophone array, direct sound and ξ-th (1 ≦ ξ ≦ Ξ) can be specifically calculated by Equation (20) based on environmental information such as the time difference from the reflected sound and the reflectance of the sound of the reflector (more precisely, θ in Equation (20) is θ _Nj , D is D _Gj ).

  The number 反射 of the reflected sound is set to an integer satisfying 1 ≦ Ξ, but the value of Ξ is not particularly limited and may be appropriately set according to the calculation ability.

  For example, the formula (20a), the formula (20b), the formula (21a), and the formula (21b) can be used for the calculation of the steering vector. In addition, as a transfer characteristic used for the filter design, for example, a transfer characteristic obtained by actual measurement in an actual environment may be used instead of the equation (20).

Then, the filter calculation unit 261 transmits the transfer characteristic a ^→ (ω, θ _s , D _h ) (ω∈Ω) or the transfer characteristic a ^→ (ω, θ _Nj , D _Gj ) (1 ≦ j ≦ B , ω∈Ω), the filter W ^→ (ω, θ _s , D _h , k) (ω∈Ω) is changed into the expression (4m), the expression (4am), the expression (27m), the expression (28m), It calculates | requires according to either Formula (31m), Formula (34m), or Formula (36m). Note that the spatial correlation matrix Q (ω) (or R _xx (ω)) can be calculated by, for example, Expression (39a) or Expression (40a). For the calculation of the spatial correlation matrix Q (ω), the frequency domain signal X ^→ (ω, ki) (i = 0, 1,..., Ζ− of the current and past frames accumulated in the storage unit 290 in total. 1) is used.

The frequency domain signal X ^→ (ω, k) is accumulated in the storage unit 290 as follows.
Sound is collected using M microphones 300-1,..., 300-M constituting the microphone array. The arrangement of the M microphones is preferably the same as that of the speaker array.
The AD converter 310 converts the analog signal (sound pickup signal) collected by the M microphones 300-1,..., 300-M into a digital signal x ^→ (t) = [x ₁ (t),. _M (t)]. t represents a discrete time index.
The frame generation unit 320 receives the digital signal x ^→ (t) = [x ₁ (t),..., X _M (t)] output from the AD conversion unit 310 and stores N samples in a buffer for each channel. The digital signal x ^→ (k) = [x ^→ ₁ (k),..., X ^→ _M (k)] is output in units of frames. k is an index of the frame number. x ^→ _m (k) = [x _m ((k−1) N + 1),..., x _m (kN)] (1 ≦ m ≦ M). N depends on the sampling frequency, but in the case of 16kHz sampling, around 512 points is reasonable.
The frequency domain transform unit 330 converts the digital signal x ^→ (k) of each frame into a frequency domain signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)]. Convert and output. ω is an index of discrete frequency. One method for converting a time domain signal to a frequency domain signal is a fast discrete Fourier transform, but the present invention is not limited to this, and other methods for converting to a frequency domain signal may be used. The frequency domain signal X ^→ (ω, k) is output for each frequency ω and for each frame k and accumulated in the storage unit 290.

[Step S16]
For each frequency ω∈Ω, the filter application unit 240 filters the frequency W of the frequency domain signal S (ω, k) corresponding to the position (θ _s , D _h ) to be reproduced ^→ (ω, θ _s). , D _h , k), and outputs the reproduction signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)] (see equation (42)). .

[Step S17]
The time domain conversion unit 250 uses the reproduction signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k)] of each frequency ω∈Ω of the kth frame in the time domain. The frame unit time domain signal x ^→ (k) = [x ₁ (k),..., X _M (k)] of the k-th frame is obtained by conversion, and the obtained frame unit time domain signal x ^→ ( k) = [x ₁ (k),..., x _M (k)] are connected in the order of the index of the frame number, and the time domain signal x ^→ in which the sound is emphasized toward the position (θ _s , D _h ) (t) = [x ₁ (t), ..., x _M (t)] is output. The method of converting the frequency domain signal into the time domain signal is an inverse transform corresponding to the transform method used in the process of step S14, and is, for example, a fast discrete inverse Fourier transform.

[Step S18]
M channel time domain signals x ₁ (t),..., X _M (t) are reproduced by the speakers corresponding to the channel among the M speakers 280-1,. Is done. That is, the time domain signal x _m (t) of the m-th (1 ≦ m ≦ M) channel is reproduced by the m-th speaker 280-m.

A description will be given of an experimental result of the audio spot reproduction according to the first embodiment of the present invention (minimum variance distortion-free response method under a single constraint condition). As shown in FIG. 5, 24 speakers are arranged linearly, and the reflector 300 is arranged so that the arrangement direction of the speakers included in the linear speaker array is a normal line of the reflector 300. Although there is no restriction | limiting in the shape of the reflecting plate 300, the reflecting surface is a plane, The flat reflecting plate which has a size of 1.0m x 1.0m, moderate thickness, and rigidity was used. The interval between adjacent speakers was 4 cm, and the reflectance α of the reflector 300 was 0.8. Listening position was set 45 degrees direction theta _s, the distance D _h to 1.13 m. As experimental results, the directivity (two-dimensional region) of the minimum dispersion beam former when the reflector 300 is not installed in FIG. 7A, and the minimum dispersion beam when the reflector 300 is installed in FIG. 7B. Indicates the directivity of the former (two-dimensional area). The sound pressure [unit: dB] is displayed in shades, and the white area indicates that sound can be collected with higher sound pressure. Therefore, ideally, only the position of 45 degrees in the direction and the distance of 1.13 m is white, and it can be said that the spot reproduction of the sound can be realized as the other area becomes black. From the comparison of the experimental results shown in FIG. 7A and FIG. 7B, when the reflector 300 is not installed, the sound spot reproduction is not sufficient, and when the reflector 300 is installed, the sound is reproduced. It can be seen that spot playback is possible.

  Next, an exemplary embodiment of the present invention will be described with reference to FIGS. In these examples, the configuration of the speaker array is illustrated as a linear speaker array, but is not limited to the configuration of the linear speaker array.

  In the embodiment shown in FIG. 8, the M speakers 280-1,..., 280-M constituting the linear speaker array are fixed to a rectangular flat plate-like support member 400. Are arranged on a certain plane (hereinafter referred to as an opening surface) of the support member 400 (M = 13 in the illustrated example). In addition, the wiring connected to each speaker 280-1, ..., 280-M is not illustrated. And the reflector 300 is being fixed to the edge part of the supporting member 400 so that the arrangement direction of each speaker 280-1, ..., 280-M may become the normal line of the reflector 300 of a rectangular flat plate shape. The opening surface of the support member 400 is a surface that forms 90 degrees with the reflector 300. 8, the preferable property of the reflector 300 is the same as the property of the reflector described above, and the property of the support member 400 is not particularly limited, and each speaker 280-1,. It is sufficient that the 280-M has sufficient rigidity to fix the 280-M.

  In the exemplary embodiment shown in FIG. 9A, the shaft portion 410 is fixed to the end portion of the support member 400, and the reflection plate 300 is rotatably attached to the shaft portion 410. According to this embodiment, it is possible to change the geometric arrangement of the reflector 300 with respect to the speaker array.

  In the exemplary embodiment shown in FIG. 9B, two additional reflectors 310 and 320 are added to the exemplary embodiment shown in FIG. The properties of the two added reflectors 310 and 320 may be the same as or different from those of the reflector 300. The properties of the reflector 310 may be the same as or different from those of the reflector 320. Hereinafter, the reflection plate 300 is referred to as a fixed reflection plate 300. The shaft 510 is fixed to the end of the fixed reflector 300 (the end opposite to the end of the fixed reflector 300 fixed to the support member 400), and the reflector 310 is rotated around the shaft 510. It is attached movably. Further, the shaft portion 520 is fixed to the end portion of the support member 400 (the end portion opposite to the end portion of the support member 400 to which the fixed reflection plate 300 is fixed), and the reflection plate 320 is attached to the shaft portion 520. It is pivotally attached. Hereinafter, the reflectors 310 and 320 are referred to as the movable reflectors 310 and 320. According to the embodiment shown in FIG. 9B, for example, when the position of the movable reflector 310 is set so that the reflection surface of the fixed reflector 300 and the reflection surface of the movable reflector 310 coincide with each other, the fixed reflector 300 and the movable reflector 300 are movable. The combination of the reflecting plates 310 can be made to function as a reflecting plate having a reflecting surface larger than that of the fixed reflecting plate 300. 9B, by setting the movable reflectors 310 and 320 to appropriate positions, for example, as shown in FIG. 10, the support member 400, the fixed reflector 300, and the movable reflector. Since the sound can be reflected many times in the space surrounded by 310 and 320, the number of reflected sounds can be controlled. In the case of the embodiment shown in FIG. 9B, the support member 400 serves as a reflector, and therefore preferably has the same properties as the properties of the reflector described above.

  11 differs from the embodiment shown in FIG. 8 in that the reflector 300 is also provided with a speaker array (linear speaker array in the illustrated example). In the exemplary embodiment shown in FIG. 11, the arrangement direction of the M speakers fixed to the support member 400 and the arrangement direction of the M ′ speakers fixed to the reflection plate 300 are on the same plane. The arrangement is not limited (M ′ = 13 in the illustrated example). For example, M ′ speakers may be fixed to the reflection plate 300 so as to have an arrangement direction orthogonal to the arrangement direction of M speakers fixed to the support member 400. 11, the speaker array provided on the support member 400 and the reflector 300 (the speaker array provided on the reflector 300 is not used and the reflector 300 is used as a reflector). The present invention is implemented in combination, or a combination of the support member 400 (the speaker member provided on the support member 400 is not used and the support member 400 is used as a reflector) and the speaker array provided on the reflector 300 The present invention can be implemented.

  Further, as an example of an extended implementation configuration of the implementation configuration example shown in FIG. 11, a configuration in which two reflectors 310 and 320 are further added to the implementation configuration example shown in FIG. 11, as in the implementation configuration example shown in FIG. 9B. (See FIG. 12). Although not shown, a speaker array may be provided on at least one of the movable reflectors 310 and 320. The loudspeaker holes of each speaker constituting the speaker array provided in the movable reflector 310 are arranged on the plane (opening surface) of the movable reflector 310 that can be opposed to the opening surface of the support member 400, for example. The loudspeaker holes of each speaker constituting the speaker array provided in the movable reflector 320 are arranged on the plane (opening surface) of the movable reflector 320 that can form the same plane as the opening surface of the support member 400, for example. Even in this embodiment configuration example, the same usage pattern as the embodiment configuration example shown in FIG. 9B is possible. Further, according to this embodiment, for example, when the position of the movable reflecting plate 320 is set so that the opening surface of the supporting member 400 and the opening surface of the movable reflecting plate 320 coincide with each other, the combination of the supporting member 400 and the movable reflecting plate 320 is changed. The speaker array can be made to function larger than the speaker array provided on the support member 400. Also in the embodiment configuration example shown in FIG. 12, in the embodiment configuration example in which at least one of the movable reflectors 310 and 320 is provided with a speaker array, the same usage pattern as in the embodiment configuration example shown in FIG. 10 is possible. Also in the example of the configuration shown in FIG. 12, in the example of the configuration in which the speaker array is provided on at least one of the movable reflectors 310 and 320, for example, the movable reflectors 310 and 320 are used as normal reflectors. A use form in which the speaker array provided on the support member 400 and the speaker array provided on the fixed reflector 300 are used as an integral speaker array is also possible. In this case, this is equivalent to an example of an embodiment in which a speaker array composed of (M + M ′) speakers and two reflectors are used.

  When the movable reflector 310 is provided with a speaker array, the loudspeaker holes of the speakers constituting the speaker array provided in the movable reflector 310 are opposite to the plane of the movable reflector 310 that can face the opening surface of the support member 400. A speaker array may be provided on the movable reflector 310 so as to be arranged on a flat surface (opening surface). Further, when a speaker array is provided on the movable reflector 320, the loudspeaker holes of the speakers constituting the speaker array provided on the movable reflector 320 can be formed on the same plane as the opening surface of the support member 400. A speaker array may be provided on the movable reflector 320 so as to be disposed on a plane (opening surface) opposite to the plane. Of course, at least one of the movable reflectors 310 and 320 may be provided with a speaker array on the movable reflector so as to have openings on both sides thereof.

[A] When the speaker array is provided on at least one of the movable reflectors 310 and 320, and the opening surface of the movable reflector 310 is a plane that can face the opening surface of the support member 400, or the movable reflector. When the opening surface of 320 is a plane that can form the same plane as the opening surface of the support member 400, in the usage mode shown in FIG. 10, the opening surface of the movable reflector 310 and / or the movable reflector 320 with respect to the line-of-sight direction. Although the apparent array size in the line-of-sight direction is reduced by disposing the movable reflector 310 and / or the movable reflector 320 so as not to be visible, the movable reflector 310 and / or the movable reflector 320 is provided. By using the speaker array, the same effect as when the array size is increased can be obtained.

[B] When the speaker array is provided on at least one of the movable reflectors 310 and 320, the opening surface of the movable reflector 310 is a plane opposite to the plane that can face the opening surface of the support member 400. If the opening surface of the movable reflecting plate 320 is a plane opposite to the plane that can form the same plane as the opening surface of the support member 400, in the usage mode shown in FIG. The same effect as when the array size is increased can be obtained while maintaining the size.

  When at least one of the movable reflectors 310 and 320 is provided with a speaker array on the movable reflector so that both sides thereof have an opening surface, the effects of both [A] and [B] can be obtained. Is possible.

<Application example>
Examples of services in which the audio spot reproduction processing technology of the present invention is useful will be described below.

  As a first example, there is an audio reproduction by digital signage. According to the present invention, since voice can be provided only in a narrow range in a specific direction as compared with the prior art, an advertisement can be transmitted only to people in the range without causing trouble to the surroundings.

  A second example is application to a TV conference system (which may be an audio conference system). According to the present invention, audio can be provided only in a narrow range in a specific direction as compared with the conventional case when a conference is performed in a situation where a room for exclusive use of a video conference cannot be prepared. It can be carried out.

<Hardware configuration example of audio spot reproduction processing device>
The audio spot reproduction processing apparatus according to the above-described embodiment may include an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a CPU (Central Processing Unit) [cache memory, or the like. ] RAM (Random Access Memory) or ROM (Read Only Memory) and external storage device as a hard disk, and data exchange between these input unit, output unit, CPU, RAM, ROM, and external storage device It has a bus that can be connected. If necessary, the audio spot reproduction processing device may be provided with a device (drive) that can read and write a storage medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the audio spot reproduction processing device stores a program for emphasizing a narrow range of audio and data necessary for the processing of the program [not limited to the external storage device, for example, reading a program It may be stored in a ROM which is a dedicated storage device. ]. Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device. Hereinafter, a storage device that stores data, addresses of storage areas, and the like is simply referred to as a “storage unit”.

  In the storage unit of the audio spot reproduction processing device, a program for obtaining a filter for each frequency using the spatial correlation matrix represented by the expressions (5a) to (5b) for the position to be subjected to spot reproduction, Corresponds to a program for performing AD conversion on analog signals, a program for performing frame generation processing, a program for converting a digital signal for each frame into a frequency domain signal in the frequency domain, and a position to be subjected to spot reproduction A program for obtaining a reproduction signal by applying a filter to the frequency domain signal for each frequency and a program for converting the reproduction signal into a time domain signal are stored.

  In the audio spot reproduction processing apparatus, each program stored in the storage unit and data necessary for processing each program are read into the RAM as necessary, and are interpreted and executed by the CPU. As a result, the audio spot reproduction process is realized by the CPU realizing predetermined functions (filter design unit, AD conversion unit, frame generation unit, frequency domain conversion unit, filter application unit, time domain conversion unit).

<Supplementary note>
The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  Further, when the processing functions in the hardware entity (audio spot reproduction processing device) described in the above embodiment are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (15)

When M is an integer of 2 or more, the sound source signal is an M channel time domain signal x reproduced by a speaker array composed of M speakers for a desired position (spot reproduction position) determined by the direction and distance. The frequency domain signal S converted into the frequency domain is applied to the frequency domain signal S obtained by applying the filter for converting the frequency domain signal S to the frequency domain signal X of the M channel for each frequency. An audio spot reproduction processing method obtained by converting to
Transmission characteristics of audio from each speaker to each position (where i is the direction for identifying each position and g is the distance) included in one or more positions assumed as audio spot playback positions a filter design step for obtaining the filter for each frequency with respect to a position to be reproduced by using a _{i, g} ;
Applying the filter obtained in the filter design step to the frequency domain signal S for each frequency to obtain the M channel frequency domain signal X, and
The sound radiated from the speaker array and having the listening position in the traveling direction of the reflected sound reflected by the reflector is a dual sound, and each of the transfer characteristics a _{i, g} has the direction i and An audio spot reproduction processing method characterized by being expressed as a sum of a transfer characteristic of a direct sound to a position determined by the distance g and each transfer characteristic of one or more dual sounds. The audio spot reproduction processing method according to claim 1,
Each of the transfer characteristics a _{i, g} is the direct sound steering vector and the one or more dual sound steering vectors in which the attenuation of the sound due to reflection and the time difference between the reflected sound and the direct sound are corrected. An audio spot reproduction processing method characterized by being a sum. The audio spot reproduction processing method according to claim 1,
Each of the transfer characteristics a _{i, g} is obtained by actual measurement in an actual environment. In the audio spot reproduction processing method according to any one of claims 1 to 3,
The audio spot reproduction processing method characterized in that, in the filter design step, the filter is obtained for each frequency so that the power of audio to a position other than the position to be reproduced is minimized. In the audio spot reproduction processing method according to any one of claims 1 to 3,
An audio spot reproduction processing method characterized in that, in the filter design step, the filter is obtained for each frequency so that the SN ratio at the position to be reproduced is maximized. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the power of the sound other than the one or a plurality of positions assumed as the spot reproduction position with the filter coefficient for one speaker among the M speakers fixed to a constant value is minimized. The audio spot reproduction processing method, wherein the filter is obtained for each frequency. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the sound reproduction is performed under the conditions of (1) full band transmission of the sound to the position to be reproduced, and (2) full band suppression of the sound to one or more suppression points. An audio spot reproduction processing method, wherein the filter is obtained for each frequency so that the power of audio to the position other than the position and the suppression points is minimized. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the filter is obtained for each frequency by normalizing the transfer characteristics a _{s, h} at the positions i = s, g = h to be reproduced. Spot playback processing method. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the filter is obtained for each frequency using the spatial correlation matrix represented by the transfer characteristics a _{i, g} corresponding to each position other than the position to be reproduced. A featured audio spot reproduction processing method. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the power of the sound to the position other than the position to be reproduced is minimized under the condition that the deterioration amount of the sound at the position to be reproduced is not more than a predetermined amount. An audio spot reproduction processing method, wherein the filter is obtained for each frequency. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the filter is obtained for each frequency by using a spatial correlation matrix represented by a frequency domain signal obtained by converting a signal obtained by observing with a microphone array into a frequency domain. A featured audio spot reproduction processing method. In the audio spot reproduction processing method according to any one of claims 1 to 3,
In the filter design step, the spatial correlation matrix represented by the transfer characteristics a _{i, g} corresponding to each position included in one or a plurality of positions assumed as spot reproduction positions is used for each frequency. An audio spot reproduction processing method characterized in that a filter is required. When M is an integer of 2 or more, the sound source signal is an M channel time domain signal x reproduced by a speaker array composed of M speakers for a desired position (spot reproduction position) determined by the direction and distance. The frequency domain signal S converted into the frequency domain is applied to the frequency domain signal S obtained by applying the filter for converting the frequency domain signal S to the frequency domain signal X of the M channel for each frequency. An audio spot reproduction processing device obtained by converting to
Transmission characteristics of audio from each speaker to each position (where i is the direction for identifying each position and g is the distance) included in one or more positions assumed as audio spot playback positions a filter design unit for obtaining the filter for each frequency with respect to a position to be reproduced by using a _{i, g} ;
A filter application unit that obtains the M channel frequency domain signal X by applying the filter obtained by the filter design unit to the frequency domain signal S for each frequency;
The sound radiated from the speaker array and having the listening position in the traveling direction of the reflected sound reflected by the reflector is a dual sound, and each of the transfer characteristics a _{i, g} has the direction i and An audio spot reproduction processing device characterized by being expressed as a sum of a transfer characteristic of a direct sound to a position determined by the distance g and each transfer characteristic of one or more dual sounds. In the audio spot reproduction processing device according to claim 13,
An audio spot reproduction processing apparatus, further comprising one or more reflectors that give each of the reflected sounds to the dual sound. A program for causing a computer to execute processing of the audio spot reproduction processing method according to any one of claims 1 to 12.