JP7555543B2 - SOUND PROCESSING DEVICE, SOUND PROCESSING SYSTEM, SOUND PROCESSING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to an audio processing device, an audio processing system, an audio processing method, and a program.

Sound source localization is a method for estimating the direction of a sound source from multiple channel acoustic signals collected using a microphone array. Various methods have been proposed as elemental technologies for acoustic signal processing. A single microphone array can estimate the direction of a sound source, but cannot estimate the position of the sound source. In recent years, methods have been proposed for estimating the position of a sound source using multiple microphone arrays.

However, each microphone array has individual differences. When using multiple microphone arrays, it is necessary to calibrate the position, orientation, and time between the multiple microphone arrays. Non-Patent Document 1 describes a method for estimating the position, orientation, and sound source position of an unknown microphone array from the observed sound source direction, using an objective function that indicates the relationship between the position and orientation of the microphone array, and the position of the sound source.

S. Wozniak and K. Kowalczyk: “Passive Joint Localization and Synchronization of Distributed Microphone Arrays”, IEEE Signal Processing Letters, Volume 26, Issue 2, pp.292-296, February 2019

However, with the objective function described in Non-Patent Document 1, it was sometimes difficult to estimate the position and orientation of the microphone array, as well as the position of the sound source. For example, with this objective function, it was not possible to determine the microphone array position or the position of the sound source to the extent of their magnitude. In addition, the objective function sometimes fell into a local optimum solution, making it difficult to obtain a more appropriate solution.

This embodiment has been made in consideration of the above points, and aims to provide an audio processing device, an audio processing system, an audio processing method, and a program that can simultaneously estimate the position and orientation of a microphone array, and the position of a sound source more accurately.

(1) The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is an acoustic processing device including an estimation unit that determines the position, direction, and offset time of the microphone array and the position of the sound source so that the sum of a function value of a first objective function indicating the magnitude of a directional difference, which is the difference between an observed sound source direction, which is the sound source direction determined for each of N (N is an integer of 2 or more), microphone arrays, and the direction of the sound source from the microphone array rotated around the direction of the microphone array, and a function value of a second objective function indicating the magnitude of a time difference, which is the difference between the time difference in propagation time of sound waves from the sound source to each microphone array and the time difference between the offset time of each microphone array, from an observed arrival time difference, which is the arrival time difference of sound from the sound source determined for each pair of two of the N microphone arrays, is smaller .

(2) Another aspect of the present invention is a method for determining the position, direction, offset time, and front time of the microphone arrays so that a function value of a first objective function indicating the magnitude of a directional difference, which is the difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer of 2 or more), microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array, and a function value of a second objective function indicating the magnitude of a time difference, which is the difference between a time difference of a sound wave propagation time from the sound source to each microphone array and a time difference of an offset time of each microphone array, from an observed arrival time difference, which is the arrival time difference of a sound from the sound source determined for each pair of two microphone arrays out of the N, are smaller. The sound processing device is characterized in that it includes an estimation unit that determines the position of the sound source, and the estimation unit determines the position, direction, and offset time of the microphone array and the position of the sound source so that the sum of the distances from the first position to the second position between the pair of microphone arrays is smaller, the first position being a position obtained by rotating the position of the sound source relative to the microphone array in the direction of the microphone array, and the second position being a position away from one microphone array of the pair of microphone arrays in the direction of the observed sound source, a distance equivalent to the difference in sound propagation distance in the time difference from the distance from the other microphone array of the pair of microphone arrays to the sound source.

(3) Another aspect of the present invention is a function value of a first objective function indicating the magnitude of a directional difference, which is the difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer of 2 or more) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array; and a second objective function indicating the magnitude of a time difference, which is the difference between a time difference between a propagation time of a sound wave from the sound source to each microphone array and a time difference between an offset time of each microphone array, from an observed arrival time difference, which is the arrival time difference of a sound from the sound source determined for each pair of two microphone arrays out of the N. The sound processing device includes an estimation unit that determines the position, orientation, and offset time of the microphone array and the position of the sound source so that the function value of the first objective function becomes smaller, respectively, and the estimation unit alternately executes a step of updating the position, orientation, and offset time of the microphone array and the position of the sound source so that the function value of the first objective function becomes smaller, and a step of updating the position, orientation, and offset time of the microphone array and the position of the sound source so that the function value of the second objective function becomes smaller .

(4) Another aspect of the present invention may be an acoustic processing system including an acoustic processing device according to any one of (1) to (3) , a microphone array, a sound source direction observation unit that determines the observed sound source direction based on an acoustic signal picked up by each of the microphone arrays, and an arrival time difference observation unit that determines the observed arrival time difference for each pair.

(5) Another aspect of the present invention may be a program for causing a computer to function as any one of the sound processing devices (1) to (3) .

(6) Another aspect of the present invention is an acoustic processing method comprising a step of determining a position, direction, and offset time of the microphone arrays, and a position of the sound source, so as to reduce the sum of a function value of a first objective function indicating the magnitude of a directional difference, which is the difference between an observed sound source direction, which is the sound source direction determined for each of N (N is an integer of 2 or more), microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array around the direction of the microphone array, and a function value of a second objective function indicating the magnitude of a time difference, which is the difference between an observed arrival time difference, which is the arrival time difference of sound from the sound source, and a time difference between the propagation time of the sound wave from the sound source to each microphone array and the offset time of each microphone array, from the observed arrival time difference, which is the arrival time difference of sound from the sound source, and
(7) In another aspect of the present invention, the position, direction, and offset time of the microphone arrays are determined so that a function value of a first objective function indicating the magnitude of a directional difference, which is the difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer of 2 or more), microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array, and a function value of a second objective function indicating the magnitude of a time difference, which is the difference between a time difference of a sound wave propagation time from the sound source to each microphone array and a time difference of an offset time of each microphone array, from an observed arrival time difference, which is the arrival time difference of a sound from the sound source determined for each pair of two microphone arrays out of the N, are smaller. and a step of determining a position of the sound source, the step determining the position, direction, and offset time of the microphone array, and the position of the sound source such that a sum of distances from a first position to a second position between the pair of microphone arrays is smaller, the first position being a position obtained by rotating the position of the sound source relative to the microphone array in the direction of the microphone array, and the second position being a position away from one microphone array of the pair of microphone arrays in the observed sound source direction, a distance equivalent to a difference in sound propagation distance in the time difference from a distance from the other microphone array of the pair of microphone arrays to the sound source.
(8) Another aspect of the present invention is a method for detecting a time difference between a sound source and an offset time of the sound wave from the sound source, the time difference being a difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer of 2 or more), microphone arrays, and a direction difference being a difference between the direction of the sound source from the microphone array and a direction obtained by rotating the direction of the microphone array with respect to the direction of the microphone array; and a second objective function indicating a magnitude of a time difference between a time difference between a propagation time of a sound wave from the sound source to each microphone array and a time difference between an offset time of each microphone array, the time difference being a difference between an observed arrival time difference, which is a difference in arrival time of a sound from the sound source, determined for each pair of two microphone arrays among the N. The acoustic processing method includes a step of determining the position, orientation, and offset time of the microphone array and the position of the sound source so that the function values of the objective functions become smaller, respectively, and the step alternately includes a step of updating the position, orientation, and offset time of the microphone array and the position of the sound source so that the function value of the first objective function becomes smaller, and a step of updating the position, orientation, and offset time of the microphone array and the position of the sound source so that the function value of the second objective function becomes smaller.

According to the above-mentioned configurations (1) to (8) , the second objective function and the first objective function are constraints on the first objective function and the second objective function, respectively, so that the uncertainty of the solution that may be included in the position of the microphone array and the position of the sound source estimated based on the first objective function and the uncertainty of the solution that may be included in the direction of the sound source from the microphone array estimated based on the second objective function are eliminated. Therefore, by making the function values of the first objective function and the second objective function smaller, it is possible to avoid a local optimum solution and more accurately determine the position, direction, and offset time of the microphone array and the position of the sound source.

In the above-mentioned configurations (1) and (6) , the sum of the function values of the first objective function and the second objective function is an index indicating the accuracy of the estimated microphone array position, direction, offset time, and sound source position. In addition, the microphone array position, sound source position, and direction from the microphone array to the sound source estimated using the sum do not include uncertainty. Therefore, it is possible to avoid a local optimum solution and more accurately determine the microphone array position, direction, offset time, and sound source position.

In the above-mentioned configurations (2) and (7) , the sum of the distances between the first and second positions of the microphone array pairs also serves as an index of the accuracy of the estimated microphone array position, direction, and offset time, and the sound source position. In addition, the microphone array position, sound source position, and sound source direction from the microphone array estimated using the sum do not include uncertainty. Therefore, it is possible to avoid a local optimum solution and more accurately determine the microphone array position, direction, offset time, and sound source position.

According to the above-mentioned configurations (3) and (8) , the step of making the function value of the first objective function smaller and the step of making the function value of the second objective function smaller are alternately executed, so that it is possible to avoid a local optimum solution and an indefinite solution that may occur if any of the steps continues. Therefore, it is possible to more accurately determine the position, direction, and offset time of the microphone array, and the position of the sound source.

1 is a block diagram showing an example of the configuration of a sound processing system according to an embodiment of the present invention. FIG. 11 is an explanatory diagram for explaining a first objective function. FIG. 11 is an explanatory diagram for explaining a second objective function. 5 is a flowchart showing a first example of an estimation process according to the present embodiment. 13 is a flowchart showing a second example of the estimation process according to the embodiment. 13 is a flowchart showing a third example of the estimation process according to the embodiment. FIG. 2 is a diagram showing an example of microphone arrangement. FIG. 13 is a diagram showing another example of microphone arrangement. FIG. 11 is a block diagram showing another example of the configuration of the sound processing system according to the embodiment. FIG. 13 is a diagram illustrating an example of a change in the average error. FIG. 13 is a diagram illustrating an example of a calculation time. FIG. 13 is a diagram showing another example of a change in the average error. FIG. 13 is a diagram illustrating another example of the calculation time.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing an example of the configuration of a sound processing system S1 according to this embodiment.
The sound processing system S1 includes a sound processing device 10 and a sound collection unit 20.

The sound collection unit 20 has a plurality of microphone arrays. The number N of the microphone arrays is an integer equal to or greater than 2. Each microphone array is installed at a different position facing a different direction. In this application, the position and direction of the microphone array may be referred to as the "array position" and the "array direction", respectively. Also, each microphone array may be distinguished by being assigned an index such as _A1 , _A2 , etc. When there is no need to distinguish between the individual microphone arrays or matters common to the N microphone arrays, they may be simply referred to as the "microphone array" or "microphone array A".

Each microphone array A includes multiple microphones. Each microphone is arranged at a different position and picks up sound waves arriving at that microphone. Each microphone includes an actuator that converts the incoming sound waves into an acoustic signal, and outputs the converted acoustic signal to the sound processing device 10. In this application, the unit of the acoustic signal picked up by each microphone is called a "channel." The microphone array A illustrated in FIG. 7 includes L microphones mic1 to micL arranged at equal intervals on a straight line. The spatial arrangement of the multiple microphones in the microphone array A is not limited to this. As illustrated in FIG. 8, the multiple microphones that make up the microphone array A may be arranged on an arc.

The direction of the microphone array may be any direction that serves as a reference for the sound source direction of the microphone array. The direction of the microphone array may be, but is not limited to, the direction in which the microphones are arranged. Furthermore, this application is based on the premise that the acoustic signals picked up by each microphone array are not synchronized. Therefore, each microphone array has an offset time. The offset time corresponds to the time difference between the time of each microphone array and the reference time for the entire acoustic processing system S1.

The sound processing device 10 determines the direction of a sound source as an observed sound source direction using the multi-channel sound signals acquired from each of the N microphone arrays (sound source localization). In this application, the direction of a sound source is called a "sound source direction", and in particular, the sound source direction determined from the multi-channel sound signals may be called an "observed sound source direction". In addition, the position of a sound source may be called a "sound source position".
The sound processing device 10 determines the difference in the arrival time of sound from the sound source to each of the microphone arrays as the observed arrival time difference (arrival time difference observation) for each pair of microphone arrays consisting of two microphone arrays out of N. In this application, a pair of microphone arrays may be referred to as an "array pair."

The sound processing device 10 estimates the array position, array direction, offset time, and sound source position as unknown information so that the function values of the first objective function and the second objective function are smaller, based on the observed sound source direction for each microphone array and the observed arrival time difference for each array pair. The first objective function and the second objective function will be described later.
The sound processing device 10 outputs output information indicating some or all of the estimated array position, array direction, offset time, and sound source position to the output destination device 30.
The output destination device 30 is a device to which some or all of the estimated array position, array direction, offset time, and sound source position are output. The output destination device 30 may be an information and communication device such as a PC (Personal Computer) or a multi-function mobile phone, or may be a measuring instrument, a monitoring device, or the like.

Next, an example of the functional configuration of the sound processing device 10 according to the present embodiment will be described.
The sound processing device 10 includes an input/output unit 110 and a control unit 120 .
The input/output unit 110 is connected wirelessly or wired to other devices so as to input and output various data. The input/output unit 110 outputs, as input data, acoustic signals of multiple channels input from each microphone array A to the control unit 120. The input/output unit 110 outputs, as output data, estimated information input from the control unit 120 to an output destination device. The input/output unit 110 may be, for example, any one of an input/output interface, a communication interface, and the like, or a combination thereof.

The control unit 120 executes processes for implementing the functions of the sound processing device 10, processes for controlling the functions, etc. The control unit 120 may be configured using a dedicated member, or may be configured as a computer system including a processor such as a CPU (Central Processing Unit) and various storage media. The processor reads out a predetermined program stored in advance in the storage medium, and executes processes instructed by various commands written in the read out program to implement the functions of the control unit 120.
The control unit 120 includes a sound source direction observation unit 122 , an arrival time difference observation unit 124 , an estimation unit 126 , and an output processing unit 128 .

The sound source direction observation unit 122 performs sound source localization processing based on the multi-channel acoustic signals input from each microphone array A using the input/output unit 110 to determine the observed sound source direction, which is the observed value of the sound source direction of each sound source. In the sound source localization processing, the sound source direction observation unit 122 can use methods such as the MUSIC (Multiple Signal Classification) method and the WDS-BF (Weighted Delay and Sum Beam Forming) method. The sound source direction observation unit 122 outputs sound source direction information indicating the observed sound source direction determined for each microphone array A to the estimation unit 126.

The arrival time difference observation unit 124 determines, for each array pair, an observed arrival time difference that is an observed value of the difference in arrival time between when a sound wave arrives from a sound source to each microphone array A, based on the multi-channel acoustic signals input from each microphone array A via the input/output unit 110. The arrival time difference observation unit 124 outputs to the estimation unit 126 arrival time difference information indicating the observed arrival time difference determined for each array pair.
When determining the observed arrival time difference, the arrival time difference observation unit 124 calculates, for example, the cross-correlation between an acoustic signal of one channel of one microphone array forming an array pair and a delayed signal obtained by delaying an acoustic signal of one channel of the other microphone array for each delay amount for each channel pair. A channel pair corresponds to a set of one channel of one microphone array and one channel of the other microphone array. The arrival time difference observation unit 124 determines, as the observed arrival time difference, an average delay amount obtained by averaging the delay amounts at which the cross-correlation is maximized between the channel pairs.

The number of sound sources in the acoustic space is not limited to one, but can be multiple. Therefore, multiple arrival time differences can be determined for each microphone array. There is a risk that the arrival time difference of sound waves from different sound sources between different microphone arrays may be erroneously estimated. However, the arrival time difference observation unit 124 may discard the delay amount that gives a cross-correlation that is equal to or less than a predetermined maximum value threshold, and adopt the delay amount that gives a cross-correlation that is equal to or greater than that threshold when determining the arrival time difference. This avoids determining the arrival time difference using the arrival times from different sound sources between microphone arrays, and the arrival time difference observation unit 124 can determine the arrival time difference using the arrival time from a sound source that is common between microphone arrays.

The estimation unit 126 receives the sound source direction information from the sound source direction observation unit 122 and the arrival time difference information from the arrival time difference observation unit 124 .
The estimation unit 126 determines the array position, array direction, offset time, and sound source position, which are estimated values, so that the function value of the first objective function and the function value of the second objective function are minimized, based on the observed sound source direction for each microphone array indicated by the sound source direction information and the observed arrival time difference for each array pair indicated by the arrival time difference information. In the "minimization", for example, a gradient method can be applied. In this application, "minimization" and "optimization" each include the meaning of searching for an estimated value so that the function value is as small as possible and approximates an optimal value, and are not limited to determining an estimated value at which the function value is absolutely minimum or optimal. Therefore, in the process of "minimization" and "optimization", the function value may temporarily increase or deviate from the optimal value. The estimation unit 126 outputs estimation information indicating the determined array position, array direction, offset time, and sound source position to the output processing unit 128.

The output processing unit 128 outputs a part or all of the estimated information input from the estimation unit 126 as output information to the output destination device 30 via the input/output unit 110 .
For example, each time estimated information is input from the estimation unit 126, the output processing unit 128 outputs information on a predetermined item (e.g., sound source position) from the input estimated information to the output destination device 30 (push-type transmission). When request information is input from the output destination device 30, the output processing unit 128 may output information on an item specified by the request information to the output destination device 30 as a response (pull-type transmission). The output processing unit 128 may store part or all of the input estimated information in a storage medium of its own device without outputting it to an external device.

(Objective function)
Next, the objective function according to this embodiment will be described. FIG. 2 is an explanatory diagram for explaining the first objective function. The first objective function L ₁ is based on the relationship that the direction vector d _i,j indicating the observation sound source direction θ _i,j of the sound source S _i from the microphone array A _j corresponds to a unit vector obtained by rotating the direction vector to the position s _i of the sound source S _i based on the position a _j of the microphone array A _j in the direction θ _j of the microphone array A _j , as shown in formula (1). However, in the example shown in FIG. 2, the microphone array A _j is placed at the origin of the x-y Cartesian coordinate system, and the direction θ _j is directed to the x-axis direction.

In formula (1), _Rj denotes a rotation matrix related to the direction _θj of the microphone array _Aj . A two-dimensional rotation matrix is shown in formula (2). ... ^T denotes the transpose of a matrix ... or a vector .... _{s i} -a _j corresponds to a position vector indicating the position s _i of the sound source Si based on the position a _j of the microphone array _Aj . || ... || ₂ denotes the L2 norm of the vector ..., that is, the square root of the sum of the squares of the element values of each dimension.

Multiplying both sides of equation (1) by ||s _i -s _j || ₂ , further transferring the left side to the right side, and squaring both sides to obtain a function D, the first objective function L ₁ is derived by taking the sum of the microphone arrays. However, equation (3) is generalized to include the case where the number of sound sources M is 2 or more. The first objective function L ₁ forms the sum of the M sound sources.

As shown in equation (3), the first objective function _L1 is a function that gives, for each microphone array _Aj , a function value indicating the magnitude of the difference between the rotated coordinate value _RjT ( _si - _{aj) which} is the coordinate value of the position obtained by rotating the coordinate value ^si _{- aj} of the sound source position based on the microphone array _Aj in the array direction θj of the microphone array _Aj , and the observed sound source direction coordinate value d i, _j || _{si - aj} || ₂ which is the coordinate value of the position away from the origin in the directional vector d _i,j of the observed sound source direction by a distance equivalent to the distance ||si _{- aj} || ₂ from the microphone array Aj to the sound source _{S i} . In other words, the first objective function _L1 is a function that gives an index value indicating the magnitude of the _difference between the observed sound source direction θ _{i,j of} sound source S _i from microphone array A _j and the direction from microphone array A j to position s _i of sound source S i rotated in the direction θ _j of microphone array A _j .

Equation (1) shows that a unit vector obtained by normalizing the difference value of the coordinate value s _i of the sound source position from the coordinate value a _j of the array position is rotated in the direction θ _j of the microphone array A _j to obtain a directional vector d _i,j . Therefore, even if the coordinate value a _j of a certain array position and the coordinate value s _i of the sound source position are enlarged or reduced by a common magnification α (α is a real number other than 0), equation (1) holds. That is, the coordinate value a _j of the array position and the coordinate value s _i of the sound source position estimated by minimizing the first objective function L ₁ shown in equation (3) are left with an uncertainty of α times. The magnification α is also called a coordinate scale. In order to determine the magnification α, it is necessary to use a separate constraint condition. For example, the arrival time difference of sound waves between microphones and the distance between microphones can be used as separate constraint conditions. In this embodiment, the second objective function described next plays the role of a separate constraint condition.

3 is an explanatory diagram for explaining the second objective function. The second objective function is based on the relationship that the arrival time of the sound wave from the sound source S _i to the microphone array A _j and the observed arrival time difference τ _{i, (j, k)} from the sound source i to the microphone array A _k correspond to the time difference between the propagation time c ^-1 || s _i - a _j || ₂ of the sound wave from the sound source S _i to the microphone array A _j and the propagation time c ^-1 || s _i - a _k || ₂ of the sound wave from the sound source S _i to the microphone array A _k , and the time difference between the offset time δ _j of the microphone array A _j and the offset time δ _k of the microphone array A _k , as shown in formula (4). In formula (4), c indicates the speed of sound.

The second objective function _L2 is derived by multiplying both sides of equation (4) by the sound speed c, and then taking the sum of the function T obtained by shifting the left side to the right side between the array pairs. However, equation (5) is generalized to include the case where the number of sound sources M is 2 or more.

As shown in equation (5), the second objective function L ₂ is a function that gives, for each array pair consisting of two microphone arrays A _j and A _k , a function value indicating the magnitude of the sum of the distance difference from _a first distance ||s _i -a _j || ₂ from one microphone array A _j to a sound source S _i constituting the array pair to a second distance ||s _i -a _k || ₂ from the other microphone array A _k to the sound source _{S i} , and the distance difference cτ i,(j, _k ) of the sound traveling distance _c at the observed arrival time difference τ _i,(j,k) of the array pair from the sound traveling distance c(δ _j -δ _{k )} in the time difference from the offset time δ j of one microphone array A j to the offset time δ k of the other microphone array A k. That is, the second objective function _L2 is a function that gives an index value indicating the magnitude of the difference between the time difference c ^-1 ||s _{i -a k} || ₂ of the sound wave from sound source i to each of the microphone arrays A _j and A _k, based on the observed arrival time difference τ _{i,(j,k) between} the microphone arrays A _j and A k for the sound wave from sound source S _i , and the time difference from the offset time δ _j of microphone array A _j to the offset time δ _k of microphone array A k.

_Incidentally , formula (4) shows that the sound arrival time difference in the distance difference between the distance ||s _i -a _j || ₂ from the sound source i to the microphone array A _j and the distance ||s _i -a _k || ₂ from the sound source i to the microphone array A _k , and the sum of the time difference (δ _j -δ _{k ) between the offset time δ j of} the microphone array A _j and the offset time δ k of the microphone array A k can be obtained as the observed arrival time difference τ _i,(j,k) . Therefore, formula (4) is established even if the direction vector d _i,j indicating the direction from the microphone array A _j to the sound source _S i is multiplied by a rotation matrix γ common between the microphone arrays. That is, the individual direction vectors d _i,j estimated by minimizing the second objective function L ₂ shown in formula (5) include uncertainty with respect to rotation. In order to determine the rotation matrix γ, it is necessary to use separate constraint conditions. For example, the position of each sound source can be used as a separate constraint condition. If the number of microphone arrays N and the number of sound sources M are determined so that the number of unknowns and the number of equations derived from the second objective function _L2 match, the estimated information should be analytically determined. However, the error contained in the observed arrival time difference τ _i,(j,k) is a factor that reduces the accuracy of the estimated information. This also applies to the observed sound source direction θ _i,j .

Both the equations based on the first objective function _L1 and the second objective function _L2 are nonlinear equations, and multiple local optimal solutions exist. Therefore, it is desirable to avoid convergence to a local optimal solution. In this embodiment, it is possible to obtain estimated information based on both the first objective function _L1 and the second objective function _L2, which have different solution uncertainties. It is expected that one of the first objective function _L1 and the second objective function _L2 acts as a constraint condition for the other, thereby avoiding a local optimal solution.

(Estimation process)
Next, a method of estimating the estimated information using both the first objective function L ₁ and the second objective function L ₂ will be described. The estimated information is the position, direction, offset time, and sound source position for each microphone array. Here, the position, direction, and offset time of each microphone array may be relative to the position, direction, and offset time of a predetermined microphone array (for example, microphone array A ₁ ) that is used as a reference. In addition, in a two-dimensional space, one position is expressed by two coordinate values. Therefore, the number of unknowns that form the estimated information is 2M for the position of the sound source, 2(N-1) for the position of the microphone array, N-1 for the orientation of the microphone array, and N-1 for the offset time of the microphone array. In estimating the estimated information, the number of simultaneous equations needs to be equal to or greater than the number of unknowns, 2M+4(N-1). The simultaneous equations are equations in which the derivative obtained by partially differentiating the objective function used in the calculation with any of the unknowns is set to 0. For example, when N=3, the numbers of sound sources required for methods 1, 2, and 3 described below are 3, 3, and 6, respectively.

The method for estimating the estimated information may be any of the following methods 1 to 3.
Method 1 is a method of estimating the estimated information by simultaneously calculating the first objective function L ₁ and the second objective function L _2. In the present application, method 1 may be simply referred to as "simultaneous evaluation". More specifically, method 1 is a method of estimating the estimated information so that the simultaneous objective function S shown in formula (6) is smaller. The simultaneous objective function H is an objective function equivalent to the sum of the first objective function L ₁ and the second objective function L ₂ as shown in formula (6). Since both the first objective function L ₁ and the second objective function L ₂ are functions that give function values that are closer to 0 as the estimated information is estimated more correctly, it is expected that the simultaneous objective function H, which is the sum of these, will also be closer to 0 as the estimated information is estimated more correctly. According to method 1, the number of equations used in one calculation is large, so the number of sound sources related to the estimation is smaller than other methods. The number of equations that set the derivative of the simultaneous objective function H to 0 is M (2N-1).

The method 2 according to the present embodiment is a method of determining the estimated information so that the function value of the integrated objective function I obtained by integrating the first objective function L ₁ and the second objective function L ₂ becomes smaller. In the present application, the method 2 may be referred to as "integration of objective functions". The integrated objective function I is shown in formula (7). The integrated objective function I is obtained by eliminating the term related to ||s _i -a _j || ₂ , which is common to the first objective function L ₁ and the second objective function L _{2, from the first objective function L 1.} Since both the first objective function L ₁ and the second _objective function L ₂ are functions that give function values that are closer to 0 as the estimated information is estimated more accurately, the integrated objective function I derived from these functions also approaches 0 as the estimated information is estimated more accurately. In the method 2, the number of equations that set the derivative of the integrated objective function I to 0 is M (2N-1).

The integrated objective function I shown in equation (7) corresponds to the sum of the magnitude of the distance between the first position R _j ^T (s _i - a _j ) and the second position d _i,j (||s _i - _{a k} || ₂ - c δ _i + c δ _k + c τ _i,(j,k) ) between the array pairs. The first position R _j ^T (s _i - _{a j} ) corresponds to the position obtained by rotating _{the position a j of} the sound source based on the microphone array A j in the direction θ _j of the microphone array A _j . The second position d _i,j (||s _i -a _k || ₂ - cδ _i + cδ _k + cτ _i,(j,k) ) corresponds to a position that is away from one microphone array A _j of the array pair in the observed sound source direction indicated by the direction vector d _{i ,j by} a distance ||s _i -a _k || ₂ - cδ i + cδ _k + cτ _i ,(j,k) that corresponds to the difference in _sound propagation distance c(δ _j - δ _k ) at the time difference δ _{j - δ k} from the offset time δ _j of _one microphone array A j to the offset time δ _k of the other microphone _{array A k .}

The method 3 according to the present embodiment is a method of alternately repeating a process 3-1 for updating the estimated information so as to make the function value of the first objective function L ₁ smaller and a process 3-2 for updating the estimated information so as to make the function value of the second objective function L ₂ smaller. In the present application, the method 3 may be referred to as "iterative estimation". In the process 3-1, the time offset of the microphone array is not estimated. Therefore, the number of unknowns is 2M+3N-3. In contrast, the number of equations in which the derivative of the first objective function L ₁ is 0 is MN. In the process 3-2, the direction of the microphone array is not estimated. Therefore, the number of unknowns is M+3N-3. In contrast, the number of equations in which the derivative of the second objective function L ₂ is 0 is M(N-1). In the repetition of the processes 3-1 and 3-2, the process 3-2 may precede the process 3-1. This is because the first objective function _L1 has a larger number of equations relative to the number of unknowns, and therefore the conditions for stably finding a solution are stricter than those for the second objective function _L2 .

Next, the estimation process according to the present embodiment will be described. Fig. 4 is a flowchart showing a first example of the estimation process according to the present embodiment. The process shown in Fig. 4 includes a process of determining the estimation information using the method 1 in step S108.
(Step S102) The sound source direction observation unit 122 determines an observed sound source direction for each microphone array A based on acoustic signals of multiple channels.
(Step S104) The arrival time difference observing unit 124 determines an observed arrival time difference, which is the difference in time when sound arrives from the sound source to each microphone array, for each array pair.
(Step S106) The estimation unit 126 determines initial values of the unknowns, that is, the position, direction, and offset time of the microphone array, and the sound source position.

(Step S108) The estimation unit 126 updates the position, direction, and offset time of the microphone array, and the sound source position so that the simultaneous objective function H that gives the sum of the first objective function L1 and the second objective function L2 becomes smaller.
(Step S110) The estimation unit 126 determines whether the position, direction, and offset time of the microphone array, and the sound source position have converged. The estimation unit 126 can determine whether the convergence has occurred based on, for example, whether the magnitude of the difference between the objective function before the update and the objective function after the update, or between any of the estimated values, is equal to or less than a predetermined threshold value of the magnitude of the difference. When it is determined that the convergence has not occurred (NO in step S110), the process returns to step S108. When it is determined that the convergence has occurred (YES in step S110), the process proceeds to step S112.
(Step S112) The output processing unit 112 outputs output information indicating the position, direction, and offset time of the microphone array, which are estimated values, and some or all of the sound source position, to the output destination device 30. Then, the process shown in FIG. 4 ends.

Figure 5 is a flowchart showing a second example of the estimation process according to this embodiment. The process shown in Figure 5 includes steps S102, S104, S106, S110, S112, and S108'. The process shown in Figure 4 includes step S108' in which estimated information is determined using method 2. The explanation of Figure 4 applies to steps S102, S104, S106, S110, and S112. In the process shown in Figure 5, after step S106 is completed, the process proceeds to step S108'.

(Step S108') The estimation unit 126 updates the position, direction, offset time, and sound source position of the microphone array so that the function value of the integrated objective function _I obtained by integrating the first objective function _L1 and the second objective function L2 becomes smaller. The integrated objective function I corresponds to the sum of the magnitude of the distance from the first position to the second position between the array pair. The first position corresponds to a position obtained by rotating the position of the sound source based on the microphone array in the direction of the microphone array. The second position corresponds to a position away from one microphone array of the array pair in the observed sound source direction by a distance equivalent to the difference in the sound propagation distance in the time difference between the offset time of one microphone array and the offset time of the other microphone array from the distance from the other microphone array of the array pair to the sound source. Then, proceed to the processing of step S110.

Figure 6 is a flowchart showing a third example of the estimation process according to this embodiment. The process shown in Figure 5 includes steps S102, S104, S106, S110, S112, S108", and S109. The process shown in Figure 6 includes steps S108" and S109 for determining estimated information using method 3. The explanation of Figure 4 is applied to steps S102, S104, S106, S110, and S112. In the process shown in Figure 6, after step S106 is completed, the process proceeds to step S108".

(Step S108'') The estimation unit 126 updates the position, direction, and offset time of the microphone array, and the sound source position so as to reduce the function value of the second objective function _L2 .
(Step S109) The estimation unit 126 updates the position, direction, and offset time of the microphone array, and the sound source position so as to reduce the function value of the first objective function L _1. Then, the process proceeds to step S110.

In the process shown in Figures 4 to 6, instead of omitting step S110, the process may proceed to step S112 after repeating steps S108, S108', or S108'' and S109 a predetermined number of times.

(Sound source localization)
Next, the MUSIC method and the WDS-BF method, which are methods for sound source localization, will be described.
The MUSIC method is a method for determining the direction θ where the spatial spectrum power P _ext (θ) is maximum and higher than a predetermined level as the sound source direction. The sound source direction observation unit 122 stores transfer functions for each direction θ distributed at predetermined intervals (for example, 1° to 3°) in advance.

The sound source direction observation unit 122 generates a transfer function vector [G(θ)] for each direction θ, whose elements are transfer functions G[q](ω) from the sound source to the microphone corresponding to each channel q (q is an integer greater than or equal to 1 and less than or equal to Q).
The sound source direction observation unit 122 calculates a transformation coefficient ξ _q (ω) by transforming the acoustic signal ξ _q of each channel q into the frequency domain for each frame consisting of a predetermined number of elements. The sound source direction observation unit 122 calculates an input correlation matrix [R _ξξ ] shown in equation (8) from an input vector [ξ(ω)] including the calculated transformation coefficients as elements.

In formula (8), E[...] denotes the expected value of.... [...] denotes that... is a matrix or a vector. [...] ^* denotes the conjugate transpose of a matrix or a vector.
The sound source direction observation unit 122 calculates the eigenvalue δ _p and eigenvector [ε _p ] of the input correlation matrix [R _ξξ ]. The input correlation matrix [R _ξξ ], the eigenvalue δ _p , and the eigenvector [ξ _p ] have the relationship shown in Equation (9).

In formula (9), p is an integer between 1 and Q. The order of the index p is the descending order of the eigenvalue δ _p .
The sound source direction observation unit 122 calculates the power P _sp (θ) of the frequency-specific spatial spectrum shown in the following equation (10) based on the transfer function vector [G(θ)] and the calculated eigenvector [εp].

In equation (10), _Dm is the maximum number of detectable sound sources (for example, 4) and is a predetermined natural number smaller than Q.
The sound source direction observation unit 122 can calculate the sum of spatial spectra P _sp (θ) in frequency bands where the S/N ratio is greater than a predetermined threshold (for example, 20 dB) as the power P _ext (θ) of the spatial spectrum over all bands.

Next, the WDS-BF method will be described. The WDS-BF method is a method for calculating the squared value of the delay sum of the acoustic signal ξ _q (t) of the entire band of each channel q as the spatial spectrum power P _ext (θ) as shown in formula (11), and searching for the sound source direction θ where the spatial spectrum power P _ext (θ) is maximized. In formula (11), the transfer function indicated by each element of [G(θ)] indicates the contribution of the phase delay from the sound source to the microphone corresponding to each channel q (q is an integer of 1 to Q). [ξ(t)] is a vector whose elements are the signal values of the acoustic signal ξ _q (t) of each channel q at time t.

(simulation)
Next, a simulation performed to evaluate the effectiveness of the sound processing device 10 will be described. In the simulation, in addition to methods 1 to 3, estimation information was estimated for each of the conventional methods. The conventional method is a method for estimating the position and direction of each microphone array and the sound source position so that the function value of the first objective function L ₁ becomes smaller. At this stage, the time offset of the microphone array is not estimated. In the conventional method, the time offset for each microphone array can be calculated from the estimated position and direction of the microphone array and the sound source position using the arrival time difference of the sound waves between the microphones in the microphone array and the distance between the microphones as constraint conditions.

In the simulation, the following conditions were assumed unless otherwise specified. In a two-dimensional space of a square with a side length of 120 m, the positions of each sound source and each microphone are randomly distributed, and the direction of each microphone array is randomly directed over the entire circumference from -180° to +180°. The true values of the positions of each sound source, each microphone, and each microphone array direction were determined using uniform random numbers. However, the position of the microphone array A ₁ is directed to the origin (0,0) and in the x-axis direction (θ=0°). In addition, the true values of the time offset of each microphone array are distributed according to a normal distribution with an average of 0 and a standard deviation of σ _b . Furthermore, the observed sound source direction includes an error following a normal distribution with an average of 0 and a standard deviation of σ _d , and the observed arrival time difference includes an error following a normal distribution with an average of 0 and a standard deviation of σ _τ . The number of sound sources M was set to 4 or 7, the number of microphone arrays N was set to 3, and the sampling frequency f _s of the acoustic signal was set to 16000 Hz. The sound source direction estimation resolution ρ _d was set to 1°, the sound source direction estimation standard deviation σ _d was set to 2°, the arrival time difference standard deviation σ _τ was set to 1 ms, and the time offset standard deviation σ _δ was set to 10°.

In the simulation, in order to verify the effectiveness of each method, the error in the position of the microphone array and the sound source estimated at each update step of the estimated value and the time required for the position to converge were defined as the calculation time. However, the root mean square error of each of the microphone array and the sound source was determined for each simulation, and the average value over 100 times was calculated as the error average. FIG. 10 is a diagram showing the change in the error average for each method. FIG. 10 shows that the error average is the smallest in method 2, and the error average increases in the order of method 1 and method 3. The error average of each of methods 1 to 3 is smaller than that of the conventional method. In the conventional method, the error average does not decrease even if the calculation proceeds, and converges to an almost constant value. This indicates that the method falls into a local optimum solution. In addition, the error average after convergence is equivalent in method ₁ and method 2. This indicates that the positions of the microphone array and the sound source are estimated with high accuracy by simultaneous or integrated first objective function L 1 and second objective function L _2. FIG. 11 is a diagram showing the average calculation time for each method. 11 shows that method 2 has the shortest calculation time, followed by the conventional method, method 1, and method 3. However, the difference in calculation time between method 2, the conventional method, and method 1 is relatively small.

Similar simulations were also performed using methods 1 and 2, assuming a three-dimensional space. It was assumed that the microphone array and sound source were randomly distributed in a three-dimensional space forming a cube with a side length of 120 m, and that the direction of the microphone array was randomly directed within the entire sphere. Figure 12 shows the change in the average error at each step in three-dimensional space. In Figure 12, the average error temporarily increases at the beginning of the simulation for both methods 1 and 2, but then begins to decrease and converges to a certain value. The average error in three-dimensional space was equal to or less than that in two-dimensional space. However, as shown in Figure 13, the calculation time required from the start of the simulation to convergence tends to be longer in three-dimensional space than in two-dimensional space.

(Summary)
As described above, the sound processing device 10 according to the present embodiment includes an estimation unit 126 that determines the position, direction, and offset time of the N microphone arrays and the position of the sound source so that the function value of the first objective function (e.g., the first objective function L ₁ ) and the function value of the second objective function (e.g., the second objective function L ₂ ) are smaller. The first objective function indicates the magnitude of the directional difference, which is the difference between the observed sound source direction, which is the sound source direction determined for each of the N microphone arrays, and the direction of the sound source from the microphone array rotated with the direction of the microphone array. The function value of the second objective function indicates the magnitude of the time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, from the observed arrival time difference, which is the arrival time difference of the sound from the sound source determined for each pair of two microphone arrays out of the N.
According to this configuration, since the second objective function and the first objective function are constraint conditions for the first objective function and the second objective function, respectively, the uncertainty of the solution that may be included in the position of the microphone array and the position of the sound source estimated based on the first objective function and the uncertainty of the solution that may be included in the direction of the sound source from the microphone array estimated based on the second objective function are eliminated. Here, the position of the microphone array and the position of the sound source estimated based on the first objective function each include uncertainty regarding the size, but do not include uncertainty regarding the rotation of the direction of the sound source from the microphone array. The position of the microphone array and the position of the sound source estimated based on the second objective function include uncertainty regarding the rotation of the direction of the sound source from the microphone array, but do not include uncertainty regarding the position of the microphone array and the size of the sound source. Therefore, by making the function values of the first objective function and the second objective function smaller, it is possible to avoid local optimal solutions and more accurately determine the position, direction, offset time, and position of the sound source of the microphone array.

The estimation unit 126 may determine the position, direction, and offset time of the microphone array, and the position of the sound source, so that the sum of the function value of the first objective function and the function value of the second objective function is smaller.
The sum of the function value of the first objective function and the function value of the second objective function also serves as an index indicating the accuracy of the estimated microphone array position, direction, and offset time, and the sound source position. In addition, the estimated microphone array position, sound source position, and direction from the microphone array to the sound source do not include uncertainty. Therefore, it is possible to avoid a local optimum solution and more accurately determine the microphone array position, direction, offset time, and sound source position.

The estimation unit 126 may determine the position, direction, and offset time of the microphone array and the position of the sound source so that the sum of the distances between the pair of microphone arrays from the first position to the second position is smaller. The first position is a position obtained by rotating the position of the sound source relative to the microphone array in the direction of the microphone array, and the second position is a position away from one microphone array of the pair of microphone arrays in the observed sound source direction by a distance equivalent to the difference in sound propagation distance in time difference from the distance from the other microphone array of the pair to the sound source.
The sum of the distances between the first and second positions of the microphone array pairs is also an index of the accuracy of the estimated microphone array position, direction, and offset time, and the sound source position. In addition, the estimated microphone array position, sound source position, and sound source direction from the microphone array do not include uncertainty. Therefore, the microphone array position, direction, and offset time, and the sound source position can be determined more accurately by avoiding a local optimum solution.

The estimation unit 126 updates the position, direction, and offset time of the microphone array and the position of the sound source so that the function value of the first objective function becomes smaller;
and updating the position, orientation and offset time of the microphone array and the position of the sound source so as to reduce the function value of the second objective function.
Since the step of reducing the function value of the first objective function and the step of reducing the function value of the second objective function are alternately performed, it is possible to avoid a local optimum solution and indefiniteness of the solution that may occur continuously in either step, and therefore it is possible to more accurately determine the position, direction, and offset time of the microphone array, and the position of the sound source.

One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the present invention.

For example, the sound processing device 10 may be configured integrally with at least one of the N microphone arrays.
In the above description, the sound processing device 10 includes all of the sound source direction observation unit 122, the arrival time difference observation unit 124, and the estimation unit 126. However, the present invention is not limited to this. For example, one or both of the sound source direction observation unit 122 and the arrival time difference observation unit 124 may be omitted in the sound processing device 10. However, the sound processing system S1 includes one or both of them, and is capable of inputting and outputting various data to and from the sound processing device 10.

9, sound source direction observation units 122-1 to 122-N are connected to the individual microphone arrays A ₁ to A _N , respectively, and the sound source direction observation unit 122 is omitted in the sound processing device 10. The sound source direction observation units 122-1 to 122-N may be integrated with the microphone arrays A ₁ to A _N , respectively, or may be separate entities.
The sound source direction observation units 122-1 to 122-N perform sound source localization processing based on the acoustic signals of multiple channels acquired by the microphone arrays A ₁ to A _N connected to each unit, and determine the sound source direction of each sound source. The sound source direction observation units 122-1 to 122-N output sound source direction information indicating the determined sound source directions as observed sound source directions to the estimation unit 126.

Note that a part of the sound processing device 10 in the above-described embodiment, for example, a part or all of the sound source direction observation unit 122, the arrival time difference observation unit 124, the estimation unit 126, and the output processing unit 128, may be realized by a computer. In that case, the control functions may be realized by recording a program for realizing the control functions in a computer-readable recording medium, and reading and executing the program recorded in the recording medium into a computer system including a processor.
In addition, a part or the whole of the sound processing device 10 in the above-mentioned embodiment and modified examples may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the sound processing device 10 may be individually processed, or a part or the whole may be integrated and processed. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on that technology may be used.

S1...sound processing system, 10...sound processing device, 110...input/output unit, 120...control unit, 122...sound source direction observation unit, 124...arrival time difference observation unit, 126...estimation unit, 128...output processing unit

Claims (8)

a function value of a first objective function indicating a magnitude of a directional difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer equal to or greater than 2) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array;
The sum of the observed arrival time difference, which is the arrival time difference of the sound from the sound source determined for each pair of two microphone arrays among the N microphone arrays, and the function value of a second objective function indicating the magnitude of the time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array , is made smaller.
An audio processing device comprising an estimation unit for determining a position, an orientation, and an offset time of the microphone array, and a position of the sound source. a function value of a first objective function indicating a magnitude of a directional difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer equal to or greater than 2) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array;
The second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, from the observed arrival time difference, which is the arrival time difference of the sound from the sound source determined for each pair of two microphone arrays out of N, so that the function value of the second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, is smaller.
an estimator for determining a position, an orientation and a time offset of the microphone array and a position of the sound source ;
The estimation unit is
determining a position, a direction, and an offset time of the microphone array and a position of the sound source such that a sum of distances between pairs of the microphone arrays from a first position to a second position is smaller;
the first position is a position obtained by rotating a position of the sound source relative to the microphone array toward the microphone array;
The second position is a position away from one of the microphone arrays in the pair in a direction toward the observed sound source by a distance corresponding to a difference in sound propagation distance in the time difference from a distance from the other microphone array of the pair to the sound source.
Sound processing equipment. a function value of a first objective function indicating a magnitude of a directional difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer equal to or greater than 2) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array;
The second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, from the observed arrival time difference, which is the arrival time difference of the sound from the sound source determined for each pair of two microphone arrays out of N, so that the function value of the second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, is smaller.
an estimator for determining a position, an orientation and a time offset of the microphone array and a position of the sound source ;
The estimation unit is
updating the position, orientation, and offset time of the microphone array and the position of the sound source so that a function value of the first objective function becomes smaller;
and updating the position, orientation, and offset time of the microphone array and the position of the sound source so as to reduce the function value of the second objective function.
Sound processing equipment. The microphone array;
a sound source direction observation unit that determines the observed sound source direction based on an acoustic signal collected by each of the microphone arrays;
an arrival time difference observer for determining the observed arrival time difference for each of the pairs;
The sound processing device according to any one of claims 1 to 3 ,
An audio processing system comprising: A program for causing a computer to function as the sound processing device according to any one of claims 1 to 3 . a function value of a first objective function indicating a magnitude of a directional difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer equal to or greater than 2) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array;
The sum of the observed arrival time difference, which is the arrival time difference of the sound from the sound source, determined for each pair of two microphone arrays among the N microphone arrays, and the function value of a second objective function indicating the magnitude of the time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array , is made smaller.
determining a position, orientation and time offset of the microphone array and a location of the sound source. a function value of a first objective function indicating a magnitude of a directional difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer equal to or greater than 2) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array;
The second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, from the observed arrival time difference, which is the arrival time difference of the sound from the sound source determined for each pair of two microphone arrays out of N, so that the function value of the second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, is smaller.
determining a position, orientation and offset time of the microphone array and a position of the sound source ;
The steps include:
determining a position, a direction, and an offset time of the microphone array and a position of the sound source such that a sum of distances between pairs of the microphone arrays from a first position to a second position is smaller;
the first position is a position obtained by rotating a position of the sound source relative to the microphone array toward the microphone array;
The second position is a position away from one of the microphone arrays in the pair in a direction toward the observed sound source by a distance corresponding to a difference in sound propagation distance in the time difference from a distance from the other microphone array of the pair to the sound source.
Acoustic processing methods. a function value of a first objective function indicating a magnitude of a directional difference between an observed sound source direction, which is a sound source direction determined for each of N (N is an integer equal to or greater than 2) microphone arrays, and a direction obtained by rotating the direction of the sound source from the microphone array with respect to the direction of the microphone array;
The second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, from the observed arrival time difference, which is the arrival time difference of the sound from the sound source determined for each pair of two microphone arrays out of N, so that the function value of the second objective function indicates the magnitude of a time difference, which is the difference between the time difference of the propagation time of the sound wave from the sound source to each microphone array and the time difference of the offset time of each microphone array, is smaller.
determining a position, orientation and offset time of the microphone array and a position of the sound source ;
The steps include:
updating the position, orientation, and offset time of the microphone array and the position of the sound source so that a function value of the first objective function becomes smaller;
and updating the position, orientation, and offset time of the microphone array and the position of the sound source so as to reduce the function value of the second objective function.
Acoustic processing methods.

Images