JP7485982B2 - Processing device, playback system, processing method, and processing program - Google Patents

Description

The present invention relates to a processing device, a playback system, a processing method and a processing program.

In recent years, area playback technology that presents sound only to specific areas has become widespread in public spaces such as train stations, art galleries, and museums. Area playback technology that presents sound only to specific areas is being considered for application to small spaces, such as delivering sound only to people watching television at home, or presenting navigation sounds only to the driver in a car and music to other passengers.

In particular, methods of controlling the sound field using ultrasonic speakers with sharp directionality or speaker arrays consisting of multiple regular speakers lined up are being considered for this type of area reproduction technology. Generally, ultrasonic speakers demodulate ultrasonic waves into audible sound, but distortion occurs during the demodulation, degrading the sound quality, making it particularly difficult to reproduce high-pitched sounds. When considering the reproduction of various content such as music, area reproduction that enables high-quality sound reproduction is required.

For example, there is an area playback method that uses a speaker array to present sound only to any specific area, preventing the sound from leaking out to other areas (see Patent Document 1).

As a method for realizing area reproduction using a speaker array, directivity control technology and acoustic contrast maximization (ACM) technology may be used. Using ACM, area reproduction that maximizes the acoustic contrast between an area where an arbitrarily set sound is presented (bright zone) and an area where the sound is suppressed (dark zone) using a speaker array is considered (Non-Patent Document 1). In Non-Patent Document 1, the same value is experimentally used for all frequencies as the regularization parameter, which is the weight of the penalty term, based on the reproduction accuracy of the desired sound field for each frequency and the magnitude of the filter gain.

There is a method for designing a filter by using a penalty term for the objective function from which the filter is derived (see Non-Patent Document 2). In Non-Patent Document 2, the sum of squares of the filter coefficients is used as the penalty term to suppress the filter gain.

JP 2013-110495 A

J. W. Choi, Y. H. Kim, “Generation of an acoustically bright zone with an illuminated region using multiple sources,” The Journal of the Acoustical Society of America 111 (4) (2001) 1695 MM. Boone, WH. Cho, and JG Ih, “Design of a highly directional endfire loudspeaker array,” Journal of the Audio Engineering Society 57.5 (2009): 309-325

When designing a filter using ACM or directional control for any frequency band, it is possible to design a filter, but the filter gain in the low range may become large. If the drive signal obtained by the designed filter gain exceeds the allowable vibration of the speaker, the output sound may become distorted.

The present invention has been made in consideration of the above circumstances, and the object of the present invention is to provide a technology that suppresses the gain applied to each of multiple speakers to achieve a target directional characteristic.

A processing device according to one embodiment of the present invention includes a reward function acquisition unit that acquires a reward function from a target directional characteristic, a constraint condition acquisition unit that acquires as constraint conditions a range of filter coefficients for generating drive signals that can be reproduced by multiple speakers that achieve the target directional characteristic, a calculation unit that calculates, for each of the multiple speakers, a filter that satisfies the constraint conditions and maximizes the reward function, and a convolution calculation unit that convolves an input acoustic signal with the calculated filter to generate drive signals to be input to each speaker.

A playback system according to one embodiment of the present invention comprises the above-mentioned processing device and a plurality of speakers that play back the drive signals calculated by the convolution calculation unit.

A processing method according to one aspect of the present invention includes a step in which a computer obtains a reward function from a target directional characteristic, a step in which the computer obtains, as a constraint condition, a range of filter coefficients for generating drive signals that can be reproduced by multiple speakers that achieve the target directional characteristic, a step in which the computer calculates, for each of the multiple speakers, a filter that satisfies the constraint condition and maximizes the reward function, and a step in which the computer convolves an input acoustic signal with the calculated filter to generate a drive signal to be input to each speaker.

One aspect of the present invention is a processing program that causes a computer to function as the above-mentioned processing device.

The present invention provides a technology that suppresses the gain applied to each of multiple speakers to achieve a desired directional characteristic.

FIG. 1 is a diagram illustrating the system configuration of a playback system according to an embodiment of the present invention and functional blocks of a processing device. FIG. 2 is a flowchart illustrating the processing of the processing device. FIG. 3 is a diagram for explaining an observation system when determining a directivity control filter. FIG. 4 is a diagram for explaining an observation system when a filter for maximizing the acoustic contrast is obtained. FIG. 5 is a diagram for explaining the hardware configuration of a computer used in the processing device.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same parts are given the same reference numerals and the description will be omitted.

(Playback System)
A reproduction system 3 using a processing device 1 according to an embodiment of the present invention will be described with reference to Fig. 1. The reproduction system 3 includes the processing device 1 and a speaker array 2.

The processing device 1 is a computer that converts the input acoustic signal into a drive signal for each speaker. The processing device 1 generates a filter corresponding to each speaker based on the directional characteristics of the desired sound field and the specifications of each speaker that makes up the speaker array 2. The processing device 1 convolves the input acoustic signal with the generated filter for each speaker, and converts it into a drive signal for each speaker.

The speaker array 2 is formed by multiple speakers. Each of the multiple speakers is installed in a venue where a sound field is reproduced by a reproduction system 3. Each speaker reproduces the drive signal for each speaker output from the processing device 1. By reproducing the drive signal here, an output acoustic signal that realizes the desired directional specification is obtained.

(Processing method)
An overview of the processing performed by the processing device 1 will be described with reference to FIG.

In step S1, the processing device 1 acquires the desired sound field and speaker specifications. In step S2, the processing device 1 acquires a reward function generated from the desired sound field acquired in step S1. In step S3, the processing device 1 acquires constraint conditions generated from the speaker specifications.

In step S4, the processing device 1 calculates a filter that controls the sound field by reinforcement learning. At this time, the processing device 1 sets the filter so as to satisfy the constraint conditions obtained in step S3 and maximize the reward function obtained in step S2.

In step S5, the processing device 1 convolves the input acoustic signal with the filter designed in step S4 to generate a drive signal to be input to each speaker. The desired sound field is realized by inputting the drive signal to each speaker.

(Processing Equipment)
The processing device 1 generates filters for each speaker by reinforcement learning to solve an optimization problem in which a function for generating a desired sound field as a reward function is maximized under a constraint condition that the filter gain falls within a certain range that can be reproduced by each speaker constituting the speaker array 2. The processing device 1 includes a reward function acquisition unit 11, a constraint condition acquisition unit 12, a calculation unit 13, and a convolution calculation unit 14.

The reward function acquisition unit 11 acquires a reward function set from the target directional characteristics. The reward function acquisition unit 11 acquires a reward function for realizing a sound field with the target directional characteristics. The reward function is arbitrarily set from functions that realize a sound field. The reward function acquisition unit 11 may acquire a reward function generated by a user, or may generate a reward function for realizing the target directional characteristics by computer processing and acquire the generated reward function. The reward function acquisition unit 11 sets the acquired reward function in the processing device 1 so that it can be referenced by the calculation unit 13 described below.

As a reward function, a function may be used that is transformed so that the larger the error is, the smaller the reward function becomes, and vice versa, for the error between the desired sound field indicated by the target function in the filter design for directional control using the least squares method and the sound field observed at the control point.

Specifically, for example, in the observation system shown in Figure 3, control points are placed around a speaker array in which multiple speakers are lined up, and a filter is designed to control the amplitude and phase of each speaker based on the transmission characteristics from the speaker to the control point. This method makes it possible to control the direction in which sound propagates strongly from the speaker, or the direction in which sound does not propagate. In this method, the signal observed at each control point is expressed by equation (1).

Here, the least squares method for finding a filter for directional control is a minimization problem for finding a filter that minimizes the sum of squares of the error between the desired directional characteristic and the directional characteristic observed at the control point. Therefore, the objective function J to be minimized is expressed by equation (3).

In an embodiment of the present invention, a reward function is derived from equation (3). The reward function is a function that is transformed so that the larger the error between the desired directional characteristic in the objective function expressed by equation (3) and the directional characteristic observed at the control point, the smaller the reward function becomes, and vice versa. The reward function is obtained, for example, by multiplying the objective function shown in equation (3) by -1.

Generally, a directional control filter shown in equation (4) is obtained by calculating a filter that minimizes the objective function shown in equation (3), but in an embodiment of the present invention, a filter obtained by calculation unit 13 described below is used.

As another example of a reward function, we will explain an example of using acoustic contrast in ACM. When using acoustic contrast, as shown in Figure 4, an area where speakers are set is arbitrarily set to present an arbitrarily set sound (bright-zone) and an area where sound is suppressed (dark-zone). In ACM, a filter is designed so that the acoustic contrast between the bright zone and the dark zone is maximized by the speaker array.

When an arbitrary speaker array is driven by a filter, the sound field observed in each area can be expressed by the following equations (5) and (6).

The acoustic energy in each area is expressed by equations (7) and (8).

Acoustic contrast is given by the ratio of acoustic energy in the bright zone to the acoustic energy in the dark zone, as shown in equation (9).

In an embodiment of the present invention, the acoustic contrast equation in equation (9) is the reward function.

Generally, this is an optimization problem of maximizing the acoustic contrast shown in equation (9) as shown in equation (10), but in an embodiment of the present invention, a filter obtained by the calculation unit 13 described below is used.

In the embodiment of the present invention, two types of reward functions are shown as examples, but the present invention is not limited to these. Any reward function that can express a sound field may be used.

Here, we will explain an example of the process in which the reward function acquisition unit 11 generates a reward function for achieving the target directional characteristic through computer processing.

For example, in the observation system shown in Figure 3, a method for generating a reward function from a filter design for directivity control will be described. First, the user inputs the direction and beam width of the beam from the speaker as the target directional characteristics. The beam direction and beam width are specified, for example, as an angle centered on the speaker. The reward function acquisition unit 11 can generate d in equation (3) from the input directional characteristics and generate the objective function of equation (3) as the reward function.

As another example, we will explain a method of generating a reward function from the acoustic contrast in ACM in the observation system shown in Figure 4. The reward function acquisition unit 11 generates each sound field of equations (5) and (6) for each control point of the bright zone and the dark zone, and generates the acoustic contrast obtained by equation (9) as a reward function.

In this way, the reward function acquisition unit 11 can also generate the reward function through computer processing.

The constraint condition acquisition unit 12 acquires, as constraint conditions, a range of filter coefficients for generating drive signals that can be reproduced by multiple speakers that achieve the target directional characteristics. The constraint condition acquisition unit 12 may acquire constraint conditions generated by a user, or may generate constraint conditions from the specifications of the speakers of the speaker array 2 by computer processing and acquire the generated constraint conditions. The constraint condition acquisition unit 12 sets the acquired constraint conditions in the processing device 1 so that the calculation unit 13 described below can refer to them.

The filter that controls the sound field using the speaker array is calculated including the filter gain that affects the output sound source from the filter. The filter gain corresponding to a certain speaker at each frequency is expressed by equation (11).

As shown in equation (11), if the filter gain is large, the signal input to the speaker also becomes proportionally large, which places a heavy load on the speaker and may make reproduction difficult. In particular, when designing an ACM or directional control filter for an arbitrary frequency band, the low-frequency filter gain becomes large. Therefore, in an embodiment of the present invention, the constraint is the range of filter coefficients for generating drive signals that can be reproduced by multiple speakers that achieve the target directional characteristics. The constraint is to restrict the filter coefficients corresponding to each speaker to fall within a certain range.

The constraints are set so that the drive signal can be reproduced by suppressing the speaker gain corresponding to each speaker. The constraints may be the same for each frequency band.

In addition, since the range of filters that the speaker can reproduce may be limited depending on the magnitude of the input acoustic signal, the constraint condition acquisition unit 12 may set constraint conditions taking into account the magnitude of the input acoustic signal.

In addition, if the sum of squares of the filter coefficients is used as a penalty term in order to suppress the filter gain in the objective function obtained by equation (3), the objective function is expressed by equation (12).

In equation (12), the regularization parameter controls the relative weight of the loss term and the penalty term. Generally, the same value of the regularization parameter is experimentally used for each frequency band based on the reproduction accuracy of the desired sound field for each frequency and the magnitude of the filter gain. Conventionally, when the same regularization parameter is used for each frequency band, there was a problem that the optimal regularization parameter could not be given for each frequency band. In addition, when determining the regularization parameter for each frequency band, it is difficult to experimentally set a value for each frequency. In addition, there is a problem that it is difficult to set the optimal parameter because there is a trade-off between the reproduction accuracy of the desired sound field and the magnitude of the filter gain.

Therefore, in an embodiment of the present invention, rather than using a penalty term, constraint conditions in the reinforcement learning method are set so that the drive signal can be reproduced by suppressing the speaker gain corresponding to each speaker and frequency band. The constraint conditions may be different for each frequency band. This restricts the filter coefficients for each speaker and each frequency band to fall within a certain range.

In general, the least squares filter shown in equation (13) can be obtained by solving the minimization problem of the objective function of equation (12) in the same manner as equation (4), but in an embodiment of the present invention, a filter obtained by calculation unit 13 described below is used.

The calculation unit 13 calculates a filter for each of the multiple speakers that satisfies the constraint conditions and maximizes the reward function. The constraint conditions are determined by the constraint condition acquisition unit 12 based on the specifications of the speakers that make up the speaker array 2. The reward function is determined by the reward function acquisition unit 11 based on the directional characteristics of the target sound field.

The calculation unit 13 calculates a filter that satisfies the constraint conditions and maximizes the reward function by a reinforcement learning method that treats the filter coefficients as continuous values of the behavior. The calculation unit 13 calculates a filter that generates a desired sound field by solving an optimization problem that maximizes the reward function under the constraint conditions by a reinforcement learning method that treats the behavior as continuous values, such as DDPG (Deep Deterministic Policy Gradient) and NAF (Normalized Advantage Function).

Below, we explain an example of a process in which the reward function is acoustic contrast and a filter is calculated using NAF.

The Q function is expressed by equation (14) using the state value function and advantage function.

In an embodiment of the present invention, the states, actions, rewards and clipping ranges in Table 1 above are defined as in equation (16).

The action a' finally obtained by the process shown in Table 1 is the filter that generates the desired sound field. By performing this process for all frequencies to be used, the calculation unit 13 can find the optimal filter for each frequency.

The convolution calculation unit 14 convolves the input sound signal with the filter calculated by the calculation unit 13 to generate a drive signal to be input to each speaker. The convolution calculation unit 14 generates a drive signal from the input sound signal and the filter calculated by the calculation unit 13. The desired sound field is realized by having multiple speakers each reproduce the drive signal calculated by the convolution calculation unit 14.

In this manner, in the processing device 1 according to the embodiment of the present invention, the reward function acquisition unit 11 sets a function that generates a sound field that realizes the reproduction of a desired area as the reward function, and the constraint condition acquisition unit 12 sets a constraint that the filter gain falls within a certain range as the constraint condition. The calculation unit 13 derives a sound field control filter that satisfies the constraint condition and maximizes the reward function from the output reward function and constraint condition. The convolution calculation unit 14 convolves the input acoustic signal and the sound field control filter output from the calculation unit 13 to reproduce the signal. This makes it possible to obtain the drive signal to be input to each speaker in order to realize the reproduction of the desired area.

Here, a range of filter coefficients is set as a constraint for generating a drive signal that can be reproduced by multiple speakers. The processing device 1 according to an embodiment of the present invention can suppress the gain applied to each of the multiple speakers to achieve the target directional characteristics.

In the embodiment of the present invention, an example in which multiple speakers are arranged in a circle in the observation system for calculating the reward function has been described, but this is not limited to this. Multiple speakers may be arranged in any position, such as in a row.

The processing device 1 of the present embodiment described above is, for example, a general-purpose computer system including a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906. In this computer system, the CPU 901 executes a processing program loaded on the memory 902, thereby realizing each function of the processing device 1.

The processing device 1 may be implemented in one computer or in multiple computers. The processing device 1 may also be a virtual machine implemented in a computer.

The processing program of the processing device 1 can be stored on a computer-readable recording medium such as a HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc), or can be distributed via a network.

The present invention is not limited to the above-described embodiments, and many variations are possible within the scope of the invention.

REFERENCE SIGNS LIST 1 Processing device 2 Speaker array 3 Playback system 11 Reward function acquisition unit 12 Constraint condition acquisition unit 13 Calculation unit 14 Convolution calculation unit 901 CPU
902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims (6)

A reward function acquisition unit that acquires a reward function set from a target directional characteristic;
a constraint condition acquisition unit that acquires, as a constraint condition, a range of filter coefficients for generating drive signals that can be reproduced by a plurality of speakers that realize the target directional characteristic;
A calculation unit that calculates a filter that satisfies the constraint condition and maximizes the reward function for each of the plurality of speakers;
a convolution calculation unit that convolves an input sound signal with the calculated filter to generate a drive signal to be input to each speaker ,
The constraint condition is set in a processing device so that a drive signal can be reproduced by suppressing a speaker gain corresponding to each speaker . The processing device according to claim 1 , wherein the calculation unit calculates a filter that satisfies the constraint condition and maximizes the reward function by a reinforcement learning method that treats a filter coefficient as a continuous value of an action. The processing device according to claim 1 , wherein the constraint condition is set so that a drive signal can be reproduced by suppressing a speaker gain corresponding to each speaker and frequency band. A processing device according to any one of claims 1 to 3 ,
A reproduction system comprising a plurality of speakers that reproduce the drive signals calculated by the convolution calculation unit. A step in which a computer obtains a reward function from a target directional characteristic;
The computer acquires, as a constraint condition, a range of filter coefficients for generating drive signals that can be reproduced by a plurality of speakers that realize the target directional characteristic;
The computer calculates, for each of the plurality of speakers, a filter that satisfies the constraint condition and maximizes the reward function;
The computer convolves an input acoustic signal with the calculated filter to generate a drive signal to be input to each speaker ,
A processing method in which the constraint conditions are set so that a drive signal can be reproduced by suppressing a speaker gain corresponding to each speaker . A processing program for causing a computer to function as the processing device according to any one of claims 1 to 3.

Images