JP7659464B2 - Audio device and audio control method - Google Patents

Description

The present invention relates to an audio device and an audio control method.

Conventionally, there is known an acoustic device that outputs various sound source signals, such as music and voice, from multiple channels (see, for example, Patent Document 1). In the conventional technology, a pseudo-reverberation sound generated from the sound source signal is actively added to the direct sound, and surround sound is reproduced on multiple channels.

Patent No. 5372142

However, conventional technology leaves room for improvement in terms of providing appropriate surround sound reproduction according to the sound source signal.

The present invention has been made in consideration of the above, and aims to provide an audio device and an audio control method that can perform appropriate surround playback according to a sound source signal.

In order to solve the above problems and achieve the object, the present invention provides an acoustic device comprising a separation unit and an output control unit. The separation unit separates a predetermined sound source signal that does not require processing to add artificial pseudo-reverberation from a sound source signal, and removes the separated predetermined sound source signal from the sound source signal. The output control unit applies a filter for generating the artificial reverberation to the sound source signal from which the predetermined sound source signal has been removed by the separation unit, and outputs the result.

The present invention makes it possible to perform appropriate surround playback according to the sound source signal.

FIG. 1A is a diagram for explaining an overview of an acoustic control method according to a first embodiment. FIG. 1B is a diagram illustrating an overview of the acoustic control method according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of an audio system including the audio device according to the first embodiment. FIG. 3 is a diagram illustrating the separation and extraction process performed by the separation unit. FIG. 4A is a diagram for explaining the sound image of the reproduced sound, etc. FIG. 4B is a diagram for explaining the sound image of the reproduced sound. FIG. 4C is a diagram for explaining the sound image of the reproduced sound. FIG. 4D is a diagram for explaining the sound image of the reproduced sound. FIG. 4E is a diagram for explaining the sound image of the reproduced sound. FIG. 4F is a diagram for explaining the sound image of the reproduced sound. FIG. 4G is a diagram for explaining the sound image of the reproduced sound. FIG. 5 is a flowchart showing a processing procedure executed by the audio device according to the first embodiment. FIG. 6 is a block diagram showing an example of the configuration of an audio system including an audio device according to the second embodiment. FIG. 7 is a diagram illustrating the gain determination process performed by the determination unit. FIG. 8 is a flowchart showing a processing procedure executed by the audio device according to the second embodiment.

Embodiments of the audio device and audio control method disclosed in this application will be described in detail below with reference to the attached drawings. Note that the present invention is not limited to the embodiments described below.

(First embodiment)
<Overview of Acoustic Control Method Using Acoustic Device According to First Embodiment>
First, an overview of an acoustic control method using an acoustic device according to a first embodiment will be described below with reference to Figures 1A and 1B. Figures 1A and 1B are diagrams for explaining the overview of the acoustic control method according to the first embodiment.

The acoustic control method according to the first embodiment is executed, for example, by an acoustic device 1 shown in FIG. 1A. FIG. 1A shows a case in which, in an in-vehicle space such as the cabin of a vehicle, direct sound, which is a sound source signal, and actual reverberation sound are output from two speakers FR, FL arranged on the left and right sides of the front, and pseudo reverberation sound is output from two speakers RL, RR arranged on the left and right sides of the rear and added to the direct sound, thereby performing surround reproduction.

The sound source signal here is a signal from a sound source that has a spatial spread (sound image width) by outputting different sounds from, for example, two speakers FR and FL. In other words, the sound source signal is a stereo signal that is reproduced in stereo on two channels (speakers FR and FL).

The sound source signal may be, for example, a signal of a mixture of multiple instrument sounds and voices (vocals), such as classical music or opera, that is, a signal of a mixture of multiple sound sources, but is not limited to this. In other words, the sound source signal may be a signal of a sound only of voice, or a signal of a sound from a single instrument (sound source), such as only a piano or only a violin.

As shown in FIG. 1A, it is known that a listener L who listens to sound from a sound source S, which is a position where a sound source such as a musical instrument or voice (vocals) is located in space, can perceive two types of spatial impressions. One spatial impression is the sound image width ASW, which is defined as the "width of the apparent sound source" that is perceived as being blended with the direct sound in both time and space, and the other spatial impression is the sense of envelopment LEV, which is defined as the "feeling that the listener is surrounded by sound sources other than the apparent sound source." The sound image width ASW is the width of the sound image A derived from the direct sound and early reflected sound components, which are the early components of the sound source signal. The sense of envelopment LEV is the sound image B derived from the reverberation sound components, which are the later components of the sound source signal.

When designing and evaluating the sound image width ASW and the sense of envelopment LEV, an index using the so-called "law of the first wave front" may be used. In this index, as shown in Figure 1B, two regions R1 and R2 are defined by two thresholds TH1 and TH2.

Region R1 is a region that contains, among the components contained in the sound source signal, components (initial components) that mainly contain direct sound. For example, if the initial components of region R1 are large, the sound image width ASW becomes large, and the listener L may feel that the sound is diffused, and depending on the sound source, the sound quality may be evaluated as poor (unclear). Note that direct sound is, for example, sound recorded directly from voice (vocals) or musical instruments, and does not include sound reflected by walls, etc.

Region R2 is a region that contains, among the components contained in the sound source signal, components that mainly contain reverberation sounds (late components). For example, if the reverberation sound components in region R2 are large, the sense of envelopment LEV increases, and the listener L will feel that the sound is diffused, which will result in a rich sense of envelopment. Note that reverberation sounds are, for example, recorded sounds of voices, musical instruments, etc. that are reflected by walls, etc., and are sounds that are delayed in time from the direct sound.

In addition, in a relatively small space such as a vehicle interior, it is difficult to separate direct sound and reverberant sound, and reverberant sound tends to be mixed into the direct sound. In such a case, the reverberant sound components in area R2 become smaller, and the envelopment LEV becomes smaller. Therefore, in a small space such as a vehicle interior where direct sound and reverberant sound are mixed, the acoustic device 1 actively adds pseudo-reverberant sound to the direct sound, thereby increasing the reverberant sound components in area R2 and ensuring the envelopment LEV.

In the above, it has been described that direct sound and the like are output from the speakers FR and FL, and pseudo reverberation sound is output from the speakers RL and RR, but this is not limited to the above. That is, for example, the acoustic device 1 may output direct sound and the like from all or some of the four speakers FR, FL, RL, and RR, and adjust the volume and the like. This allows, for example, the acoustic device 1 to move the spatial position of the sound image A of the direct sound and the like. Similarly, the acoustic device 1 may output pseudo reverberation sound from all or some of the four speakers FR, FL, RL, and RR, and adjust the volume and the like. This allows, for example, the acoustic device 1 to move the spatial position of the sound image B of the pseudo reverberation sound. In this way, the acoustic device 1 can localize the sound image A of the direct sound and the sound image B of the pseudo reverberation sound at any position.

In the prior art, there have been cases where a process for adding pseudo-reverberation is performed by extracting surround components including early reflection sound components from a sound source signal based on the correlation between the left and right channels of the sound source, and then delaying and outputting the extracted surround components. However, in such cases, it was not always possible to extract the intended surround components for various types of sound source signals. Furthermore, when the surround components are delayed and output as described above, the early reflection sounds that were originally present may not be obtained, and as a result, appropriate surround reproduction may not be possible.

Therefore, the audio device 1 according to this embodiment is configured to perform appropriate surround playback according to the sound source signal.

The processing of the acoustic device 1 will now be described in detail with reference to FIG. 1A. First, the acoustic device 1 separates a predetermined sound source signal that does not require processing to add artificial reverberation from the sound source signal of the sound source device 50, and removes the separated predetermined sound source signal from the sound source signal (step S1). Note that a detailed method for separating the predetermined sound source signal will be described later. In addition, the predetermined sound source signal here includes, for example, a voice (vocal) component.

Next, the acoustic device 1 applies a filter for generating a pseudo-reverberation sound to the sound source signal from which the predetermined sound source signal has been removed (hereinafter, may be referred to as the "removed sound source signal"), generates a reverberation signal indicating the pseudo-reverberation sound, and outputs it from the speakers RL and RR (step S2). Note that, as the filter, for example, an impulse response filter such as an FIR (Finite Impulse Response) filter or an IIR (Infinite Impulse Response) filter can be used, but is not limited to this.

In this manner, in this embodiment, a filter is applied only to a sound source signal from which a predetermined sound source signal (e.g., a voice component) that does not require processing to add artificial reverberation has been removed, and a reverberation signal is generated. In other words, a filter is not applied to a predetermined sound source signal among the sound source signals, and a reverberation signal is not generated.

As a result, even if the sound source signal includes a sound source signal that requires processing to add artificial reverberation and a sound source signal that does not (here, a specified sound source signal), a reverberation signal is generated only for the sound source signal that requires the processing, so that appropriate surround playback can be performed according to the sound source signal (more specifically, according to the content (type) of the sound source signal).

The above-mentioned filter is not applied to the audio components contained in the specified sound source signal, and they are reproduced as direct sound. Therefore, the listener L does not feel the envelopment LEV in the reproduced specified sound source signal (sound in this case). This is because the listener L can hear the reproduced sound more clearly if there is no envelopment LEV for sound (vocals) and the like.

In the above, an example in which a filter is applied only to a sound source signal from which a predetermined sound source signal has been removed has been shown, but the present invention is not limited to this. That is, for example, the acoustic device 1 may apply a filter to both the sound source signal from which a predetermined sound source signal has been removed and the predetermined sound source signal. In this case, for the sound source signal from which a predetermined sound source signal has been removed, a reverberation signal may be generated that relatively increases the reverberation level of the corresponding artificial reverberation sound, while for the predetermined sound source signal, a reverberation signal may be generated that relatively decreases the reverberation level of the corresponding artificial reverberation sound. The above-mentioned "reverberation level" is an index value that indicates the degree of reverberation of the artificial reverberation sound in a room (here, a vehicle interior) when the reverberation signal is reproduced from the speakers RL and RR, for example.

In the above, the specified sound source signal includes a voice component, but is not limited to this. In other words, the specified sound source signal may include a component of a sound that reverberates too much and sounds unnatural when the surround sound level is increased during surround playback, specifically, a transient sound component such as the sound of a percussion instrument such as a drum, in which there is silence between striking points.

<Configuration of an audio system including an audio device according to the first embodiment>
Next, the configuration of an audio system including the audio device 1 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of an audio system including the audio device 1 according to the first embodiment. In Fig. 2, only components necessary for explaining the features of this embodiment are shown as functional blocks, and descriptions of general components are omitted.

In other words, each component shown in FIG. 2 is a functional concept, and does not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each functional block is not limited to that shown, and all or part of it can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

As shown in FIG. 2, the acoustic system 100 includes an acoustic device 1, a sound source device 50, various sensors 60, and multiple speakers FL, FR, RL, and RR. Note that the acoustic system 100 according to this embodiment is mounted in a vehicle, but is not limited to this.

The sound source device 50 outputs a sound source signal to the acoustic device 1. The sound source signal is, for example, a stereo signal. The sound source signal becomes a spatially expanding sound image by outputting different signals from two speakers FL and FR, which are two channels, via the acoustic device 1.

The various sensors 60 include various sensors that detect the state of the vehicle. The various sensors 60 include sensors that can detect, for example, the running state of the vehicle, such as the vehicle speed, the open/closed state of the windows, the operating state of the air conditioner, the seating state of the occupants, fader adjustment instructions for the speakers FL, FR, RL, and RR (instructions for adjusting the front/rear balance by the occupants), whether or not the vehicle is in automatic driving, and output information indicating the detected state of the vehicle to the audio device 1. Note that, although the running state and the open/closed state of the windows are specifically shown above as the state of the vehicle, these are merely examples and are not limiting.

Multiple speakers FL, FR, RL, and RR are connected to the audio device 1. These speakers FL, FR, RL, and RR output the signal output from the audio device 1 as sound. For example, the speakers FL and FR output direct sound, which is a sound source signal, and the speakers RL and RR output pseudo-reverberation sound generated from the sound source signal, but are not limited to this.

The audio device 1 includes a control unit 2 and a storage unit 3. The control unit 2 includes an acquisition unit 21, a separation unit 22, and an output control unit 23. The audio device 1 includes a computer having, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, input/output ports, and various other circuits.

The computer's CPU functions as the acquisition unit 21, separation unit 22, and output control unit 23 of the control unit 2, for example, by reading and executing a program stored in the ROM.

In addition, at least one or all of the acquisition unit 21, separation unit 22, and output control unit 23 of the control unit 2 can be configured with hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The storage unit 3 corresponds to a RAM or a flash memory. The RAM or the flash memory can store information on various programs. The audio device 1 may also acquire the above-mentioned programs and various information via other computers or portable recording media connected via a wired or wireless network.

The acquisition unit 21 acquires various information and signals. For example, the acquisition unit 21 acquires a sound source signal from the sound source device 50. For example, the acquisition unit 21 acquires a sound source signal that is a stereo signal. Specifically, the acquisition unit 21 acquires sound source signals output from two speakers FL and FR, which are two channels. The acquisition unit 21 outputs the acquired sound source signals to the separation unit 22 and the output control unit 23.

In the following, among the stereo sound source signals, the sound source signal output (played) from the speaker FL, which is the left channel, may be referred to as the "Lch sound source signal," and the sound source signal output from the speaker FR, which is the right channel, may be referred to as the "Rch sound source signal."

The acquisition unit 21 acquires information indicating the vehicle state (e.g., the driving state such as the vehicle speed, the open/closed state of the windows, etc.) output from the various sensors 60, and outputs the acquired information to the output control unit 23.

The separation unit 22 performs a process of separating and extracting various sound source signals, including a predetermined sound source signal that does not require processing to add artificial reverberation, from the sound source signal. For example, in addition to the predetermined sound source signal described above, the separation unit 22 can separate and extract an L component sound source signal, an R component sound source signal, a reverberation component sound source signal, and the like.

The L component sound source signal is a sound source signal that includes a sound component (L component) that is reproduced on one of the two channels (speaker FL) of the speakers FL and FR. The R component sound source signal is a sound source signal that includes a sound component (R component) that is reproduced on the other channel (speaker FR). The L component sound source signal is an example of a first sound source signal, and the R component sound source signal is an example of a second sound source signal. The reverberation component sound source signal is a sound source signal that includes actual reverberation sound components such as early reflection sounds.

In the above, the separation unit 22 separates and extracts the predetermined sound source signal, the L component sound source signal, the R component sound source signal, and the reverberation component sound source signal from the sound source signal, but this is not limited thereto, and the separation unit 22 may be configured to separate and extract, for example, a portion of these various sound source signals. In addition, since the predetermined sound source signal includes a voice component as described above, hereinafter, the predetermined sound source signal may be referred to as the "voice sound source signal."

Here, the separation and extraction process by the separation unit 22 will be described with reference to FIG. 3. FIG. 3 is a diagram illustrating the separation and extraction process by the separation unit 22. As shown in FIG. 3, first, the separation unit 22 receives an Lch sound source signal and an Rch sound source signal from the acquisition unit 21.

The separation unit 22 then performs time-frequency analysis on the Lch sound source signal and the Rch sound source signal, respectively, to calculate Lch sound source data and Rch sound source data. For example, the separation unit 22 performs a short-time Fourier transform on the Lch sound source signal to convert it from the time domain to the time-frequency domain and calculate the Lch sound source data. Similarly, the separation unit 22 performs a short-time Fourier transform on the Rch sound source signal to calculate the Rch sound source data.

The Lch sound source data and the Rch sound source data indicate the frequency characteristics of the corresponding audio signal over time. In more detail, the Lch sound source data and the Rch sound source data are data in which the audio signal is divided into predetermined frequency bands and elapsed times, and a relative decibel value (not shown) is set for each divided region.

In this way, the separation unit 22 can accurately calculate the Lch sound source data and the Rch sound source data by performing time-frequency analysis on the sound source signal. In this embodiment, by using the accurately calculated sound source data, it becomes possible to perform appropriate surround playback according to the sound source signal.

The separation unit 22 then calculates difference information between the Lch sound source data and the Rch sound source data. The difference information is, for example, information indicating the L/R channel difference obtained by subtracting the Lch sound source data from the Rch sound source data, and is specifically level difference information and phase difference information. In more detail, the level (amplitude) difference information is the inter-channel level difference (ICLD (Inter-channel Level Difference)), and the phase difference information is the inter-channel phase difference (ICPD (Inter-channel Phase Difference)).

In this way, the separation unit 22 calculates difference information indicating the difference between the sound source signals reproduced by each of the two channels (speakers FL and FR), and by using this difference information, it becomes possible to perform accurate separation of audio sound source signals, etc., as described below.

Then, the separation unit 22 generates masks for separating and extracting various sound source signals based on the difference information. In detail, the separation unit 22 generates masks for separating and extracting the voice sound source signal, the L component sound source signal, the R component sound source signal, and the reverberation sound component sound source signal, based on the difference information. Note that, as the mask, for example, a binary mask that takes only two values, 0 or 1, can be used, but is not limited to this.

In the following, a mask for separating and extracting an audio source signal may be referred to as an "audio mask." Furthermore, masks for separating and extracting an L component audio source signal, an R component audio source signal, and a reverberation component audio source signal may be referred to as an "L component mask," an "R component mask," and a "reverberation mask," respectively.

First, we will explain how to generate an "audio mask." Note that, because audio (vocals) is usually recorded in mono, we will assume here that the Lch audio source signal and the Rch audio source signal contain the same audio audio source signal.

The separation unit 22 sets the audio mask value (more precisely, the value in each region) based on ICLD, which is level difference information, and ICPD, which is phase difference information, from the difference information, and on the setting condition 1 below, to generate the audio mask.

[Setting condition 1]
Mask value 1: Region where |ICLD|<threshold a and |ICPD|<threshold b Mask value 0: Regions other than the above Here, thresholds a and b are both set to relatively small values. More specifically, thresholds a and b are set to values that allow for estimation that both ICLD and ICPD are 0 or close to 0, but are not limited thereto and can be set to any value.

As described above, the Lch sound source signal and the Rch sound source signal contain the same audio sound source signal, so setting condition 1 sets the audio mask to a value of 1 in the area containing the audio sound source signal (in other words, the area where ICLD and ICPD are both 0 or close to 0) and to 0 in other areas.

Next, the separation unit 22 separates and extracts the audio source signal using the generated audio mask. Specifically, the separation unit 22 separates and extracts the audio source signal from the time-frequency domain where the value of the audio mask is set to 1.

For example, the separation unit 22 applies an audio mask to the Lch sound source data and the Rch sound source data and filters them to separate the audio sound source signal from each of the Lch sound source data and the Rch sound source data. The separation unit 22 then averages the two separated audio sound source signals and extracts the averaged signal as the audio sound source signal.

In the above, an example was shown in which the audio mask is applied to both the Lch sound source data and the Rch sound source data, but this is not limited to this. For example, the audio mask may be applied to one of the Lch sound source data and the Rch sound source data, and the audio source signal may be separated and extracted from one of the data.

In the above, in the voice mask, values in areas other than the area where ICLD and ICPD are 0 or near 0 are set to 0. However, in addition to or instead of this, values in frequency areas other than the voice band (300 to 3400 Hz, as an example) may be set to 0 in advance. That is, here, a process of separating and extracting a voice source signal is performed. For this reason, by setting values in frequency areas other than the voice band, in other words frequency areas where a voice source signal cannot exist, to 0 in advance as described above, it is possible to avoid erroneous detection that a voice source signal exists in a frequency area where a voice source signal cannot exist, and therefore to separate and extract the voice source signal with high accuracy.

In addition, in the audio mask, the value of the frequency range below the audio band may be preset to 1. This makes it possible to separate and extract the audio source signal of an instrument (such as a bass drum) that contains a mono component and has a frequency below the audio band.

Next, we will explain how to generate an "L component mask." For example, there may be cases where the components of a certain instrument sound (a piano as an example) are contained in greater quantities in the Lch sound source signal than in the Rch sound source signal. The L component mask is a mask for separating and extracting such instrument sound components as the L component sound source signal.

Specifically, the separation unit 22 sets the value of the L component mask (more precisely, the value in each region) based on the level difference information ICLD, the phase difference information ICPD, and the setting condition 2 below, and generates the L component mask.

[Setting condition 2]
Mask value 1: Area where ICLD<threshold c and ICPD<threshold d Mask value 0: Area other than the above Here, threshold c and threshold d are both set to values equal to or less than 0. In detail, threshold c and threshold d are set to values that allow estimation that ICLD and ICPD are both negative values, but are not limited to this and can be set to any value.

That is, as described above, the Lch sound source signal contains a large amount of a certain instrument sound (piano in this case), so in the difference information obtained by subtracting the Lch sound source data from the Rch sound source data, ICLD and ICPD in the area containing the certain instrument sound components are negative values. Therefore, according to setting condition 2, the L component mask is set to a value of 1 in the area containing the certain instrument sound (piano in this case) components (in other words, areas where ICLD and ICPD are both negative values), and to 0 in other areas.

Then, the separation unit 22 separates and extracts the L component sound source signal using the generated L component mask. Specifically, the separation unit 22 separates and extracts the L component sound source signal from the time-frequency domain where the value of the L component mask is set to 1. For example, the separation unit 22 applies the L component mask to the Lch sound source data and filters it, thereby separating the L component sound source signal from the Lch sound source data.

Next, we will explain how to generate an "R component mask." For example, there may be cases where the Rch sound source signal contains more components of a certain instrument sound (a guitar as an example) than the Lch sound source signal. The R component mask is a mask for separating and extracting such instrument sound components as the R component sound source signal.

Specifically, the separation unit 22 sets the value of the R component mask (more precisely, the value in each region) based on the level difference information ICLD, the phase difference information ICPD, and the following setting condition 3, and generates the R component mask.

[Setting condition 3]
Mask value 1: Area where ICLD > threshold e and ICPD > threshold f Mask value 0: Area other than the above Here, threshold e and threshold f are both set to values equal to or greater than 0. In detail, threshold e and threshold f are set to values that allow for estimation that both ICLD and ICPD are positive values, but are not limited thereto and can be set to any values.

That is, as described above, the Rch sound source signal contains a large amount of a certain instrument sound (here, a guitar), so in the difference information obtained by subtracting the Lch sound source data from the Rch sound source data, ICLD and ICPD in the area containing the certain instrument sound component are positive values. Therefore, according to setting condition 3, the R component mask is set to a value of 1 in the area containing the certain instrument sound (here, a guitar) component (in other words, the area where ICLD and ICPD are both positive values), and to 0 in other areas.

Then, the separation unit 22 separates and extracts the R component sound source signal using the generated R component mask. Specifically, the separation unit 22 separates and extracts the R component sound source signal from the time-frequency domain where the value of the R component mask is set to 1. For example, the separation unit 22 applies the R component mask to the Rch sound source data and filters it, thereby separating the R component sound source signal from the Rch sound source data.

Next, we will explain the "reverberation mask." For example, the Lch sound source signal and the Rch sound source signal may contain actual reverberation components such as early reflection sounds in addition to the monaural components such as the above-mentioned voice, L components, and R components. Some of these reverberation components tend to make the sound image unclear when played back, and the reverberation mask is a mask for separating and extracting such reverberation components as reverberation component sound source signals.

Specifically, the separation unit 22 sets the reverberation mask value (more precisely, the value in each region) based on the level difference information ICLD, the phase difference information ICPD, and the setting condition 4 below, and generates the reverberation mask.

[Setting condition 4]
Mask value 0: Region where |ICLD|<threshold a, and |ICPD|<threshold b Mask value 0: Region where ICLD<threshold c, and ICPD<threshold d Mask value 0: Region where ICLD>threshold e, and ICPD>threshold f Mask value 1: Regions other than the above As can be seen from the contents of setting condition 4, the regions where the mask value is set to 1 in setting conditions 1 to 3 described above are set to 0 in setting condition 4, and the values of the other regions are set to 1. That is, for example, a region where ICLD and ICPD have reversed positive and negative values, or a region where ICPD is ≈ 180°, is a region that contains reverberation that tends to make the sound image unclear during playback, so the value of that region is set to 1.

Next, the separation unit 22 separates and extracts the reverberation component sound source signal using the generated reverberation mask. Specifically, the separation unit 22 separates and extracts the reverberation component sound source signal from the time-frequency domain where the value of the reverberation mask is set to 1.

For example, the separation unit 22 applies a reverberation mask to the Lch sound source data and the Rch sound source data and filters them to separate the reverberation component sound source signals from the Lch sound source data and the Rch sound source data. The separation unit 22 then averages the two separated reverberation component sound source signals and extracts the averaged signal as the reverberation component sound source signal.

In the above, an example was shown in which the reverberation mask is applied to both the Lch sound source data and the Rch sound source data, but this is not limited to the above. For example, the reverberation mask may be applied to either the Lch sound source data or the Rch sound source data, and the reverberation component sound source signal may be separated and extracted from one of the data.

In the above, since the value of each mask is set to 1 or 0, separation distortion may occur in the separation process of each sound source signal. In such a case, separation distortion may be reduced by combining a low-pass filter with each mask. Also, for areas where the value of each mask is set to 0, a relaxed value such as 0.1 may be set to reduce separation distortion.

Continuing with the explanation of FIG. 2, the separation unit 22 outputs the separated and extracted voice sound source signal, L component sound source signal, R component sound source signal, and reverberation component sound source signal to the output control unit 23. The separation unit 22 also removes the voice sound source signal (predetermined sound source signal) from the sound source signal to generate a removed sound source signal, and outputs the generated removed sound source signal to the output control unit 23.

The output control unit 23 performs predetermined processing on the sound source signal input from the acquisition unit 21, and the voice sound source signal, L component sound source signal, R component sound source signal, reverberation sound component sound source signal, and removal sound source signal input from the separation unit 22, and outputs them from the speakers FL, FR, RL, and RR.

For example, the output control unit 23 performs D/A conversion on a sound source signal including a direct sound, etc., amplifies the D/A converted sound source signal, and outputs (plays) it from the speakers FL and FR. Here, the sound image of the played sound will be described with reference to Figures 4A to 4G.

Figures 4A to 4G are diagrams for explaining the sound images of the sound reproduced by the output control unit 23. Note that Figure 4A shows a state in which only the sound source signal including the direct sound etc. is output and reproduced, in other words, a state in which the pseudo reverberation sound etc. is not reproduced (added). Also, in Figures 4A to 4F, the sound image of the voice (vocals) is indicated by the symbol A1. Also, as described above, the sound source signal is a stereo signal, and includes an Lch sound source signal and an Rch sound source signal. In Figures 4A to 4F, the sound image of the instrument sound etc. contained more in the Lch sound source signal than in the Rch sound source signal is indicated by the symbol AL, and the sound image of the instrument sound etc. contained more in the Rch sound source signal than in the Lch sound source signal is indicated by the symbol AR.

As shown in FIG. 4A, when the output control unit 23 outputs a sound source signal including direct sound etc. from the speakers FL and FR, the sound image A1 of the voice and the sound images AL and AR of the instrument sound are localized at a relatively narrow interval between the speakers FL and FR.

Continuing with the explanation of FIG. 2, the output control unit 23 has a filter 23a, and generates and outputs a pseudo-reverberation sound using the filter 23a. The filter 23a is a filter for generating a pseudo-reverberation sound. Note that, as the filter, for example, an FIR filter or an IIR filter can be used, but is not limited to these.

For example, the output control unit 23 applies a filter 23a to the sound source signal from which a specific sound source signal has been removed, i.e., the removed sound source signal, to generate a reverberation signal that represents a pseudo-reverberation sound. The output control unit 23 then performs D/A conversion on the reverberation signal, amplifies the D/A converted reverberation signal, and outputs it from the speakers RL and RR. This adds the pseudo-reverberation sound to the direct sound, etc.

Therefore, as shown in FIG. 4B, for example, the output control unit 23 adjusts the volume of the pseudo-reverberation sound output from the speakers RL and RR, and positions the sound image B of the pseudo-reverberation sound around the listener L, so that the listener L can feel a sufficient sense of being enveloped LEV.

In this way, the output control unit 23 according to this embodiment applies the filter 23a only to the sound source signal (removed sound source signal) from which the voice sound source signal that does not require processing to add artificial reverberation has been removed, and generates and outputs a reverberation signal.

As a result, even if the sound source signal contains sound source signals that require processing to add artificial reverberation and sound source signals that do not (here, voice sound source signals), a reverberation signal is generated and output only for the sound source signals that require processing, making it possible to perform appropriate surround playback according to the sound source signal (more specifically, according to the content (type) of the sound source signal).

In the above, the output control unit 23 applies the filter 23a only to the sound source signal from which the voice source signal has been removed (removed sound source signal), but this is not limited to this. That is, for example, the output control unit 23 may apply the filter 23a to both the sound source signal from which the voice source signal (predetermined sound source signal) has been removed and the voice source signal. At this time, the output control unit 23 outputs the pseudo reverberation sound corresponding to the sound source signal from which the voice source signal has been removed (removed sound source signal) so that the reverberation level of the pseudo reverberation sound corresponding to the voice source signal is higher than the reverberation level of the pseudo reverberation sound corresponding to the voice source signal. In other words, the output control unit 23 outputs the pseudo reverberation sound corresponding to the voice source signal so that the reverberation level of the pseudo reverberation sound corresponding to the removed sound source signal is lower than the reverberation level of the pseudo reverberation sound corresponding to the removed sound source signal. In other words, the filter 23a is set to generate a reverberation signal that has the reverberation level of the pseudo reverberation sound described above.

This makes it possible to suppress the listener L from getting an unnatural impression that the sound image and tone of the voice (vocals) are extremely clear and appear to be in front. In detail, for example, when a music sound source signal containing vocals and instrumental sounds is given a large reverberation only to the instrumental sound source signal (i.e., the removed sound source signal) and no reverberation is given to the vocal sound source signal, the listener L may get an unnatural impression that the sound image and tone of only the vocals are extremely clear and appear to be in front. This is because there is a psychological characteristic that when there is a large reverberation and there are many reflected sounds, the sound image (here, the sound image of the instrument) feels far away, and accordingly, the sound image without any reverberation (here, the vocal sound image) feels closer to the listener. Therefore, by applying the filter 23a to both the sound source signal from which the voice sound source signal has been removed and the voice sound source signal, and by making the reverberation levels different as described above, it is possible to suppress the listener L from getting the unnatural impression described above.

The output control unit 23 may select the speaker that outputs the removed sound source signal to be played in surround from among the speakers FL, FR, RL, and RR depending on the content (type) of the removed sound source signal. As an example, the output control unit 23 may improve the sense of envelopment LEV by selecting the speaker that outputs the sound source signal and the reverberation signal so that the direct sound and the artificial reverberation sound are played on either side of the listener L.

To be more specific, the reverberation signal (in other words, the removed sound source signal to which the filter 23a is applied and which is played in surround) includes the L component sound source signal and the R component sound source signal described above, and the L component sound source signal and the R component sound source signal are separated and extracted by the separation unit 22. Therefore, of the reverberation signal (the removed sound source signal to which the filter 23a is applied), the output control unit 23 plays (outputs) the L component sound source signal in surround from the speaker RR, and plays the R component sound source signal in surround from the speaker RL.

As a result, the listener L is positioned between the speaker FL, which reproduces the L component of the direct sound, and the speaker RR, which reproduces the L component of the artificial reverberation sound in surround sound, and between the speaker FR, which reproduces the R component of the direct sound, and the speaker RL, which reproduces the R component of the artificial reverberation sound in surround sound. Therefore, the sound image B of the artificial reverberation sound is more likely to be localized around the listener L, and as a result, it is possible to improve the sense of envelopment LEV for the listener L.

The output control unit 23 also applies a filter 23a to the separated reverberation component sound source signal to generate a reverberation signal. The output control unit 23 may then output the reverberation signal generated from the reverberation component sound source signal from the speakers RL and RR. This causes a pseudo-reverberation sound due to the reverberation component sound source signal to be added to the direct sound, etc. In more detail, for example, a pseudo-reverberation sound of a reverberation component that is likely to cause an unclear sound image during playback among the actual reverberation components is added to the direct sound, thereby clarifying the sound image of the reverberation component and further improving the sense of envelopment LEV.

Incidentally, for example, in the passenger compartment of a vehicle, as shown by the imaginary lines in FIG. 4B, another listener Lx may be seated at the rear of the passenger compartment (e.g., in the back seat). In this case, if the above-described process of adding artificial reverberation is performed, there is a risk that the reverberation sound, including the artificial reverberation sound, will become excessive for the listener Lx.

Therefore, the output control unit 23 according to this embodiment may suppress the reverberation sound from becoming excessive for the listener Lx by performing a process of adding a pseudo reverberation sound according to the state of the vehicle. In detail, the output control unit 23 may suppress the reverberation sound from becoming excessive by applying a filter 23a to the removed sound source signal according to the state of the vehicle and outputting the applied removed sound source signal (i.e., the reverberation signal) from a channel (e.g., speakers RL, RR, etc.).

For example, as shown by the imaginary lines in FIG. 4B, when the vehicle is in a state in which the speaker Rx is located behind the listener Lx in the rear seat, the output control unit 23 outputs the removed sound source signal (reverberation signal) to which the filter 23a has been applied from the speaker Rx. This makes it possible to localize the sound image of the pseudo-reverberation sound around the listener Lx, although this is not shown in the figure. Therefore, the sense of envelopment LEV can be ensured for the listener Lx as for the listener L, and excessive reverberation can be prevented.

For example, the output control unit 23 may add pseudo-reverberation in response to a fader adjustment instruction (an example of the vehicle state) for the speakers FL, FR, RL, and RR. For example, when a fader adjustment instruction indicating that playback should be focused on the rear seats is detected from the various sensors 60 as the vehicle state, the output control unit 23 may weaken the reverberation signal output from the speakers RL and RR or prohibit the output of the reverberation signal. This reduces or eliminates the pseudo-reverberation, thereby preventing the reverberation from becoming excessive for the listener Lx.

For example, the output control unit 23 may add a pseudo-reverberation sound depending on the seating state of the occupants (one example of the state of the vehicle). For example, when the various sensors 60 detect a seating state of the occupants indicating that no occupants are seated in the rear seats, i.e., that no listener Lx is present, as the state of the vehicle, the output control unit 23 may output a reverberation signal from the speakers RL and RR. This prevents the excessive reverberation sound described above from occurring.

For example, the output control unit 23 may add a pseudo reverberation sound according to at least one of the vehicle conditions, such as the vehicle window opening/closing state, the air conditioner operating state, and the vehicle speed. For example, when the rear seat window is open, the air conditioner is operating at a relatively strong wind, or the vehicle is running at a relatively high speed, a relatively loud noise is generated in the rear seat. Therefore, it is estimated that the listener Lx in the rear seat is in an environment where the above-mentioned reverberation sound is difficult to be felt. Therefore, the output control unit 23 may output a reverberation signal from the speakers RL and RR when the above-mentioned open window, strong wind operation of the air conditioner, high speed driving, etc. are detected as the vehicle conditions by the various sensors 60. As a result, the listener Lx is in an environment where the reverberation sound is difficult to be felt, and therefore the reverberation sound can be prevented from becoming excessive for the listener Lx.

In the above description, the reverberation signal is output from the speakers RL and RR located behind the listener L, but this is not limited to the above. The reverberation signal may be output from other speakers, such as a speaker (not shown) located above the listener L.

The output control unit 23 can also perform localization control of the sound image. Specifically, the output control unit 23 can output the audio source signal so that the sound image A1 of the sound generated by the separated audio source signal is localized at a predetermined position. For example, as shown in FIG. 4C, the output control unit 23 can localize the sound image A1 of the sound (vocals) generated by the audio source signal at a position displaced toward the listener L by emphasizing and outputting the separated audio source signal from the speakers RL and RR. This makes it possible to make it difficult for the sound image A1 of the sound and the sound images AL and AR of the instrument sound to overlap, and thus makes it possible to clarify the sounds corresponding to each of the sound images A1, AL, and AR. Note that "emphasis" in this specification means, for example, adding and reproducing new sounds corresponding to various signals, or increasing the volume of the reproduced sound, but is not limited thereto.

In the above, the specified position is set to a position displaced toward the listener L, but this is not limited to this and can be set to any position. For example, although not shown in the figure, a speaker may be placed above the listener L and the audio source signal may be emphasized and output from the speaker, so that the audio image A1 is localized at a position displaced upward. As a result, the sound image A1 and the sound images AL and AR are shifted in the height direction, which can increase the three-dimensional effect of the sound and improve the sense of realism, for example.

The output control unit 23 may also emphasize and output the separated audio source signal from, for example, speakers FL and FR. This adds the sound of the emphasized audio source signal to the sound (vocals) contained in the direct sound, thereby making it possible to make the sound image A1 of the audio clearer.

The output control unit 23 can also control the sound image width of the sound image. Specifically, as shown in FIG. 4D, the output control unit 23 may output the separated L component sound source signal from one speaker (channel) FL and control the sound image width of the sound due to the L component sound source signal. For example, the output control unit 23 emphasizes and outputs the separated L component sound source signal from the speaker FL. This adds the instrument sound of the emphasized L component sound source signal to an instrument sound (e.g., a piano), which is an example of an L component contained in the direct sound, and thus makes it possible to expand the sound image width of the sound image AL of the instrument sound to the left.

Similarly, the output control unit 23 may output the separated R component sound source signal from the other speaker (channel) FR and control the sound image width of the sound due to the R component sound source signal. For example, the output control unit 23 emphasizes and outputs the separated R component sound source signal from the speaker FR. This adds the instrument sound of the emphasized R component sound source signal to an instrument sound (e.g., a guitar), which is an example of the R component contained in the direct sound, thereby making it possible to expand the sound image width of the sound image AR of the instrument sound to the right.

Also, as shown by the imaginary lines in FIG. 4D, the output control unit 23 may output the separated L component sound source signal from the speaker RL, and control the sound image width of the sound image ALx of the sound (here, the instrument sound) due to the L component sound source signal to be expanded rearward. Similarly, the output control unit 23 may output the separated R component sound source signal from the speaker RR, and control the sound image width of the sound image ARx of the sound (here, the instrument sound) due to the R component sound source signal to be expanded rearward. This makes it possible to localize the sound images ALx and ARx at a position that envelops the listener L.

Here, for example, noises such as driving sounds and air conditioner sounds (driving noises) are generated inside the vehicle cabin. In the example of FIG. 4E, the sound image of the noise is indicated by the symbol C. The sound image C of the noise may be localized near the listener L. In this case, the sound image C of the noise may be close to or overlap with the sound image A1 of the voice or the sound images AL and AR of the musical instrument sound, so that the noise may mix with the voice or music, causing the sound images A1, AL, and AR to become unclear.

For example, the output control unit 23 may output the separated audio source signal from the speakers FL and FR with emphasis, or may weaken or prohibit the output of the audio source signal that was output with emphasis from the speakers RL and RR, thereby localizing the audio image A1 of the audio source signal at a position displaced in a direction away from the noise image C (for example, forward) (see arrow D1). This makes it possible, for example, to make it difficult for the audio image A1 and the noise image C to overlap, in other words, to make it difficult for the noise to mix with the audio, and to clarify the audio image A1.

For example, the output control unit 23 may emphasize the separated L component sound source signal and output it from the speaker FL, or may weaken or prohibit the output of the L component sound source signal that was emphasized and output from the speaker RL. As a result, the output control unit 23 may localize the sound image AL of the instrument sound of the L component sound source signal at a position displaced in a direction away from the sound image C of the noise (for example, forward) (see arrow D2a), or may expand the sound image width of the sound image AL to the left (see arrow D2b). As a result, for example, it becomes possible to make it difficult for the sound image AL of the instrument sound and the sound image C of the noise to overlap, in other words, it becomes possible to make it difficult for the noise to mix with the instrument sound, and it becomes possible to clarify the sound image AL of the instrument sound.

Similarly, for example, the output control unit 23 may emphasize the separated R component sound source signal and output it from the speaker FR, or weaken or prohibit the output of the R component sound source signal that was emphasized and output from the speaker RR. As a result, the output control unit 23 may localize the sound image AR of the instrument sound of the R component sound source signal at a position displaced away from the sound image C of the noise (see arrow D3a), or expand the sound image width of the sound image AR to the right (see arrow D3b). As a result, for example, it becomes possible to make it difficult for the sound image AR of the instrument sound and the sound image C of the noise to overlap, in other words, it becomes possible to make it difficult for the noise to mix with the instrument sound, and the sound image AR of the instrument sound can be made clearer.

The output control unit 23 may control the position and width of the sound images A1, AL, and AR depending on the position of the noise sound image C. That is, for example, the position of the noise sound image C changes depending on the driving conditions such as the vehicle speed, the open/closed state of the windows, the operating state of the air conditioner, etc. Therefore, for example, the correlation between the driving conditions, the open/closed state of the windows, the operating state of the air conditioner, and the position of the noise sound image C may be calculated in advance through experiments, etc., and the output control unit 23 may control the position and width of the sound images A1, AL, and AR based on information indicating the calculated correlation. The position of the noise sound image C may be detected by analyzing sounds collected using a microphone (not shown) or the like.

Here, for example, a vehicle may be configured to be able to run using automatic driving control that does not require the occupant (driver) to operate the vehicle. When such automatic driving control is being executed, the driving burden on the occupant (listener L) is reduced, and image 31 may be projected on the vehicle window or outside the vehicle, as shown in FIG. 4F. Note that image 31 can be set to any type of image, but in this case it includes an image of an instrument being played that corresponds to the instrument sound included in the sound source signal, or an image of a singer emitting sound (vocals).

In this way, when the vehicle is in a state in which automatic driving control is being executed and image 31 is being displayed, output control unit 23 may perform localization control and sound image width control of sound images A1, AL, and AR in accordance with image 31. For example, output control unit 23 localizes sound image A1 of the sound from the sound source signal at a position where it overlaps with image 31 by emphasizing and outputting the separated audio sound source signal from speakers FL and FR. Also, for example, output control unit 23 localizes sound image AL of the instrument sound of the L component sound source signal at a position where it overlaps with image 31 or controls the sound image width of sound image AL by emphasizing and outputting the separated L component sound source signal from speakers FL and RL. Similarly, the output control unit 23 emphasizes and outputs the separated R component sound source signal from the speaker FR or the speaker RR, localizes the sound image AR of the instrument sound of the R component sound source signal at a position where it overlaps with the image 31, and controls the sound image width of the sound image AR. As a result, the sound images A1, AL, and AR overlap with the image 31 of the singer and the instrument, making it possible for the listener L to feel the spread of the sound images A1, AL, and AR in accordance with the image 31.

As shown in FIG. 4G, the seat orientation of the occupant, the listener L, may be changed, for example, when performing automatic driving control or when parking. In such a case, the output control unit 23 may perform localization control or sound image width control of the sound images A1, AL, and AR according to the seat orientation (or the face orientation of the listener L). For example, the output control unit 23 may localize the sound image A1 of the sound by the sound source signal in a position in front of the listener L by emphasizing and outputting the separated voice sound source signal from a speaker selected from the speakers FL, FR, RL, and RR according to the seat orientation. Also, for example, the output control unit 23 may localize the sound image AL of the instrument sound of the L component sound source signal in a position to the left of the listener L or control the sound image width of the sound image AL by emphasizing and outputting the separated L component sound source signal from a speaker selected according to the seat orientation. Similarly, the output control unit 23 localizes the sound image AR of the instrument sound of the R component sound source signal to the right of the listener L and controls the sound image width of the sound image AR by emphasizing and outputting the separated R component sound source signal from a speaker selected according to the seat orientation, etc. This makes it possible to localize (place) the sound images A1, AL, and AR in positions that correspond to the state of the vehicle, including the orientation of the seat (in other words, the orientation of the listener L). Note that the orientation of the face of the listener L described above is detected by an in-car camera (not shown), but is not limited to this.

In the above, the localization control of the sound images A1, AL, and AR is performed according to the orientation of the seats and the orientation of the face of the listener L, but the present invention is not limited to this and the localization control may be performed according to instructions from the listener L (passenger), for example. In the localization control, the sound images A1, AL, and AR can be made clearer by preventing the sound images A1, AL, and AR from overlapping with the noise sound image C, as shown in FIG. 4G.

Continuing with the explanation of Figure 2, for example, the audio contained in the sound source signal may include vocals and voice guidance in navigation, etc. In such a case, the voice guidance may overlap with the vocals, making it difficult for the listener L to hear the voice guidance.

For example, the output control unit 23 outputs the sound source signal from which the audio sound source signal including vocals has been removed (removed sound source signal) and the sound source signal including the audio guidance as direct sounds from the speakers FL and FR. As a result, although not shown in the figure, the sound images AL and AR of the instrument sounds and the sound image of the audio guidance are localized in the vehicle cabin. Therefore, the audio guidance does not overlap with the vocals, and the listener L can easily hear the audio guidance. In addition, the output control unit 23 at this time can also provide audio guidance without lowering the audio volume.

<Control process of the audio device according to the first embodiment>
Next, a specific processing procedure in the audio device 1 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing procedure executed by the audio device 1 according to the first embodiment.

As shown in FIG. 5, the control unit 2 of the acoustic device 1 acquires a sound source signal from the sound source device 50 (step S10). Next, the control unit 2 separates a voice sound source signal, an L component sound source signal, an R component sound source signal, and a reverberation sound component sound source signal from the sound source signal (step S11).

Next, the control unit 2 removes the voice source signal from the sound source signal, and applies the filter 23a to the sound source signal from which the voice source signal has been removed and to the reverberation sound component sound source signal to generate a reverberation sound signal (step S12).

Next, the control unit 2 controls the output of the sound source signal, the separated voice sound source signal, the L component sound source signal, the R component sound source signal, and the generated reverberation sound signal (step S13).

As described above, the acoustic device 1 according to the first embodiment includes a separation unit 22 and an output control unit 23. The separation unit 22 separates a predetermined sound source signal (audio sound source signal) that does not require processing to add artificial pseudo-reverberation from the sound source signal, and removes the separated predetermined sound source signal from the sound source signal. The output control unit 23 applies a filter 23a for generating pseudo-reverberation to the sound source signal from which the predetermined sound source signal has been removed by the separation unit 22, and outputs the result. This makes it possible to perform appropriate surround reproduction according to the sound source signal.

In addition, because the separation unit 22 uses time-frequency analysis, it is possible to separate various sound source signals such as audio source signals with a relatively low amount of calculation. In addition, in this embodiment, sound image localization control, sound image width control, optimization of the envelopment feeling LEV, etc. are performed for each of the various sound source signals such as the separated audio source signals, so that the overall spatial impression of the sound reproduced from the speakers FL, FR, RL, and RR can be improved.

In addition, in this embodiment, the process of separating various sound source signals, such as a voice sound source signal, from the sound source signal is used in combination with the filter 23a. This allows the filter length of the filter 23a to be shorter than when only the filter 23a is used, for example, and therefore makes it possible to reduce the amount of calculation in the control unit 2.

For example, if the sound source signal contains a short-duration instrument sound such as a percussion instrument, the frequency characteristics may contain prominent components whose levels stand out near a certain frequency. If such prominent components are present, the reverberation sound, including pseudo-reverberation sound, tends to become excessive during surround playback. Also, the higher the frequency, the easier it is for the reverberation sound to be separated, and the more likely it is that the reverberation sound will become excessive. Therefore, for example, the control unit 2 can reduce the influence of the prominent components by performing a process that creates frequency characteristics that decay more linearly or smoothly as the frequency increases, and therefore it becomes possible to prevent the reverberation sound from becoming excessive, even for a sound source signal that contains a short-duration instrument sound such as a percussion instrument.

In addition, the control unit 2 may, for example, when outputting the reverberation signal to each speaker FR, FL, RL, RR (surround channel), introduce a random delay or impart randomness to the time characteristics such as the rise time of the filter 23a, thereby reducing the cross-correlation. This allows the listener L to experience the artificial reverberation sound as a more natural reverberation sound.

Second Embodiment
<Configuration of the audio device according to the second embodiment>
Next, the configuration of the acoustic device 1 according to the second embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the configuration of an acoustic system 100 including the acoustic device 1 according to the second embodiment. Note that, in the following, the same reference numerals are used for the configuration common to the first embodiment, and the description thereof will be omitted.

The control unit 2 of the acoustic device 1 according to the second embodiment includes a filter setting unit 24 that sets the gain of the filter 23a that generates the pseudo-reverberation sound. In the second embodiment, the filter setting unit 24 changes the gain of the filter 23a for each sound source (musical instrument or voice), thereby outputting a more natural pseudo-reverberation sound from the sound source signal of one song. Specifically, in the second embodiment, the filter setting unit 24 sets a filter with an optimized gain for each sound source based on the characteristics of each sound source contained in the sound source signal, thereby realizing high-quality surround playback.

To explain more specifically below, the acquisition unit 21 of the control unit 2 acquires sound source information related to the sound source signal. The sound source information is, for example, the type (genre) of the sound source signal and information on the recording environment, but is not limited to this. The type of sound source signal is, for example, voice, musical instruments (percussion instruments, wind instruments, etc.), classical music, etc. The recording environment is, for example, information on the location where the recording was made, such as a recording studio or concert hall.

The acquisition unit 21 acquires sound source information, for example, through input by a user such as the listener L, or acquires sound source information from a server or the like via the Internet. The acquisition unit 21 may also acquire sound source information by analyzing the acquired sound source signal. The acquisition unit 21 outputs the acquired sound source information to the filter setting unit 24.

The filter setting unit 24 includes a detection unit 24a and a determination unit 24b. The detection unit 24a detects feature information, which is a feature of the acoustic components contained in the sound source signal, based on the sound source signal.

For example, the detection unit 24a detects channel relationship information related to the relationship between the sound source signals reproduced on each of the two channels as feature information related to the features of the acoustic components. Specifically, the channel relationship information includes information on the LR channel difference and the LR channel correlation.

The LR channel difference is the above-mentioned inter-channel level difference (ICLD) or inter-channel time difference (Inter-channel Time Difference), but is not limited to these. The LR channel correlation is information about the correlation components of two sound source signals reproduced in stereo on two channels. Specifically, the LR channel correlation is the inter-channel cross correlation (ICC), but is not limited to this.

In this way, the detection unit 24a can detect the acoustic components with high accuracy by detecting channel relationship information as feature information, and therefore can generate optimal pseudo-reverberation sounds by filter processing in the output control unit 23.

The detection unit 24a then continuously detects the channel relationship information at regular intervals, arranges it in a time series, and calculates a moving average of the detected time-series channel relationship information, thereby detecting the moving average as feature information. This makes it possible to smooth out any sudden changes in the channel relationship information, and therefore to smooth out any sudden changes in the gain determined by the subsequent determination unit 24b. As a result, it is possible to smooth out the changes in the pseudo-reverberation sound generated by the output control unit 23, thereby achieving more natural surround playback.

The detection unit 24a maximizes the channel relationship information for sections of the time-series channel relationship information where the sound pressure level (amplitude) of the sound source signal is less than a predetermined value, and then calculates a moving average. In other words, the detection unit 24a calculates a moving average so as to control the time-series channel relationship information so that surround sound is not output for sections where the sound pressure level of the sound source signal is less than a predetermined value, and surround sound is not output around the section where the sound pressure level is less than the predetermined value.

This lowers the upper limit at which the law of the first wave front applies when there is silence between beats, such as in a solo percussion performance, so the filter gain for the percussion instruments is lowered to control the surround level to an appropriate level.

The detection unit 24a also detects information related to the duration and frequency of the acoustic components as feature information. The duration is the time that the sound source signal continues. Specifically, the detection unit 24a calculates the envelope of the sound source signal, differentiates the calculated envelope, and then calculates the envelope again to detect the duration from the slope of the obtained envelope. The frequency is information related to the frequency components of the sound source signal. Specifically, the detection unit 24a detects the center of gravity of the frequency of the sound source signal in the sound source data obtained by performing a short-time Fourier transform on the sound source signal.

In this way, the detection unit 24a can detect acoustic components with high accuracy by detecting duration and frequency information as feature information, and therefore can generate optimal pseudo-reverberation sounds through filtering in the output control unit 23 at the downstream stage.

The determination unit 24b determines the gain of the filter 23a used by the output control unit 23. Specifically, the determination unit 24b determines the gain of the filter 23a based on at least one of the feature information detected by the detection unit 24a and the sound source information acquired by the acquisition unit 21.

Specifically, the determination unit 24b corrects the threshold range determined from the first wave front law based on at least one of the feature information and the sound source information, and determines the gain based on the corrected threshold range. Here, the process of determining the gain based on the first wave front law is described with reference to FIG. 7.

Figure 7 is a diagram explaining the gain determination process performed by the determination unit 24b. As shown in Figure 7, the determination unit 24b first corrects the threshold range obtained from the first wave front law based on the feature information (channel relationship information, duration, and frequency) and sound source information.

For example, the threshold ranges shown in the graph in the middle of Figure 7, regions R1 and R2, are set as the reference ranges. In this case, the determination unit 24b corrects regions R1 and R2 by correcting the thresholds TH1 and TH2 on the two axes of time and amplitude based on the feature information and sound source information.

For example, the larger the channel relationship information, the more the determination unit 24b corrects the thresholds TH1 and TH2 so that the time and amplitude become larger. In other words, the larger the channel relationship information, the more the determination unit 24b corrects the regions R1 and R2 so that they become larger.

On the other hand, the smaller the channel relationship information, the more the determination unit 24b corrects the thresholds TH1 and TH2 in the direction of decreasing the time and amplitude. In other words, the smaller the channel relationship information, the more the determination unit 24b corrects the regions R1 and R2 to become smaller.

The channel relationship information becomes larger the greater the inter-channel level difference, or the greater the inter-channel time difference, or the lower the cross-correlation (the more uncorrelated components there are).

Furthermore, the determination unit 24b corrects the thresholds TH1 and TH2 in the direction of increasing the time and amplitude as the duration of the sound component becomes longer. In other words, the determination unit 24b corrects the regions R1 and R2 so that they become larger as the duration of the sound component becomes longer.

On the other hand, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction that reduces the time and amplitude as the duration of the sound component becomes shorter. In other words, the determination unit 24b corrects the regions R1 and R2 so that they become smaller as the duration of the sound component becomes shorter.

In addition, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude increase as the frequency (center of gravity) of the sound component decreases. In other words, the determination unit 24b corrects the regions R1 and R2 to increase as the frequency (center of gravity) of the sound component decreases.

On the other hand, the determination unit 24b corrects the thresholds TH1 and TH2 in the direction of decreasing the time and amplitude as the frequency (center of gravity) of the sound component increases. In other words, the determination unit 24b corrects the regions R1 and R2 to become smaller as the frequency (center of gravity) of the sound component increases.

In addition, when the sound source signal is determined to be classical based on the sound source information, the determination unit 24b corrects the thresholds TH1 and TH2 in the direction of increasing the time and amplitude. In other words, when the sound source signal is classical, the determination unit 24b corrects the regions R1 and R2 to become larger.

On the other hand, when the sound source information indicates that the sound source signal is voice, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction that reduces the time and amplitude. In other words, when the sound source signal is voice, the determination unit 24b corrects the regions R1 and R2 to be smaller.

Then, the determination unit 24b determines the gain of the filter 23a used in the output control unit 23 at the subsequent stage based on the corrected threshold range. Specifically, the determination unit 24b determines the gain so that the pseudo reverberation sound generated by the filter 23a falls within the corrected threshold range.

In other words, when regions R1 and R2 are corrected to be larger, the determination unit 24b determines the gain so that the time and amplitude of the artificial reverberation sound are increased. On the other hand, when regions R1 and R2 are corrected to be smaller, the determination unit 24b determines the gain so that the time and amplitude of the artificial reverberation sound are decreased.

In this way, the determination unit 24b can determine the gain for generating the optimal pseudo-reverberation sound by determining the gain from the threshold range of the first wave front law corrected based on the feature information and sound source information.

Then, the output control unit 23 generates and outputs a reverberation signal indicating the artificial reverberation sound using the filter 23a to which the gain determined by the determination unit 24b is set.

As a result, in the second embodiment, even if the characteristics of the reverberation of the sound source are different, such as for voice, percussion, classical music, etc., it is possible to generate an optimal pseudo-reverberation sound for each sound source, and optimal surround playback according to the characteristics of the sound source is possible. In other words, in the second embodiment, high-quality surround playback can be performed.

<Control process of the audio device according to the second embodiment>
Next, a specific processing procedure in the audio device 1 according to the second embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the processing procedure executed by the audio device 1 according to the second embodiment.

As shown in FIG. 8, the control unit 2 of the audio device 1 performs the same processes as in the first embodiment in steps S10 and S11. Next, the control unit 2 acquires sound source information related to the sound source signal (step S11a).

Next, the control unit 2 detects feature information that is a feature of the acoustic components contained in the sound source signal based on the acquired sound source signal and sound source information (step S11b). Next, the control unit 2 determines a threshold range in the first wave front law based on the detected feature information (step S11c).

Next, the control unit 2 determines the gain of the filter 23a based on the determined threshold range so that the pseudo reverberation sound falls within the threshold range (step S11d). Then, the control unit 2 executes the processes from step S12 onward using the filter 23a whose gain has been determined and set.

In the above description, when removing the audio source signal from the sound source signal, a time-frequency analysis is performed on the Lch sound source signal and the Rch sound source signal, but this is not limited to this. In other words, instead of performing a time-frequency analysis, a process of removing the audio source signal from the sound source signal may be performed using the difference or ratio between the Lch sound source signal and the Rch sound source signal, L/R channel correlation, etc., thereby reducing the amount of calculation.

Further advantages and modifications may readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

Reference Signs List 1 Acoustic device 2 Control unit 22 Separation unit 23 Output control unit 24a Detection unit 24b Determination unit

Claims (9)

a separation unit that separates a predetermined sound source signal that does not require a process of adding a pseudo reverberation sound from a sound source signal, and removes the separated predetermined sound source signal from the sound source signal;
an output control unit that applies a filter for generating the pseudo reverberation sound to the sound source signal from which the predetermined sound source signal has been removed by the separation unit, and outputs the resultant signal ;
The output control unit is
applying the filter to the sound source signal from which the predetermined sound source signal has been removed and to the predetermined sound source signal, and outputting the artificial reverberation sound corresponding to the sound source signal from which the predetermined sound source signal has been removed such that a reverberation level of the artificial reverberation sound is higher than a reverberation level of the artificial reverberation sound corresponding to the predetermined sound source signal.
Sound equipment. The output control unit is
outputting the predetermined sound source signal so that a sound image of the sound due to the predetermined sound source signal separated by the separation unit is localized at a predetermined position ;
2. An acoustic device according to claim 1. The separation unit is
performing a time-frequency analysis on the sound source signal to separate the predetermined sound source signal ;
3. An acoustic device according to claim 1 or 2 . The sound source signal is
A stereo signal that is played back in stereo on two channels.
The separation unit is
calculating difference information indicating a difference between the sound source signals reproduced on the two channels, and separating the predetermined sound source signal based on the calculated difference information ;
The acoustic device according to any one of claims 1 to 3 . The sound source signal is
A stereo signal that is played back in stereo on two channels.
The separation unit is
Separating a first sound source signal including a component of a sound to be reproduced in one of the two channels from the sound source signal, and a second sound source signal including a component of a sound to be reproduced in the other of the two channels;
The output control unit is
outputting the first sound source signal separated by the separation unit from one of the channels to control a sound image width of the sound due to the first sound source signal, and outputting the second sound source signal from the other of the channels to control a sound image width of the sound due to the second sound source signal ;
The acoustic device according to any one of claims 1 to 4 . a detection unit that detects, based on the sound source signal, feature information indicating features of an acoustic component included in the sound source signal;
a determination unit that determines a gain of the filter based on the feature information detected by the detection unit,
The output control unit is
generating and outputting a reverberation signal indicative of the artificial reverberation sound using the filter to which the gain determined by the determination unit is set ;
An acoustic device according to any one of claims 1 to 5 . An acquisition unit for acquiring a state of the vehicle,
The output control unit is
applying the filter to the sound source signal from which the predetermined sound source signal has been removed in accordance with the state of the vehicle acquired by the acquisition unit, and outputting the sound source signal from a channel disposed at least one of behind and above a listener .
An acoustic device according to any one of claims 1 to 6 . The predetermined sound source signal includes a voice component .
An acoustic device according to any one of claims 1 to 7 . a separation step of separating a predetermined sound source signal that does not require addition of a pseudo reverberation sound from the sound source signal, and removing the separated predetermined sound source signal from the sound source signal;
an output control step of applying a filter for generating the artificial reverberation sound to the sound source signal from which the predetermined sound source signal has been removed by the separation step, and outputting the resultant sound;
The output control step includes:
applying the filter to the sound source signal from which the predetermined sound source signal has been removed and to the predetermined sound source signal, and outputting the artificial reverberation sound corresponding to the sound source signal from which the predetermined sound source signal has been removed such that a reverberation level of the artificial reverberation sound is higher than a reverberation level of the artificial reverberation sound corresponding to the predetermined sound source signal.
Acoustic control methods.

Images