JP5654955B2 - Direct sound extraction device and reverberation sound extraction device - Google Patents

Description

  The present invention relates to a direct sound extraction device and a reverberation sound extraction device. More specifically, the present invention relates to a direct sound extraction device capable of extracting a direct sound from an input signal including the reverberation sound, and to extract the reverberation sound. The present invention relates to a reverberant sound extraction device.

  When music or speech performed in an environment where reverberation is likely to occur such as a hall is recorded, not only direct sound but also reverberation sound is often recorded and recorded in the recorded acoustic signal. For this reason, when an acoustic signal in which reverberant sound is convoluted is reproduced in another acoustic environment, the intelligibility of the direct sound is lowered, and it may be very difficult to hear during reproduction.

  In addition, when speech recognition or the like is performed using speech speech in which reverberant sound is convoluted, there is a problem in that the recognition rate of speech content is reduced due to a decrease in intelligibility due to reverberant sound. .

  In the acoustic signal in which the reverberant sound is convoluted as described above, a technique for reducing the reverberant sound has been conventionally known (see, for example, Patent Document 1). By using this technology, it is possible to reduce the reverberant sound and to clarify the sound directly.

JP 2010-74531 A

  However, in the method shown in Patent Document 1, it is necessary to perform various signal processing such as pseudo whitening processing, multi-step linear prediction processing, and rear reverberation prediction processing in order to reduce reverberation in the input signal. Processing burden is required. For this reason, in order to reduce actual reverberation, a high-performance device such as a microprocessor or a digital signal processor is required, and from the viewpoint of cost and the like, it is not easy to use the method according to Patent Document 1 as it is. was there.

  The present invention has been made in view of the above problems, and provides a direct sound extraction device and a reverberation sound extraction device that can easily extract a direct sound or a reverberation sound from an acoustic signal including a reverberation sound. This is the issue.

  In order to solve the above problems, a direct sound extraction apparatus according to the present invention includes a Fourier transform unit that performs a Fourier transform process on an input signal including a reverberant sound in the direct sound, and a Fourier transform process performed by the Fourier transform unit. Further, spectrum conversion means for converting the input signal into a first amplitude spectrum signal and a phase spectrum signal based on the real and imaginary frequency spectra of the input signal, and preset for the first amplitude spectrum signal Low-pass filter means for performing low-pass filtering processing for each frequency using the normalized cut-off frequency, and first amplitude to limit the negative amplitude of the second amplitude spectrum signal low-pass filtered by the low-pass filter means to zero. The limiter means and the first limiter means limit the amplitude on the minus side. First subtraction means for obtaining a third amplitude spectrum signal by subtracting the second amplitude spectrum signal from the first amplitude spectrum signal, and a negative amplitude of the third amplitude spectrum signal obtained by the first subtraction means Of the real and imaginary numbers based on the second limiter means for limiting the value to zero, the phase spectrum signal, and the third amplitude spectrum signal whose amplitude on the negative side is limited by the second limiter means. Inverse spectrum transforming means for obtaining a signal composed of a frequency spectrum and inverse Fourier transform for generating a direct sound signal obtained by extracting the direct sound from the input signal by subjecting the signal obtained by the inverse spectrum transforming means to inverse Fourier transform processing. Conversion means.

  The direct sound extraction apparatus according to the present invention performs a Fourier transform on an input signal including a reverberation sound in the direct sound, and sets a normalized cutoff frequency that is set in advance for the first amplitude spectrum signal obtained by the spectrum conversion means. Is used to obtain a signal integrated for each spectrum (integrated signal: second amplitude spectrum signal). The signal integrated in this way corresponds to a spectrum signal that forms a stationary component in a time change in the input signal, that is, a reverberation signal.

  Accordingly, the third amplitude spectrum signal obtained by subtracting the second amplitude spectrum signal from the first amplitude spectrum signal in the first subtraction means is a signal obtained by subtracting the reverberation sound from the input signal. A signal corresponding to the sound signal can be obtained.

  For this reason, the signal generated by the inverse spectrum transforming means and the inverse Fourier transforming means is a signal obtained by extracting the direct sound from the input signal, and the direct sound can be easily extracted from the input signal including the reverberant sound in the direct sound. Is possible.

  Furthermore, it is possible to adjust the extraction time of the direct sound in the input signal by adjusting the normalized cutoff frequency. When the value of the normalized cut-off frequency is reduced, the time for extracting the direct sound from the input signal becomes longer, and the direct sound can be extracted in a form including not only a non-stationary sound but also a stationary sound. By extracting direct sounds that include stationary sounds, it is possible to add timbre, ease of hearing, etc. to the direct sounds, compared to direct sounds that do not include any stationary sounds. In this case, it is possible to recognize the sound as having no sense of incongruity.

  Further, the direct sound extraction device described above performs amplification or attenuation for the amplitude of the third amplitude spectrum signal, on which the amplitude on the negative side is limited by the second limiter unit, for each frequency, thereby the third amplitude. First gain means for weighting the spectrum signal, wherein the inverse spectrum conversion means is based on the phase spectrum signal and the third amplitude spectrum signal weighted by the first gain means, You may obtain | require the signal which consists of an imaginary number frequency spectrum.

  In the above direct sound extraction apparatus, the first gain means performs amplification or attenuation for the amplitude of the third amplitude spectrum signal whose amplitude on the negative side is limited by the second limiter means for each frequency. Weighting is performed on the spectrum signal. By performing weighting by amplitude amplification or attenuation in this way for each frequency, when a direct sound is extracted from an input signal, it is possible to obtain a direct sound with a little reverberation sound included.

  For example, if the direct sound contains a lot of reverberant sound (convolved), it may be difficult to hear the direct sound due to the reverberant sound, but if the direct sound contains a slight amount of reverberant sound, In some cases, the direct sound heard by the listener is added with a timbre associated with the reverberant sound to make it easy to hear. As described above, by adjusting the weighting, it is possible to adjust the reverberation sound included in the extracted direct sound, and it is possible to adjust the tone quality such as the timbre.

  Furthermore, the reverberant sound extraction apparatus according to the present invention includes a Fourier transform unit that performs a Fourier transform process on an input signal that includes a reverberant sound in a direct sound, and a real number of the input signal that has undergone a Fourier transform process by the Fourier transform unit. And a spectrum conversion means for converting the input signal into a first amplitude spectrum signal and a phase spectrum signal based on the imaginary frequency spectrum, and a normalized cutoff frequency preset for the first amplitude spectrum signal. A high-pass filter means for performing high-pass filtering processing for each frequency, a third limiter means for limiting the negative amplitude of the fourth amplitude spectrum signal subjected to high-pass filtering by the high-pass filter means to zero, and the third limiter means The fourth amplitude spectrum in which the amplitude on the negative side is limited by the 3 limiter means. A second subtracting means for obtaining a fifth amplitude spectrum signal by subtracting a torque signal from the first amplitude spectrum signal, and limiting a negative amplitude of the fifth amplitude spectrum signal obtained by the second subtracting means. Based on the fourth limiter means for zeroing, the phase spectrum signal, and the fifth amplitude spectrum signal whose amplitude on the negative side is limited by the fourth limiter means, it is composed of frequency spectra of real and imaginary numbers. An inverse spectrum transforming means for obtaining a signal, and an inverse Fourier transforming means for generating a reverberation sound signal obtained by extracting the reverberation sound from the input signal by performing an inverse Fourier transform process on the signal obtained by the inverse spectrum transforming means. It is characterized by providing.

  The reverberant sound extraction apparatus according to the present invention performs a Fourier transform on an input signal including a reverberant sound in a direct sound, and sets a normalized cutoff frequency that is set in advance for the first amplitude spectrum signal obtained by the spectrum converting means. Is used to obtain a signal differentiated for each spectrum (differentiated signal: fourth amplitude spectrum signal). The signal thus differentiated corresponds to a spectral signal that forms an unsteady component in the time change in the input signal, that is, a direct sound signal.

  Therefore, the fifth amplitude spectrum signal obtained by subtracting the fourth amplitude spectrum signal from the first amplitude spectrum signal in the second subtracting means is a signal obtained by subtracting the direct sound from the input signal. A signal corresponding to the signal can be obtained.

  For this reason, the signal generated by the inverse spectrum transform unit and the inverse Fourier transform unit is a signal obtained by extracting the reverberation sound from the input signal, and the reverberation sound can be easily extracted from the input signal in which the reverberation sound is included in the direct sound. Is possible.

  Furthermore, by adjusting the normalized cutoff frequency, it is possible to adjust the reverberation sound extraction time in the input signal. When the value of the normalized cut-off frequency is increased, the extraction time of the reverberant sound in the input signal becomes longer, and it becomes possible to extract the reverberant sound including not only the stationary sound but also the unsteady sound. Also, if the value of the normalized cut-off frequency is reduced, the reverberation sound extraction time in the input signal is shortened, and reverberation sound that does not contain much non-stationary sound can be extracted.

  Furthermore, the reverberation sound extraction apparatus described above performs amplification or attenuation for the amplitude of the fifth amplitude spectrum signal, on which the amplitude on the negative side is limited by the fourth limiter unit, for each frequency, thereby the fifth amplitude. Second gain means for weighting the spectrum signal, wherein the inverse spectrum conversion means is based on the phase spectrum signal and the fifth amplitude spectrum signal weighted by the second gain means, You may obtain | require the signal which consists of an imaginary number frequency spectrum.

  In the above reverberant sound extraction device, the second gain means performs amplification or attenuation for the amplitude of the fifth amplitude spectrum signal whose amplitude on the negative side is limited by the fourth limiter means for each frequency, thereby obtaining the fifth amplitude. Weighting is performed on the spectrum signal. By performing weighting by amplitude amplification or attenuation for each frequency in this way, when reverberation is extracted from an input signal, it is possible to obtain a reverberation that contains a small amount of direct sound. Thus, by adjusting the weighting, the direct sound included in the extracted reverberation sound can be adjusted, and the tone quality such as the timbre can be adjusted.

  According to the direct sound extraction device of the present invention, it is possible to easily extract a direct sound from an input signal in which the direct sound includes a reverberant sound. Moreover, according to the reverberation sound extraction apparatus according to the present invention, it is possible to easily extract a reverberation sound from an input signal in which the direct sound includes the reverberation sound.

It is the block diagram which showed schematic structure of the sound processing apparatus which concerns on embodiment. It is the figure which showed typically the Fourier-transform length and overlap length in the case of performing a short-time Fourier-transform process with respect to an input signal in the FFT part which concerns on embodiment. It is the block diagram which showed schematic structure of the frequency spectrum domain filtering part which concerns on embodiment. (A) shows an example of the filter coefficient for each amplitude spectrum in the LPF unit according to the embodiment, and (b) shows an example of the filter coefficient for each amplitude spectrum in the HPF unit. (A) is the figure which showed an example of the frequency change of the weighting amount of the amplification of the 1st gain part which concerns on embodiment, and (b) is an example of the frequency change of the weighting amount of the 2nd gain part. FIG. The amplitude of the input signal input to the frequency spectrum domain filtering unit according to the embodiment, the amplitude of the integrated signal Lfa1, the amplitude of the differential signal Lfa2, the amplitude of the direct sound signal Lfd, and the amplitude of the reverberant sound signal Lfr It is the 1st figure which illustrated time change. The amplitude of the input signal input to the frequency spectrum domain filtering unit according to the embodiment, the amplitude of the integrated signal Lfa1, the amplitude of the differential signal Lfa2, the amplitude of the direct sound signal Lfd, and the amplitude of the reverberant sound signal Lfr It is the 2nd figure which illustrated time change. The amplitude of the input signal input to the frequency spectrum domain filtering unit according to the embodiment, the amplitude of the integrated signal Lfa1, the amplitude of the differential signal Lfa2, the amplitude of the direct sound signal Lfd, and the amplitude of the reverberant sound signal Lfr It is the 3rd figure which illustrated time change. The amplitude of the input signal input to the frequency spectrum domain filtering unit according to the embodiment, the amplitude of the integrated signal Lfa1, the amplitude of the differential signal Lfa2, the amplitude of the direct sound signal Lfd, and the amplitude of the reverberant sound signal Lfr It is the 4th figure which illustrated time change. It is the 1st figure which illustrated the time change of the amplitude of the input signal in the reverberation sound extraction device concerning an embodiment, the amplitude of the direct sound signal extracted in the reverberation sound extraction device, and the amplitude of the reverberation sound signal. It is the 2nd figure which illustrated the time change of the amplitude of the input signal in the reverberation sound extraction device concerning an embodiment, the amplitude of the direct sound signal extracted in the reverberation sound extraction device, and the amplitude of the reverberation sound signal. It is the 3rd figure which illustrated the time change of the amplitude of the input signal in the reverberation sound extraction device concerning an embodiment, the amplitude of the direct sound signal extracted in the reverberation sound extraction device, and the amplitude of the reverberation sound signal. It is the 4th figure which illustrated time change of the amplitude of the input signal in the reverberation sound extraction device concerning an embodiment, the amplitude of the direct sound signal extracted in the reverberation sound extraction device, and the amplitude of the reverberation sound signal. It is the 5th figure which illustrated time change of the amplitude of the input signal in the reverberation sound extraction device concerning an embodiment, the amplitude of the direct sound signal extracted in the reverberation sound extraction device, and the amplitude of the reverberation sound signal. (A) is the figure which showed typically a mode that the waveform of the direct sound signal shown in FIG. 14 changed according to the adjustment state of the value of a normalization cut-off frequency, and an input signal, (b) FIG. 15 is a diagram schematically showing a state in which the waveform of the reverberation signal shown in FIG. 14 changes according to the adjustment state of the value of the normalized cutoff frequency, and the input signal.

  Hereinafter, an acoustic processing apparatus which is an example of a direct sound extraction apparatus and a reverberation sound extraction apparatus according to the present invention will be described and described in detail with reference to the accompanying drawings.

  Note that when a reverberant sound is convoluted with a direct sound such as a voice or a musical instrument, a signal having a steadiness according to the reverberation time is added to a signal such as an unsteady voice or a musical instrument in the frequency spectrum. The sound processing apparatus according to the present embodiment extracts direct sound by extracting and separating an unsteady signal from an input signal, and extracts reverberation sound by extracting and separating a steady signal from the input signal. is there.

  FIG. 1 is a block diagram showing a schematic configuration of the sound processing apparatus. As shown in FIG. 1, the sound processing apparatus 1 includes an FFT unit (Fourier transform unit, spectrum transform unit) 3, a frequency spectrum domain filtering unit 4, an IFFT unit (inverse Fourier transform unit, inverse spectrum transform unit) 5a, 5b.

  The FFT unit 3 has two channels (L channel and L channel) in which a reverberant sound (for example, a reflected sound in speech) is convoluted with a direct sound (for example, speech such as speech) from a sound source unit (not shown). R channel) input signals L and R are input. The FFT unit 3 has a role of weighting each of the two-channel input signals L and R in which reverberation sound is convoluted with a window function.

  The FFT unit 3 weights the input signals L and R after performing weighting using a window function, thereby performing a short-time Fourier transform process on the input signals L and R, thereby changing the input signals L and R from the time domain to the frequency domain. To obtain the real and imaginary frequency spectra. FIG. 2 is a diagram schematically showing the Fourier transform length and the overlap length when the FFT unit 3 performs short-time Fourier transform processing on the input signal L (or input signal R). Here, since the FFT unit 3 performs a Fourier transform process on the input signal, it corresponds to a Fourier transform unit according to the present invention.

  Further, the FFT unit 3 converts the frequency spectra of the two channels obtained by the frequency domain conversion into amplitude spectrum signals Lfa and Rfa (first amplitude spectrum signal) and phase spectrum signals Lfp and Rfp, respectively. The FFT unit 3 then outputs the converted two-channel amplitude spectrum signals Lfa and Rfa to the frequency spectrum domain filtering unit 4. The FFT unit 3 also outputs two-channel phase spectrum signals Lfp and Rfp to the IFFT unit 5a and the IFFT unit 5b, respectively. Here, since the FFT unit 3 converts the input signal into the amplitude spectrum signals Lfa and Rfa and the phase spectrum signals Lfp and Rfp, it corresponds to the spectrum conversion means according to the present invention.

  FIG. 3 is a block diagram showing a schematic configuration of the frequency spectrum domain filtering unit 4. The frequency spectrum domain filtering unit 4 has a role of extracting a non-stationary signal and a stationary signal by a simple filtering process for each spectrum. In the process in the frequency spectrum domain filtering unit 4, the filtering process is performed only on the amplitude spectrum signals Lfa and Rfa, and the filtering process is not performed on the phase spectrum signals Lfp and Rfp.

  As shown in FIG. 3, the frequency spectrum domain filtering unit 4 includes an LPF unit (low-pass filter unit) 10, an HPF unit (high-pass filter unit) 11, a first limiter unit (first limiter unit) 12, and a second A limiter unit (second limiter unit) 13, a third limiter unit (third limiter unit) 14, a fourth limiter unit (fourth limiter unit) 15, a first gain unit (first gain unit) 16, A second gain unit (second gain unit) 17, a first subtraction unit (first subtraction unit) 18, and a second subtraction unit (second subtraction unit) 19 are provided. FIG. 3 shows only functional units (LPF unit 10, HPF unit 11, limiter units 12 to 15, gain units 16 and 17, subtracting units 18 and 19) that process the amplitude spectrum signal Lfa. A functional unit that performs processing on the amplitude spectrum signal Rfa is not shown. However, a similar functional unit is provided in the functional unit that performs processing on the amplitude spectrum signal Rfa, and a similar filtering process is performed.

  The LPF unit 10 has a role of performing low-pass filtering processing for each spectrum (for each frequency) on the amplitude spectrum signal Lfa input from the FFT unit 3 based on a predetermined normalized cutoff frequency. The first limiter unit 12 has a role of limiting the amplitude on the negative side of the amplitude spectrum signal (second amplitude spectrum signal) subjected to low-pass filtering by the LPF unit 10 to zero. The first gain unit 16 has a role of amplifying or attenuating the amplitude of the amplitude spectrum signal on which the minus side amplitude restriction is performed. In this way, by performing the low-pass filtering process on the amplitude spectrum signal Lfa in the LPF unit 10, a signal (integrated signal: second amplitude spectrum signal) Lfa1 integrated for each spectrum is generated.

  Then, the first subtraction unit 18 subtracts the integration signal Lfa1 from the amplitude spectrum signal Lfa input from the FFT unit 3 to obtain a non-stationary spectrum signal (third amplitude spectrum signal) with time variation. Thereafter, the second limiter unit 13 limits the minus-side amplitude of the spectrum signal (third amplitude spectrum signal) obtained by the first subtraction unit 18 to zero. The signal whose amplitude is limited in the second limiter unit 13 is output to the IFFT unit 5a as a direct sound signal Lfd.

  The HPF unit 11 has a role of performing high-pass filtering processing for each spectrum (for each frequency) on the amplitude spectrum signal Lfa input from the FFT unit 3 based on a predetermined normalized cutoff frequency. The third limiter unit 14 has a role of limiting the amplitude on the negative side of the amplitude spectrum signal (fourth amplitude spectrum signal) subjected to the high-pass filtering processing by the HPF unit 11 to zero. The second gain unit 17 has a role of amplifying or attenuating the amplitude of the amplitude spectrum signal on which the minus side amplitude restriction is performed. In this way, by performing high-pass filtering on the amplitude spectrum signal Lfa in the HPF unit 11, a signal differentiated for each spectrum (differentiated signal: fourth amplitude spectrum signal) Lfa2 is generated.

  Then, the second subtractor 19 subtracts the differential signal Lfa2 from the amplitude spectrum signal Lfa input from the FFT unit 3 to obtain a steady spectrum signal (fifth amplitude spectrum signal) with little time variation. Thereafter, the fourth limiter unit 15 limits the minus-side amplitude of the spectrum signal (fifth amplitude spectrum signal) obtained by the second subtraction unit 19 to zero. The signal whose amplitude is limited in the fourth limiter unit 15 is output to the IFFT unit 5b as a reverberation sound signal Lfr.

  The normalization cutoff frequency of the low pass filter for each amplitude spectrum in the LPF unit 10 and the normalization cutoff frequency of the high pass filter for each amplitude spectrum in the HPF unit 11 adjust the division time of the direct sound and the reverberation sound. (Adjusting the extraction time of the direct sound and adjusting the extraction time of the reverberation sound). Further, the first gain unit 16 and the second gain unit 17 adjust the mixing ratio of the direct sound and the reverberant sound by changing the weighting amount of the amplification and attenuation (the ratio of the reverberant sound included in the direct sound is changed). And the ratio of the direct sound included in the reverberant sound can be adjusted.

  4A shows an example of the filter coefficient for each amplitude spectrum in the LPF unit 10 according to the present embodiment, and FIG. 4B shows the filter coefficient for each amplitude spectrum in the HPF unit 11. An example is shown. Each of the LPF unit 10 and the HPF unit 11 shown in FIGS. 4A and 4B is a first-order Butterworth filter. As shown in FIG. 4, the normalized cutoff frequencies are 0.000001, 0.000002, 0.000004,..., 0.0655. The lower the cutoff frequency value, the longer the direct sound extraction time and the reverberation sound extraction time. In the frequency spectrum domain filtering unit 4 according to the present embodiment, the cut-off frequencies of the LPF unit 10 and the HPF unit 11 are set in common for the amplitude spectrum, but are configured to be set independently for each amplitude spectrum. It is also possible to do.

  FIG. 5A is a diagram showing an example of a frequency change in the weighting amount of amplification and attenuation of the first gain unit 16 according to the present embodiment, and FIG. It is the figure which showed an example of the frequency change of weighting amount. As shown in FIGS. 5A and 5B, in the first gain unit 16 and the second gain unit 17 according to the present embodiment, the amount of mixing increases as the gain (signal level) decreases. Further, as shown in FIGS. 5A and 5B, the first gain unit 16 on the direct sound side hardly separates the direct sound and the reverberant sound in the amplitude spectrum of 500 Hz or less.

  6 to 9 show an example of the operation of each part of the frequency spectrum domain filtering unit 4. The amplitude of the input signal (amplitude spectrum signal Lfa) input to the frequency spectrum domain filtering unit 4 and the integrated signal Lfa1 are shown. FIG. 6 is a diagram exemplifying time variation of the amplitude of the differential signal Lfa2, the amplitude of the direct sound signal Lfd, and the amplitude of the reverberation sound signal Lfr. Each of the waveforms in FIGS. 6 to 9 is obtained by observing the time change of the amplitude spectrum near 1 kHz.

  6 to 9, the sampling rate of the input signal is 44.1 kHz, the Fourier transform length of the FFT unit 3 is 4,096 sample (sample), and the overlap length is 15 / of the Fourier transform length. The window function of 16 times 3,840 sample and Fourier transform is Blackman. 6 to 8 is a 1 kHz sine wave with a reproduction time of 1 sec (seconds), and the input signal shown in FIG. 9 is music.

  8 and 9, the first gain unit 16 and the second gain unit 17 perform weighting for each spectrum (for each frequency) shown in FIGS. 5 (a) and 5 (b). FIGS. 6 and 7 show the case where the gain (signal level) for all amplitude spectra is set to 0 dB without weighting in the first gain unit 16 and the second gain unit 17. ing.

  First, in the direct sound side signal shown in FIG. 6A, the rectangular input signal is integrated by the low pass filtering process of the LPF unit 10. For this reason, the rising portion of the rectangular input signal is extracted, and the integrated signal Lfa1 having a gentle rise in amplitude is generated. Thereafter, the first subtraction unit 18 subtracts the integration signal Lfa1 from the input signal, and the amplitude of the gentle rising portion of the integration signal Lfa1 is subtracted from the rectangular shape of the input signal. The portion, that is, the unsteady component is extracted as the direct sound signal Lfd.

  Although the amplitude of the direct sound signal Lfd becomes negative due to the subtraction processing in the first subtracting unit 18, the amplitude is limited to 0 by the second limiter unit 13, so that as shown in FIG. In addition, the direct sound signal Lfd is not a negative value.

  Next, in the reverberation sound side signal shown in FIG. 6B, the rectangular input signal is differentiated by the high-pass filtering process of the HPF unit 11. Therefore, a differential signal Lfa2 composed of a sharp rising portion of the rectangular input signal and a subsequent gentle attenuation portion is generated. Thereafter, the second subtraction unit 19 subtracts the differential signal Lfa2 from the input signal and subtracts the amplitude of the sharp rising portion of the differential signal Lfa2 from the rectangular shape of the input signal. The other part, that is, the stationary component is extracted as the reverberation sound signal Lfr.

  In the subtraction processing in the second subtraction unit 19, the amplitude of the reverberation sound signal Lfr is negative, but the amplitude is limited to 0 by the fourth limiter unit 15, so that it is shown in FIG. Thus, the reverberation signal Lfr is not a negative value.

  FIG. 7 is a diagram in which the normalized cutoff frequencies of the HPF unit 11 and the LPF unit 10 are changed in the state shown in FIG. Specifically, the normalized cutoff frequency of the HPF unit 11 shown in FIG. 7B is 0.0041, which is lower than the normalized cutoff frequency 0.0082 of the HPF unit 11 shown in FIG. Is set to Also, the normalized cutoff frequency of the LPF unit 10 shown in FIG. 7A is set to 0.0164, which is a higher value than the normalized cutoff frequency 0.0082 of the LPF unit shown in FIG. ing.

  As shown in FIGS. 6 and 7, when the normalized cut-off frequency is lowered, the response of the filter becomes slow and the response of the rising edge of the signal becomes long. Further, when the normalized cutoff frequency is increased, the response of the filter becomes faster and the response of the rising edge of the signal becomes shorter. In this way, by adjusting the cut-off frequency, it is possible to adjust the division time of the direct sound and the reverberation sound (adjustment of the extraction time of the direct sound and the extraction time of the reverberation sound). .

  FIG. 8 is a diagram showing a case where the weighting amount for each spectrum in the first gain unit 16 and the second gain unit 17 is set in the state shown in FIG. By setting the weighting amount, an offset (amplification of the amplitude) corresponding to the weighting amount occurs in the direct sound and the reverberant sound. For this reason, a reverberant sound accompanying an offset is added to the direct sound signal shown in FIG. 8 (a) (the amplitude of the height L1 shown in FIG. 8 (a) is increased), and the reverberation shown in FIG. 8 (b). A direct sound accompanying an offset is added to the sound signal (the amplitude of the height L1 shown in FIG. 8B is increased). In this way, the mixing ratio of the direct sound and the reverberation sound is adjusted by the offset generated according to the setting of the weighting amount (the ratio of the reverberation sound included in the direct sound is adjusted, and the direct sound included in the reverberation sound is adjusted). Can be adjusted).

  FIG. 9 is a diagram in which components in the vicinity of 1 kHz that attenuate with time are extracted from the input signal as a music signal in the state shown in FIG. In the direct sound side signal, as shown in FIG. 9A, the direct sound signal is extracted in the first half portion having a large amplitude, and in the reverberant sound side signal, as shown in FIG. 9B. Further, it can be seen that a reverberant signal is extracted in the latter half of the case where the amplitude of the input signal is attenuated.

  The IFFT unit 5a is a real number based on the amplitude spectrum signal (direct sound signal Lfd, Rfd) made of the direct sound filtered by the frequency spectrum domain filtering unit 4 and the phase spectrum signal Lfp, Rfp acquired from the FFT unit 3. Are converted into an imaginary frequency spectrum, and then weighted by a window function. Then, the IFFT unit 5a performs a short-time inverse Fourier transform process and an overlap addition on the weighted signal, thereby converting the signal from the frequency domain to the time domain to generate a direct sound signal composed of a direct sound. Ld and Rd are generated.

  Similarly, the IFFT unit 5b also uses amplitude spectrum signals (reverberation sound signals Lfr, Rfr) that are filtered by the frequency spectrum domain filtering unit 4 and phase spectrum signals Lfp, Rfp acquired from the FFT unit 3. Is converted to a real and imaginary frequency spectrum, and then weighted by a window function. Then, the IFFT unit 5b performs a short-time inverse Fourier transform process and an overlap addition on the weighted signal, thereby converting the signal from the frequency domain to the time domain to generate a reverberant sound signal composed of a reverberant sound. Lr and Rr are generated.

  The IFFT units 5a and 5b correspond to the inverse spectrum conversion means according to the present invention because they perform processing for conversion into a real number and an imaginary number frequency spectrum based on the amplitude spectrum signal and the phase spectrum signal. Furthermore, since the IFFT units 5a and 5b perform short-time inverse Fourier transform processing on the weighted signal, they correspond to the inverse Fourier transform means according to the present invention.

  10 to 14 are diagrams exemplifying the time change of the amplitude of the input signal to the sound processing apparatus 1 and the amplitude of the direct sound signal and the reverberation sound signal extracted (generated) by the sound processing apparatus 1. 10 and 11 show a case where a 1 kHz sine wave having a playback time of 1 sec is inputted as an input signal, and FIGS. 12 and 13 show a case where music is inputted as an input signal, and FIG. The impulse response in the hall (environment where reverberation is likely to occur) is input as an input signal.

  10 to 14, the normalized cut-off frequencies in the HPF unit 11 and the LPF unit 10 are all 0.0082, and FIGS. 10, 12, and 14 show cases in which weighting processing for each spectrum is not performed. 11 and 13 show a case where there is a weighting process for each spectrum (for each frequency).

  10 to 14, the inverse Fourier transform length of the IFFT unit 5 is 4,096 sample, the overlap length is 3,840 sample which is 15/16 times the Fourier transform length, and the window function of the inverse Fourier transform is Blackman. The same applies to the FFT unit 3.

  FIGS. 10 and 11 show a state in which a direct sound signal that is an unsteady component and a reverberation sound signal that is a stationary component are extracted with respect to a temporal change in the amplitude of a rectangular input signal. Yes. Further, with respect to the direct sound signal and the reverberation sound signal shown in FIG. 10, the amplitude values of the direct sound signal and the reverberation sound signal shown in FIG. 11 are offset by the weighting process for each spectrum. For this reason, in the offset portion (in FIG. 11, the portion where the amplitudes of the direct sound signal and the reverberation sound signal are raised by the height of L2), a portion where the direct sound and the reverberation sound are mixed is included. Thus, the mixing ratio of the direct sound and the reverberant sound can be adjusted according to the weighting process by the first gain unit 16 and the second gain unit 17.

  In FIG. 12 and FIG. 13, it is possible to confirm a waveform obtained by extracting a direct sound and a reverberation sound from the music (input signal) waveform. In addition, even when the separated direct sound and reverberation sound are listened to separately, the direct sound and reverberation sound of music can be confirmed respectively, and the extraction (or separation) of the direct sound and reverberation sound can be performed. Auditory recognition is possible.

  In FIG. 13, since the weighting is set for each spectrum, a waveform in which a part of the reverberation sound is added to the direct sound and a part of the direct sound is added to the reverberation sound (the direct sound signal in FIG. 13). And the amplitude of the reverberant signal is higher than that of FIG. 12), and it is confirmed that the mixing ratio of the direct sound and the reverberant sound can be adjusted by the weight setting for each spectrum. it can. Further, even when the direct sound and the reverberation sound shown in FIG. 13 are respectively listened to, the output sound in which the direct sound and the reverberation sound are mixed can be confirmed according to the mixing ratio.

  In FIG. 14, the impulse response in the hall is input as an input signal. Since it is an impulse response, it has an output when an extremely short signal is input and the amplitude converges in a short time. In addition to this, a reverberant sound is probably included.

  In FIG. 14, compared to the convergence of the amplitude of the input signal, the direct sound in which the amplitude is converged in a shorter time and the reverberant sound in which the amplitude is maintained for a longer time than the convergence of the amplitude of the direct sound. And can be confirmed. In FIG. 14, the normalized cutoff frequency of the HPF unit 11 and the LPF unit 10 is set to 0.0082. By adjusting the value of the normalized cutoff frequency, the direct sound extraction time and reverberation are set. It is possible to adjust the sound extraction time.

  FIG. 15A is a diagram schematically showing a state in which the waveform of the direct sound shown in FIG. 14 changes according to the adjustment state of the value of the normalized cutoff frequency and the input signal. As shown in FIG. 15A, when the value of the normalized cutoff frequency increases, the time until the amplitude of the impulse response converges decreases, and when the value of the normalized cutoff frequency decreases, the amplitude of the impulse response decreases. The time until convergence becomes longer, and a waveform shape close to the convergence state of the amplitude of the input signal is shown.

  In this way, by adjusting the value of the normalized cutoff frequency, it is possible to change the direct sound extraction time in the input signal. Therefore, if the value of the normalized cut-off frequency is reduced, the time for extracting the direct sound from the input signal becomes longer, and it becomes possible to extract the direct sound in a form including not only the unsteady sound but also the stationary sound. For example, by extracting a direct sound including a stationary sound as shown in FIG. 14, it is possible to add a timbre, ease of hearing, etc. to the direct sound as compared to a direct sound that does not include a stationary sound at all. Thus, when the listener listens directly to the sound, it can be recognized as a sound without any sense of incongruity.

  On the other hand, FIG. 15 (b) is a diagram schematically showing how the reverberant waveform shown in FIG. 14 changes according to the adjustment state of the value of the normalized cutoff frequency and the input signal. . As shown in FIG. 15 (b), when the value of the normalized cut-off frequency increases, the amplitude of the reverberant sound increases from an early time, and the increase in the amplitude of the reverberant sound tends to rise greatly from an early stage. is there. On the other hand, when the value of the normalized cut-off frequency becomes small, the increase (rise part) of the amplitude of the reverberant sound becomes moderate.

  For this reason, it is possible to change the extraction time of the direct sound in the input signal by adjusting the value of the normalized cutoff frequency. When the value of the normalized cutoff frequency is decreased, the direct sound included in the reverberant sound signal is changed. It becomes possible to reduce the influence of sound, and if the value of the normalized cut-off frequency is increased, it is possible to extract a reverberant sound signal containing a little direct sound.

  Although the direct sound extraction device and the reverberation sound extraction device according to the present invention have been described in detail with reference to the drawings, the direct sound extraction device and the reverberation sound extraction device according to the present invention are described in the above-described embodiments. It is not limited to the configuration shown. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  Various acoustic environments can also be constructed by using the direct sound extraction device and the reverberation sound extraction device according to the present invention. For example, a direct sound extraction device extracts a direct sound signal from an input signal that includes a reverberant sound in the direct sound, and outputs it from a speaker installed at a position close to the listener so that the input signal is directly output from the speaker. Compared to the case, the vocal sound and the like can be made clearer and easier to hear. Moreover, a reverberant sound can be effectively output by extracting a reverberant sound signal from an input signal by the reverberant sound extracting device and outputting it from a speaker installed at a position far from the listener.

1 ... Sound processing device (direct sound extraction device and reverberation sound extraction device)
3 ... FFT section (Fourier transform means, spectrum transform means)
4 ... frequency spectrum domain filtering units 5a, 5b ... IFFT units (inverse Fourier transform means, inverse spectrum transform means)
10 ... LPF section (low-pass filter means)
11 ... HPF section (high-pass filter means)
12 ... 1st limiter part (1st limiter means)
13 ... 2nd limiter part (2nd limiter means)
14 ... Third limiter section (third limiter means)
15 ... Fourth limiter section (fourth limiter means)
16 ... 1st gain part (1st gain means)
17 ... 2nd gain part (2nd gain means)
18 ... 1st subtraction part (1st subtraction means)
19 ... 2nd subtraction part (2nd subtraction means)
L, R ... input signals Lfa, Rfa ... amplitude spectrum signals Lfp, Rfp ... phase spectrum signal Lfa1 ... integration signal Lfa2 ... differential signals Lfd, Ld, Rfd, Rd ... direct sound signals Lfr, Lr, Rfr, Rr ... reverberation sound signals

Claims (4)

Fourier transform means for performing a Fourier transform process on an input signal in which reverberant sound is included in the direct sound;
Spectrum conversion means for converting the input signal into a first amplitude spectrum signal and a phase spectrum signal based on the real and imaginary frequency spectra of the input signal Fourier-transformed by the Fourier transform means;
Low-pass filter means for performing a low-pass filtering process for each frequency using a preset normalization cutoff frequency for the first amplitude spectrum signal;
First limiter means for limiting the negative amplitude of the second amplitude spectrum signal low-pass filtered by the low-pass filter means to zero,
First subtraction means for obtaining a third amplitude spectrum signal by subtracting from the first amplitude spectrum signal the second amplitude spectrum signal in which the negative-side amplitude is limited by the first limiter;
Second limiter means for limiting the minus-side amplitude of the third amplitude spectrum signal obtained by the first subtraction means to zero, and
Inverse spectrum conversion means for obtaining a signal composed of a real number and an imaginary number frequency spectrum based on the phase spectrum signal and the third amplitude spectrum signal on which the amplitude of the negative side is limited by the second limiter means;
Direct sound extraction comprising: an inverse Fourier transform means for generating a direct sound signal obtained by extracting the direct sound from the input signal by subjecting the signal obtained by the inverse spectrum transform means to an inverse Fourier transform process. apparatus. First gain means for weighting the third amplitude spectrum signal by amplifying or attenuating the amplitude of the third amplitude spectrum signal whose amplitude on the negative side is limited by the second limiter means for each frequency. With
The inverse spectrum converting means obtains a signal composed of a real number and an imaginary number frequency spectrum based on the phase spectrum signal and the third amplitude spectrum signal weighted by the first gain means. The direct sound extraction device according to claim 1. Fourier transform means for performing a Fourier transform process on an input signal in which reverberant sound is included in the direct sound;
Spectrum conversion means for converting the input signal into a first amplitude spectrum signal and a phase spectrum signal based on the real and imaginary frequency spectra of the input signal Fourier-transformed by the Fourier transform means;
High-pass filter means for performing high-pass filtering processing for each frequency using a normalization cutoff frequency set in advance for the first amplitude spectrum signal;
Third limiter means for limiting the negative amplitude of the fourth amplitude spectrum signal subjected to high-pass filtering by the high-pass filter means to zero,
Second subtracting means for obtaining a fifth amplitude spectrum signal by subtracting the fourth amplitude spectrum signal, the amplitude of which has been negatively limited by the third limiter means, from the first amplitude spectrum signal;
Fourth limiter means for limiting the minus-side amplitude of the fifth amplitude spectrum signal obtained by the second subtracting means to zero, and
Inverse spectrum conversion means for obtaining a signal composed of a real number and an imaginary number frequency spectrum based on the phase spectrum signal and the fifth amplitude spectrum signal in which the negative amplitude is limited by the fourth limiter means;
Reverberation sound extraction, comprising: inverse Fourier transform means for generating a reverberation sound signal obtained by extracting the reverberation sound from the input signal by performing an inverse Fourier transform process on the signal obtained by the inverse spectrum conversion means apparatus. Second gain means for weighting the fifth amplitude spectrum signal by amplifying or attenuating the amplitude of the fifth amplitude spectrum signal, the amplitude of which is negatively limited by the fourth limiter means, for each frequency. With
The inverse spectrum conversion unit obtains a signal composed of a real number and an imaginary number frequency spectrum based on the phase spectrum signal and the fifth amplitude spectrum signal weighted by the second gain unit. The reverberation sound extraction device according to claim 3.

Images