JP4540290B2 - A method for moving a three-dimensional space by localizing an input signal. - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
In the present invention, a sound image obtained by processing and localizing an input acoustic signal so that a listener feels a sense of hearing as if the listener is listening to the sound of the real sound source in a three-dimensional space where the real sound source is present, The present invention relates to a moving method of a localized sound image that is freely moved in a space.
[0002]
[Prior art]
The inventor of the present invention first focuses on the frequency characteristics of the head related transfer function (HRTF), divides the frequency band of the input signal into a plurality of parts, and performs appropriate processing on the signal of each band. For example, Japanese Patent Application Laid-Open No. 10-228520 proposes a method in which the amount of calculation is relatively small, and therefore processing for sound image localization of an input signal can be executed in real time.
[0003]
[Problems to be solved by the invention]
The present invention further extends the sound image localization technique previously proposed by the present inventor and provides a method that allows the listener to hear the sound as if the localized sound image is moving freely. , That is the subject.
[0004]
[Means for Solving the Problems]
The configuration of the method of the present invention made for the purpose of solving the above problem is to divide a monaural input signal into a plurality of bands between a low frequency band and a high frequency band by a digital filter for both the left and right channels, and to divide the signal. If the band is a low frequency band, use a digital filter with time difference and volume difference as parameters, and if it is an intermediate frequency band, use a parametric equalizer to follow the frequency characteristics of the head-related transfer function, In this case, processing for sound image localization is performed for each band using a comb filter, and the delay taking into account the difference in delay due to sound diffraction and the difference in transmission time of sound reaching both ears for each frequency band of both channels After the processing, the input signal is determined by adjusting the ratio (gain) of the amount of sound processed for each frequency band and mixing with a mixer. When obtaining a sound image, at least two sound image localization points where sound image localization can be performed by this method, the difference between the delay parameter and the gain parameter at the time of mixing is obtained among the sound image movement parameters for each frequency band at both points. The amount of change per unit time between the time for moving the localization sound image from one localization point to the other localization point and the parameter difference at this time is obtained as a sound image movement parameter between the two localization points, and the sound image movement parameter By changing the delay difference and gain difference at the one localization point in the movement time by replacing the sound image movement parameters other than the delay difference and gain difference with the sound image localization parameter at that point at the other localization point It is characterized by generating a sound that gives a feeling of movement.
[0005]
In the present invention, in order to make it feel as if the localized sound image is continuously moving within a range where the sound image can be localized, as an example, the sound image is continuously moved from any sound image localization point to the next localization possible point. It is conceivable to interpolate between the two points with respect to all sound image localization parameters using all the parameters used in the sound image localization method described above. However, if all parameters are interpolated, the amount of computation for interpolation control between two points will increase dramatically, and therefore, a large capacity memory is also essential.
[0006]
Therefore, in the present invention, in the sound image localization method described above, delay control for each frequency band that plays an important role in the movement of the sound image in terms of audibility, and processing of each frequency band processed when mixing signals of each frequency band By interpolating the ratio (gain) of the amount of sound, a sense of hearing that moves in the three-dimensional space of the localized sound image is realized.
[0007]
DETAILED DESCRIPTION OF THE INVENTION
Prior to the description of the embodiments of the method of the present invention, the sound image localization method of the audio input signal previously proposed by the inventor of the present invention will be described.
[0008]
In general, when a person listens to the sound of any real sound source with his / her ears, physics such as the person's head, both ears on the left and right sides of the head, and sound transmission structures in both ears It is known that the specific factor is deeply related to the sound image localization of the real sound source.
[0009]
The inventor of the present invention diligently investigated the effects or effects on the sound image of the human head and the auricles attached to the left and right sides of the above physical factors affecting sound image localization, and conducted experiments. As a result of repeated research, the following knowledge was obtained.
[0010]
That is, the human head, although there are individual differences, is considered to be a vibration transmitting sphere of about 150 to 200 mmφ, the frequency of which this diameter is a half wavelength (hereinafter, this frequency is referred to as “aHz”) or less Since the half wavelength is larger than the diameter of the sphere at the frequency of, the first finding is that the sound of the frequency below the aHz is less influenced by the human head. In other words, the sound below aHz ignores the reflection and diffraction of the sound from the human head, and the time difference and volume difference between the sound from the sound source below aHz and the person's ears is aHz. It is related to the sound image localization of the following sounds.
[0011]
On the other hand, when the pinna of both ears of a human being is considered to be a conical shape whose bottom surface is approximately 35 to 55 mmφ in diameter, the half wavelength exceeds the diameter of the pinna (hereinafter referred to as “bHz”) or more. The second finding is that the sound of a frequency is less influenced by the auricle as a physical factor of sound image localization of the sound of that frequency. Incidentally, when the inventor measured the influence characteristic of the sound in the frequency band of bHz or more using the dummy head, it was found that the characteristic closely resembled the acoustic characteristic of the sound that passed through the comb filter. For sounds in the frequency band above bHz, the input signal in this band is processed through a comb filter, and it is also known that the time difference between the left and right sound entering the ear and the volume difference are deeply related to the sound image localization of the sound in this band. Got.
[0012]
This has led to the finding that for sounds in the frequency band of bHz or in the vicinity thereof, it is necessary to consider acoustic characteristics having different elements from those of the above-mentioned frequency band of bHz.
[0013]
About the frequency band examined above, that is, the band other than the band below aHz and the band above bHz, that is, a narrow band between aHz and bHz, a conventionally known human head and left and right auricles are used. We learned that it is sufficient to process and control the input signal by simulating the frequency characteristics by reflection and diffraction as physical factors.
[0014]
Based on the above knowledge, the inventor of the present invention parameterized control elements such as time difference and volume difference of sounds entering the left and right ears for each frequency band of aHz or less, bHz or more, and aHz to bHz. As a result, the following test results were obtained.
[0015]
Test result of frequency band below aHz The audio signal in this band could be localized to some extent by simply controlling the two parameters of the time difference and volume difference of the sound entering the left and right ears. However, the localization of any space including the vertical direction is not sufficient only by the control of this element. Therefore, the time difference between the left and right ears should be controlled in units of 1/4 x 10 ^-5 to 1/5 x 10 ^-5 seconds, and the volume difference in units of ndB (where n is a natural number from 0.1 to 2 digits). Thus, it was found that the position of the sound image localization in the horizontal plane, the vertical plane, and the distance can be realized arbitrarily. Note that if the time difference between the left and right ears is increased for the sound in this band, the position of the sound image localization will be on the side of the listener.
[0016]
Effect of time difference between aHz and bHz test results Time difference of input signal in this band was attempted with parametric equalizer (hereinafter referred to as PEQ) disabled to give time difference to left and right ears. . As a result, a sound image localization such as that obtained by the control in the band below aHz was not obtained. It has been found that the sound image in this band moves linearly to the left and right by the control that gives the time difference.
When the input audio signal is processed through PEQ, control using the time difference between the left and right ears as a parameter becomes important. Here, there are three types of acoustic characteristics that can be corrected by PEQ: fc (center frequency), Q (sharpness), and Gain (gain).
Effect of volume difference When the volume difference for both the left and right ears is controlled around ndB (n is a natural number of one digit), the distance from the listener to the sound image localization point increases. The larger the volume difference, the smaller the distance to the sound image localization point.
Effect of fc When a sound source is placed at an angle of 45 degrees ahead of the listener and the audio signal input from the sound source is subjected to PEQ processing according to the listener's head-related transfer function, fc is shifted higher in the band of aHz to bHz. Then, it was found that the distance to the sound image localization position tends to increase. Conversely, it has also been found that when fc is shifted to a lower side, the distance to the sound image localization position tends to decrease.
Effect of Q When performing PEQ processing of an audio signal in the band of aHz to bHz under the same conditions as in the case of fc, if the Q near 1 kHz of the audio signal for the right ear is increased from the original value to about four times, The angle decreased, but conversely the distance increased and the vertical angle did not change. As a result, in this aHz to bHz band, the sound image can be localized forward at around 1 m. It was also found that when the gain of PEQ is negative, increasing the Q to be corrected tends to widen the sound image and shorten the distance.
Effect of Gain When PEQ processing is performed under the same conditions as the effects of fc and Q described above, the horizontal angle is smaller than 45 degrees when the gain at the peak portion near 1 kHz of the audio signal for the right ear is lowered by several dB. And the distance increases. A sound image localization position substantially equivalent to that obtained when Q was raised was realized. It should be noted that there was no change in the distance of sound image localization even when processing was performed so that the effects of Q and Gain were obtained simultaneously by PEQ.
[0017]
Effect of time difference in test results in frequency band above bHz Sound image localization could hardly be realized by controlling only the time difference entering the left and right ears. However, after comb filter processing, control that gives a time difference between the left and right ears was effective for sound image localization.
Effect of volume difference It was found that, if a volume difference between the left and right ears is given to an audio signal in this band, the effect is very effective compared to the case of other bands. That is, in order to localize the sound image in this band, it is necessary to control the left and right ears with a corresponding level, for example, a volume difference of 10 dB or more.
Effect of comb filter spacing When the test was performed with different comb filter spacing, the position of the sound image localization changed significantly. Further, the comb filter interval was varied only for one channel of the left ear or the right ear, but in this case, the left and right sound images were separated, and it was difficult to sense the sound image localization. Therefore, it is necessary to change the interval of the comb filter simultaneously for both channels for the left and right ears.
The relationship between the depth of influence of the depth of the comb filter and the vertical angle was a reverse characteristic.
The relationship between the depth and the horizontal angle was also reversed on the left and right.
The depth was found to be proportional to the distance of sound image localization.
[0018]
Crossover bandwidth test results
No discontinuity was observed in the band below aHz and the intermediate band between aHz and bHz, and the crossover portion between this intermediate band and the band above bHz, and there was no sense of antiphase. The frequency characteristics obtained by mixing the three bands were almost flat.
[0019]
From the above test results, it was found that the sound image localization can be controlled by different elements for each band by dividing the input audio signal into a plurality of frequency bands for both left and right ears. That is, for example, the effect of the time difference between the sounds entering the left and right ears on the sound image localization is significant in the band below aHz, but the effect of the time difference is small in the high band above bHz. One. In addition, in the high frequency band above bHz, it was also found that the use of the comb filter and the difference in volume between the left and right ears are significant for sound image localization. In the intermediate band from aHz to bHz, although the distance is short, parameters other than the above-described control element that is localized forward have also been found.
[0020]
Next, an example in which a monaural audio signal is localized at an arbitrary three-dimensional position based on the method described above will be described with reference to FIG. Reference numeral 1 denotes a monaural audio input signal obtained by picking up the sound of an arbitrary sound source with a microphone.
[0021]
Reference numeral 2 denotes a band division filter that divides the input signal 1 sent from the microphone into two channels for left and right ears and divides the signals of both channels into a plurality of frequency bands. In this embodiment, an input audio signal 1 is divided into three bands, for example, a low frequency band of about 1000 Hz or less, an intermediate frequency band of about 1000 to about 4000 Hz, and a high frequency band of about 4000 Hz or more. Three 33rd-order FIR filters 2L, 2M, and 2H were used. In the present invention, the band division of the input signal 1 is not limited to the above three bands. For example, the input signal 1 is divided into two of a low frequency band and a medium high frequency band, or the entire band of the input signal 1 is 4-5. It may be divided into bands. In the following, it is assumed that the processing is also performed on the signals in the respective bands for the left and right channels.
[0022]
3L, 3M, and 3H are signal processing for processing and controlling the audio signals of the respective bands for each band divided by the filters 2L, 2M, and 2H by the sound image localization method described above. Part. In the low frequency band, the primary IIR filter 31 is used to process the time difference and the volume difference as parameters for sound image localization for the left and right ear channels. In the intermediate frequency band, in order to accurately reproduce the frequency characteristic of the head-related transfer function, the PEQ is constituted by three stages of second-order IIR filters 32 in series for processing by a parametric equalizer (PEQ). In the high frequency band, since the head-related transfer function exhibits comb-like characteristics, the comb filter 33 is used.
[0023]
After sound image localization processing for each frequency band with the above configuration, each frequency band is subjected to delay control that takes into account the delay difference due to sound diffraction and the transmission time difference of the sound reaching both ears. Delay control units 4L, 4M, and 4H are provided, and the acoustic signals of the processed frequency bands output from the delay control units 4L, 4M, and 4H are mixed in the mixer 5 with the ratio (gain) of the amount of sound in each band. By adjusting and mixing, an acoustic signal that is localized in three dimensions can be obtained from the mixer 5.
[0024]
As illustrated in FIG. 2, the sound image can be localized at an arbitrary position on a concentric circle with the listener M as the center by the above three-dimensional sound image localization method. Therefore, in the present invention, this sound image localization method is used. The function is further expanded so that the sound image localized at a certain point can be moved to the next point.
[0025]
In order to move the localization sound image from the point A illustrated in FIG. 2 to the point B, or vice versa, all parameters used in the sound image localization method of FIG. Interpolation between points is sufficient, but this is not practical because the amount of calculation and memory becomes enormous.
[0026]
Therefore, in the present invention, in the sound image localization method described above, the delay control in the sound image localization processing for each frequency band, which plays an important role in expressing the moving effect of the sound image, and the ratio of each frequency band in the mixing processing ( The gain) is interpolated between the two points. The interpolation procedure will be described below.
[0027]
First, a delay difference and a gain difference between two points A and B in which sound image localization parameters are defined are obtained. Next, the delay difference and the gain difference are divided by the time required to move the localization sound image from the point A to the point B, and a time interval at which the difference between the minimum unit of the delay and the gain is obtained. In the example of FIG. 2, point A and point B are points where sound image localization is possible. For convenience of explanation, “5” in FIG. 2 is a coefficient of a delay difference or gain difference parameter at point A, and “15” is point B. It represents the coefficient of the parameter at. Here, when moving the sound image from the sound image localization point A to the point B, by dividing the time required for moving the sound image by the difference “10” between the coefficients at both points, the minimum unit of the coefficient is obtained by the movement of the sound image. Since a time interval where “1” changes is obtained, if the coefficient is changed from “5” at point A to 6, 7, 8,... , A and B are linearly interpolated between the delay difference and the gain difference. At this time, other parameters used in the sound image localization method illustrated in FIG. 1 are replaced with the localization parameter coefficients at the B point when the localized sound image moves to the B point.
[0028]
In other words, when the localization sound image is moved from point A to point B, the linearly interpolated values are used for the delay difference and the gain difference. For other parameters, the localization parameter at point A is used during movement. Every time it arrives at point B, it is replaced with the parameter at point B. On the contrary, in order to move the localization sound image from the point B to the point A, the localization parameter of the point B is used until the point A is reached.
[0029]
The processing necessary for moving the localization sound image includes processing for obtaining a delay difference for the left and right ears for each frequency band, processing for obtaining a difference in gain, and the delay difference and gain obtained in each processing. A process for obtaining a time interval at which each minimum unit of the difference changes, and a process for changing the values of the delay and gain by the values of the obtained minimum unit of delay and gain.
[0030]
Therefore, by adding the algorithm for executing the above-described processes for moving the sound image to the sound image localization method algorithm of FIG. 1, the sound image-localized sound that allows the listener M to feel a sense of movement by the same processing flow as FIG. Can be generated arbitrarily.
[0031]
Although the example of the method described above makes the listener M feel the sensation of moving the sound image localized on a concentric circle or a sphere, the present invention changes the three-dimensional radial distance around the listener M. The sound image can also be moved at.
[0032]
【The invention's effect】
The present invention is as described above. The input audio signal is divided into left and right binaural channels, and the audio signal of each channel is taken as an example in each of the low, medium and high frequency ranges. It is divided into three bands, and a localization sound image is obtained by performing a process of controlling the time difference between the left and right ears, a volume difference, etc. as a parameter having a sound image localization element for each frequency band, and for each frequency band described above, It is possible to perceive the sense of movement of the stereophonic sound image by changing the delay difference with respect to the left and right ears, which is an element for perceiving the moving effect of the sound image, and the ratio (gain) difference when mixing the sounds in each band for each unit time. Since sound is generated, it is possible to obtain a reproduced sound excellent in sound image localization from input audio signals for the left and right ears input from appropriate sound sources. By moving a three-dimensional sound image was, it audibility a stereoscopic effect of depth to the change in change in localized sound image is possible to obtain a reproduced sound. In addition to the above-described method, if the sound image localization and the control for moving the sound image are superimposed during reproduction, more effective and highly accurate sound image localization and movement of the sound image can be easily realized.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of an example for carrying out the method of the present invention.
FIG. 2 is a schematic diagram for explaining the method of the present invention.
[Explanation of symbols]
1 Input signal 2 Band division filter
3L, 3M, 3H signal processor
4L, 4M, 4H Delay control unit 5 Mixer

Claims (1)

The monaural input signal is divided into multiple bands between the low frequency band and the high frequency band by the digital filter for both the left and right channels,
If the divided band is a low frequency band, use a digital filter with the time difference and volume difference as parameters, and if it is an intermediate frequency band, use a parametric equalizer to follow the frequency characteristics of the head-related transfer function. In the case of, perform processing for sound localization for each band using a comb filter,
After performing delay processing considering the difference in sound diffraction for each frequency band of both channels and the transmission time difference of sound reaching both ears, the ratio of the amount of sound processed for each frequency band ( When the stereo image of the input signal is obtained by adjusting the (Gain) and mixing with a mixer,
In at least two sound image localization points where sound image localization is possible by this method, among the sound image transfer parameters for each frequency band at both points, a difference in delay parameter and a difference in gain parameter during mixing are obtained.
The amount of change per unit time of the difference between the time during which the localization sound image is moved from one localization point to the other localization point and the parameter at this time is obtained as a sound image movement parameter between the two localization points;
With this sound image movement parameter, the delay difference and gain difference at the one localization point are changed in the movement time, and the sound image movement parameters other than the delay difference and gain difference are changed to the sound image localization parameter at that point at the other localization point. A method of moving a three-dimensional space by localizing an input signal, characterized by generating a sound that gives a sense of movement by switching.

Images