JP6904197B2 - Signal processing equipment, signal processing methods, and programs - Google Patents

Description

The present invention relates to signal processing devices, signal processing methods, and programs.

As a sound image localization technique, there is an out-of-head localization technique in which a sound image is localized on the outside of the listener's head using headphones. In the out-of-head localization technology, the sound image is localized out of the head by canceling the characteristics from the headphones to the ears and giving four characteristics (spatial acoustic transmission characteristics) from the stereo speaker to the ears.

In the out-of-head localization reproduction, the measurement signal (impulse sound, etc.) emitted from the speaker of 2 channels (hereinafter referred to as ch) is recorded by a microphone (hereinafter referred to as a microphone) installed in the listener's ear. Then, the processing device creates a filter based on the sound pick-up signal obtained by the impulse response. By convolving the created filter into a 2ch audio signal, out-of-head localization reproduction can be realized.

Patent Document 1 discloses a method of obtaining a set of personalized indoor impulse responses. In Patent Document 1, a microphone is installed near each ear of the listener. Then, the left and right microphones record the impulse sound when the speaker is driven.

Japanese Patent Application Laid-Open No. 2008-512015

A head-related transfer function (HRTF) is used as such a spatial acoustic transmission characteristic from the speaker to the ear. The head-related transfer function is acquired by measuring the dummy head or the user himself / herself. Many analyzes and studies have been conducted on HRTFs and hearing and localization.

Spatial acoustic transmission characteristics are classified into two types: direct sound from the sound source to the listening position, and reflected sound (and diffracted sound) that is reflected and reaches an object such as a wall surface or a bottom surface. The direct sound, the reflected sound itself, and their relationship are components that represent the entire spatial acoustic transmission characteristic. Even in the simulation of acoustic characteristics, the overall characteristics may be calculated by simulating the direct sound and the reflected sound individually and integrating them. Also, in the analysis and research, it is very useful to be able to handle the transmission characteristics of two types of sounds individually.

Therefore, it is desired to appropriately separate the direct sound and the reflected sound from the sound picked up signal picked up by the microphone.

The present invention has been made in view of the above points, and an object of the present invention is to provide a signal processing device, a signal processing method, and a program capable of appropriately processing a sound pick-up signal.

The signal processing device according to the present embodiment has a sound collecting signal acquisition unit that acquires a sound collecting signal including a direct sound that directly reaches the microphone from the sound source and a reflected sound, and a first unit based on the time amplitude data of the sound collecting signal. 1 The first rough shape calculation unit that calculates the rough shape, the second rough shape calculation unit that calculates the second rough shape of the sound pick-up signal by smoothing the first rough shape, and the first rough shape. A time determining unit for determining the bottom time from the direct sound of the pick-up signal to the initial reflected sound and the peak time of the initial reflected sound based on at least one of the shape and the second general shape is provided. It is a thing.

The signal processing method according to the present embodiment includes a step of acquiring a sound collection signal including a direct sound directly reaching the microphone from the sound source and a reflected sound, and the sound collection signal based on the time amplitude data of the sound collection signal. A step of calculating the first rough shape, a step of calculating the second rough shape of the sound pick-up signal by smoothing the first rough shape, and at least the first rough shape and the second rough shape. Based on one of them, it is provided with a step of determining the bottom time from the direct sound of the pick-up signal to the initial reflected sound and the peak time of the initial reflected sound.

The program according to the present embodiment is a program for causing a computer to execute a signal processing method for processing a sound collection signal obtained by collecting sound output from a sound source with a microphone, and the signal processing method is , A step of acquiring a sound collection signal including a direct sound directly reaching the microphone from the sound source and a reflected sound, and a step of calculating the first rough shape of the sound collection signal based on the time amplitude data of the sound collection signal. , The sound collection signal is calculated based on the step of calculating the second outline of the sound collection signal by smoothing the first outline, and at least one of the first outline and the second outline. It has a step to determine the bottom time from the direct sound to the early reflected sound and the peak time of the early reflected sound.

According to the present invention, it is possible to provide a signal processing device, a signal processing method, and a program capable of appropriately processing a sound pick-up signal.

It is a block diagram which shows the out-of-head localization processing apparatus which concerns on this embodiment. It is a figure which shows the measuring apparatus of a spatial acoustic transmission characteristic. It is a control block diagram which shows the structure of a signal processing apparatus. It is a flowchart which shows the signal processing method in the signal processing apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows the signal processing method in the signal processing apparatus which concerns on Embodiment 1. FIG. It is a waveform diagram for demonstrating the processing in a signal processing apparatus. It is a flowchart which shows the signal processing method in the signal processing apparatus which concerns on Embodiment 2. It is a flowchart which shows the signal processing method in the signal processing apparatus which concerns on Embodiment 2. It is a waveform diagram for demonstrating the processing in a signal processing apparatus. It is a waveform diagram for demonstrating the process of finding a convergence point by an iterative search method.

An outline of the sound image localization process using the filter generated by the signal processing device according to the present embodiment will be described. The extra-head localization process according to the present embodiment is to perform the extra-head localization process using the spatial acoustic transmission characteristic and the external auditory canal transmission characteristic. The spatial acoustic transmission characteristic is a transmission characteristic from a sound source such as a speaker to the ear canal. The ear canal transmission characteristic is the transmission characteristic from the entrance of the ear canal to the eardrum. In the present embodiment, the spatial acoustic transmission characteristics when the headphones or earphones are not worn are measured, and the external auditory canal transmission characteristics when the headphones or earphones are worn are measured, and the head is used using these measurement data. Realizes external localization processing.

The out-of-head localization process according to this embodiment is executed on a user terminal such as a personal computer, a smart phone, or a tablet PC. A user terminal is an information processing device having a processing means such as a processor, a storage means such as a memory or a hard disk, a display means such as a liquid crystal monitor, and an input means such as a touch panel, a button, a keyboard, and a mouse. The user terminal may have a communication function for transmitting and receiving data. Further, an output means (output unit) having headphones or earphones is connected to the user terminal.

Embodiment 1.
(Out-of-head localization processing device)
FIG. 1 shows an out-of-head localization processing device 100 which is an example of the sound field reproducing device according to the present embodiment. FIG. 1 is a block diagram of the out-of-head localization processing device 100. The out-of-head localization processing device 100 reproduces the sound field for the user U who wears the headphones 43. Therefore, the out-of-head localization processing device 100 performs sound image localization processing on the stereo input signals XL and XR of Lch and Rch. The Lch and Rch stereo input signals XL and XR are analog audio reproduction signals output from a CD (Compact Disc) player or the like, or digital audio data such as mp3 (MPEG Audio Layer-3). The out-of-head localization processing device 100 is not limited to a physically single device, and some of the processing may be performed by different devices. For example, a part of the processing may be performed by a personal computer or the like, and the remaining processing may be performed by a DSP (Digital Signal Processor) built in the headphones 43 or the like.

The out-of-head localization processing device 100 includes an out-of-head localization processing unit 10, a filter unit 41, a filter unit 42, and headphones 43. The out-of-head localization processing unit 10, the filter unit 41, and the filter unit 42 can be specifically realized by a processor or the like.

The out-of-head localization processing unit 10 includes convolution calculation units 11 to 12, 21 to 22, and adders 24 and 25. The convolution calculation units 11 to 12 and 21 to 22 perform a convolution process using the spatial acoustic transmission characteristic. Stereo input signals XL and XR from a CD player or the like are input to the out-of-head localization processing unit 10. Spatial acoustic transmission characteristics are set in the out-of-head localization processing unit 10. The out-of-head localization processing unit 10 convolves a filter having spatial acoustic transmission characteristics (hereinafter, also referred to as a spatial acoustic filter) with the stereo input signals XL and XR of each channel. The spatial acoustic transmission characteristic may be a head-related transfer function HRTF measured by the head or auricle of the person to be measured, or may be a dummy head or a third-party head-related transfer function.

The spatial acoustic transfer function is a set of four spatial acoustic transfer characteristics Hls, Hlo, Hro, and Hrs. The data used for convolution by the convolution calculation units 11, 12, 21, and 22 serves as a spatial acoustic filter. A spatial acoustic filter is generated by cutting out the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs with a predetermined filter length.

Each of the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs has been acquired in advance by impulse response measurement or the like. For example, the user U wears microphones on the left and right ears, respectively. The left and right speakers arranged in front of the user U output impulse sounds for measuring the impulse response. Then, the measurement signal such as the impulse sound output from the speaker is picked up by the microphone. Spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs are acquired based on the sound pick-up signal of the microphone. Spatial sound transmission characteristic Hls between the left speaker and the left microphone, Spatial sound transmission characteristic Hlo between the left speaker and the right microphone, Spatial sound transmission characteristic Hro between the right speaker and the left microphone, Right speaker and the right microphone The spatial acoustic transmission characteristic Hrs between and is measured.

Then, the convolution calculation unit 11 convolves the spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hls with respect to the stereo input signal XL of the Lch. The convolution calculation unit 11 outputs the convolution calculation data to the adder 24. The convolution calculation unit 21 convolves a spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hro with respect to the stereo input signal XR of Rch. The convolution calculation unit 21 outputs the convolution calculation data to the adder 24. The adder 24 adds two convolution operation data and outputs the data to the filter unit 41.

The convolution calculation unit 12 convolves a spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hlo with respect to the Lch stereo input signal XL. The convolution calculation unit 12 outputs the convolution calculation data to the adder 25. The convolution calculation unit 22 convolves a spatial acoustic filter corresponding to the spatial acoustic transmission characteristic Hrs with respect to the stereo input signal XR of Rch. The convolution calculation unit 22 outputs the convolution calculation data to the adder 25. The adder 25 adds two convolution operation data and outputs the data to the filter unit 42.

Inverse filters that cancel the headphone characteristics (characteristics between the headphone playback unit and the microphone) are set in the filter units 41 and 42. Then, the inverse filter is convoluted into the reproduced signal (convolution calculation signal) processed by the out-of-head localization processing unit 10. The filter unit 41 convolves the inverse filter with respect to the Lch signal from the adder 24. Similarly, the filter unit 42 convolves the inverse filter with respect to the Rch signal from the adder 25. The reverse filter cancels the characteristics from the headphone unit to the microphone when the headphone 43 is attached. The microphone may be placed anywhere between the ear canal entrance and the eardrum. As will be described later, the inverse filter is calculated from the measurement result of the characteristics of the user U himself / herself.

The filter unit 41 outputs the processed Lch signal to the left unit 43L of the headphones 43. The filter unit 42 outputs the processed Rch signal to the right unit 43R of the headphones 43. The user U is wearing the headphones 43. The headphone 43 outputs the Lch signal and the Rch signal toward the user U. As a result, the sound image localized outside the head of the user U can be reproduced.

As described above, the out-of-head localization processing device 100 performs the out-of-head localization processing by using the spatial acoustic filter corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs and the inverse filter of the headphone characteristics. In the following description, the spatial acoustic filter corresponding to the spatial acoustic transmission characteristics Hls, Hlo, Hro, and Hrs and the inverse filter of the headphone characteristics are collectively referred to as an out-of-head localization processing filter. In the case of a 2ch stereo reproduction signal, the out-of-head localization filter is composed of four spatial acoustic filters and two inverse filters. Then, the out-of-head localization processing device 100 executes the out-of-head localization processing by performing a convolution calculation process on the stereo reproduction signal using a total of six out-of-head localization filters.

(Filter generator)
A filter generation device that measures spatial acoustic transmission characteristics (hereinafter referred to as transmission characteristics) and generates a filter will be described with reference to FIG. FIG. 2 is a diagram schematically showing the configuration of the filter generation device 200. The filter generation device 200 may be a device common to the out-of-head localization processing device 100 shown in FIG. Alternatively, a part or all of the filter generation device 200 may be a device different from the out-of-head localization processing device 100.

As shown in FIG. 2, the filter generation device 200 includes a stereo speaker 5, a stereo microphone 2, and a signal processing device 201. The stereo speaker 5 is installed in the measurement environment. The measurement environment may be the user U's home room, an audio system sales store, a showroom, or the like. In the measurement environment, sound is reflected by the floor and walls.

In the present embodiment, the signal processing device 201 of the filter generation device 200 performs arithmetic processing for appropriately generating a filter according to the transmission characteristics. The signal processing device 201 may be a personal computer (PC), a tablet terminal, a smart phone, or the like.

The signal processing device 201 generates a measurement signal and outputs it to the stereo speaker 5. The signal processing device 201 generates an impulse signal, a TSP (Time Streched Pure) signal, or the like as a measurement signal for measuring the transmission characteristic. The measurement signal includes a measurement sound such as an impulse sound. Further, the signal processing device 201 acquires the sound pick-up signal picked up by the stereo microphone 2. The signal processing device 201 has a memory and the like for storing measurement data of transmission characteristics.

The stereo speaker 5 includes a left speaker 5L and a right speaker 5R. For example, a left speaker 5L and a right speaker 5R are installed in front of the user U. The left speaker 5L and the right speaker 5R output an impulse sound or the like for measuring an impulse response. Hereinafter, in the present embodiment, the number of speakers serving as sound sources will be described as 2 (stereo speakers), but the number of sound sources used for measurement is not limited to 2, and may be 1 or more. That is, the present embodiment can be similarly applied to a so-called multi-channel environment such as 1ch monaural or 5.1ch, 7.1ch, etc.

The stereo microphone 2 has a left microphone 2L and a right microphone 2R. The left microphone 2L is installed in the left ear 9L of the user U, and the right microphone 2R is installed in the right ear 9R of the user U. Specifically, it is preferable to install microphones 2L and 2R at positions from the entrance of the ear canal to the eardrum of the left ear 9L and the right ear 9R. The microphones 2L and 2R collect the measurement signal output from the stereo speaker 5 and output the sound collection signal to the signal processing device 201. The user U may be a person or a dummy head. That is, in the present embodiment, the user U is a concept including not only a person but also a dummy head.

As described above, the measurement signals output from the left and right speakers 5L and 5R are picked up by the microphones 2L and 2R, and an impulse response is obtained based on the picked up sound signals. The filter generation device 200 stores the sound pick-up signal acquired based on the impulse response measurement in a memory or the like. As a result, the transmission characteristic Hls between the left speaker 5L and the left microphone 2L, the transmission characteristic Hlo between the left speaker 5L and the right microphone 2R, the transmission characteristic Hro between the right speaker 5R and the left microphone 2L, and the right speaker The transmission characteristic Hrs between the 5R and the right microphone 2R is measured. That is, the transmission characteristic Hls is acquired by the left microphone 2L collecting the measurement signal output from the left speaker 5L. The transmission characteristic Hlo is acquired by the right microphone 2R collecting the measurement signal output from the left speaker 5L. The transmission characteristic Hro is acquired by the left microphone 2L collecting the measurement signal output from the right speaker 5R. The transmission characteristic Hrs is acquired by the right microphone 2R picking up the measurement signal output from the right speaker 5R.

Then, the filter generation device 200 generates a filter according to the transmission characteristics Hls, Hlo, Hro, and Hrs from the left and right speakers 5L and 5R to the left and right microphones 2L and 2R based on the sound pick-up signal. By doing so, the filter generation device 200 generates a filter used for the convolution calculation of the out-of-head localization processing device 100. As shown in FIG. 1, the out-of-head localization processing device 100 uses a filter corresponding to the transmission characteristics Hls, Hlo, Hro, and Hrs between the left and right speakers 5L and 5R and the left and right microphones 2L and 2R. Perform external localization processing. That is, the out-of-head localization process is performed by convolving the filter according to the transmission characteristic into the audio reproduction signal.

Further, in the measurement environment, when the measurement signal is output from the speakers 5L and 5R, the sound pick-up signal includes a direct sound and a reflected sound. The direct sound is a sound that directly reaches the microphones 2L and 2R (ears 9L, 9R) from the speakers 5L and 5R. That is, the direct sound is the sound that reaches the microphones 2L and 2R from the speakers 5L and 5R without being reflected by the floor surface or the wall surface or the like. The reflected sound is a sound that is output from the speakers 5L and 5R, is reflected on the floor surface or the wall surface, and reaches the microphones 2L and 2R. The sound pick-up signals corresponding to the transmission characteristics Hls, Hlo, Hro, and Hrs each include direct sound and reflected sound. Direct sound reaches the ear faster than reflected sound. Then, the reflected sound reflected by an object such as a wall surface or a floor surface appears after the direct sound.

The signal processing device 201 performs processing for separating the direct sound and the reflected sound. Specifically, the signal processing device 201 calculates the separation boundary point between the direct sound and the arrival of the initial reflected sound. The initial reflected sound is the reflected sound that reaches the ear 9 (microphone 2) earliest among the reflected sounds reflected by an object such as a wall or a wall surface. Then, by separating the transmission characteristics Hls, Hlo, Hro, and Hrs at the separation boundary point, the direct sound and the reflected sound are separated. That is, the signal (characteristic) before the separation boundary point includes the direct sound, and the signal (characteristic) after the separation boundary point includes the reflected sound.

The signal processing device 201 performs processing for calculating a separation boundary point for separating the direct sound and the initially reflected sound. Specifically, the signal processing device 201 calculates the bottom time (bottom position) between the direct sound and the initial reflected sound and the peak time (peak position) of the initial reflected sound in the sound pick-up signal. Then, the signal processing device 201 sets a search range for searching the separation boundary point based on the bottom position and the peak position. The signal processing device 201 calculates the separation boundary point based on the value of the evaluation function in the search range.

The signal processing device 201 of the filter generation device 200 and its processing will be described in detail below. FIG. 3 is a control block diagram showing the signal processing device 201 of the filter generation device 200. Since the filter generation device 200 performs the same measurement for each of the left speaker 5L and the right speaker 5R, a case where the left speaker 5L is used as a sound source will be described here. That is, since the measurement using the right speaker 5R as a sound source can be performed in the same manner as the measurement using the left speed 5L as a sound source, the right speaker 5 is omitted in FIG.

The signal processing device 201 includes a measurement signal generation unit 211, a sound collection signal acquisition unit 212, a signal selection unit 221, a first rough shape calculation unit 222, a second rough shape calculation unit 223, and an extreme value calculation unit 224. , Time determination unit 225, search range setting unit 226, evaluation function calculation unit 227, separation boundary point calculation unit 228, characteristic separation unit 229, environment information setting unit 230, characteristic analysis unit 241 and characteristic adjustment. A unit 242, a characteristic generation unit 243, and an output device 250 are provided.

The signal processing device 201 is an information processing device such as a personal computer or a smart phone, and includes a memory and a CPU. The memory stores processing programs, various parameters, measurement data, and the like. The CPU executes a processing program stored in the memory. When the CPU executes the processing program, the measurement signal generation unit 211, the sound pick-up signal acquisition unit 212, the signal selection unit 221, the first rough shape calculation unit 222, the second rough shape calculation unit 223, the extreme value calculation unit 224, Search range setting unit 226, evaluation function calculation unit 227, separation boundary point calculation unit 228, characteristic separation unit 229, environment information setting unit 230, characteristic analysis unit 241, characteristic adjustment unit 242, characteristic generation unit 243, and output device 250. Each process is carried out.

The measurement signal generation unit 211 generates a measurement signal. The measurement signal generated by the measurement signal generation unit 211 is D / A converted by the D / A converter 215 and output to the left speaker 5L. The D / A converter 215 may be built in the signal processing device 201 or the left speaker 5L. The left speaker 5L outputs a measurement signal for measuring the transmission characteristics. The measurement signal may be an impulse signal, a TSP (Time Streched Pure) signal, or the like. The measurement signal includes a measurement sound such as an impulse sound.

The left microphone 2L and the right microphone 2R of the stereo microphone 2 each collect the measurement signal and output the sound collection signal to the signal processing device 201. The sound collection signal acquisition unit 212 acquires the sound collection signals from the left microphone 2L and the right microphone 2R. The sound pick-up signals from the microphones 2L and 2R are A / D-converted by the A / D converter 213L and 213R and input to the sound pick-up signal acquisition unit 212. The sound pick-up signal acquisition unit 212 may synchronously add the signals obtained by a plurality of measurements. Here, since the impulse sound output from the left speaker 5L is picked up, the sound pick-up signal acquisition unit 212 acquires the sound pick-up signal corresponding to the transmission characteristic Hls and the sound pick-up signal corresponding to the transmission characteristic Hlo. To do.

Hereinafter, signal processing in the signal processing device 201 will be described with reference to FIGS. 4 to 6 together with FIG. 4 and 5 are flowcharts showing a signal processing method. FIG. 6 is a waveform diagram showing signals in each process. In FIG. 6, the horizontal axis represents time and the vertical axis represents signal strength. The horizontal axis (time axis) is normalized so that the time of the first data is 0 and the time of the last data is 1.

First, the signal selection unit 221 selects the sound collection signal closer to the sound source from the pair of sound collection signals acquired by the sound collection signal acquisition unit 212 (S101). Since the left microphone 2 is closer to the left speaker 5L than the right microphone 2R, the signal selection unit 221 selects a sound pick-up signal corresponding to the transmission characteristic Hls. As shown in Graph I of FIG. 6, in the microphone 2L close to the sound source (speaker 5L), the direct sound arrives earlier than the microphone 2R. Therefore, it is possible to select a sound pick-up signal that is close to the sound source by comparing the arrival times at which the sound arrives earliest among the two sound pick-up signals. It is also possible to input the environmental information from the environmental information setting unit 230 into the signal selection unit 221 and have the signal selection unit 221 collate the selection result with the environmental information.

The first outline calculation unit 222 calculates the first outline based on the time amplitude data of the sound collection signal. In order to calculate the first outline, first, the first outline calculation unit 222 calculates the time amplitude data by Hilbert transforming the selected sound collection signal (S102). Next, the first outline calculation unit 222 linearly interpolates between the peaks (maximum values) of the time amplitude data to calculate the linear interpolation data (S103).

Then, the first outline calculation unit 222 sets the cutout width T3 based on the arrival prediction time T1 of the direct sound and the arrival prediction time T2 of the initial reflected sound (S104). Environmental information related to the measurement environment is input to the first outline calculation unit 222 from the environment information setting unit 230. Environmental information includes geometric information about the measurement environment. For example, one or more information of the distance and angle from the user U to the speaker 5L, the distance from the user U on the wall surfaces on both sides, the installation height of the speaker 5L, the ceiling height, and the ground clearance of the user U is included. The first outline calculation unit 222 predicts the arrival prediction time T1 of the direct sound and the arrival prediction time T2 of the initial reflected sound, respectively, using the environmental information. The first scheme calculation unit 222 sets, for example, twice the difference between the two estimated arrival times as the cutout width T3. That is, the cutout width T3 = 2 × (T2-T1). The cutout width T3 may be preset in the environment information setting unit 230.

The first outline calculation unit 222 calculates the rise time T4 of the direct sound based on the linear interpolation data (S105). For example, the first scheme calculation unit 222 can set the time (position) of the earliest peak (maximum value) in the linear interpolation data as the rise time T4.

The first outline calculation unit 222 calculates the first outline by cutting out the linear interpolation data of the cutout range and performing window hanging (S106). For example, the time before the rise time T4 by a predetermined time is the cutout start time T5. Then, the linear interpolation data is cut out with the time of the cutout width T3 as the cutout range from the cutout start time T5. The first scheme calculation unit 222 calculates the cutout data by cutting out the linear interpolation data of the cutout range of T5 to (T5 + T3). Then, the first outline calculation unit 222 calculates the first outline by window-hanging the data so that both ends of the data converge to 0 outside the cutout range. Graph II of FIG. 6 shows the waveform of the first outline.

The second outline calculation unit 223 calculates the second outline from the first outline by a smoothing filter (cubic function approximation) (S107). That is, the second rough shape calculation unit 223 calculates the second rough shape by performing a smoothing process on the first rough shape. Here, the second scheme calculation unit 223 uses the data obtained by smoothing the first scheme by cubic function approximation as the second scheme. Graph II of FIG. 6 shows the waveform of the second outline. Of course, the second scheme calculation unit 223 may calculate the second scheme using a smoothing filter other than the cubic function approximation.

The extreme value calculation unit 224 obtains all the maximum values and the minimum values of the second outline (S108). Next, the extremum calculation unit 224 excludes the extremum before the maximum value that takes the maximum (S109). The maximum value that takes the maximum corresponds to the peak of the direct sound. The extreme value calculation unit 224 excludes extreme values in which two consecutive extreme values are within a certain level difference (S110). In this way, the extremum calculation unit 224 extracts the extremum. Graph II of FIG. 6 shows the extremum extracted from the second scheme. The extreme value calculation unit 224 extracts the minimum value that is a candidate for the bottom time Tb.

For example, from an early time, 0.8 (maximum value), 0.5 (minimum value), 0.54 (maximum value), 0.2 (minimum value), 0.3 (maximum value), 0.1 ( An example of numerical values arranged in the order of (minimum value) will be described. When a certain level difference (threshold value) is 0.05, in the pair of [0.5 (minimum value), 0.54 (maximum value)], two consecutive extreme values are less than or equal to a certain level difference. It becomes. As a result, the extreme value calculation unit 224 eliminates the extreme values of 0.5 (minimum value) and 0.54 (maximum value). The extreme values remaining without being excluded are 0.8 (maximum value), 0.2 (minimum value), 0.3 (maximum value), and 0.1 (minimum value) in order from the earliest time. In this way, the extremum calculation unit 224 eliminates unnecessary extremums. By excluding extreme values in which two consecutive extreme values are equal to or less than a certain level difference, only appropriate extreme values can be extracted.

The time determination unit 225 calculates the bottom time Tb from the direct sound to the initial reflected sound and the peak time Tp of the initial reflected sound based on the first outline and the second outline. Specifically, the time determination unit 225 sets the time (position) of the earliest minimum value among the extreme values of the second rough form obtained by the extreme value calculation unit 224 as the bottom time Tb (S111). ). That is, among the extreme values of the second rough form not excluded by the extreme value calculation unit 224, the time of the minimum value at the earliest time is the bottom time Tb. The bottom time Tb is shown in Graph II of FIG. In the above numerical example, the time of 0.2 (minimum value) is the bottom time Tb.

The time determination unit 225 obtains the differential value of the first outline, and sets the time at which the differential value takes the maximum after the bottom time Tb as the peak time Tp (S112). Graph III of FIG. 6 shows the waveform of the differential value of the first scheme and its maximum point. As shown in Graph III, the maximum point of the differential value of the first scheme is the peak time Tp.

The search range setting unit 226 determines the search range Ts from the bottom time Tb and the peak time Tp (S113). For example, the search range setting unit 226 sets the time before the specified time T6 from the bottom time Tb as the search start time T7 (= Tb-T6), and sets the peak time Tp as the search end time. In this case, the search range Ts is T7 to Tp.

Then, the evaluation function calculation unit 227 calculates the evaluation function (third outline) using the data of the pair of sound pick-up signals and the reference signal in the search range Ts (S114). The pair of sound collecting signals is a sound collecting signal corresponding to the transmission characteristic Hls and a sound collecting signal corresponding to the transmission characteristic Hlo. The reference signal is a signal in which all the values in the search range Ts are 0. Then, the evaluation function calculation unit 227 calculates the average value of the absolute values and the sample standard deviation for the three signals of the two sound pickup signals and the one reference signal.

For example, the absolute value of the sound collection signal of the transmission characteristic Hls at time T is ABS _Hls (t), the absolute value of the sound collection signal of the transmission characteristic Hlo is ABS _Hlo (t), and the absolute value of the reference signal is ABS _Ref (ABS Ref (t). t). The average value of the three absolute values is ABS _ave = (ABS _Hls (t) + ABS _Hlo (t) + ABS _Hls (t)) / 3. _{Further, let} σ (t) be the sample standard deviations of the three absolute values ABS Hls (t), ABS _Hlo (t), and ABS _{Ref (t).} Then, the evaluation function calculation unit 227 uses _{the added value (ABS ave} (t) + σ (t)) of _{the average value ABS ave} of the absolute values and the sample standard deviation σ (t) as the evaluation function. The evaluation function is a signal that changes with time in the search range Ts. The evaluation function is shown in Graph IV of FIG.

Then, the separation boundary point calculation unit 228 searches for the point where the evaluation function is the minimum, and sets that time as the separation boundary point (S115). The point (T8) where the evaluation function is minimized is shown in Graph IV of FIG. By doing so, it is possible to calculate the separation boundary point for appropriately separating the direct sound and the initially reflected sound. By calculating the evaluation function using the reference signal, the point where the pair of sound pickup signals are close to 0 can be set as the separation boundary point.

Then, the characteristic separation unit 229 separates the pair of sound pick-up signals at the separation boundary point. As a result, the sound pick-up signal is separated into a transmission characteristic (signal) including the direct sound and a transmission characteristic (signal) including the initial reflected sound. That is, the signal before the separation boundary point exhibits the direct sound transmission characteristic. In the signal after the separation boundary point, the transmission characteristic of the reflected sound reflected by an object such as a wall surface or a floor surface becomes dominant.

The characteristic analysis unit 241 analyzes the frequency characteristics of the signals before and after the separation boundary point. The characteristic analysis unit 241 performs a discrete Fourier transform or a discrete cosine transform to calculate the frequency characteristics. The characteristic adjustment unit 242 adjusts the frequency characteristics of the signal before and after the separation boundary point. For example, the amplitude of the frequency band in which there is a response can be adjusted to either of the signals before and after the characteristic adjustment unit 242 separation boundary point. The characteristic generation unit 243 generates transmission characteristics by synthesizing the characteristics analyzed and adjusted by the characteristic analysis unit 241 and the characteristic adjustment unit 242.

Since a known method can be used for the processing in the characteristic analysis unit 241 and the characteristic adjustment unit 242 and the characteristic generation unit 243, the description thereof will be omitted. The transmission characteristic generated by the characteristic generation unit 243 serves as a filter corresponding to the transmission characteristics Hls and Hlo. Then, the output device 250 outputs the characteristics generated by the characteristic generation unit 243 to the out-of-head low-level processing device 100 as a filter.

As described above, in the present embodiment, the sound collection signal acquisition unit 212 acquires the sound collection signal including the direct sound directly reaching the microphone 2L from the left speaker 5L which is the sound source and the reflected sound. The first outline calculation unit 222 calculates the first outline based on the time amplitude data of the sound collection signal. The second outline calculation unit 223 calculates the second outline of the sound pick-up signal by smoothing the first outline. Based on the first outline and the second outline, the time determination unit 225 has a bottom time (bottom position) from the direct sound of the pick-up signal to the initial reflected sound, and a peak time (peak position) of the initial reflected sound. And have been decided.

The time determination unit 225 can appropriately obtain the bottom time from the direct sound of the pick-up signal to the initial reflected sound and the peak time of the initial reflected sound. That is, the bottom time and the peak time, which are information for appropriately separating the direct sound and the reflected sound, can be appropriately obtained. According to this embodiment, the sound pick-up signal can be appropriately processed.

Further, in the present embodiment, the first scheme calculation unit 222 performs the Hilbert transform of the sound collection signal in order to obtain the time amplitude data of the sound collection signal. Then, the first outline calculation unit 222 interpolates the peaks of the time amplitude data in order to obtain the first outline. The first scheme calculation unit 222 is windowed so that both ends of the interpolated data obtained by interpolating the peaks converge to 0. Thereby, the first outline for obtaining the bottom time Tb and the peak time Tp can be appropriately obtained.

The second scheme calculation unit 223 calculates the second scheme by performing a smoothing process on the first scheme using a cubic function approximation or the like. Thereby, the second outline for obtaining the bottom time Tb and the peak time Tp can be appropriately obtained. A polynomial other than the cubic function or another function may be used as the approximate expression for calculating the second scheme.

The search range Ts is set based on the bottom time Tb and the peak time Tp. As a result, the separation boundary point can be calculated appropriately. In addition, the separation boundary point can be automatically calculated by a computer program or the like. In particular, even in a measurement environment in which the initial reflected sound arrives at a timing when the reflected sound does not converge, appropriate separation is possible.

Further, in the present embodiment, the environmental information setting unit 230 sets the environmental information regarding the measurement environment. Then, the cutout width T3 is set based on the environmental information. Thereby, the bottom time Tb and the peak time Tp can be obtained more appropriately.

The evaluation function calculation unit 227 calculates the evaluation function based on the sound collection signals acquired by the two microphones 2L and 2R. As a result, an appropriate evaluation function can be calculated. Therefore, an appropriate separation boundary point can be obtained for the sound pick-up signal of the microphone 2R far from the sound source. Of course, when the sound from the sound source is picked up by three or more microphones, the evaluation function may be obtained from the three or more picked up signals.

Further, the evaluation function calculation unit 227 may obtain an evaluation function for each sound collection signal. In this case, the separation boundary point calculation unit 228 calculates the separation boundary point for each sound collection signal. Thereby, an appropriate separation boundary point can be determined for each sound pick-up signal. For example, in the search range Ts, the evaluation function calculation unit 227 calculates the absolute value of the sound collection signal as the evaluation function. The separation boundary point calculation unit 228 can set the point where the evaluation function is the minimum as the separation boundary point. The separation boundary point calculation unit 228 can set a point at which the fluctuation of the evaluation function becomes small as the separation boundary point.

The right speaker 5R is also subjected to the same processing as the left speaker 5L. As a result, it can be obtained as a filter in the convolution calculation units 11, 12, 21, and 22 shown in FIG. Therefore, it is possible to perform high-precision extra-head low-level processing.

Embodiment 2.
The signal processing method according to the present embodiment will be described with reference to FIGS. 7 to 9. 7 and 8 are flowcharts showing the signal processing method according to the second embodiment. FIG. 9 is a diagram showing a waveform for explaining each process. Since the configurations of the filter generation device 200 and the signal processing device 201 in the second embodiment are the same as those in FIGS. 2 and 3 shown in the first embodiment, the description thereof will be omitted.

In the present embodiment, the processing in the first outline calculation unit 222, the second outline calculation unit 223, the time determination unit 225, the evaluation function calculation unit 227, and the separation boundary point calculation unit 228 is the processing of the first embodiment. Is different. The same processing as in the first embodiment will be omitted as appropriate. For example, the processing of the extreme value calculation unit 224, the characteristic separation unit 229, the characteristic analysis unit 241 and the characteristic adjustment unit 242, the characteristic generation unit 243 and the like is the same as the processing of the first embodiment, and thus detailed description thereof will be omitted.

First, the signal selection unit 221 selects the sound collection signal closer to the sound source from the pair of sound collection signals acquired by the sound collection signal acquisition unit 212 (S201). As a result, the signal selection unit 221 selects the sound pick-up signal corresponding to the transmission characteristic Hls, as in the first embodiment. A pair of sound pick-up signals is shown in Graph I of FIG.

The first outline calculation unit 222 calculates the first outline based on the time amplitude data of the sound collection signal. In order to calculate the first outline, first, the first outline calculation unit 222 smoothes the data of the absolute value of the amplitude of the selected sound collection signal by taking a simple moving average (). S202). Here, the data of the absolute value of the amplitude of the sound pick-up signal is used as the time amplitude data. Then, the data obtained by smoothing the time amplitude data is used as the smoothed data. The smoothing process is not limited to the simple moving average.

The first outline calculation unit 222 sets the cutout width T3 based on the estimated arrival time T1 of the direct sound and the estimated arrival time T2 of the initial reflected sound (S203). The cutout width T3 can be set based on the environmental information as in S104.

The first outline calculation unit 222 calculates the rise time T4 of the direct sound based on the smoothing data (S104). For example, the first scheme calculation unit 222 can set the position (time) of the earliest peak (maximum value) in the smoothed data as the rise time T4.

The first outline calculation unit 222 calculates the first outline by cutting out the smoothing data of the cutout range and performing window hanging (S205). Since the processing in S205 is the same as the processing in S106, the description thereof will be omitted. Graph II of FIG. 9 shows the waveform of the first outline.

The second scheme calculation unit 223 calculates the second scheme from the first scheme by cubic spline interpolation (S206). That is, the second scheme calculation unit 223 calculates the second scheme by smoothing the first scheme by applying third-order spline interpolation. Graph II of FIG. 9 shows the waveform of the second outline. Of course, the second scheme calculation unit 223 may smooth the first scheme by using a method other than the third-order spline interpolation. For example, the smoothing method is not particularly limited, such as B-spline interpolation, approximation by Bezier curve, Lagrange interpolation, and smoothing by Savitzky-Goray filter.

The extreme value calculation unit 224 obtains all the maximum values and the minimum values of the second outline (S207). Next, the extremum calculation unit 224 excludes the extremum before the maximum value that takes the maximum (S208). The maximum value that takes the maximum corresponds to the peak of the direct sound. The extreme value calculation unit 224 excludes extreme values in which two consecutive extreme values are within a certain level difference (S209). As a result, candidates for a minimum value that is a candidate for the bottom time Tb and a maximum value that is a candidate for the peak time Tp are obtained. Since the processing of S207 to S209 is the same as the processing of S108 to S110, the description thereof will be omitted. Graph II of FIG. 9 shows the extremum of the second outline.

Next, the time determination unit 225 obtains an extremum pair that maximizes the difference between two consecutive extrema values (S210). The difference between extrema is a value defined by the slope in the time axis direction. The extreme value pair obtained by the time determination unit 225 is in the order in which the maximum value is obtained after the minimum value. That is, in the order in which the maximum value is followed by the minimum value, the difference between the extreme values is negative. Therefore, the extreme value pair obtained by the time determination unit 225 is in the order in which the maximum value is obtained after the minimum value.

The time determination unit 225 sets the time of the minimum value of the obtained extreme value pair as the bottom time Tb from the direct sound to the initial reflected sound, and sets the time of the maximum value as the peak time Tp of the initial reflected sound (S211). Graph III of FIG. 9 shows the bottom time Tb and the peak time Tp.

The search range setting unit 226 determines the search range Ts from the bottom time Tb and the peak time Tp (S212). For example, similarly to S113, the search range setting unit 226 sets the time before the bottom time Tb by the specified time T6 as the search start time T7 (= Tb-T6), and sets the peak time Tp as the search end time.

The evaluation function calculation unit 227 calculates the evaluation function (third outline) using the data of the pair of sound pick-up signals in the search range Ts (S213). The pair of sound collecting signals is a sound collecting signal corresponding to the transmission characteristic Hls and a sound collecting signal corresponding to the transmission characteristic Hlo. Therefore, in the present embodiment, unlike the first embodiment, the evaluation function calculation unit 227 calculates the evaluation function without using the reference signal.

Here, the sum of the absolute values of the pair of sound collecting signals is used as the evaluation function. For example, the absolute value of the sound pick-up signal of the transmission characteristic Hls at time T is ABS _Hls (t), and the absolute value of the sound pick-up signal of the transmission characteristic Hlo is ABS _Hlo (t). The evaluation function is ABS _Hls (t) + ABS _Hlo (t). The evaluation function is shown in Graph III of FIG.

The separation boundary point calculation unit 228 obtains the convergence point of the evaluation function by the iterative search method, and sets the time as the separation boundary point (S214). Graph III of FIG. 9 shows the time T8 of the convergence point of the evaluation function. For example, in the present embodiment, the separation boundary point calculation unit 228 calculates the separation boundary point by iteratively searching as follows.
(1) Data of a constant window width is extracted from the beginning of the search range Ts, and the total is calculated.
(2) The window is shifted in the time axis direction, and the sum of the window width data is sequentially obtained.
(3) Determine the window position where the obtained sum is the minimum, cut out the data, and use it as a new search range.
(4) The processes (1) to (3) are repeated until the convergence point is obtained.

By using the iterative search method, the time when the fluctuation of the evaluation function becomes small can be set as the separation boundary point. FIG. 10 is a waveform diagram showing data cut out by the iterative search method. FIG. 10 shows the waveform obtained by the process of repeating the first search to the third search three times. In FIG. 10, the time axis, which is the horizontal axis, is shown by the number of samples.

In the first search, the separation boundary point calculation unit 228 sequentially obtains the total sum in the search range Ts with the first window width. In the second search, the separation boundary point calculation unit 228 sets the first window width at the window position obtained in the first search as the search range Ts1, and sequentially obtains the total sum in the second window width. The width of the second window is narrower than the width of the first window.

Similarly, in the third search, the separation boundary point calculation unit 228 sets the second window width at the window position obtained in the second search as the search range Ts2, and sequentially obtains the total sum in the third window width. The width of the third window is narrower than the width of the second window. The window width in each search may be any value as long as it is set appropriately. Further, the window width may be appropriately changed for each iteration. Further, as in the first embodiment, the minimum value of the evaluation function may be set as the separation boundary point.

As described above, in the present embodiment, the sound collection signal acquisition unit 212 acquires the sound collection signal including the direct sound directly reaching the microphone 2L from the left speaker 5L which is the sound source and the reflected sound. The first outline calculation unit 222 calculates the first outline based on the time amplitude data of the sound collection signal. The second outline calculation unit 223 calculates the second outline of the sound pick-up signal by smoothing the first outline. The time determination unit 225 determines the bottom time (bottom position) from the direct sound of the pick-up signal to the initial reflected sound and the peak time (peak position) of the initial reflected sound based on the second outline. ing.

By doing so, the bottom time from the direct sound of the pick-up signal to the initial reflected sound and the peak time of the initial reflected sound can be appropriately obtained. That is, the bottom time and the peak time, which are information for appropriately separating the direct sound and the reflected sound, can be appropriately obtained. As described above, the sound pick-up signal can be appropriately processed by the processing of the second embodiment as well as the first embodiment.

The time determination unit 225 may determine the bottom time Tb and the peak time Tp based on at least one of the first outline and the second outline. Specifically, the peak time Tp may be determined based on the first outline as in the first embodiment, or may be determined based on the second outline as in the second embodiment. .. Further, in the first and second embodiments, the time determination unit 225 determines the bottom time Tb based on the second outline, but the bottom time Tb may be determined based on the first outline. ..

The process of the first embodiment and the process of the second embodiment can be combined as appropriate. For example, instead of the processing of the first scheme calculation unit 222 in the first embodiment, the processing of the first scheme calculation unit 222 in the second embodiment may be used. Similarly, the processing of the second rough shape calculation unit 223, the extreme value calculation unit 224, the time determination unit 225, the search range setting unit 226, the evaluation function calculation unit 227, or the separation boundary point calculation unit 228 in the first embodiment. Instead, the processing of the second rough shape calculation unit 223, the extreme value calculation unit 224, the time determination unit 225, the search range setting unit 226, the evaluation function calculation unit 227, or the separation boundary point calculation unit 228 in the second embodiment is performed. You may use it.

Alternatively, the first rough shape calculation unit 222, the second rough shape calculation unit 223, the extreme value calculation unit 224, the time determination unit 225, the search range setting unit 226, the evaluation function calculation unit 227, or the separation boundary in the second embodiment. Instead of the processing of the point calculation unit 228, the first rough shape calculation unit 222, the second rough shape calculation unit 223, the extreme value calculation unit 224, the time determination unit 225, the search range setting unit 226, and the evaluation in the first embodiment. The processing of the function calculation unit 227 or the separation boundary point calculation unit 228 may be used. In this way, the first outline calculation unit 222, the second outline calculation unit 223, the extreme value calculation unit 224, the time determination unit 225, the search range setting unit 226, the evaluation function calculation unit 227, and the separation boundary point calculation unit 228. It is possible to replace at least one or more of the processes of the above with the first embodiment and the second embodiment.

Part or all of the above processing may be executed by a computer program. The programs described above can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-transient computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)) is included. The program may also be supplied to the computer by various types of temporary computer readable media. Examples of temporary computer-readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Although the invention made by the present inventor has been specifically described above based on the embodiment, the present invention is not limited to the above embodiment and can be variously modified without departing from the gist thereof. Needless to say.

U User 2 Microphone unit 2L Left microphone 2R Right microphone 5L Left speaker 5R Right speaker 9L Left ear 9R Right ear 10 Out-of-head localization processing unit 11 Convolution calculation unit 12 Convolution calculation unit 21 Convolution calculation unit 22 Convolution calculation unit 24 Adder 25 Addition Instrument 41 Filter unit 42 Filter unit 200 Filter generation device 201 Signal processing device 211 Measurement signal generation unit 212 Sound collection signal acquisition unit 221 Signal selection unit 222 1st rough shape calculation unit 223 2nd rough shape calculation unit 224 Extreme value calculation unit 225 Time determination unit 226 Search range setting unit 227 Evaluation function calculation unit 228 Separation boundary point calculation unit 229 Characteristic separation unit 230 Environmental information setting unit 241 Characteristic analysis unit 242 Characteristic adjustment unit 243 Characteristic generator 250 Output unit

Claims (5)

A sound collection signal acquisition unit that acquires a sound collection signal including the direct sound that reaches the microphone directly from the sound source and the reflected sound,
The first outline calculation unit that calculates the first outline based on the time amplitude data of the sound collection signal, and
A second outline calculation unit that calculates the second outline of the sound collection signal by smoothing the first outline, and
A time determination unit that determines the bottom time from the direct sound of the sound collection signal to the initial reflected sound and the peak time of the initial reflected sound based on at least one of the first rough shape and the second rough shape. A signal processing device equipped with and. The signal processing apparatus according to claim 1, further comprising a search range determining unit for determining a search range for searching a separation boundary point based on the bottom time and the peak time. An evaluation function calculation unit that calculates an evaluation function based on the sound collection signal in the search range,
The signal processing device according to claim 2, further comprising a separation boundary point calculation unit that calculates the separation boundary point based on the evaluation function. The step of acquiring a pick-up signal including the direct sound that reaches the microphone directly from the sound source and the reflected sound,
A step of calculating the first outline of the sound collection signal based on the time amplitude data of the sound collection signal, and
A step of calculating the second outline of the sound collection signal by smoothing the first outline, and
Based on at least one of the first outline and the second outline, a step of determining the bottom time from the direct sound of the sound collection signal to the initial reflected sound and the peak time of the initial reflected sound is determined. Provided signal processing method. A program that causes a computer to execute a signal processing method that processes a sound collection signal obtained by collecting the sound output from a sound source with a microphone.
The signal processing method is
The step of acquiring a pick-up signal including the direct sound that reaches the microphone directly from the sound source and the reflected sound,
A step of calculating the first outline of the sound collection signal based on the time amplitude data of the sound collection signal, and
A step of calculating the second outline of the sound collection signal by smoothing the first outline, and
Based on at least one of the first outline and the second outline, a step of determining the bottom time from the direct sound of the pick-up signal to the initial reflected sound and the peak time of the initial reflected sound is determined. Prepared, program.

Images