JP7497040B2 - AUDIO SIGNAL PROCESSING APPARATUS, AUDIO SIGNAL PROCESSING METHOD, AND PROGRAM - Google Patents

Description

Article 30, Paragraph 2 of the Patent Act applied. Proceedings of the 2020 Spring Research Presentation of the Acoustical Society of Japan (held at Saitama University) held on March 2, 2020.

The present invention relates to an audio signal processing device, an audio signal processing method, and a program.

Various techniques have been proposed to generate a time series showing a sound in which the influence of sounds other than the target sound (i.e., noise) has been reduced from a time series showing the sound when the target sound is recorded (Non-Patent Documents 1 and 2).

R. Martin, “Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics” IEEE Trans. on SAP, vol. 9, no. 5, pp.504-512, 2001. T. Gerkmann, et al., “Unbiased MMSE-Based Noise Power Estimation With Low Complexity and Low Tracking Delay” IEEE/ACM Trans. on ASLP,vol. 20, no. 4, pp.1383|1393, 2012.

One such proposal is a technique for removing non-stationary noise from mono recordings. However, this technique cannot be said to remove noise sufficiently. As such, it is difficult to generate a time series of sound with reduced noise from a time series of sound recorded with a single microphone (i.e., mono recording).

In view of the above circumstances, the present invention aims to provide a technology for generating a time series of sound with reduced influence of noise from a time series of sound recorded in mono.

One aspect of the present invention is a processing target spectrogram, which is a time series of complex spectra of the processing target sound time series that satisfies an origin coincidence condition that the start position of the processing target sound in the processing target sound time series that indicates the processing target sound recorded in mono coincides with the time origin of the reference time series that indicates the processing target sound that has been recorded in advance, and a sampling frequency coincidence condition that the sampling frequency of the processing target sound time series and the sampling frequency of the reference time series coincide, and a reference spectrogram, which is a time series of complex spectra of the reference time series that satisfies the origin coincidence condition and the sampling frequency coincidence condition. The acoustic signal processing device includes: a compensated spectrogram acquisition unit that acquires a spectrogram to be processed; a target complex spectrogram estimation unit that estimates a time series that maximizes the likelihood, for a predetermined probability distribution, of a target complex spectrum, which is a complex spectrum in a target complex spectrogram obtained as the difference between the complex spectrum at the same time of the processing target spectrogram and the reference spectrogram multiplied by a frequency transfer function; and a target sound time series generation unit that generates a time series of a sound having the target complex spectrogram estimated by the target complex spectrogram estimation unit.

The present invention makes it possible to generate a time series of sound with reduced noise effects from a time series of sound recorded in mono.

1 is an explanatory diagram illustrating an overview of an acoustic signal processing device 1 according to an embodiment. FIG. 1 is a diagram showing an example of a functional configuration of an acoustic signal processing device 1 according to an embodiment. FIG. 2 is a diagram showing an example of a functional configuration of a control unit 10 according to the embodiment. 4 is a flowchart showing an example of a flow of processing executed by the acoustic signal processing device 1 of the embodiment. FIG. 2 is an explanatory diagram for explaining an experiment environment for an evaluation experiment according to the embodiment. 11A to 11C are diagrams showing examples of results of evaluation experiments in the embodiment.

(Embodiment)
1 is an explanatory diagram for explaining an overview of an audio signal processing device 1 according to an embodiment. The audio signal processing device 1 generates a time series of sound in which the influence of noise has been reduced, using a time series indicating a sound (hereinafter referred to as a "processing target sound") recorded when a target sound (hereinafter referred to as a "target sound") is recorded in monaural. Hereinafter, the time series indicating the processing target sound is referred to as the processing target sound time series. Monaural recording means recording with one microphone.

Noise is any sound other than the target sound. Noise can be any sound other than the target sound. Noise includes not only random sounds, but also sounds other than the target sound, such as classical music playing from a speaker. Reducing the effect of sound specifically means reducing the amplitude of the sound.

Specifically, the audio signal processing device 1 uses a reference time series to generate a time series indicating a sound in which the influence of the sound to be removed has been suppressed from the sound to be processed time series (hereinafter referred to as the "target sound time series"). The sound to be removed is a sound other than the target sound among sounds included in the sound to be processed, and is a sound to be removed from the sound to be processed. The reference time series is a time series indicating the reference sound. The reference sound is a sound to be removed that has been pre-recorded on a sound recording medium such as a compact disc. Since music data in a specified format such as MP3 (MPEG-1 Audio Layer-3) indicates pre-recorded sound, the reference sound may be the sound to be removed that has been pre-recorded in the form of music data.

The sound to be processed is, for example, a mono recorded conversation between two speakers while a classical piece of music recorded on a compact disc is played from the speakers. In this case, the target sound is the conversation between the two speakers, the sound to be removed is the sound of the classical music played from the speakers, and the reference sound is the sound recorded on the compact disc.

The sound time series to be processed is a signal in which an analog signal representing the sound to be processed has been discretized at a predetermined sampling frequency. Therefore, in FIG. 1, x(n) represents the amplitude x of the nth sample in the sound time series to be processed.

The reference time series is a signal in which an analog signal representing a reference sound is discretized at a predetermined sampling frequency. Therefore, in FIG. 1, o(n) represents the amplitude o of the nth sample of the reference time series.

The sound time series to be processed and the reference time series are not necessarily discretized at the same sampling frequency. Rather, the sampling frequencies of the sound time series to be processed and the reference time series are generally not the same. This is because, even if the sampling frequency set during recording is the same, the sampling frequency deviates from the set value due to environmental changes in the performance of the hardware used for recording, such as transducers. In this way, the sound time series to be processed and the reference time series do not necessarily satisfy the sampling frequency matching condition. The sampling frequency matching condition is a condition in which the sampling frequency of the sound time series to be processed and the sampling frequency of the reference time series are the same.

In addition, the sound to be removed does not necessarily originate from the timing when recording of the sound to be processed started. For this reason, the position where the sound to be removed starts in the sound to be removed time series (hereinafter referred to as the "start position of the sound to be removed") does not necessarily coincide with the time origin of the sound to be processed time series (i.e. n = 0). Therefore, the sound to be processed time series and the reference time series do not necessarily satisfy the origin coincidence condition. The origin coincidence condition is a condition in which the start position of the sound to be removed and the time origin of the reference time series coincide with each other.

A more detailed flow of the processing executed by the acoustic signal processing device 1 will be described. The acoustic signal processing device 1 includes a compensated spectrogram acquisition unit 110, a target complex spectrogram estimation unit 120, and a target sound time series generation unit 130.

The compensated spectrogram acquisition unit 110 acquires a processing target sound time series and a reference time series, and acquires a processing target spectrogram and a reference spectrogram based on the processing target sound time series and the reference time series. The processing target spectrogram is a time series of complex spectra of the processing target sound time series that satisfies the sampling frequency matching condition and the origin matching condition. The reference spectrogram is a time series of complex spectra of the reference time series that satisfies the sampling frequency matching condition and the origin matching condition.

The compensated spectrogram acquisition unit 110 acquires the target spectrogram and the reference spectrogram by performing, for example, a time origin compensation process, an amplitude-frequency conversion process, and a sampling frequency compensation process.

Time origin compensation processing is processing that matches the start position of the sound to be removed with the time origin of the reference time series. In time origin compensation processing, for example, the time of the time origin of the reference time series is shifted so as to maximize the cross-correlation between the sound time series to be processed and the reference time series, thereby matching the time origin of the reference time series with the start position of the sound to be removed (see Reference 1). In such a case, the sound time series to be processed does not change, but the reference time series changes. By executing the time origin compensation processing, the origin matching condition is satisfied.

Reference 1: JP 2014-174393 A

The amplitude frequency conversion process includes a first sub-amplitude frequency conversion process and a second sub-amplitude frequency conversion process. The first sub-amplitude frequency conversion process is a process for acquiring a complex spectrum for each first division period of a plurality of first division periods for a processing target sound time series that satisfies the origin coincidence condition. The first division period is a continuous part of the entire period of the processing target sound time series. One first division period does not include other first division periods, and the union of all the first division periods is equal to the entire period of the processing target sound time series. The process for acquiring a complex spectrum for each first division period of a plurality of first division periods is, for example, a short-time Fourier transform. By executing the first sub-amplitude frequency conversion process, the processing target sound time series that satisfies the origin coincidence condition is converted into a time series of complex spectra of the processing target sound time series that satisfies the origin coincidence condition (hereinafter referred to as the "first spectrogram").

The second sub-amplitude frequency conversion process is a process of acquiring a complex spectrum for each second segment period of a plurality of second segment periods for a reference time series that satisfies the time origin coincidence condition. The second segment period is a continuous portion of the entire period of the reference time series. One second segment period does not include other second segment periods, and the union of all the second segment periods is equal to the entire period of the reference time series. The process of acquiring a complex spectrum for each second segment period of a plurality of second segment periods is, for example, a short-time Fourier transform. By executing the second sub-amplitude frequency conversion process, the reference time series that satisfies the origin coincidence condition is converted into a time series of complex spectra of the reference time series that satisfies the origin coincidence condition (hereinafter referred to as the "second spectrogram").

The sampling frequency compensation process is a process of obtaining a target spectrogram and a reference spectrogram based on a first spectrogram and a second spectrogram. The sampling frequency compensation process is, for example, a process of applying blind synchronization (a blind compensation audio signal processing method described in Reference 1) to the first spectrogram and the second spectrogram. In blind synchronization, the sampling frequency of the reference time series changes, while the sampling frequency of the target sound time series does not change.

In this way, by performing the time origin compensation process, the amplitude frequency conversion process, and the sampling frequency compensation process in this order on the pair of the sound time series to be processed and the reference time series, the spectrogram to be processed and the reference spectrogram are obtained.

The symbol in the following equation (1) in Figure 1 represents the spectrogram to be processed.

The symbols in equation (1) represent the amplitude and phase of the frequency component of frequency ω in the complex spectrum represented by the mth time frame of the spectrogram to be processed (m is an integer equal to or greater than 1).

The symbol in the following equation (2) in Figure 1 represents the reference spectrogram.

The symbols in equation (2) represent the amplitude and phase of the frequency component of frequency ω in the complex spectrum represented by the m-th time frame of the reference spectrogram. The symbol ε ₀ in equation (2) represents the difference in sampling frequency between the sound time series to be processed and the reference time series.

The target complex spectrogram estimation unit 120 estimates a time series of target complex spectra that maximizes the likelihood for a predetermined probability distribution (hereinafter referred to as the "reference probability distribution") based on the processing target spectrogram and the reference spectrogram. The target complex spectrum is a complex spectrum obtained as the difference between the complex spectra at the same time of the processing target spectrogram and the reference spectrogram multiplied by the frequency transfer function. Hereinafter, the time series of target complex spectra is referred to as the target complex spectrogram.

The target complex spectrum represented by the m-th time frame of the target complex spectrogram is expressed, for example, by the following formula (3). The symbol on the left side of formula (3) represents the target complex spectrum.

H(ω) on the right hand side of equation (3) represents the frequency transfer function (i.e., a function that represents the frequency response of the playback/recording environment of the reference sound).

The reference probability distribution is expressed, for example, by the following formula (4). Formula (4) is a zero-mean generalized complex normal distribution.

α and β are auxiliary variables that determine the shape of the reference probability distribution and are predetermined values. In particular, α represents the variance. β is an integer equal to or greater than 1. When β = 2, equation (4) represents a normal distribution, and when β = 1, equation (4) represents a Laplace distribution. In equation (4), Γ represents the Gamma function.

Hereinafter, the distribution of the occurrence probability of the target complex spectrum in the target complex spectrogram is referred to as the target complex spectrum occurrence probability distribution. Maximizing the likelihood for a predetermined probability distribution means maximizing the log-likelihood function of the frequency transfer function expressed by, for example, the following equation (5) when the reference probability distribution is expressed by, for example, equation (4).

The left side of equation (5) is the logarithmic likelihood function of the frequency transfer function. The second term on the right side of equation (5) represents a predetermined constant. Increasing the left side of equation (5) means decreasing the magnitude of the negative of the first term on the right side. Reducing the negative of the first term on the right side of equation (5) means decreasing the average of the β power of the absolute value of S(ω, m). Since S(ω, m) becomes smaller as the removal target sound is removed from the processing target sound, the smaller the first term on the right side of equation (5) is, the more the time series of the processing target sound in which the influence of the removal target sound is suppressed is obtained. Therefore, obtaining a target complex spectrum that maximizes the logarithmic likelihood function of the frequency transfer function represented by equation (5) means obtaining a time series of the processing target sound in which the influence of the removal target sound is suppressed to the greatest extent possible.

The target sound time series generation unit 130 generates a time series of a sound having the target complex spectrogram estimated by the target complex spectrogram estimation unit 120 as a target sound time series based on the target complex spectrogram estimated by the target complex spectrogram estimation unit 120.

The target sound time series is a time series generated using the processing target sound time series, and is a time series in which the components of the reference time series are suppressed from the processing target sound time series. Since the reference sound is a sound to be removed that has been recorded in advance, the target sound time series is a sound time series in which the components of the sound to be removed are suppressed from the processing target sound.

FIG. 2 is a diagram showing an example of the functional configuration of the acoustic signal processing device 1 of the embodiment. The acoustic signal processing device 1 includes a control unit 10 including a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected by a bus, and executes a program. The acoustic signal processing device 1 functions as a device including the control unit 10, input unit 11, communication unit 12, storage unit 13, and output unit 14 by executing the program. More specifically, the processor 91 reads out a program stored in the storage unit 13, and stores the read out program in the memory 92. The processor 91 executes the program stored in the memory 92, and the acoustic signal processing device 1 functions as a device including the control unit 10, input unit 11, communication unit 12, storage unit 13, and output unit 14.

The control unit 10 controls the operation of various functional units of the audio signal processing device 1. The control unit 10 generates a target sound time series using, for example, the processing target sound time series and the reference time series.

The input unit 11 includes input devices such as a mouse, a keyboard, and a touch panel. The input unit 11 may also include an interface that connects these input devices to the device itself. The input unit 11 accepts input of various information for the device itself.

The communication unit 12 includes a communication interface for connecting the device to an external device. The communication unit 12 communicates with the connected external device via wired or wireless communication. The communication unit 12 acquires, for example, a reference time series from the external device. The communication unit 12 acquires, for example, a processing target sound time series from the external device. The external device is, for example, a computer that plays back music data of the reference sound and transmits data of the time series being played back to the audio signal processing device 1. The external device is, for example, a computer that transmits the processing target sound time series. The external device is, for example, a microphone that records the processing target sound time series in mono.

The storage unit 13 is configured using a non-transitory computer-readable storage medium device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 13 stores various information related to the acoustic signal processing device 1. The storage unit 13 stores, for example, information indicating a reference probability distribution in advance. The storage unit 13 stores, for example, a frequency transfer function in advance.

The output unit 14 outputs various information. The output unit 14 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 14 may be configured as an interface that connects these display devices to the device itself. The output unit 14 outputs information that is input to the input unit 11, for example. The output unit 14 may include a sound output device such as a speaker. The output unit 14 may be configured as an interface that connects these sound output devices to the device itself.

Figure 3 is a diagram showing an example of the functional configuration of the control unit 10 in the embodiment. The control unit 10 includes a compensated spectrogram acquisition unit 110, a target complex spectrogram estimation unit 120, a target sound time series generation unit 130, a processing target sound time series acquisition unit 140, a reference time series acquisition unit 150, a communication control unit 160, an output control unit 170, and a recording unit 180.

The processing target sound time series acquisition unit 140 acquires the processing target sound time series input to the communication unit 12. When the processing target sound time series has been stored in advance in the storage unit 13, the processing target sound time series acquisition unit 140 may acquire the processing target sound time series by reading the processing target sound time series from the storage unit 13.

The reference time series acquisition unit 150 acquires the reference time series input to the communication unit 12. If the reference time series has been previously stored in the storage unit 13, the reference time series acquisition unit 150 may acquire the reference time series by reading the reference time series from the storage unit 13.

The communication control unit 160 controls the operation of the communication unit 12. The output control unit 170 controls the operation of the output unit 14. The recording unit 180 records information in the memory unit 13.

The compensated spectrogram acquisition unit 110 includes a time origin compensation unit 111, an amplitude frequency conversion unit 112, and a sampling frequency compensation unit 113.

The time origin compensation unit 111 performs a time origin compensation process on the processing target sound time series acquired by the processing target sound time series acquisition unit 140 and the reference time series acquired by the reference time series acquisition unit 150.

The amplitude frequency conversion unit 112 performs an amplitude frequency conversion process on the processing target sound time series that satisfies the origin matching condition and the reference time series that satisfies the origin matching condition. By performing the amplitude frequency conversion process, the processing target sound time series that satisfies the origin matching condition is converted into a first spectrogram, and the reference time series that satisfies the origin matching condition is converted into a second spectrogram.

The sampling frequency compensation unit 113 executes a sampling frequency compensation process. By executing the sampling frequency compensation process, a processing target spectrogram and a reference spectrogram are generated based on the first spectrogram and the second spectrogram.

Figure 4 is a flowchart showing an example of the flow of processing executed by the acoustic signal processing device 1 of the embodiment. A processing target sound time series is input to the communication unit 12, and the processing target sound time series acquisition unit 140 acquires the input processing target sound time series (step S101). Next, a reference time series is input to the communication unit 12, and the input reference time series is acquired by the reference time series acquisition unit 150 (step S102). Next, the compensated spectrogram acquisition unit 110 acquires the processing target sound time series and the reference time series, and acquires a processing target spectrogram and a reference spectrogram based on the processing target sound time series and the reference time series (step S103).

In step S103, the compensated spectrogram acquisition unit 110 acquires a processed spectrogram and a reference spectrogram based on the processed sound time series and the reference time series, for example by executing the following compensated spectrogram acquisition process. In the compensated spectrogram acquisition process, the time origin compensation unit 111 first executes a time origin compensation process on the processed sound time series and the reference time series to match the start position of the processed sound time series with the time origin of the reference time series. In this way, the time origin compensation unit 111 acquires the processed sound time series that satisfies the origin matching condition and the reference time series that satisfies the origin matching condition.

In the compensated spectrogram acquisition process, the amplitude frequency conversion unit 112 then performs an amplitude frequency conversion process on the processing target sound time series that satisfies the origin matching condition and the reference time series that satisfies the origin matching condition. By performing the amplitude frequency conversion process, the processing target sound time series that satisfies the origin matching condition is converted into a first spectrogram, and the reference time series that satisfies the origin matching condition is converted into a second spectrogram.

In the compensated spectrogram acquisition process, the sampling frequency compensation unit 113 then executes the sampling frequency compensation process. By executing the sampling frequency compensation process, a processing target spectrogram and a reference spectrogram are generated based on the first spectrogram and the second spectrogram.

After step S103, the target complex spectrogram estimation unit 120 estimates a target complex spectrogram that maximizes the likelihood with respect to the reference probability distribution based on the target spectrogram and the reference spectrogram (step S104).

Next, the target sound time series generation unit 130 generates a time series of a sound having the target complex spectrogram estimated by the target complex spectrogram estimation unit 120 as a target sound time series based on the target complex spectrogram estimated by the target complex spectrogram estimation unit 120 (step S105).

Next, the output control unit 170 controls the operation of the output unit 14 to output the sound indicated by the target sound time series from the output unit 14 (step S106). Note that the target sound time series generated in step S105 may be recorded in the memory unit 13 by the recording unit 180 after step S105.

<Experimental Results>
An example of the results of an experiment (hereinafter referred to as an "evaluation experiment") in which a time series of sound in which the influence of noise is reduced is generated using the acoustic signal processing device 1 of the embodiment will be shown.

Figure 5 is an explanatory diagram illustrating the experimental environment of the evaluation experiment in the embodiment. The target sound and the reference sound for the evaluation experiment were recorded by the microphone 903 in a room measuring 4.1 x 3.8 x 2.8 cubic meters and equipped with two speakers 901 and 902 and a microphone 903 (hereinafter referred to as the "experimental room"). The sound absorption coefficient of the experimental room was 0.2. The speaker 901 was a sound source that output (sound) the target sound, and the speaker 902 was a sound source that output (sound) the reference sound.

Speaker 902 was installed 100 cm from one of the walls of the experimental room (hereinafter referred to as the "vertical reference wall"), and speaker 901 was installed 140 cm from the vertical reference wall. The distance between speaker 901 and speaker 902 in the direction perpendicular to the vertical reference wall was 40 cm. Microphone 903 was installed 120 cm from the vertical reference wall.

The distance from the horizontal reference wall to speakers 901 and 902 was 120 cm. The horizontal reference wall is one of the walls of the laboratory and is perpendicular to the vertical reference wall. The distance from the horizontal reference wall to microphone 903 was 290 cm. In other words, microphone 903 was installed at a position 120 cm from the wall of the laboratory that faces the horizontal reference wall.

In the laboratory, recording was performed with only speaker 901 operating and only the target sound being output by microphone 903 (hereinafter referred to as "target sound recording"). The sampling frequency setting value of microphone 903 during target sound recording was 16000 Hz. Also in the laboratory, at a timing separate from the target sound recording, recording was performed with only speaker 902 operating and only the reference sound being output by microphone 903 (hereinafter referred to as "reference sound recording"). The sampling frequency setting value of microphone 903 during reference sound recording was (16000 + 1) Hz. In the evaluation experiment, a sampling frequency mismatch was simulated by shifting the sampling frequency during target sound recording from the sampling frequency during reference sound recording by 1.

In the evaluation experiment, the sound recorded by the target sound recording and the sound recorded by the reference sound recording were mixed so that the input SNR (Signal to Noise Ratio) was -5, 0, 5, and 10 decibels, and used as the sound to be processed in the evaluation experiment. Note that "mixed so that the input SNR is -5, 0, 5, and 10 decibels" means that the input SNR is changed to four conditions: -5 decibels, 0 decibels, 5 decibels, and 10 decibels.

In the evaluation experiment, the short-time Fourier transform was used for the amplitude-frequency conversion process. The short-time Fourier transform in the evaluation experiment had a window length of 4096 points, and was performed at 8192 points under the condition that the shift length was 1/2 the window length and zero-padded. The Hamming window was used as the window for the short-time Fourier transform.

In the evaluation experiments, instead of using a time series of sounds output from speakers 901 and 902 and recorded by microphone 903, an environment similar to that of a laboratory, including input SNR, was modeled, and the acoustic signal processing device 1 was evaluated using a time series generated by a numerical simulation such as the finite element method. Hereinafter, among the evaluation experiments, an experiment in which the acoustic signal processing device 1 is evaluated using a time series of sounds output from speakers 901 and 902 and recorded by microphone 903 is referred to as an actual experiment. Hereinafter, among the evaluation experiments, an experiment in which the acoustic signal processing device 1 is evaluated using a time series generated by a numerical simulation is referred to as a simulation experiment.

FIG. 6 is a diagram showing an example of the results of an evaluation experiment in the embodiment. Results R1 to R3 each show an example of the experimental result of a simulation experiment. Results R4 to R6 each show an example of the experimental result of an actual experiment. The horizontal axis of results R1 to R6 represents the input SNR, and the vertical axis represents the output SNR. The output SNR was defined as (10×log ₁₀ (σ _q ² /σ _o ² )) decibels. _{σ q} ² is the variance of the target sound time series and represents the variance of the target sound time series in the period in which the sound to be removed was present. σ _o ² represents the variance of the reference sound recorded by microphone 903.

The bar graphs in results R1 to R6 show the results when β = 0.2, 0.4, ..., 2.0 (from left to right) in each graph. Also, the solid areas (Sync.(off)) show the results when blind synchronization was not performed, and the unshaded areas (Sync.(on)) show the results when blind synchronization was performed.

Results R1 to R6 show that the output SNR is improved by compensating for the deviation in the sampling frequency. One reason for this is that when there is a deviation in the sampling frequency, the frequency response (frequency transfer function) appears to be time-invariant, and the model expressed by equation (3) does not hold. Another reason is that β at which the output SNR is maximized varies depending on the type of sound to be processed and the input SNR. Thus, the acoustic signal processing device 1 is a more flexible model than one that assumes that the probability distribution of the target complex spectrum is a normal distribution or a Laplace distribution, and can obtain a higher output SNR.

<Method of obtaining a target complex spectrogram using equation (5)>
Here, an example of a method for estimating a target complex spectrogram using formula (5) will be described. The target complex spectrogram is estimated by differentiating formula (5) with respect to H(ω) and obtaining S(ω, m) that gives a maximum value as an estimation result. However, the value of the differential of the power of the absolute value cannot be analytically obtained.

Of course, since it is simply not possible to obtain an analytical solution, the audio signal processing device 1 may estimate the target complex spectrogram using a more basic method of treating equation (5) as is in the numerical calculations to obtain an approximate value. However, in such a case, the amount of calculation is large, and rounding errors and the like are likely to occur, making it highly likely that the calculation error will be large. Therefore, instead of performing numerical calculations on equation (5) as is, the audio signal processing device 1 may estimate the target complex spectrogram by transforming equation (5) into an equivalent equation that requires less calculation.

The acoustic signal processing device 1 may obtain the target complex spectrogram by applying, for example, the following auxiliary function as a method for transforming equation (5) into an equivalent equation with less computational complexity to estimate the target complex spectrogram. The auxiliary function method is also called the Majorization-Minimization Algorithm or MM Algorithm (see References 2 and 3).

Reference 2: David R Hunter & Kenneth Lange “A Tutorial on MM Algorithms”, The American Statistician, 58:1, 30-37.
Reference 3: Ono, "Optimization algorithm using auxiliary function method and its application to acoustic signal processing," Journal of the Acoustical Society of Japan, Vol. 68, No. 11, pp. 566-571, 2012.

In the auxiliary function method, it is necessary to find an appropriate auxiliary function for the objective function. Here, the objective function is a function to be maximized or minimized, given for each optimization problem. An appropriate auxiliary function in the auxiliary function method can be found, for example, by using the method described in Reference 4.

Reference 4: Nobutaka Ono and Shigeki Miyabe, “Auxiliary-Function-Based Independent
Component Analysis for Super-Gaussian Sources”, V. Vigneron et al. (Eds.): LVA/ICA 2010, LNCS 6365, pp. 165-172, 2010, Springer-Verlag Berlin Heidelberg 2010

Now, let us consider the mathematical properties of function G(x), which is a continuous, differentiable function expressed in terms of amplitude x and is an even function of amplitude x. Here, we will treat the amplitude x as a variable x that can have negative values and explain the mathematical properties of even function G(x). If a function that divides the derivative of G(x) with respect to variable x by variable x is continuous within the domain, positive for x>0, and monotonically decreasing, then the following equation (6) holds for any x.

The equality condition in equation (6) is expressed as equation (7) below.

The first term of equation (5) satisfies the condition of equation (7). Therefore, an appropriate auxiliary function Q in the auxiliary function method expressed by the following equation (8) is expressed as the following equation (9).

H ₀ (ω) is an auxiliary variable. The second term on the right side of formula (9) is a term that depends only on the auxiliary variable H ₀ (ω). Therefore, it is a term that is not related to the optimization of H(ω). Formula (9) is a quadratic function of H(ω). The formula obtained by differentiating the auxiliary function Q with respect to H(ω) is set to 0, and the following update formula (11) is obtained by substituting H ₀ (ω) = H(ω) ^(k) and transforming the formula. k represents the number of iterations.

In equation (11), the asterisk symbol indicates a complex conjugate. The target complex spectrogram is estimated by substituting the frequency transfer function estimated in equation (11) into equation (3). The auxiliary function application method is a method of estimating the target complex spectrogram by repeatedly executing such estimation using equations (811) and (3) until a predetermined termination condition is satisfied. The termination condition is, for example, that the process has been repeated a predetermined number of times. The initial value of the repetitions may be any condition as long as the denominator of equation (11) is not set to 0. For example, the initial value of H(ω) may be 1, and the denominator of equation (11) may not be set to 0.

The acoustic signal processing device 1 of the embodiment configured in this manner estimates a target complex spectrogram that minimizes the difference between the distribution of occurrence probability of the target complex spectrum in the target complex spectrogram and the reference probability distribution, based on the processing target spectrogram and the reference spectrogram. The target sound time series is a time series generated using the processing target sound time series, in which the components of the reference time series are suppressed from the processing target sound time series. The reference sound is a sound to be removed that has been recorded in advance. Therefore, the acoustic signal processing device 1 can generate a target sound time series in which the components of the sound to be removed are suppressed from the processing target sound. In other words, the acoustic signal processing device 1 can generate a time series of sound in which the influence of noise is further reduced from a time series of sound recorded in mono.

(Modification)
The acoustic signal processing device 1 may be implemented using a plurality of information processing devices communicably connected via a network. In this case, each functional unit of the acoustic signal processing device 1 may be distributed and implemented in the plurality of information processing devices.

All or part of the functions of the audio signal processing device 1 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

The above describes an embodiment of the present invention in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs that do not deviate from the gist of the present invention.

1...acoustic signal processing device, 10...control unit, 11...input unit, 12...communication unit, 13...storage unit, 14...output unit, 110...compensated spectrogram acquisition unit, 111...time origin compensation unit, 112...amplitude frequency conversion unit, 113...sampling frequency compensation unit, 120...target complex spectrogram estimation unit, 130...target sound time series generation unit, 140...processing target sound time series acquisition unit, 150...reference time series acquisition unit, 160...communication control unit, 170...output control unit, 180...recording unit, 91...processor, 92...memory

Claims (5)

a compensated spectrogram acquisition unit that acquires a processed spectrogram, which is a time series of complex spectra of the processing target sound time series that satisfies an origin coincidence condition that a start position of a processing target sound time series that indicates a processing target sound recorded in mono is coincident with a time origin of a reference time series that indicates the processing target sound that has been recorded in advance, and a sampling frequency coincidence condition that a sampling frequency of the processing target sound time series is coincident with a sampling frequency of the reference time series, and a reference spectrogram, which is a time series of complex spectra of the reference time series that satisfies the origin coincidence condition and the sampling frequency coincidence condition;
a target complex spectrogram estimating unit that estimates a time series that maximizes a likelihood of a predetermined probability distribution of a target complex spectrum, the target complex spectrum being a complex spectrum in a target complex spectrogram obtained as a difference between a complex spectrum at the same time between the processing target spectrogram and the reference spectrogram multiplied by a frequency transfer function;
An acoustic signal processing device comprising: a target sound time series generation unit that generates a time series of a sound having the target complex spectrogram estimated by the target complex spectrogram estimation unit;
The audio signal processing apparatus according to claim 1 , the compensated spectrogram acquisition unit acquires a first spectrogram, which is a time series of complex spectra of the processing target sound time series that satisfies an origin coincidence condition, and a second spectrogram, which is a time series of complex spectra of the reference time series that satisfies an origin coincidence condition, after matching a start position of the processing target sound time series with a time origin of the reference time series, and acquires the processing target spectrogram and the reference spectrogram using the first spectrogram and the second spectrogram.
3. The audio signal processing device according to claim 1 or 2. a compensated spectrogram acquisition step of acquiring a processed spectrogram, which is a time series of complex spectra of the processing target sound time series that satisfies an origin coincidence condition that a start position of the processing target sound in a processing target sound time series that indicates a mono-recorded processing target sound coincides with a time origin of a reference time series that indicates the processing target sound that has been recorded in advance, and a sampling frequency coincidence condition that a sampling frequency of the processing target sound time series coincides with a sampling frequency of the reference time series, and a reference spectrogram, which is a time series of complex spectra of the reference time series that satisfies the origin coincidence condition and the sampling frequency coincidence condition;
a target complex spectrogram estimating step of estimating a time series that maximizes the likelihood of a target complex spectrum, which is a complex spectrum in a target complex spectrogram obtained as a difference between complex spectra at the same time between the processing target spectrogram and the reference spectrogram multiplied by a frequency transfer function, for a predetermined probability distribution;
An acoustic signal processing method comprising the steps of: A program for causing a computer to function as an audio signal processing device according to any one of claims 1 to 3.

Images