JP5744992B2 - Audio signal output device, audio signal output method, and audio signal output program - Google Patents

Description

  The present invention relates to an apparatus for outputting an audio signal, a method for outputting an audio signal, and an audio signal output program.

  Currently, the use of IP telephones and Internet telephones using VoIP (Voice over Internet Protocol) technology for voice communication over IP networks is increasing. VoIP enables voice communication by packetizing voice data and sending / receiving it via an IP network.

  In Patent Document 1, in a signal of a plurality of channels, a signal of a channel other than the signal of the deteriorated channel is sound (not deteriorated), and the signal of the sound channel and the channel of the deteriorated channel are assumed. A method for compensating a signal of a deteriorated channel using a correlation with a signal is disclosed.

  In Non-Patent Document 1, in a single channel signal, when a packet is lost at a certain point in time (that is, the quality is deteriorated), a pitch is determined from a sound signal at a point before that point (quality deterioration point). A method is disclosed in which a pitch waveform is extracted by detecting a period, and the pitch waveform is repeatedly used as a compensation signal at the time of quality degradation.

Patent No. 3559454

ITU-T Reccomendation G.711 Appendix I, “Ahigh quality low-complexity algorithm for packet loss concealment with G.711” C. Parkins, et al., “A Survey of Packet Loss Recovery Techniques for Streaming Audio,” IEEE Network, Sept./Oct. 1998 ISO / IEC 14496-3: 2001 / Amd.2.2004: “ParametricCoding for High Quality Audio” 3GPP TS 26.405 “General audio codec audioprocessing functions; Enhanced aacPlus general audio codec; Encoderspecification; Parametric stereo part” ISO / IEC 23003-1: 2007 “MPEG Surround” F. Baumgarte, et al., “Binaural Cue Coding-Part I: Psychoacoustic Fundamentals and Design Principles,” IEEE Trans. On Speech ahd Audio Proc., Vol. 11, No. 6, Nov.2003 C. Faller, et al., “Binaural Cue Coding-Part II: Schemes and Applications,” IEEETrans. On Speech ahd Audio Proc., Vol. 11, No. 6, Nov. 2003 ISO / IEC 14496-3: 2005: “Information technology- Coding of audio-visual objects-Part3: Audio”

  However, the method of Patent Document 1 and the method of Non-Patent Document 1 have the following problems.

  The method of Patent Literature 1 is based on the premise that a signal of a channel other than a deteriorated channel signal is sound in a plurality of channel signals. For this reason, for example, signals of a plurality of channels at the same time are stored in the same packet, and when the quality of the signals of the plurality of channels at the same time deteriorates simultaneously due to loss of the packet or the like, it cannot be handled.

  Further, when the method of Non-Patent Document 1 is applied to a plurality of channel signals, every time the quality deteriorates, pitch detection and pitch waveform extraction are performed on each channel signal, and the quality of the compensation signal (that is, the signal quality of the plurality of channels) It is necessary to generate a signal to compensate. For this reason, the amount of calculation increases as the number of channels increases.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to stably generate a compensation signal while suppressing an increase in the amount of computation accompanying an increase in the number of channels.

The audio signal output device according to the present invention generates a multi-channel audio signal that is an audio signal of two or more channels using multi-channel audio signal related information that is information related to the multi-channel audio signal. Generating means, multi-channel audio signal holding means for holding a multi-channel audio signal generated by the multi-channel audio signal generating means for a predetermined period, and the multi-channel audio signal based on predetermined quality control information. Quality control means for determining whether or not the multi-channel audio signal is deteriorated from the multi-channel audio signal related information when the quality control means determines that the multi-channel audio signal is deteriorated. Correlation detecting means for detecting the correlation between the channels, and based on the detection result by the correlation detecting means. Compensation signal generation method determination means for determining a signal generation method, compensation signal generation means for generating a compensation signal based on the generation method determined by the compensation signal generation method determination means, and generation by the compensation signal generation means And output the compensation signal, and when the quality control means determines that the multi-channel audio signal has not deteriorated, the signal output means outputs the multi-channel audio signal, and the compensation The signal generation method determination means determines a generation method for generating a common compensation signal as a generation method of the compensation signal for two or more channel audio signals whose correlation between detected channels is higher than a predetermined reference. The compensation signal generating means generates the compensation signal from the multi-channel audio signal related information.

The audio signal output device further includes a multi-channel audio signal-related information holding unit that holds the multi-channel audio signal-related information for a predetermined period, and the correlation detection unit includes the multi-channel audio signal-related information. It is desirable to detect the correlation between the channels of the multi-channel audio signal from the multi-channel audio signal related information held in the holding means .

  In the audio signal output device, it is preferable that the correlation detecting unit detects a correlation between the channels of the multi-channel audio signal from at least one of the multi-channel audio signal and the multi-channel audio signal related information. .

  The multi-channel audio signal-related information may include at least one mixed audio signal obtained by mixing the multi-channel audio signal by a predetermined method. The multi-channel audio signal related information may include at least one of an intensity difference between channels of the multi-channel audio signal, a phase difference between channels, and a coherence between channels. The multi-channel audio signal related information may include a correlation detection result between the channels of the multi-channel audio signal.

  The invention relating to the above-described audio signal output apparatus can also be regarded as an invention relating to an audio signal output method, and can be described as follows.

An audio signal output method according to the present invention is an audio signal output method executed by an audio signal output device, wherein a multi-channel audio signal that is an audio signal of two or more channels is represented by information on the multi-channel audio signal. A multi-channel audio signal generation step generated using certain multi-channel audio signal related information, a step of holding the multi-channel audio signal generated in the multi-channel audio signal generation step for a predetermined period, and a predetermined quality control Determining whether or not the multi-channel audio signal is degraded based on the information for use, and outputting the multi-channel audio signal when it is determined that the multi-channel audio signal is not degraded when the plurality when channel audio signal is determined to have deteriorated, the plural-channel audio signals associated information From the steps of detecting correlation between channels of the plurality of channels audio signals, on the basis of the detection result of the correlation between channels, determining a method of generating the compensation signal, the determined the generation method And generating a compensation signal, and outputting the generated compensation signal. In the step of determining a method of generating the compensation signal, a correlation between detected channels is a predetermined reference. For the two or more channel audio signals higher than the above, in the step of determining a generation method for generating a common compensation signal as the generation method of the compensation signal, Then, the compensation signal is generated.

  Further, the present invention can be understood as an invention relating to an audio signal output program as follows.

That is, an audio signal output program according to the present invention provides a computer provided in an audio signal output device, a multi-channel audio signal that is an audio signal of two or more channels, Multi-channel audio signal generating means for generating using the audio signal-related information, multi-channel audio signal holding means for holding the multi-channel audio signal generated by the multi-channel audio signal generating means for a predetermined period, and predetermined quality Based on management information, quality management means for determining whether or not the multi-channel audio signal is degraded, and when the quality management means determines that the multi-channel audio signal is degraded, a plurality channel audio signal relationship, the correlation for detecting a correlation between channels of the plurality of channels audio signals And a compensation signal generation method determining means for determining a compensation signal generation method based on a detection result by the correlation detection means, and a compensation signal based on the generation method determined by the compensation signal generation method determination means. A compensation signal generating means for generating the plurality of channels when the compensation signal generated by the compensation signal generating means is output and the quality control means determines that the multi-channel audio signal is not degraded. The compensation signal generation method determining means functions as signal output means for outputting an audio signal, and the compensation signal generation method determining means is configured to detect the compensation signal for two or more channel audio signals whose correlation between detected channels is higher than a predetermined reference. A generation method for generating a common compensation signal is determined as a generation method, and the compensation signal generation means From distribution to generate the compensation signal, characterized in that. That is, as shown in FIG. 15A, the audio signal output program 20A includes a multi-channel audio signal holding module 21 that causes a computer provided in the audio signal output device to function as multi-channel audio signal holding means, and the computer. A quality control module 22 that functions as quality control means, a correlation detection module 23 that causes the computer to function as correlation detection means, a compensation signal generation method determination module 24 that causes the computer to function as compensation signal generation method determination means, and the computer Is provided with a compensation signal generation module 25 that functions as a compensation signal generation means, and a signal output module 26 that causes the computer to function as signal output means.

  As another aspect of the audio signal output program, as shown in FIG. 15B, a multi-channel audio signal related information holding module that causes the computer to function as a multi-channel audio signal related information holding unit is further provided. Also good. Further, as shown in FIG. 15C, in addition to the multi-channel audio signal related information holding module, a multi-channel audio signal generation module that causes the computer to function as a multi-channel audio signal generation unit may be further provided.

  According to the present invention, it is possible to stably generate a compensation signal while suppressing an increase in calculation amount accompanying an increase in the number of channels.

It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 1st Embodiment. It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 2nd Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 3rd Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 3rd Embodiment. It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 4th Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 4th Embodiment. It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 5th Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 5th Embodiment. It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 6th Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 6th Embodiment. It is a block diagram which shows the structure of the audio | voice signal output device which concerns on 7th Embodiment. It is a flowchart which shows operation | movement of the audio | voice signal output device which concerns on 7th Embodiment. It is a figure which shows the structure of an audio | voice signal output program. It is a figure which shows the hardware structural example of an audio | voice signal output device.

  Hereinafter, various embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[First Embodiment]
FIG. 1 shows the configuration of an audio signal output apparatus 1A according to the first embodiment of the present invention.

  As shown in FIG. 1, the audio signal output apparatus 1A includes a multi-channel audio signal holding unit 101 (multi-channel audio signal holding unit), a quality management unit 102 (quality management unit), a correlation detection unit 103 (correlation detection unit), and compensation. It has a signal generation method determination unit 104 (compensation signal generation method determination unit), a compensation signal generation unit 105 (compensation signal generation unit), and a signal output unit 106 (signal output unit). As shown in FIG. 16, for example, as shown in FIG. 16, the audio signal output device 1A has a CPU 11 that executes an operating system, an application program, etc., a main storage unit 12 composed of ROM and RAM, and a non-volatile memory. Auxiliary storage unit 13 composed of, etc., a communication control unit 14 for controlling communication with the outside, a display unit 15 composed of a liquid crystal monitor, etc., and a key for inputting characters and numbers and executing instructions The operation unit 16, an input unit 17 that inputs various information and signals from the outside, and an output unit 18 that outputs various information and signals to the outside can be used. However, the configuration of FIG. 16 is an example and is not limited to this configuration. The hardware configuration of the audio signal output devices of the second to seventh embodiments to be described later is also the same.

  The multi-channel audio signal holding unit 101 holds a multi-channel audio signal for a predetermined period.

  The quality management unit 102 determines whether or not an N-channel (N is an integer greater than 1) multi-channel audio signal has deteriorated based on quality management information described later, and the multi-channel audio signal has deteriorated In addition, the correlation detection unit 103 is notified of the deterioration.

  When the correlation detecting unit 103 receives a notification that the multi-channel audio signal is deteriorated, the correlation detecting unit 103 receives the multi-channel audio signal from the multi-channel audio signal holding unit 101, and the correlation between the channels of the multi-channel audio signal is strong. And the detection result is notified to the compensation signal generation method determination unit 104.

  The compensation signal generation method determination unit 104 determines a compensation signal generation method based on the detection result, and notifies the compensation signal generation unit 105 of the determined compensation signal generation method.

  The compensation signal generation unit 105 receives the multi-channel audio signal from the multi-channel audio signal holding unit 101, generates a compensation signal from the multi-channel audio signal based on the compensation signal generation method, and outputs the generated compensation signal to the signal output unit 106.

  The signal output unit 106 outputs the compensation signal when receiving the compensation signal from the compensation signal generation unit 105 (that is, when the multi-channel audio signal is deteriorated), and receives the compensation signal from the compensation signal generation unit 105. If not (ie, if the multi-channel audio signal is not degraded), the multi-channel audio signal is output.

  FIG. 2 shows a flowchart for explaining the operation of the audio signal output apparatus 1A according to the first embodiment of the present invention. The operation of the audio signal output device 1A will be described with reference to FIG.

  In the audio signal output device 1A, the multi-channel audio signal holding unit 101 holds a multi-channel audio signal (S101). However, if a multi-channel audio signal to be newly held is not input, step S101 is omitted and the process proceeds to step S102.

  Next, the quality management unit 102 determines whether or not the multi-channel audio signal is deteriorated based on quality management information described later. If the multi-channel audio signal is deteriorated, the process proceeds to step S103. If the multi-channel audio signal is not degraded, the process proceeds to step S107 (S102). Note that the quality management information is information related to the quality of the multi-channel audio signal, for example, information indicating that the packet storing the multi-channel audio signal is lost, or information indicating that the arrival of the packet is not in time by the reproduction time. And information indicating that part of the information of the multi-channel audio signal is incorrect. However, the content of the quality control information is not limited to the above example.

Rijを予め決められた値Rthと比較し、RijがRthよりも大きければ相関が強いとする方法である。別の方法としては、xi(t)とxj(t)の和信号mij(t)と差信号sij(t)を式(２)、式(３)で算出する。

さらにt=tl,…,tl+T-1の期間のmij(t)とsij(t)のそれぞれの電力Mij、Sijを(式４)、(式５)で算出する。

上記の方法は、MijとSijの差Mij-Sijを予め決められた値Dthと比較し、Mij-SijがDthよりも大きければ、相関が強いとする方法である。さらに、MijとSijの比Mij/Sijを予め決められた値Fthと比較し、Mij/SijがFthよりも大きければ、相関が強いとする方法でもよい。なお、相関検出の方法は、上記の方法と等価な方法でもよく、また別の方法でもよく、相関検出の方法は上記の方法に限定されない。 In step S103, correlation detection section 103 detects the correlation between the channels of the held multi-channel audio signal. Examples of the correlation detection method here include the following methods. If the audio signal of the i-th channel of the multi-channel audio signal is xi (t), the audio signal of the j-th channel is xj (t), and the target period for correlation detection is t = tl, ..., tl + T-1, The cross-correlation value Rij is calculated by equation (1).

In this method, Rij is compared with a predetermined value Rth, and if Rij is larger than Rth, the correlation is strong. As another method, the sum signal mij (t) and the difference signal sij (t) of xi (t) and xj (t) are calculated by Equations (2) and (3).

Further, the powers Mij and Sij of mij (t) and sij (t) during the period of t = tl,..., Tl + T−1 are calculated by (Expression 4) and (Expression 5).

In the above method, the difference Mij-Sij between Mij and Sij is compared with a predetermined value Dth, and if Mij-Sij is larger than Dth, the correlation is strong. Further, the ratio Mij / Sij between Mij and Sij is compared with a predetermined value Fth, and if Mij / Sij is larger than Fth, the correlation may be strong. The correlation detection method may be a method equivalent to the above method or may be another method, and the correlation detection method is not limited to the above method.

  Next, the compensation signal generation method determination unit 104 determines a compensation signal generation method based on the correlation detection result (S104). As a compensation signal generation method here, there are, for example, the following three methods.

  As a first example of the compensation signal generation method, there is a method of generating a compensation signal for each channel audio signal of a multi-channel audio signal. According to this method, a compensation signal can be generated for each channel audio signal of the multi-channel audio signal.

  Further, as a second example of the compensation signal generation method, there is a method of generating a common compensation signal for two or more channel audio signals of a plurality of channel audio signals. According to this method, a common compensation signal can be generated without separately generating a compensation signal for two or more channel sound signals having a strong correlation with each other among a plurality of channel sound signals. Therefore, the compensation signal can be generated more efficiently than the first example described above. Since the audio signal of the target channel has a strong correlation between channels, the quality does not deteriorate even if a common compensation signal is used.

  As a third example of the compensation signal generation method, there is a method of adjusting one or more of amplitude and phase after generating a compensation signal according to the second example. According to this method, as in the second example, a compensation signal can be efficiently generated for two or more channel sound signals having a strong correlation with each other among the plurality of channel sound signals. Furthermore, the quality of the compensation signal of each channel can be improved by adjusting one or more of the amplitude and phase. In this way, it is possible to improve the quality of the compensation signal for each channel while suppressing an increase in the amount of computation.

  Further, as a fourth example of the compensation signal generation method, after generating a one-channel compensation signal according to the second example, a non-correlated signal obtained by decorrelating the one-channel compensation signal is generated, and the compensation signal and Compensation signals for two or more channel audio signals may be generated by up-mixing the uncorrelated signals by matrix calculation. Non-patent document 3 and non-patent document 5 describe the above-mentioned decorrelation of a one-channel audio signal and up-mixing by matrix calculation.

  Note that the compensation signal generation method may be any of the above methods, and may employ a different method among the above methods for each channel of the multi-channel audio signal. Further, methods other than those described above may be employed as the compensation signal generation method, and the compensation signal generation method employed is not limited.

  Next, the compensation signal generation unit 105 generates a compensation signal from the held multi-channel audio signal according to the compensation signal generation method determined by the compensation signal generation method determination unit 104 (S105). Here, as a one-channel compensation signal generation algorithm, for example, there is one described in Non-Patent Document 1 or Non-Patent Document 2.

  The compensation signal in the first example of the compensation signal generation method described above can be generated by applying the algorithm to the audio signal of each channel.

  The compensation signal in the second example of the compensation signal generation method described above generates a one-channel representative signal from the target channel audio signal for generating a common compensation signal, and uses the representative signal to compensate the one-channel compensation signal. It can be generated by a signal generation algorithm.

  As a representative signal generation method here, for example, a sum signal of a target channel audio signal may be generated. When the number of channels to be processed is 2, the sum signal can be calculated by the above equation 2, and when the number of channels to be processed is larger than 2, it can be calculated similarly. Furthermore, a so-called downmixing signal may be used, which is a signal obtained by adding the weights of the audio signals of the target channels to each channel after adding weights. Furthermore, it is possible to select an audio signal of one channel from the audio signals of the target channel and use the audio signal of the channel as a representative signal. As a method for selecting a one-channel audio signal, for example, the one having the largest signal power may be selected. Furthermore, when the target channel is 3 or more, the channel having the strongest correlation with other channels may be selected. For example, consider a case where the target channel is 3, and the i-th channel audio signal, the j-th channel audio signal, and the k-th channel audio signal are targets. The j-th channel audio signal has a strong correlation with the i-th channel audio signal and the k-th channel audio signal, and the correlation between the i-th channel audio signal and the k-th channel audio signal is relatively weak. In this case, the j-th channel audio signal is selected as the representative signal. When the target channel is larger than 3, the same selection can be made. As another method of selecting the audio signal of one channel, when the audio signal of the target channel is related to the arrangement position of the speaker to be reproduced, the selection may be made from the speaker arrangement position. For example, consider a case where the target channel is 3, and the i-th channel audio signal, the j-th channel audio signal, and the k-th channel audio signal are targets. When the i-th channel audio signal is associated with the left channel speaker L, the j-th channel audio signal is associated with the center speaker C, and the k-th channel audio signal is associated with the right channel speaker R, the i-channel audio signal is placed between the right channel speaker L and the left channel speaker R. The arranged j-th channel audio signal associated with the central speaker C is selected as the representative signal. When the target channel is larger than 3, the same selection can be made. The method for selecting the representative signal may be the above method or another method, and the method for selecting the representative signal is not limited.

  The compensation signal in the third example of the compensation signal generation method described above adjusts one or more of the amplitude and phase of the compensation signal for the compensation signal generated in the same manner as in the second example of the compensation signal generation method. Can be generated. Amplitude and phase adjustment amounts may be calculated from the held multi-channel audio signals. Furthermore, the adjustment amount of the amplitude or phase may be changed according to one or more of the frequency band and the short time section. The compensation signal generation algorithm may be the above algorithm or another algorithm, and the compensation signal generation algorithm is not limited.

  The compensation signal in the fourth example of the compensation signal generation method described above is generated by generating an uncorrelated signal and upmixing the one-channel compensation signal generated in the same manner as in the second example of the compensation signal generation method. it can. Information (for example, amplitude difference, phase difference, coherence) necessary for generation between target channels may be calculated from the held multi-channel audio signal. Further, the information between the target channels may be changed according to one or more of the frequency band and the short time interval. The compensation signal generation algorithm may be the above algorithm or another algorithm, and the compensation signal generation algorithm is not limited.

  Next, in step S106, the signal output unit 106 outputs the compensation signal generated by the compensation signal generation unit 105.

  On the other hand, in step S107, which proceeds when the multi-channel audio signal is not degraded in step S102, the signal output unit 106 outputs the multi-channel audio signal.

  In S108, the presence / absence of a multi-channel signal to be output is confirmed. If there is a multi-channel audio signal to be output, the process returns to step S101. If there is no multi-channel audio signal to be output, the process of FIG. Exit. However, the processing for instructing the end here is not limited.

  According to the first embodiment described above, when it is determined that the multi-channel audio signal is deteriorated, the correlation between the channels of the multi-channel audio signal is detected, and the compensation signal is detected based on the detection result. A generation method is determined, and a compensation signal is generated and output based on the generation method. Since the compensation signal is generated based on an appropriate generation method determined according to the correlation between the channels as described above, the compensation signal can be stably generated. For two or more channel audio signals, an increase in the amount of computation can be suppressed by, for example, generating a common compensation signal.

[Second Embodiment]
FIG. 3 shows the configuration of an audio signal output apparatus 1B according to the second embodiment of the present invention.

  As shown in FIG. 3, the audio signal output apparatus 1B includes a multi-channel audio signal holding unit 101, a multi-channel audio signal related information holding unit 201 (multi-channel audio signal related information holding unit), a quality management unit 102, and a correlation detection unit 202. A compensation signal generation method determination unit 104, a compensation signal generation unit 105, and a signal output unit 106. Since the configuration of the audio signal output device 1B has many parts in common with the configuration of the audio signal output device 1A according to the first embodiment described above, the following block is different from the configuration of the audio signal output device 1A. Will be described.

  The multi-channel audio signal related information holding unit 201 holds multi-channel audio signal related information described later for a predetermined period. The multi-channel audio signal related information is not the multi-channel audio signal itself, but is information related to the multi-channel audio signal, and its format is not limited. The multi-channel audio signal related information is information that can generate a multi-channel audio signal, for example, (a) a signal that has been operated on the multi-channel audio signal and represented as another signal, and (b) a multi-channel audio signal Information expressed by a signal indicating a small number of channels and a parameter indicating characteristics between channels may be used. Examples of the signal (a) obtained by performing computation on the multi-channel audio signal and expressing it as another signal include MS stereo of Non-Patent Document 8. In addition, as (b) information expressing the multi-channel audio signal with parameters indicating the characteristics between the signal having a smaller number of channels and the channel, parametric stereo, MPEG described in Non-Patent Documents 3 to 7, for example, There are surround and BCC formats. Furthermore, the multi-channel audio signal related information may be a parameter representing the characteristics of the multi-channel audio signal, for example. The multi-channel audio signal related information may include a mixed audio signal of at least one channel obtained by mixing the multi-channel audio signal by a predetermined method. The multi-channel audio signal related information may include at least one of an intensity difference between channels of the multi-channel audio signal, a phase difference between channels, and a coherence between channels. The multi-channel audio signal related information may include a correlation detection result between the channels of the multi-channel audio signal.

  The correlation detection unit 202 detects the correlation between the channels of the multi-channel audio signal in the section corresponding to the multi-channel audio signal related information that is held.

  FIG. 4 shows a flowchart for explaining the operation of the audio signal output apparatus 1B according to the second embodiment of the present invention. The operation of the audio signal output device 1B will be described with reference to FIG. However, since the operation of the audio signal output device 1B has many parts common to the operation of the audio signal output device 1A according to the first embodiment described above, the following steps are different from the operation of the audio signal output device 1A. S201 and step S202 will be described.

  After the multi-channel audio signal holding unit 101 holds the multi-channel audio signal (S101), in step S201, the multi-channel audio signal related information holding unit 201 holds the multi-channel audio signal related information. However, if there is no multi-channel audio signal related information to be newly held, the process may be omitted and the process may proceed to step S102.

  If it is determined in step S102 that the multi-channel audio signal has deteriorated, in step S202, the correlation detection unit 202 determines that the multi-channel audio signal in the corresponding section is based on the held multi-channel audio signal related information. The correlation between each channel is detected. Examples of the correlation detection method using the multi-channel audio signal related information here include the following methods.

  As a first example of the correlation detection method based on the multi-channel audio signal related information, the multi-channel audio signal related information may include a correlation value between the channels of the multi-channel audio signal represented by (Equation 1) described above. is there. In this case, the correlation can be detected by the above method.

  As a second example of the correlation detection method using the multi-channel audio signal related information, the multi-channel audio signal related information may include another signal obtained by calculating the multi-channel audio signal. Here, for example, when the sum signal and the difference signal are obtained by the above-described (Expression 2) and (Expression 3), and the correlation is detected from these signals, the correlation can be detected by the above method.

  As a third example of the correlation detection method based on the multi-channel audio signal related information, the multi-channel audio signal related information may include a parameter representing characteristics between the channels of the multi-channel audio signal. As an example of the parameter, there is Inter-Channel Coherence (ICC) described in Non-Patent Document 3. The ICC of Non-Patent Document 3 is a parameter indicating the similarity of a two-channel audio signal in a certain frequency band. The calculation method of ICC is described in Non-Patent Document 3. As a correlation detection method using ICC, there is a method of comparing ICC with a predetermined value and determining that there is a correlation if ICC is larger than the predetermined value, and determining that there is no correlation if ICC is smaller than the predetermined value. . Further, as described above, since the ICC is a value calculated for each frequency band, it may be determined from the comparison result for each frequency band after performing the above comparison for each frequency band. Moreover, after performing the comparison only for a part of the frequency bands, the determination may be made from the comparison result for the part of the frequency bands. Furthermore, the average value of the ICC of each frequency band may be calculated for the entire frequency band or a part of the frequency band, and the obtained average value may be compared with a predetermined value. Furthermore, it is good also as a weighted average which added the weight for every frequency band in the case of the said average value calculation. However, the correlation detection method using ICC is not limited. Non-Patent Documents 4 to 7 use parameters that are equivalent to or similar to the characteristics of the ICC, and these parameters can also be used for correlation detection as with the ICC.

  As a fourth example of the correlation detection method using the multi-channel audio signal related information, the multi-channel audio signal related information may include a correlation detection result between the channels. The correlation detection result is a result of performing correlation detection between the channels of the multi-channel audio signal outside the audio output device according to the present embodiment. In this case, the correlation detection unit 202 may notify the compensation signal generation method determination unit 104 of the detection result as a correlation detection result between the channels of the multi-channel audio signal. Furthermore, determination of further correlation detection may be performed using the detection result. For example, when the detection result is a detection result for each frequency band, the detection result of all frequency bands may be determined by majority decision to determine the presence or absence of correlation. The voting weight for the band may be changed. However, the correlation detection method using the external correlation detection result is not limited to the above.

  Furthermore, in the first to fourth examples of the correlation detection method using the multi-channel audio signal related information described above, the correlation detection result of the section before the section and the section after the section are used for the correlation detection determination of the correlation detection target section. Both or one of the correlation detection results in the interval may be used. However, the correlation detection method using multi-channel audio signal related information is not limited to the above method.

  As described above, the correlation detection unit 202 can detect the correlation between the channels of the multi-channel audio signal from the multi-channel audio signal-related information by various methods.

[Third Embodiment]
FIG. 5 shows the configuration of an audio signal output device 1C according to the third embodiment of the present invention.

  As shown in FIG. 5, the audio signal output apparatus 1C includes a multi-channel audio signal holding unit 101, a multi-channel audio signal related information holding unit 201, a quality management unit 102, a correlation detection unit 103, a compensation signal generation method determination unit 104, a compensation A signal generation unit 301 and a signal output unit 106 are included. Since the configuration of the audio signal output device 1C has many parts in common with the configurations of the audio signal output devices according to the first and second embodiments described above, the first and second embodiments will be described below. A block (compensation signal generation unit 301) different from the configuration of the audio signal output apparatus will be described.

  The compensation signal generation unit 301 generates a compensation signal using at least one of the held multi-channel audio signal and the held multi-channel audio signal related information.

  FIG. 6 shows a flowchart for explaining the operation of the audio signal output apparatus 1C according to the third embodiment of the present invention. The operation of the audio signal output device 1C will be described with reference to FIG. However, since the operation of the audio signal output device 1C has many parts in common with the operation of the audio signal output device according to the first and second embodiments described above, hereinafter, the operation will be described in the first and second embodiments. Step S301 different from the operation of the audio signal output apparatus will be described.

  In step S301, the compensation signal generation unit 301 generates a compensation signal using at least one of the held multi-channel audio signal and the held multi-channel audio signal related information. Here, the following is an example of generating a compensation signal using information related to a plurality of channel audio signals.

  As a first example of the compensation signal generation by the multi-channel audio signal related information, the multi-channel audio signal-related information includes a signal expressed as another signal by performing an operation on the multi-channel audio signal. You may use as a representative signal of the process of the compensation signal generation in the 2nd example of the said compensation signal generation method.

  As a second example of the compensation signal generation by the multi-channel audio signal related information, the multi-channel audio signal-related information is obtained by converting the multi-channel audio signal into a characteristic between a signal having a smaller number of channels than the multi-channel audio signal and the channel. And a signal having a smaller number of channels than the multi-channel audio signal may be used as a representative signal in the process of generating a compensation signal in the second example of the compensation signal generating method. For example, in the parametric stereo described in Non-Patent Document 3, a two-channel audio signal is represented by a parameter representing the characteristics between the one-channel audio signal and the channel. The 1-channel audio signal is a signal obtained by down-mixing a 2-channel audio signal, and can be used as a representative signal in the process of generating a compensation signal in the second example of the compensation signal generating method. Also, Non-Patent Documents 4 to 7 describe a method of expressing a multi-channel audio signal with a signal having a smaller number of channels than the multi-channel audio signal and a parameter indicating characteristics between channels, as in the parametric stereo. . In these methods, a signal obtained by down-mixing a multi-channel audio signal is used as a signal having a smaller number of channels than the multi-channel audio signal, and the down-mixed signal is used as the signal of the compensation signal generation method as in the parametric stereo. It can be used as a representative signal in the process of generating a compensation signal in the second example.

  As a third example of compensation signal generation based on multi-channel audio signal-related information, the multi-channel audio signal-related information is obtained by converting the multi-channel audio signal into a characteristic between a signal having a smaller number of channels than the multi-channel audio signal and the channel. And a signal having a smaller number of channels than a multi-channel audio signal is used as a representative signal in the process of generating a compensation signal in the third example of the compensation signal generating method. A parameter representing the characteristic between the channels may be used in one or more adjustments of the amplitude and phase of the generated compensation signal. For example, in the parametric stereo described in Non-Patent Document 3, there is a parameter representing the intensity difference between the left and right channels called Inter-channel Intensity Difference (IID), and thereby the intensity (that is, amplitude) of the left and right channels. ) Can be adjusted. In addition, the IID held by the multi-channel audio signal related information holding unit 201 (that is, the compensation signal generation target period (the section before the section where the multi-channel audio signal is degraded, the back section, both the front and back sections, The IID of the compensation signal generation target period is calculated by performing prediction or interpolation from the IID) of any one of the sections), and the calculated IID of the compensation signal generation target period can be used. The method for calculating the IID of the compensation signal generation target section is not limited. Further, for example, when the information related to the multi-channel audio signal includes a parameter indicating the strength of each channel audio signal such as the power or gain of the audio signal of each channel of the multi-channel audio signal, The intensity difference between them can be calculated and used for adjusting the amplitude of the common compensation signal. Furthermore, in the parametric stereo described in Non-Patent Document 3, the phase difference between the down-mixing audio signal called Overall Phase Difference (OPD) and Inter-channel Phase Difference (IPD) and the left channel audio signal, and the level of the audio signal between the left and right channels. There is a parameter indicating a phase difference, and thereby the phase of the audio signal of the left and right channels can be adjusted. The OPD and IPD of the compensation signal generation target section are calculated from the OPD and IPD of one of the sections before, behind, and both before and after the compensation signal generation target section, similarly to IID. Note that the calculation method of OPD and IPD in the compensation signal generation target section is not limited. Further, Non-Patent Documents 4 to 7 describe a method of expressing with parameters indicating characteristics between channels, like the parametric stereo IID, OPD, and IPD. In these methods, similar to the parametric stereo, the parameter can be used to adjust the representative signal in the process of generating the compensation signal in the third example of the compensation signal generating method.

  As described above, the compensation signal generation unit 301 can generate a compensation signal using the held multi-channel audio signal related information. Of course, the compensation signal generation unit 301 may generate a compensation signal using the held multi-channel audio signal, similarly to the compensation signal generation unit 105 in the first and second embodiments described above. Further, the compensation signal generation unit 301 may generate a compensation signal by combining a compensation signal generation method using multi-channel audio signal related information and a compensation signal generation method using multi-channel audio signals, Further, either one of the compensation signal generation methods may be selectively used.

[Fourth Embodiment]
FIG. 7 shows the configuration of an audio signal output device 1D according to the fourth embodiment of the present invention.

  As shown in FIG. 7, the audio signal output apparatus 1D includes a multi-channel audio signal holding unit 101, a multi-channel audio signal related information holding unit 201, a quality management unit 102, a correlation detection unit 401, a compensation signal generation method determining unit 104, a compensation A signal generation unit 301 and a signal output unit 106 are included. Since the configuration of the audio signal output device 1D has many parts in common with the configuration of the audio signal output device according to the first to third embodiments described above, the following description will be applied to the first to third embodiments. A block (correlation detection unit 401) different from the configuration of the audio signal output apparatus will be described.

  The correlation detection unit 401 uses at least one of the held multi-channel audio signal and the held multi-channel audio signal related information to calculate the correlation between the channels of the multi-channel audio signal in the corresponding section. To detect.

  FIG. 8 shows a flowchart for explaining the operation of the audio signal output apparatus 1D according to the fourth embodiment of the present invention. The operation of the audio signal output device 1D will be described with reference to FIG. However, since the operation of the audio signal output device 1D has many parts in common with the operation of the audio signal output device according to the first to third embodiments described above, in the following, the first to third embodiments will be described. Step S401 that is different from the operation of the audio signal output apparatus will be described.

  In step S401, correlation detecting section 401 uses at least one of the held multi-channel audio signal and the held multi-channel audio signal related information, and uses each channel of the multi-channel audio signal in the corresponding section. Detect correlation between. An example of a correlation detection method using a multi-channel audio signal here will be described in the first embodiment, and an example of a correlation detection method using multi-channel audio signal related information is described in the second embodiment. Explained. The correlation detection unit 401 may detect the correlation by combining the correlation detection method using the multi-channel audio signal and the correlation detection method using the multi-channel audio signal related information. Either correlation detection method may be selectively used.

[Fifth Embodiment]
FIG. 9 shows the configuration of an audio signal output device 1E according to the fifth embodiment of the present invention.

  As shown in FIG. 9, the audio signal output device 1E includes a multi-channel audio signal generation unit 501 (multi-channel audio signal generation unit), a multi-channel audio signal holding unit 101, a multi-channel audio signal related information holding unit 201, and a quality management unit. 102, a correlation detection unit 202, a compensation signal generation method determination unit 104, a compensation signal generation unit 301, and a signal output unit 106. Since the configuration of the audio signal output device 1E has many parts in common with the configurations of the audio signal output devices according to the first to fourth embodiments described above, the following description will be made on the first to fourth embodiments. A block (that is, a multi-channel audio signal generation unit 501) different from the configuration of the audio signal output apparatus will be described.

  The multi-channel audio signal generation unit 501 generates a multi-channel audio signal from the multi-channel audio signal related information.

  FIG. 10 shows a flowchart for explaining the operation of the audio signal output apparatus 1E according to the fifth embodiment of the present invention. The operation of the audio signal output device 1E will be described with reference to FIG. However, since the operation of the audio signal output device 1E has many parts in common with the operation of the audio signal output device according to the first to fourth embodiments described above, the following description will be made on the first to fourth embodiments. Step S501 different from the operation of the audio signal output apparatus will be described.

  In step S501, the multi-channel audio signal generation unit 501 generates a multi-channel audio signal from the multi-channel audio signal related information. As the multi-channel audio signal related information here, the multi-channel audio signal generation unit 501 can generate a multi-channel audio signal, and the correlation detection unit 202 can detect the correlation between the channels of the multi-channel audio signal. Furthermore, it is only necessary that the compensation signal generation unit 301 can generate a compensation signal using at least one of the multi-channel audio signal and the multi-channel audio signal related information.

  The multi-channel audio signal related information can include, for example, a parametric stereo 1-channel audio signal of Non-Patent Document 3 and a parameter representing characteristics between channels. Among these, the correlation between channels can be detected using the above-described ICC, which is one of the parameters representing the characteristics between channels. Further, a 2-channel audio signal that is a multi-channel audio signal can be generated from the multi-channel audio signal-related information. This generation method is described in Non-Patent Document 3.

  Further, Non-Patent Documents 4 to 7 use an audio signal having a smaller number of channels than a multi-channel audio signal, and parameters representing characteristics equivalent to or similar to the ICC. Similarly, it can be included as multi-channel audio signal related information.

  Furthermore, for example, the MS stereo of Non-Patent Document 8 can be included as the multi-channel audio signal related information. MS stereo includes the sum signal and difference signal of (Expression 2) and (Expression 3) described above, and correlation can be detected by the same method as in the first embodiment. (Expression 2), (Expression 3) A multi-channel audio signal can be generated by the inverse operation. Details of the generation method are described in Non-Patent Document 8. The multi-channel audio signal related information may be the information in the above example or other information, and the format is not limited.

[Sixth Embodiment]
FIG. 11 shows the configuration of an audio signal output device 1F according to the sixth embodiment of the present invention.

  As shown in FIG. 11, the audio signal output device 1F includes a multi-channel audio signal generation unit 501, a multi-channel audio signal holding unit 101, a multi-channel audio signal related information holding unit 201, a quality management unit 102, a correlation detection unit 103, and compensation. The signal generation method determination unit 104, the compensation signal generation unit 301, and the signal output unit 106 are included.

  The audio signal output device 1F is similar to the audio signal output device 1E of the fifth embodiment, but the operation of the correlation detection unit is different. The correlation detection unit 103 included in the audio signal output device 1F is the same as the correlation detection unit 103 in the first and third embodiments, and the correlation detection unit 103 converts a multi-channel audio signal into a multi-channel audio signal. The correlation between each channel is detected.

  FIG. 12 shows a flow chart for explaining the operation of the audio signal output device 1F according to the sixth embodiment, which is correlated with the operation (FIG. 10) of the audio signal output device 1E of the fifth embodiment. The processing steps by the detection unit are different. In the audio signal output device 1F, in step S103, the correlation detection unit 103 detects the correlation between the channels of the multi-channel audio signal from the multi-channel audio signal.

  At this time, as the multi-channel audio signal related information, it is only necessary that the multi-channel audio signal generation unit 501 can generate the multi-channel audio signal, and the correlation detection unit 103 uses the multi-channel audio signal to generate the multi-channel audio signal. It is only necessary that the correlation between the channels can be detected, and it is only necessary that the compensation signal generation unit 301 can generate the compensation signal using at least one of the multi-channel audio signal and the multi-channel audio signal related information.

[Seventh Embodiment]
FIG. 13 shows the configuration of an audio signal output device 1G according to the seventh embodiment of the present invention.

  As shown in FIG. 13, the audio signal output device 1G includes a multi-channel audio signal generation unit 501, a multi-channel audio signal holding unit 101, a multi-channel audio signal related information holding unit 201, a quality management unit 102, a correlation detection unit 401, and compensation. The signal generation method determination unit 104, the compensation signal generation unit 301, and the signal output unit 106 are included.

  The audio signal output device 1G is similar to the audio signal output device 1E of the fifth embodiment, but the operation of the correlation detection unit is different. The correlation detection unit 401 included in the audio signal output device 1G is the same as the correlation detection unit 401 in the fourth embodiment, and the correlation detection unit 401 includes the multi-channel audio signal and the multi-channel audio signal related information. The correlation between each channel of the multi-channel audio signal is detected from at least one.

  FIG. 14 is a flowchart for explaining the operation of the audio signal output device 1G according to the seventh embodiment, and is correlated with the operation of the audio signal output device 1E according to the fifth embodiment (FIG. 10). The processing steps by the detection unit are different. In audio signal output apparatus 1G, in step S401, correlation detection section 401 detects the correlation between the channels of the multi-channel audio signal from at least one of the multi-channel audio signal and the multi-channel audio signal related information.

  At this time, as the multi-channel audio signal related information, the multi-channel audio signal generation unit 501 only needs to be able to generate a multi-channel audio signal, and the correlation detection unit 401 can generate the multi-channel audio signal and the multi-channel audio signal related information. It is sufficient that the correlation between the channels of the multi-channel audio signal can be detected using at least one of them, and the compensation signal generation unit 301 uses the at least one of the multi-channel audio signal and the multi-channel audio signal related information to obtain the compensation signal. Can be generated.

  1A, 1B, 1C, 1D, 1E, 1F, 1G ... audio signal output device, 11 ... CPU, 12 ... main storage unit, 13 ... auxiliary storage unit, 14 ... communication control unit, 15 ... display unit, 16 ... operation unit , 17 ... input unit, 18 ... output unit, 20A, 20B, 20C ... audio signal output program, 21 ... multi-channel audio signal holding module, 22 ... quality control module, 23 ... correlation detection module, 24 ... determination of compensation signal generation method Module, 25 ... Compensation signal generation module, 26 ... Signal output module, 27 ... Multi-channel audio signal related information holding module, 28 ... Multi-channel audio signal generation module, 101 ... Multi-channel audio signal holding unit, 102 ... Quality management unit, DESCRIPTION OF SYMBOLS 103 ... Correlation detection part, 104 ... Compensation signal generation method determination part, 105 ... Compensation signal generation part, 106 ... Signal output part, 20 ... multiple channel audio signals related information holding unit, 202 ... correlation detection unit, 301 ... compensation signal generation unit, 401 ... correlation detection unit, 501 ... multiple-channel audio signal generator.

Claims (8)

Multi-channel audio signal generating means for generating a multi-channel audio signal that is an audio signal of two or more channels using multi-channel audio signal related information that is information related to the multi-channel audio signal;
Multi-channel audio signal holding means for holding the multi-channel audio signal generated by the multi-channel audio signal generating means for a predetermined period;
Quality management means for determining whether or not the multi-channel audio signal is deteriorated based on predetermined quality control information;
Correlation detection means for detecting a correlation between the channels of the multi-channel audio signal from the multi-channel audio signal related information when the quality management means determines that the multi-channel audio signal is deteriorated;
Compensation signal generation method determination means for determining a compensation signal generation method based on the detection result by the correlation detection means;
Compensation signal generation means for generating a compensation signal based on the generation method determined by the compensation signal generation method determination means;
A signal output unit that outputs the compensation signal generated by the compensation signal generation unit and outputs the multi-channel audio signal when the quality control unit determines that the multi-channel audio signal is not degraded; ,
Comprising
The compensation signal generation method determining means includes:
For two or more channel audio signals whose correlation between detected channels is higher than a predetermined reference, a generation method for generating a common compensation signal is determined as a generation method of the compensation signal,
The compensation signal generation means generates the compensation signal from the multi-channel audio signal related information.
Audio signal output device. The audio signal output device is
A multi-channel audio signal related information holding means for holding the multi-channel audio signal related information for a predetermined period;
The correlation detection means detects a correlation between the channels of the multi-channel audio signal from the multi-channel audio signal related information held in the multi-channel audio signal related information holding means .
The audio signal output device according to claim 1. The correlation detection means detects a correlation between the channels of the multi-channel audio signal from at least one of the multi-channel audio signal and the multi-channel audio signal-related information.
The audio signal output apparatus according to claim 1, wherein the audio signal output apparatus is an audio signal output apparatus. The multi-channel audio signal related information includes a mixed audio signal of at least one channel obtained by mixing the multi-channel audio signal by a predetermined method.
The audio signal output device according to claim 1, wherein the audio signal output device is an audio signal output device. The multi-channel audio signal related information includes at least one of an intensity difference between channels of the multi-channel audio signal, a phase difference between channels, and a coherence between channels.
The audio signal output device according to claim 1, wherein the audio signal output device is an audio signal output device. The multi-channel audio signal related information includes a correlation detection result between the channels of the multi-channel audio signal.
The audio signal output device according to claim 1, wherein the audio signal output device is an audio signal output device. An audio signal output method executed by an audio signal output device,
A multi-channel audio signal generating step of generating a multi-channel audio signal that is an audio signal of two or more channels using multi-channel audio signal related information that is information related to the multi-channel audio signal;
Holding the multi-channel audio signal generated in the multi-channel audio signal generating step for a predetermined period;
Determining whether the multi-channel audio signal is degraded based on predetermined quality control information; and
Outputting the multi-channel audio signal when it is determined that the multi-channel audio signal is not degraded;
Detecting the correlation between the channels of the multi-channel audio signal from the multi-channel audio signal-related information when it is determined that the multi-channel audio signal is degraded;
Determining a compensation signal generation method based on a detection result of correlation between the channels;
Generating a compensation signal based on the determined generation method;
Outputting the generated compensation signal;
Comprising
In the step of determining a method for generating the compensation signal,
For two or more channel audio signals whose correlation between detected channels is higher than a predetermined reference, a generation method for generating a common compensation signal is determined as a generation method of the compensation signal,
In the step of generating the compensation signal, the compensation signal is generated from the multi-channel audio signal related information.
Audio signal output method. A computer provided in the audio signal output device,
Multi-channel audio signal generating means for generating a multi-channel audio signal that is an audio signal of two or more channels using multi-channel audio signal related information that is information related to the multi-channel audio signal;
Multi-channel audio signal holding means for holding the multi-channel audio signal generated by the multi-channel audio signal generating means for a predetermined period;
Quality management means for determining whether or not the multi-channel audio signal is deteriorated based on predetermined quality control information;
Correlation detection means for detecting a correlation between the channels of the multi-channel audio signal from the multi-channel audio signal related information when the quality management means determines that the multi-channel audio signal is deteriorated;
Compensation signal generation method determination means for determining a compensation signal generation method based on the detection result by the correlation detection means;
Compensation signal generation means for generating a compensation signal based on the generation method determined by the compensation signal generation method determination means;
A signal output unit that outputs the compensation signal generated by the compensation signal generation unit and outputs the multi-channel audio signal when the quality control unit determines that the multi-channel audio signal is not degraded;
Function as
The compensation signal generation method determination means generates a common compensation signal as the compensation signal generation method for two or more channel audio signals whose detected correlation between channels is higher than a predetermined reference. Decide
An audio signal output program for generating the compensation signal from the multi-channel audio signal related information.

Images