JP6921832B2 - Efficient format conversion in audio coding - Google Patents

Description

This document relates to methods and devices for format conversion (including downmixing) of audio signals. In particular, this document relates to, for example, simplified and more efficient format conversion methods and devices for the MPEG-H 3D audio standard.

Format converter of the main profile of MPEG-H 3D audio standards, the desired playback format having N _out ≦ N _in number of channels from the transmission channel configuration having N _in number of channels, converts the multi-channel audio signal It is a down mixer. This downmixer features a phase-alignment active downmix algorithm that ensures energy conservation, as well as uncontrolled signal cancellations or signal amplification that result in unwanted signal coloration and artifacts. Functions to prevent or reduce. On the downside, the computational complexity of the downmix algorithm is high, giving the decoder a high computational load. For example, in the case of a 22.2 channel to stereo downmix, about 50 percent of the overall complexity of the decoder in the main profile is allocated to the downmix algorithm.

High computational complexity occurs especially when the number of input channels N _{in is large.} Computational load, which typically has relatively small computational power, is especially significant for devices that render high quality broadcast content featuring a large number of channels.

Therefore, there is a need for a format converter that is simple and efficient, yet retains favorable properties in terms of audio signal quality and energy conservation.

According to one aspect of the present disclosure, a method of downmixing a plurality of input audio channels will be described. The method may include obtaining multiple frequency coefficients in a plurality of corresponding frequency bins (conversion region frequency bins) for each of the input audio channels. This acquisition step receives those frequency coefficients from the bitstream, or applies time-frequency conversion to each of the input audio channels, and multiple frequencies in multiple corresponding frequency bins (for each of the input audio channels). It may be related to obtaining the coefficient. This time frequency transform may be a short time Fourier transform (STFT). In particular, this time-frequency transform may be a discrete Fourier transform (DFT) (eg, a fast Fourier transform (FFT)). The method applies a downmix matrix to a first array formed by the frequency coefficients of multiple input audio channels in each frequency bin for at least one frequency bin, and multiple intermediate audios in each frequency bin. It may further include acquiring a second array formed by the frequency coefficients of the channel. These first and second arrays can be linear arrays (vectors). The number of entries (elements) in the first array can be given by the number of input audio channels. The number of entries in the second array can be given by the number of output audio channels. The downmix matrix may be different for each different frequency bin, or may be different for each different audio band (processing band). In some embodiments, the downmix matrix may be uniform over the frequency bin or audio band. The method may further include finding a third array containing only nonzero entries (elements) of the downmix matrix. The method may further include finding for each entry in the third array a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix. Applying the downmix matrix to the first array involves multiplying at least one frequency bin the entry of the third array of each frequency bin and the corresponding entry of the first array. May be good. The corresponding entry in the first array can be determined based on the fourth array. Here, the corresponding entry in the first array indicates the entry to be multiplied by each non-zero entry in the downmix matrix in the normal matrix multiplication of the downmix matrix with the first array. Understood. Applying a downmix matrix may further include properly calculating the sum of the obtained products and assigning the sum (one or more) of the obtained products to the entries in the second array. ..

As will be appreciated by those skilled in the art by being configured as described above, the proposed method is under normal use, especially when the number of channels is large (eg, N _in , N _out ≥ 6). Take advantage of the fact that a downmix matrix is typically a sparse matrix, since typically not all output channels depend on all input channels. In other words, in a normal application scenario, the downmix matrix has entries equal to zero. The sparsity of the downmix matrix, i.e. the percentage of zero entries, depends on the format conversion use case. However, typically, sparsity increases with increasing number of input and output channels. For example, in a practical use case of downmixing 22.2 channel audio to 5.1 channel audio, 34 non-zeros out of 144 entries in a downmix matrix with a downmixing matrix of 6x24. With the entry of, the degree of sparseness S is given by S = (144-34) / 144≈76%. By utilizing sparsity, the complexity of calculation of a format that converts one audio band can be reduced _{from approximately O (N out} N _in ² ) to O ((1-S) N _out N _in ^2). Can be done. Note that some additional overhead may be involved, depending on, for example, the hardware platform used for the sparse downmix matrix and the exact representation.

In particular, the proposed method minimizes access to the downmix matrix. It avoids computational steps that increase the overall computational cost but do not affect the desired computational results. Therefore, the overall calculation of the downmixed audio channel can be done more efficiently in terms of both calculation cost and memory access rate, for example, without degrading the accuracy of the calculation and without affecting the calculation result. can. Therefore, the format converter of an audio decoder using the method according to the present proposal may have reduced computing power that allows for a slim design of the format converter.

The proposed method is useful in all examples of sparse downmix matrices, but in particular reduces the worst-case complexity that occurs with the maximum number of input and output channels supported by the system. It is beneficial to. Generally speaking, the method according to the present proposal is particularly useful for complex use cases involving a large number of input and output channels.

In the above and for the rest of this section, the method according to the present proposal shall be performed for at least one frequency bin. However, the method uses a single frequency bin for time-frequency conversion, all frequency bins within a group of frequency bins (eg, within a given audio band (processing band)), or all frequency bins (eg, within a given audio band (processing band)). For example, for each frequency bin) it is understood that it can be done.

In some exemplary embodiments, the method further comprises the step of finding (calculating) the energy index value of each intermediate audio channel for at least one frequency bin, for each intermediate audio channel. May be good. The method further comprises adjusting the magnitude of the frequency coefficient of each intermediate audio channel in each frequency bin based on the respective energy index value for each intermediate audio channel for at least one frequency bin. May be good.

In some exemplary embodiments, the method may further include determining whether the downmix matrix is a sparse matrix. After that, processing involving a third array and a fourth array (and a fifth array as defined below) may be performed only if the downmix matrix is found to be a sparse matrix.

In some exemplary embodiments, determining the energy index value is a power of the entries in the third array and a power of the absolute values of the corresponding entries in the first array for at least one frequency bin. Multiplying may be included and the corresponding entry in the first array can be determined based on the fourth array. For example, the energy index value may involve multiplying the square of the entry in the third array by the square of the absolute value (absolute square) of the corresponding entry in the first array. Finding the energy index value may further include summing the obtained products appropriately. The energy index value may be the same for all frequency bins in a given audio band. In such cases, finding the above energy index values for frequency bins sums the respective results obtained for each of the frequency bins within a given audio band (eg, the appropriate sum of the acquired products). It may further include that.

This makes it possible to efficiently calculate the energy index value in a particularly simple way with respect to calculation cost and memory access rate. Moreover, when the calculation of the energy index value is performed in this way, both the calculation of the intermediate audio channel and the calculation of the energy index value can be performed collectively. This batch calculation makes it possible to use the same calculation tool for both calculations and avoid repeated loops across the indexes of the first and third arrays. As a result, the complexity of calculation and the memory access rate can be further reduced.

In some exemplary embodiments, the third array is a linear array (vector) containing non-zero entries in the downmix matrix in their order of appearance in a given scan order across the entries in the downmix matrix. obtain. This scanning order may be, for example, a row-direction scanning order or a column-direction scanning order.

In some exemplary embodiments, each entry in the fourth array may indicate the position of each entry in the third array in the scan order in the downmix matrix. This position may be a linear position. Further, this position can be an absolute position in the scan order, for example, in the case of a row direction scan order, the row length is modal (mod), or in the case of a column direction scan order. It can also be an absolute position relative to the length of the row (height of the row).

In some exemplary embodiments, each entry in the fourth array, in the case of row-wise scan order, indicates a column of downmix matrices in which each entry in the third array is located in the downmix matrix. You may. For columnar scan order, each entry in the fourth array may represent a row in the downmix matrix in which each entry in the third array is located in the downmix matrix.

In another embodiment, each entry in the fourth array is offset by the scan order position of each entry in the third array in the downmix matrix from the first position in the scan order. It may be shown. In yet another embodiment, each entry in the fourth array is a position on the scan order of each entry in the third array in the downmix matrix, and each entry in the third array in the downmix matrix. It may indicate the offset from the position in the scanning order of the immediately preceding entry.

By making such a representation of position in the fourth array, the entries in the first array corresponding to a given entry in the third array with a small amount of computation and reduced memory access rate. It can be easily estimated.

In some exemplary embodiments, the method may further include the step (s) of finding a fifth array. The entries in the fifth array may indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order and downmix in the case of column scan order. It may indicate the number of non-zero entries for each column of the matrix. Here, the entries in the fifth array may in any case have the same degree as the rows or columns of the downmix matrix have in the downmix matrix.

In some exemplary embodiments, applying the downmix matrix to the first array is an entry in the third array of each frequency bin and a corresponding entry in the first array for at least one frequency bin. It may further include assigning the sum of one or more products of and to the entries of the second array based on the entries of the fifth array.

In some exemplary embodiments, applying the downmix matrix to the first array continuously increments the entry index for the third array, one at a time, for at least one frequency bin. For each such entry index, further including finding (calculating) the product of the entry of the third array represented by each entry index of each frequency bin and the corresponding entry of the first array. You may. Applying the downmix matrix to the first array can further entail assigning the sum of one or more of these products sequentially to the entries of the second array for at least one frequency bin. This allocation is based on the entry in the fifth array. Here, the increment may start from the smallest possible value of the entry index (eg, 1).

In some exemplary embodiments, applying the downmix matrix to the first array means that for each entry index for at least one frequency bin and the third array, by each entry index for each frequency bin. It can further be accompanied by finding the product of the square of the entries in the third array shown and the absolute squares of the corresponding entries in the first array as the second product. Applying the downmix matrix to the first array is the sum of one or more of these second products for each entry index for at least one frequency bin and the third array for each frequency bin. It can also be accompanied by sequential allocation to the entries in the sixth array. This allocation may be based on the entry of the fifth array. The sum of the sixth array over the frequency bins in a given processing band can give the first energy index value of the frequency bins in the given processing band.

In some exemplary embodiments, the step of finding a frequency bin within a given processing band can involve referring to a look-up table. This lookup table may contain, for each processing band, an entry indicating the number of frequency bins within each processing band.

In some exemplary embodiments, the method is the number of non-zero entries in the first row of the downmix matrix from the first entry in the fifth array for the first entry in the second array. Can be further accompanied by seeking. The entry index for the second and fifth arrays can be initialized to 1. The method further comprises finding (calculating) the product of the first entry of the third array and the entry of the first array represented by the first entry of the fourth array as the first product. obtain. The method finds the product of the square of the first entry in the third array and the absolute square of the entry in the first array indicated by the first entry in the fourth array as the second product (calculation). ) Can be further included. The method finds an entry index for the third array and an entry index for the fourth array when the number of first products obtained is less than the number of non-zero entries in the first row of the downmix matrix. Is further incremented by one, and the product of the entries in the third array and the entries in the fourth array indicated by each incremented entry index is further calculated (calculated) as the first product. Can include. The method may further include summing the obtained first products and assigning the sum of the obtained first products to the first entry in the second array. The method may further include summing the obtained second products and assigning the sum of the obtained second products to the first entry in the sixth array. The entry index for each of the third, fourth, and fifth arrays may then be incremented by one for each of the second entry in the second array and the second entry in the sixth array. The steps described above may be repeated, and may be repeated until the final entry of the second array (and the sixth array) is reached.

It should be noted that the methods described in this document can be applied to corresponding devices such as format converters or audio decoders. It is understood that all of the above statements regarding the method apply equally to devices that downmix multiple input audio channels.

As a result, according to another aspect, a device that downmixes a plurality of input audio channels is described. The device (eg, a format converter or audio decoder) is adapted (configured) to acquire multiple frequency coefficients in a plurality of corresponding frequency bins (conversion region frequency bins) for each of the input audio channels. A frequency coefficient acquisition unit can be provided. The frequency coefficient acquisition unit may be a receiving unit configured to receive the above frequency coefficient from the bitstream, or apply time-frequency conversion to each of the input audio channels for each of the input audio channels. , It may be a time-frequency conversion unit configured to acquire a plurality of frequency coefficients in a plurality of corresponding frequency bins. The time frequency transform may be, for example, a short time Fourier transform (STFT). In particular, the time-frequency transform may be a discrete Fourier transform (DFT), such as a fast Fourier transform (FFT). The apparatus may further include a downmix matrix acquisition unit configured to acquire the downmix matrix. The device applies a downmix matrix to a first array formed by the frequency coefficients of multiple input audio channels in each frequency bin for at least one frequency bin, and multiple intermediate audios in each frequency bin. Further downmixing units configured to acquire a second array formed by the frequency coefficients of the channels can be provided. The first array and the second array can be linear arrays (vectors). The number of entries (elements) in the first array can be given by the number of input audio channels and the number of entries in the second array can be given by the number of output audio channels. The downmix matrix may be different for each different frequency bin, or may be different for each different audio band (processing band).

In some exemplary embodiments, the downmix matrix can be uniform throughout the frequency bin or audio band. The downmixing unit may be configured to find a third array containing only nonzero entries in the downmix matrix. The downmixing unit can be further configured to find for each entry in the third array a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix. The downmixing unit can be further configured to multiply at least one frequency bin by the entry of the third array of each frequency bin and the corresponding entry of the first array, the first array. The corresponding entry for is found based on the fourth array. Here, the corresponding entry in the first array is interpreted to indicate the entry to be multiplied by each non-zero entry in the downmix matrix in a normal matrix multiplication of the downmix matrix and the first array. Will be done. The downmix unit can be further configured to appropriately sum the acquired products and assign the sum of the acquired products (one or more) to the entries in the second array.

In some exemplary embodiments, the device may further include, for each intermediate audio channel, an energy calculation unit configured to determine the energy index value of each intermediate audio channel for at least one frequency bin. can. The apparatus is configured to adjust the magnitude of the frequency coefficient of each intermediate audio channel in each frequency bin based on the respective energy index value for each intermediate audio channel for at least one frequency bin. Further units can be provided.

In some exemplary embodiments, the downmixing unit may be further configured to determine if the downmix matrix is a sparse matrix. After that, processing involving a third array and a fourth array (and also a fifth array as defined below) may be performed only if the downmix matrix is found to be a sparse matrix.

In some exemplary embodiments, the third array can be a linear array that includes non-zero entries in the downmix matrix in their order of appearance in a given scan order across the entries in the downmix matrix.

In some exemplary embodiments, the downmixing unit may be further configured to require a fifth array. The entries in the fifth array may indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order, and in the case of column scan order, respectively. It may indicate the number of non-zero entries in the downmix matrix per column.

In some embodiments, the downmixing unit is the sum of one or more products of the entries of the third array of each frequency bin and the corresponding entries of the first array for at least one frequency bin. It may be further configured to be assigned to a second array entry based on the array entry of.

According to another aspect, the software program is described. The software program may be configured to run on a processor and, when run on a computing device, to perform the steps of the method outlined in this document.

According to another aspect, the storage medium is described. This storage medium may include a software program configured to run on a processor and, when implemented on a computing device, to perform the steps of the method outlined in this document. .. For example, a non-transitory computer-readable medium that stores instructions that can be executed by one or more processors.

According to a further aspect, a computer program product is described. This computer program may include executable instructions that, when run on a computer, perform the steps of the methods outlined in this document.

It should be noted that the methods and devices, including the preferred embodiments outlined herein, can be used standalone or in combination with other methods and systems disclosed in this document. Moreover, all aspects of the methods and devices outlined in this document can be arbitrarily combined. In particular, the features of the claims can be combined with each other in any way.

Through the following detailed description of the accompanying drawings, the above-mentioned objectives, features and advantages as well as other objectives, features and advantages of the above-exemplified embodiments will be better understood. In the drawings, some embodiments are shown as examples and non-limiting.

It is a figure which shows typically the example of the format converter to which the example of the embodiment of this disclosure can be applied. It is a flowchart which shows typically an example of the process of the format conversion by the format converter by the embodiment of this disclosure. FIG. 5 is a flowchart schematically showing a detailed example of steps in the process of FIG. 2 according to an embodiment of the present disclosure. It is a figure which shows typically an example of the downmix matrix and the array derived from the downmix matrix by embodiment of this disclosure. FIG. 5 is a flowchart schematically showing a detailed example of steps in the process of FIG. 3 according to an embodiment of the present disclosure. FIG. 5 is a flowchart schematically showing a detailed example of another step in the process of FIG. 2 according to an embodiment of the present disclosure. It is a flowchart which shows schematic example of the detail of the step in the process of FIG. FIG. 5 is a diagram schematically showing an apparatus according to an embodiment of the present disclosure that executes the process of FIG.

First, the relevant aspects of the main profile format converter of the MPEG-H 3D audio standard will be briefly described. For a broader description of the MPEG-H 3D audio standard, see Reference ISO / IEC DIS 23008-3 "Information technology --High efficiency coding and media delivery in heterogeneous environments --Part 3: 3D audio". The entire document is incorporated herein by reference. Section 10 of this document relates to format converters.

In summary, the MPEG-H 3D audio standard main profile format transducer provides a phase-aligned quadrature mirror filter (QMF) region downmix. This downmix involves a QMF filter bank (hybrid QMF) and phase alignment processing. The latter includes analysis of covariance of the input signal (input channel). In particular, the main profile format converter, along with the Nyquist filter bank, which increases frequency resolution in the lowest processing band, allows input audio to be able to more effectively address and prevent signal cancellation or signal boost. The QMF representation of the channel (hybrid QMF) is used.

The QMF filter bank of the main profile format converter is designed to provide relatively high time resolution and has a hop size of 64 samples for an analysis window length of 640 samples. Therefore, there is a 90% overlap between subsequent filter bank analysis results, resulting in relatively high computational costs and memory access rates. The resulting high time resolution is effective in preventing modulation artifacts related to phase alignment, but is not required for actual downmix operation.

From the viewpoint of avoiding high computational complexity, a low complexity (LC) profile of the MPEG-H 3D audio standard incorporating a format conversion function is assumed. Generally speaking, LC profile format converters reduce the number of audio channels by downmixing when the number of loudspeaker channels available is less than the number of input channels. Format converters attempt to maintain a faithful audio image that resembles the original audio image, despite the reduced number of channels. To this end, the down mixer involves adaptive equalization that controls the energy of the output channel relative to the energy of the input channel across the individual audio bands (processing bands).

The LC format converter acts on the frequency representation of the input signal with the appropriate time-frequency resolution for downmixing. Contrary to the main profile format converter, the format conversion function built into the LC profile can be without the QMF filter bank, thereby reducing the computational complexity of the format conversion. Instead, the LC profile format transducer may operate in a short-time Fourier transform (STFT) such as the Discrete Fourier Transform (DFT) region. This time-frequency representation can be obtained by applying a complex Fast Fourier Transform (FFT) to a windowed input audio channel. The FTFT can have a length of 512 samples (analysis window length). The hop size can be 256 samples. That is, the overlap between consecutive analysis windows can be 50%. The FTFT can further perform sqrt (Hann) analysis and synthesis windows.

Frequency bins for time-frequency conversion (conversion region frequency bins) can be grouped into audio bands (processing bands) for equalization. For example, the FTFT may have 58 processing bands. These processing bands can be defined according to the psychoacoustic model. For example, in order to reduce the computational complexity, the frequency resolution at high frequencies can be reduced.

The actual down-mixing of the input audio channel can then be performed on the frequency coefficients obtained by the time-frequency conversion. The downmixing matrices used in this downmixing may differ from each other for different processing bands. The intermediate audio channels (eg, their frequency coefficients) obtained by applying each downmix matrix may then undergo energy equalization to ensure energy conservation.

LC format transducers, on the other hand, do not involve phase alignment and therefore analysis of covariance. To compensate for the omission of phase alignment, the processing band of the STFT can be further defined (eg, compared to the hybrid QMF downmix processing of the main profile format converter) in the intermediate frequency range. , A range of about 100 Hz to about 3.5 kHz, that is, a range of about 100 Hz to 3.5 kHz).

In particular, the MPEG-H 3D audio standard LC format transducer is compatible with the design of the main profile format converter, provides good audio signal quality, meets energy conservation, and above all, computational complexity. Is designed to reduce.

The present disclosure relates to actual down-mixing and equalization, and teaches advantageous strategies for efficiently performing these steps, thereby further reducing the computational complexity of the format transducer.

However, many aspects of the disclosure are independent of the particular time-frequency conversion used by LC format converters and can also be applied directly to format converters that use the QMF representation of audio channels, for example. Is understood.

As shown in FIG. 1, the format converter 100 of the audio decoder (for example, the audio processing block of the format converter) has, for example, N _in audio channels (input audio channels) 10 (decoded). A time domain audio sample is received from the core decoder of the audio decoder to generate a downmixed time domain audio output signal consisting of _{N out output audio channels 20.} The number of audio output channels is less than or equal to the number of input audio channels (eg, N _out ≤ N _in ). The actual down-mixing process can be performed in the frequency domain. A downmix matrix M _{DMX that downmixes N in} input audio channels can be provided to the format converter 100 (eg, by a core decoder). Therefore, the processing of the format converter 100 takes the audio data decoded by the core decoder and the (static) downmix matrix M _DMX as inputs. The downmix matrix M _DMX can be optionally provided to the format converter (eg, given as a signal) with the desired output channel configuration of the output audio channel during the initial processing (startup) of the format converter. .. The downmix matrix M _DMX or its index value may be transmitted in an audio bitstream provided to the audio decoder. In particular, the MPEG-H 3D audio standard allows transmission of downmix details (including index values for downmix matrices) for a particular target channel configuration. The downmix matrix M _DMX may include real-valued non-negative downmix coefficients. It may also have dimensions N _out x N _in. That is, the number of rows in the downmix matrix can be given by _{N out.} The number of columns in the downmix matrix can be given by _{N in.} The specific examples in the rest of this document assume that the _{downmix matrix has dimensions N out} x N _in. However, these particular examples also apply to downmix matrices of _{dimension N in} x N _out. In that case, the concepts of columns and rows in that particular example must be exchanged with each other.

FIG. 2 is a flowchart schematically showing a format conversion (downmixing) process (method) that can be performed (for example, by the format converter 100). This method relates to the processing of one block (eg, a frame) of the time domain sample and can be repeated for subsequent blocks of the time domain sample.

を取得することもできる。ステップＳ２０１０は、本方法（例えば、フォーマット変換器）の初期処理の際に実行することができ、ダウンミックス行列がこれよりも以前に取得されている場合には省略することができる。ダウンミックス行列は、オーディオ復号化器のフォーマット変換器モジュール又は受信側全体が（例えば、出力チャネル構成の変化の際に）リセットされるまで、セッション中は時間的に一定に維持され得る。 _{In step S2010, the (static) downmix matrix M DMX} can be obtained, for example, by signal communication from an external source of the format converter. In other words, the method (or format converter) can take the downmix matrix M _DMX as input. A single downmix matrix _{Rather than a single downmix matrix M DMX} , one or more downmix matrices for each frequency bin of the frequency coefficients obtained in step S2020

Can also be obtained. Step S2010 can be performed during the initial processing of the method (eg, format converter) and can be omitted if the downmix matrix has been acquired earlier than this. The downmix matrix can remain constant in time during the session until the audio decoder's format converter module or the entire receiver is reset (eg, during changes in output channel configuration).

の（静的な）セットが取得される場合にも関係し、このセットから、ダウンミキシングに用いられるダウンミックス行列をフレーム単位で用いることができる。したがって、上述したダウンミックス行列

は、（静的な）セットからフレーム単位で選ぶことができる。所与のフレームに用いられるダウンミックス行列の選択は、ビットストリームと共に受信され得る、当該所与のフレームのレンダリングタイプを示す情報に基づいて行われ得る。 The notion that the downmix matrix is temporally constant is that the downmix matrix (two or more) is 2 ≤ l.

It is also relevant when a (static) set of is obtained, from which the downmix matrix used for downmixing can be used on a frame-by-frame basis. Therefore, the downmix matrix described above

Can be selected on a frame-by-frame basis from a (static) set. The selection of the downmix matrix used for a given frame can be based on information indicating the rendering type of the given frame that can be received with the bitstream.

The method may further include receiving a plurality of input audio channels (input audio signals) as inputs. In particular, the method may include receiving time domain samples (sequences) of multiple input audio channels.

In step S2020, a plurality of frequency coefficients (one frequency coefficient for each frequency bin) in a plurality of frequency bins can be acquired for each of the input audio channels. For example, the plurality of frequency coefficients in the plurality of frequency bins for each of the input audio channels may be received from the bitstream. Alternatively, time-frequency conversion may be applied to each of the plurality of input audio channels (eg, each time domain sample of the plurality of input audio channels). This time-frequency conversion can provide multiple frequency coefficients in multiple frequency bins for each of the input audio channels.

Format conversion can work on consecutive non-overlapping frames. The frame may have a length L (eg, L = 256 time domain samples) in units of time domain samples of the input audio signal. The format conversion may output one frame with L samples per processed input frame of length L. This output frame has fewer audio signal channels (or at most the same number of audio signal channels) than input frames.

The time-frequency conversion can be an STFT such as DFT. The length N of the FTFT can be N = 512 samples. The hop size of the FTFT can be 256 samples. In this case, the overlap between subsequent STFTs is 256 samples, or 50%. The FTFT region processing can be performed on, for example, K = 256 frequency bins. These frequency bins can be divided into a plurality of audio bands (processing bands) to be processed, for example, B = 58 processing bands.

と表す。複数の入力オーディオチャネルの時間領域サンプルを、

と表す。νは時間領域サンプルを示すインデックスである。（・）^Tは転置行列を示す。すると、所与の入力オーディオチャネルｉの入力バッファーは、以下の式によって与えられ得る。

ここで、Ｆは、フレームインデックスを示す。最初の処理フレームについては、

である。入力バッファー内のＮ_in個の入力オーディオチャネルのそれぞれは、（例えば、正弦ウィンドウを用いて）ウィンドウ処理され得る。長さＮ（例えば、Ｎ＝５１２個のサンプル）の離散フーリエ変換（ＤＦＴ）等の時間周波数変換は、入力バッファー（例えば、ウィンドウ処理された入力バッファー）内のＮ_in個の入力オーディオチャネル（入力オーディオ信号）のそれぞれについて計算され得る。その結果、周波数係数

が得られる。ここで、ｋは周波数ビンを示す。周波数ビンごとに、そのエントリー（要素）として複素周波数係数を有するＮ_in次元ベクトルを取得することができる。例えば、周波数係数は、１≦ｉ≦Ｎ_in及び０≦ｋ≦Ｋ−１について、以下の式で与えられ得る。

以下の記載において、同じ処理が全ての周波数ビンに適用される場合、周波数インデックスｋを省略することがある。ウィンドウ関数は、例えば、ｗ［ν］＝ｓｉｎ（（ν＋０．５）π／Ｎ）によって与えられ得る。 In some exemplary embodiments, applying time-frequency conversion is performed by _{one frame of an input audio channel in} N in time domains (L time domain samples, eg, 256 time domain samples). May involve updating an input buffer of length N (eg, 512 time domain samples) (eg, a first-in first-out (FIFO) buffer). Here, the input buffer of a given input audio channel 1 ≦ i ≦ N _{in is used.}

It is expressed as. Time domain samples for multiple input audio channels,

It is expressed as. ν is an index indicating a time domain sample. (・) ^T indicates the transposed matrix. Then, the input buffer of a given input audio channel i can be given by the following equation.

Here, F indicates a frame index. For the first processing frame,

Is. Each of the N _{in input audio channels in the} input buffer can be windowed (eg, using a sinusoidal window). Time-frequency transforms such as the Discrete Fourier Transform (DFT) of length N (eg, N = 512 samples) are N _in input audio channels (inputs) in the input buffer (eg, windowed input buffer). Can be calculated for each of the audio signals). As a result, the frequency coefficient

Is obtained. Here, k indicates a frequency bin. _{For each frequency bin, an N in} dimensional vector having a complex frequency coefficient as its entry (element) can be obtained. For example, the frequency coefficient _{can be given by the following equation for 1 ≦ i ≦ N in} and 0 ≦ k ≦ K-1.

In the following description, the frequency index k may be omitted when the same processing is applied to all frequency bins. The window function can be given, for example, by w [ν] = sin ((ν + 0.5) π / N).

と表記され、次元Ｎ_inを有する（例えば、Ｎ_in個のエントリーを有する）。ダウンミックス行列は時間的に一定であるが、各周波数ビンｋは、個々の（特定の）Ｎ_out×Ｎ_inダウンミックス行列

に関連付けられ得る。すなわち、ダウンミックス行列Ｍ_DMXは、周波数ビンのインデックスに依存し得る。つまり、０≦ｋ≦Ｋ−１として、

によって与えられ得る。ダウンミックス行列を適用することにより、それぞれの周波数ビンにおける、複数の中間オーディオチャネル（中間ダウンミックスチャネル）の周波数係数によって形成される第２のアレイ（例えば、ベクトル）を取得することができる。 In step S2030, the downmix matrix is applied to a first array (eg, a vector) formed by the frequency coefficients of a plurality of input audio channels in each frequency bin (eg, for each frequency bin). In other words, the actual downmix can be performed (eg, in each frequency bin, i.e., frequency bin by frequency). In the notation adopted here, the first array of the kth frequency bin is

It is written as and has the dimension N _in (for example, it has N _in entries). The downmix matrix is temporally constant, but each frequency bin k is an individual (specific) N _out x N _in downmix matrix.

Can be associated with. That is, the downmix matrix M _DMX may depend on the index of the frequency bin. That is, with 0 ≦ k ≦ K-1,

Can be given by. By applying the downmix matrix, it is possible to obtain a second array (eg, a vector) formed by the frequency coefficients of a plurality of intermediate audio channels (intermediate downmix channels) in each frequency bin.

In the above and in the rest of this section, the proposed method is described as being performed on a frequency bin basis. However, the method generally involves at least one frequency bin (eg, a single frequency bin), all frequency bins within a group of frequency bins (eg, within a given audio band), or, in time-frequency conversion. It is understood that this can be done for all frequency bins.

The number of intermediate audio channels can be equal to the number of output audio channels N _out. The intermediate audio channel and the output audio channel can have a one-to-one relationship. The output audio channels can be obtained from the intermediate audio channels by adjusting the magnitude of their frequency coefficients.

と表記する。すると、第ｏの中間オーディオチャネルの第ｋの周波数ビンにおける周波数係数は、以下の式によって与えることができる。

ここで、

は、ダウンミックス行列

のエントリーである。ここで採用した表記では、第ｋの周波数ビンの第２のアレイは、

と表記され、次元Ｎ_out（すなわち、Ｎ_out個のエントリー）を有する。この第２のアレイは、ダウンミックス行列と第１のアレイとの行列乗算（例えば、

）によって取得されると言える。特に、ダウンミックス行列

のエントリーは実数であり得る。これに対して、

及び

は複素数であり得る。更に、周波数ビンにわたって一様であるダウンミックス行列については、前述の式におけるダウンミックス行列のインデックスｋを除去できることが理解される。 Here, the frequency coefficient of the intermediate audio channel of the o (1 ≦ o ≦ N _{out) in the kth frequency bin is set.}

Notated as. Then, the frequency coefficient in the kth frequency bin of the first intermediate audio channel can be given by the following equation.

here,

Is a downmix matrix

It is an entry of. In the notation adopted here, the second array of the kth frequency bin is

It is written as and has a dimension N _out (that is, N _out entries). This second array is a matrix multiplication of the downmix matrix and the first array (eg,

) Can be said to be acquired. Especially the downmix matrix

Entry can be real. On the contrary,

as well as

Can be a complex number. Further, for a downmix matrix that is uniform across frequency bins, it is understood that the index k of the downmix matrix in the above equation can be removed.

In order to reduce the audible artifacts that may occur in the intermediate audio channel (intermediate audio signal), the intermediate audio channel may be further processed by adaptive equalization to control the energy of the downmixed audio band. For this purpose, the format converter calculates a first energy index value and a second energy index value as described below.

は、（処理帯域への割り当て関数ｂａｎｄ（）がｂａｎｄ（ｋ）＝ｂとなる周波数ビンｋによって表現すると）以下の式によって与えられ得る。

In step S2040, the first energy index value of each intermediate audio channel (eg, the energy index value referred to in the claims) can be determined for each frequency bin. Here, the same energy index value may be assigned to a plurality of frequency bins. For example, frequency bins composed of the same processing band may have the same energy index value. For example, for each processing band 0 ≦ b ≦ B-1 (B ≦ N), the energy index value of the o-th intermediate audio channel of all frequency bins k in the b-th processing band.

Can be given by the following equation (expressed by the frequency bin k such that the allocation function band () to the processing band is band (k) = b).

According to this example, the first energy index value of a given intermediate audio channel in a given frequency bin is for all input audio channels and all frequency bins in the same processing band as the given frequency bin. It can depend on the sum of the absolute squares of the product of the frequency coefficients of the input audio channel and the corresponding entries of the downmix matrix over. In general, the first energy index value for a given intermediate audio channel in a given frequency bin spans all input audio channels and optionally all frequency bins in the same processing band as the given frequency bin. , The power of the absolute value of the frequency coefficient of the input audio channel (eg, 1st, 2nd, 3rd, etc.) and the power of the corresponding entry in the downmix matrix (eg, 1st, 2nd, 3rd, etc.) Can depend on the sum of the products of. In particular, the entries in the downmix matrix can be real, while the entries in the first array can be complex. Here, the "corresponding entry" of arrays (vectors, matrices) means entries that are multiplied by each other in normal matrix multiplication of these arrays.

は、以下の式によって与えることができる。

Further, a second energy index value can be obtained for each frequency bin. Again, the same second energy index value can be assigned to multiple frequency bins. For example, frequency bins composed of the same processing band may have the same second energy index value. For example, for each processing band 0 ≦ b ≦ B-1, the second energy index value of the o-th intermediate audio channel of all frequency bins k in the b-th processing band.

Can be given by the following equation.

Therefore, the second energy index value of a given intermediate audio channel and a given frequency bin is the frequency coefficient of a given intermediate audio channel over all frequency bins in the same processing band as the given frequency bin. It may depend on the sum of the absolute squares with each frequency bin. For example, the sum of the absolute squares of the sum of the products of the frequency coefficients of the input audio channels and the corresponding entries in the downmix matrix over all input audio channels, over all frequency bins in the same processing band as a given frequency bin. Can depend on.

The above calculation (determination) of the first energy index value and the second energy index value requires to obtain the frequency bins assigned to the same audio band and access those frequency bins. This can be done by referring to a lookup table that lists the audio bands to which each frequency bin belongs for each frequency bin. However, referencing such a table for obtaining the first energy index value and the second energy index value is expensive because it requires indirect addressing multiple times. Can be. In order to solve this problem, another lookup table may be provided for each audio band b, which lists the number Nb (b) of frequency bins belonging to each audio band b. Since each audio band contains multiple adjacent frequency bins with no overlap or gap between adjacent audio bands, the information contained in this separate lookup table is a first energy index value and a second energy index. Enough to find the value. When calculating the first energy index value and the second energy index value starting from the first audio band, the first Nb (0) terms in the sum of k items are summed to obtain the first audio. It can be assigned to the first energy index value or the second energy index value of the band (each intermediate audio channel), and the same applies to the second audio band and thereafter. In general, starting from band index b = 0, band index b is iteratively incremented one at a time, and Nb (b) in the sum of k pieces in equations [4] and / or [5]. The consecutive terms can be summed and assigned to each energy index value in the bth audio band (each intermediate audio channel). That is, according to the number Nb obtained for the audio band (for example, estimated from another look-up table), the consecutive terms in the sum over k are summed and assigned to each energy index value of each audio band. Can be done.

ここで、

は、第ｂのオーディオ帯域内の周波数ビン（すなわち、ｂａｎｄ（ｋ）＝ｂを有する周波数ビン）にわたって一定であるダウンミックス行列である。同様の簡略化は、式［５］にも適用することができる。 _{In the case of the downmix matrix M DMX} which is constant over a plurality of frequency bins or audio bands, the downmix matrix in the above equations [4] and [5] can be excluded from the calculation for the frequency bin or audio band in question. .. Thereby, the calculation cost can be further reduced. For example, the calculation of equation [4] can be simplified to the following equation.

here,

Is a downmix matrix that is constant across frequency bins within the bth audio band (ie, frequency bins having band (k) = b). Similar simplifications can be applied to equation [5].

を求めることができる。この調整ファクターを用いると、（第ｏの中間オーディオチャネルに対応する）第ｏの出力オーディオチャネル及び第ｋの周波数ビンのエネルギー調整された周波数係数は、以下の式によって与えられ得る。

In step S2050, the energy of the intermediate downmix signal can be adjusted. That is, the magnitude of each frequency coefficient can be adjusted for each intermediate audio channel and each frequency bin. This adjustment can be made based on one or both of the first energy index value and the second energy index value of each intermediate audio channel and each frequency bin. Generally, it can be said that this adjustment is made based on the energy index value of each intermediate audio channel and each frequency bin. Adjustment factor based on one or both of the first energy index value and the second energy index value of each intermediate audio channel and each frequency bin.

Can be sought. Using this tuning factor, the energy-tuned frequency coefficients of the o-th output audio channel (corresponding to the o-th intermediate audio channel) and the k-th frequency bin can be given by the following equation.

は、第１のエネルギー指標値

と、第２のエネルギー指標値及び第２のエネルギー指標値

との関数として求めることができる。ここで、ｂａｎｄ（ｋ）＝ｂである。第１のエネルギー指標値は、現在のフレーム及び以前のフレームの第１のエネルギー指標値の加重和を求めることによって平滑化関数にかけてもよい。同様に、第２のエネルギー指標値も、現在のフレーム及び以前のフレームの第２のエネルギー指標値の加重和を求めることによって平滑化関数にかけてもよい。例えば、第ｏの中間オーディオチャネル及び第ｋの周波数ビンの調整ファクター

は、以下の式によって与えられ得る。

ここで、ｂａｎｄ（ｋ）＝ｂについて、

である。加重和

及び

は、以下の式によって与えられ得る。

ここで、αは、フィルターパラメーター（例えば、α＝０．０４３５）であり、εは、０による除算を防止するための小さな数値定数（例えば、ε＝１０^-35）であり、Ｆは、現在のフレームを示す。特に、

及び

は、最初の処理フレームについて０に初期化され得る。 Adjustment factor of the oth intermediate audio channel and the kth frequency bin

Is the first energy index value

And the second energy index value and the second energy index value

It can be obtained as a function of. Here, band (k) = b. The first energy index value may be subjected to a smoothing function by obtaining the weighted sum of the first energy index values of the current frame and the previous frame. Similarly, the second energy index value may also be subjected to a smoothing function by obtaining the weighted sum of the second energy index values of the current frame and the previous frame. For example, the adjustment factor of the intermediate audio channel of o and the frequency bin of k.

Can be given by the following equation.

Here, about band (k) = b

Is. Weighted sum

as well as

Can be given by the following equation.

Where α is a filter parameter (eg α = 0.0435), ε is a small numerical constant (eg ε = 10 ^-35 ) to prevent division by 0, and F is currently Indicates the frame of. especially,

as well as

Can be initialized to 0 for the first processing frame.

In step S2060, frequency time conversion (reverse time frequency conversion, eg, inverse conversion of time frequency conversion in step S2020) can be applied to the energy adjusted frequency coefficient generated in step S2050. This reverse time frequency conversion can be performed for each processed frame (eg, 256 samples). The energy-tuned downmix signal can be transformed into the time domain by applying an inverse DFT, eg, an inverse FFT. This may further involve windowing and duplicate updates. As a result, step S2060 can generate L time domain output samples per output signal channel.

ここで、

である。ここで、（・）^*は複素共役を示す。この場合、出力オーディオチャネルの時間領域サンプルは、０≦ν＜Ｌについて、以下の式によって与えられ得る。

ここで、

は、最初の処理フレームについては０に初期化され得る。 For the current frame F, the process of step S2060 can be given for 0 ≦ ν <N by the following equation.

here,

Is. Here, (・) ^{* Indicates} the complex conjugate. In this case, the time domain sample of the output audio channel can be given by the following equation for 0 ≦ ν <L.

here,

Can be initialized to 0 for the first processing frame.

It is understood that the above method can be repeated for subsequent frames of the time domain sample. In this case, it is not necessary to acquire the downmix matrix for each frame, and step S2010 can be omitted (for example, for subsequent frames).

Next, refer to FIG. A flowchart illustrating an exemplary example of the details of the steps in the process of FIG. 2 is provided. That is, the details of step S2030 will be described with reference to the flowchart shown in FIG. More specifically, step S2030, which applies the downmix matrix to the first array, may include some or all of steps S301-10.3030.

In step S3010, a third array (eg, a vector, i.e., a linear array) M containing only non-zero entries (elements) of the _{downmix matrix M DMX can be determined.} Non-zero entries in the downmix matrix can be extracted in a predetermined scan order and arranged in a third array in the order in which they appear in the scan order in the downmix matrix. In other words, the nth entry in the third array M represents the nth nonzero entry in the scan order in the _{downmix matrix M DMX.} The scanning order may be, for example, the row direction or the column direction. If the downmix matrix has dimensions N _out x N _in , the scan order can be in the row direction. The embodiments of the present disclosure provided in the rest of this document presuppose a row-wise scan order. However, these embodiments should not be construed as being limited to row-wise scanning order. It is understood that by swapping and reversing the columns and rows, these embodiments can also be easily adapted in the case of columnwise scanning order.

Next, refer to FIG. An example of a downmix matrix according to an embodiment of the present disclosure and an array derived from the downmix matrix is schematically shown. That is, an example of the downmix matrix M _DMX and the corresponding third array M is shown in FIG. FIG. 4 also shows an example of the fourth array s and the fifth array r. In this example, the downmix matrix M _DMX has 6 non-zero entries, i.e. 1, 4, 2, 6, 3, 5 according to the row scanning order. Thus, for row scan order, the third array M has 6 entries and ^{is given by (M) T} = [1,4,2,6,3,5].

In step S3020, a fourth array (eg, a vector, i.e., a linear array) s can be determined. Here, the entry of the fourth array s may correspond to the entry of the third array M. Each entry in the fourth array s may indicate a position (eg, a linear position) that each corresponding entry in the third array M has in the downmix matrix M _{DMX by scan order.} In other words, the nth entry in the 4th array s is the nth entry in the 3rd array M (eg, _{the nth non-zero entry in the downmix matrix M DMX} by scan order) is the downmix matrix. Represents the position held by the scanning order in M _DMX. In some exemplary embodiments, in the case of row-wise scan order, the (linear) position by scan order can be given by method of row length (eg, depending on the remainder of the division by row length). obtain). For example, it may indicate a sequence of downmix matrices in which each entry in the third array is located. Alternatively, in the case of a column-wise scan order, the (linear) position by the scan order can be given the column height modulo (ie, can rely on the remainder of the column height division). For example, it may indicate the row of the downmix matrix in which each entry in the third array is located. For a _{downmix matrix of dimensions N out} x N _in , the row length is _{given by N in} and the column height is given by _{N out.}

In another embodiment, the entry of the fourth array s may indicate an offset from the first position due to the scan order in the downmix matrix. In yet another embodiment, the entry of the fourth array s is a relative offset (ie,) between the position of one non-zero entry in the _{downmix matrix M DMX and the position of the next non-zero entry due to the scan order.} , Offset from the position of the entry immediately preceding each entry in the third array in the downmix matrix by scan order). In yet another embodiment, the entries in the fourth array s, for each corresponding non-zero entry in the third array M, from or correspond to the position of their respective immediately preceding entry in their respective rows. If the non-zero entry is the first entry in that row, it may indicate the relative offset of the position of the corresponding non-zero entry in that row from the first position in each row. The above-described embodiment of the fourth array s is applied in the case of a row-direction scan order, but can be easily adapted to a column-direction scan order by exchanging rows and columns.

In the example of FIG. 4, the fourth array s ^{can be given by (s) T} = [1,4,2,6,3,5], the entry of which is each entry of the third array. It is possible to show the columns in the downmix matrix that are located (ie, were located prior to extraction into the third array M). Alternatively, if the entry of the fourth array s indicates the respective offset from the first position by the (row direction) scan order in the _{downmix matrix M DMX, the fourth array s is (s).} ^{Given by T} = [0,3,7,11,14,16], the entry of the fourth array s is the position and (row direction) of some non-zero entry in the _{downmix matrix M DMX.} If the fourth array s indicates a relative offset from the position of the next non-zero entry in the scan order, then the fourth array s is by (s) ^T = [0,3,4,4,3,2]. Will be given. If the entries in the fourth array s indicate a relative offset from the position of the previous entry or the first entry in each row, then the fourth array s is (s) ^T = [0,3. , 1, 4, 2, 2].

Although not shown in FIG. 3, step S2030 may further include a step of finding a fifth array r. The entries in this fifth array r indicate the number of nonzero entries per row or column in the _{downmix matrix M DMX.} Here, whether the entries of the fifth array r indicate the number of non-zero entries per row or the number of non-zero entries per column depends on the scanning order. That is, in the row scan order, the entries in the fifth array r _{indicate the number of non-zero entries in the downmix matrix M DMX} per row, and in the column scan order, the entries in the fifth array r. Indicates the number of non-zero entries in the downmix matrix M _{DMX per column.} In the example of FIG. 4, the fifth array is ^{given by (r) T} = [2,2,2]. For columnar scan order, the fifth array is ^{given by (r) T} = [1,1,1,1,1,1]. Needless to say, the entries in the fifth array r all have the same degree as the rows or columns of the downmix matrix have in the downmix matrix. That is, the mth entry of the fifth array r indicates the number of nonzero coefficients in the mth row (or column) of the _{downmix matrix M DMX.} Here, m ranges from 1 to the number of rows (or columns) in the _{downmix matrix M DMX.} For example, the first entry in the fifth array r indicates the number of non-zero coefficients in the first row (or column) of the _{downmix matrix M DMX, and so on for the other entries.}

は、第６のアレイ（線形アレイ、例えば、ベクトル）

にまとめることができ、所与の周波数ビンｋの第２のエネルギー指標値

は、第７のアレイ（線形アレイ、例えば、ベクトル）

にまとめることができる。 In addition, the first energy index value for a given frequency bin k

Is a sixth array (linear array, eg vector)

Can be summarized in, the second energy index value of a given frequency bin k

Is a seventh array (linear array, eg vector)

Can be summarized in.

が周波数ビンに依存する場合、第３のアレイＭ^k、第４のアレイｓ^k、及び第５のアレイｒ^kも、それらのそれぞれの周波数ビンによってインデックス付けすることができる。問題とするダウンミックス行列のそれぞれの第３のアレイ、第４のアレイ又は第５のアレイが既に取得済みである場合、前述のステップのいずれかを省略できることが更に理解される。すなわち、取得されたダウンミックス行列Ｍ_DMX又は

ごとに、第３のアレイ、第４のアレイ及び第５のアレイは、フォーマット変換全体を通して（例えば、フォーマット変換器モジュールの初期処理の際）１度求めるだけで十分であり、スタティックメモリに記憶することができ、フォーマット変換器モジュールが起動されると、１フレームに１回参照され得る。換言すれば、取得されたダウンミックス行列ごとに、スパース行列表現（第３のアレイ及び第４のアレイを含むか又はこれらのアレイによって表される）は、１度だけ決定され得る。そして、フォーマット変換器モジュールの現在のセッション全体を通して参照され得る。 It is understood that the determination of the third array, the fourth array and the fifth array can be made in any order. If the downmix matrix M _DMX is frequency bin dependent, the steps described above can be performed for each frequency bin, and if the downmix matrix M _DMX is the same for each group of frequency bins. , It is further understood that it can be performed for each group of frequency bins (processing band). Downmix matrix

If There frequency dependent bins, the third array M ^k, fourth array s ^k, and also the fifth array r ^k, can be indexed by their respective frequency bin. It is further understood that any of the steps described above can be omitted if each third array, fourth array or fifth array of the downmix matrix in question has already been acquired. That is, the acquired downmix matrix M _DMX or

Each time, the third array, the fourth array, and the fifth array need only be obtained once throughout the format conversion (for example, during the initial processing of the format converter module) and are stored in static memory. It can be referenced once per frame once the format converter module is activated. In other words, for each acquired downmix matrix, the sparse matrix representation (including or represented by a third array and a fourth array) can be determined only once. It can then be referenced throughout the current session of the format converter module.

See FIG. 3 again. In step S3030, the entry of the third array M can be multiplied by the corresponding entry of the first array (eg, for each frequency bin). These multiplications can actually determine the frequency factor of the intermediate audio channel (eg, actual downmixing), or at least part of this determination. That is, for each frequency bin, this determination multiplies the entry of the third array M (eg, of each frequency bin) with the corresponding entry of the first array (eg, of each frequency bin). Can include. Here, if the normal matrix multiplication of the first array and the downmix matrix is multiplied by _{the corresponding non-zero entries of the downmix matrix M DMX} , then the entries of the first array are of the third array. It can be said that it corresponds to the entry. The corresponding entry in the first array can be determined based on the fourth array (eg, based on the entry in the fourth array). The entries in the fourth array may indicate the positions that each entry in the third array M _{had in the downmix matrix M DMX.} Thereby, for each entry of the third array M, the entry of the first array to be multiplied by the entry of the third array M in the normal matrix multiplication of the first array and the downmix matrix M _DMX. Can be determined.

は、次元Ｎ_outを有する（例えば、それぞれがＮ_out個のエントリーを有する）。 The appropriate product of entries can be summed for each entry in the second array (in each frequency bin). The assignment of the sum of products (one or more) to the entry of the second entry can be made based on the fifth array r. Each entry in the fifth array r may indicate how many products need to be summed up and assigned to each entry in the second array for each entry in the second array. If starting from the first entry in the second array, the sum of the calculated products is assigned consecutively to the entries in the second array in the order in which they were calculated, thereby that of the second array. All entries are available. Here, the number of terms in each sum is determined by each entry in the fifth array r. Therefore, generally speaking, applying the downmix matrix to the first array is, for each frequency bin, one of the entries in the third array of each frequency bin and the corresponding entries in the first array. It can be said to further include assigning the sum of one or more products to the entries of the second array based on the entries of the fifth array. Here, the fifth array r and the second array

_Has dimensions N out (eg, each has N _out entries).

In some exemplary embodiments, applying the downmix matrix to the first array continuously increments the entry index to the third array one at a time, and such an entry index. Each may further include finding the product of the entries in the third array, indicated by the respective entry indexes in each frequency bin, and the corresponding entries in the first array. The corresponding entry in the first array can be found by reference to the fourth array. For any given entry in the third array M, the position of the entry in the downmix matrix can be determined from the corresponding entry in the fourth array (the jth entry in the fourth array is Corresponds to the jth entry in the third array, where j is an integer between 1 and the number of non-zero entries in _{M DMX).} For example, the column number of the entry in question can be determined from each entry in the fourth array s. The entry in the first array corresponding to the entry in question in the third array M can then be determined as the entry in the first array having the determined column number as the entry index in the first array. .. Applying the downmix matrix to the first array may further include assigning the sum of one or more of these products sequentially to the entries of the second array. This allocation may be based on the entry of the fifth array. That is, the number of products in the sum assigned to the entries in the second array can be determined by the entries in the fifth array that have the same entry index as the entry in question in the second array. If the product of the entries in the third array and the corresponding entries in the first array is continuously determined, then these calculated products are placed in the entries in the second array and in their order in the second array. It can be added continuously (assuming that the entries in the second array are pre-initialized to 0). Here, the number of continuously obtained products to be added to each entry in the second array can be determined from the corresponding entry in the fifth array. In particular, these steps can be performed on a frequency bin basis.

FIG. 5 shows an example according to the embodiment of the present disclosure in which the frequency coefficient of the intermediate audio channel is actually obtained (for example, actually down-mixed) in step S3030 in FIG. 3 in accordance with the above. That is, the downmixing may include some or all of steps S501-1090. In particular, the example of FIG. 5 relates to the case of row-direction scanning order, but can be easily adapted as in the case of column-direction scanning order. Also, the entries in the first and second arrays are typically complex numbers, and the operations described below need to take into account both the real and imaginary parts.

The method starts from step S5000. In step S5010, the row index can be initialized to row = 1. That is, the first entry of the second array is first found (calculated). In step S5020, the entry index (for the third and fourth arrays) can be initialized to j = 1.

In step S5030, the number of nonzero entries in the row of the _{downmix matrix M DMX} indicated by the row index row can be determined. This determination can be made by reference to the entry of the fifth array r indicated by the row index row (eg, by reference to r _row ). The entry in the fifth array r represented by the row index row directly indicates the number of non-zero coefficients in the row of the _{downmix matrix M DMX represented by the row index row.}

If the rows of the downmix matrix M _DMX indicated by the row index row do not contain non-zero entries, the method assigns a (complex) value of 0 to the entries in the second array indicated by the row index row, step S5040. ~ S5070 can be skipped.

In step S5040, the jth entry of the third array M and the corresponding entry of the first array can be multiplied together to determine their product. The entry of the first array corresponding to the jth entry of the third array M can be obtained by referring to the jth entry of the fourth array s. The jth entry of the fourth array s can indicate the column number that _{the jth entry of the third array M had in the downmix matrix M DMX.} This column number (ie, the jth entry of the fourth array s) directly indicates the location of the corresponding entry in question in the first array (ie, in the case of column number m, of the first array). The third entry corresponds to the problematic entry of the third array M). In particular, _{the number of columns in the downmix matrix M DMX} is equal to the number of entries in the first array.

In step S5050, it can be determined whether the number of products obtained is less than the number of non-zero entries in the rows of the _{downmix matrix M DMX} indicated by the row index row. That is, the number of products already calculated for the current value of the row index row is non-zero in _{the current row of the downmix matrix M DMX} (eg, the row of the downmix matrix M _{DMX indicated by the row index row).} Can be compared to the number of entries in. If the number of products is less than the above number of non-zero entries found in step S5030, then in step S5060 the entry index j may be incremented by one. The method can then be returned to step S5040.

に割り当てられ得る。 On the other hand, if the number of products is not less than the above number of non-zero entries found in step S5030, the method can proceed to step S5070. In step S5070, the products calculated for the current value of row index row can be summed. The resulting sum can be assigned to the entries in the second array indicated by the row index row. Alternatively, assuming that the problematic entry in the second array has been pre-initialized to 0, the product obtained for the current value of the row index row will be the problematic entry in the second array. Can be added continuously. In the example of FIG. 4, for low = 1, _{the product of r 1} = 2 (M ₁ · x _S1 and M ₂ · x _S2 ).
Can be sought. The sum of them is the first entry in the second array

Can be assigned to.

In step S5080, it can be determined whether all the rows have been processed, that is, whether the row index row has reached the _{value N out.} If all rows have been processed, the method ends in step S5100. On the other hand, if all rows have not been processed, the row index row may be incremented by one in step S5090. Then, this method returns to step S5030.

In particular, steps S501 to S5090 in the example of FIG. 5 can be performed for each frequency bin.

Next, exemplary pseudo-code (written in the C programming language) that implements the above steps is provided below.
// of z, M, x, and s, z
// Assuming proper type definition, memory allocation, and initialization.
// Representation of sparse matrix
sp = & s [0]; // Pointer increment (4th array)
Mp = & M [0]; // Nonzero sparse matrix (third array)
rp = & r [0]; // Number of nonzeros in each row (fifth array)

// Input and output

xp = & x [0]; // Input (first array)
zp = & z [0]; // Output (second array)

// Vector multiplication of sparse matrix: z = Mdmx x

for (row = 0; row <Nout; row ++) [// Loop over M rows
xp = & x [0]; // address of x
* zp = 0;
for (j = 0; j <* rp ++; j ++) [// Loop across non-zero lines
xp = * sp ++; // Jump into x
* zp + = (* Mp ++) * (* xp);
]
* zp ++;
]

In the above exemplary code, if the entry of the fourth array s indicates a relative offset between positions, then the line
xp = * sp ++; // Jumps within x should be replaced by, for example, the following line.
xp + = * sp ++; // Jump into x

In particular, this pseudocode assumes that x and z are real numbers, but this is just an example. It is understood that x and z can typically have complex entries. It is further understood that this exemplary code can be easily adapted to take into account the complex entries of x and z.

In another embodiment, performing the actual downmix may include determining _{for each entry in the downmix matrix M DMX} whether or not each entry is zero. The actual multiplication of the entries and the allocation of the second array to each entry are made only if the entry in question in the downmix matrix turns out to be non-zero. An exemplary code written in a C programming language that implements the above approach is given below.
// Type definition.
float Mdmx [] [], x [], z [];
int S [] []; // Non-zero entry in _{M dmx}
// Binary matrix to mark
int I, j;

// Here, N _out x N _in matrix M _DMX and S, and N _in vector x
// Assuming proper memory allocation and initialization. The matrix S is
// If Mdmx [i] [j] is zero, then S [i] [j] is zero and
// Otherwise, it will be initialized to be non-zero.

// run z = Mdmx x

for (i = 0; i <Nout; i ++) [
z [i] = 0.0;
for (j = 0; j <Nin; j ++) [
if (S [i] [j] == 0) [
z [i] + = Mdmx [i] [j] * x [j];
]
]
]

Next, the details of step S2040 in FIG. 2 will be described with reference to the exemplary flowchart shown in FIG. That is, finding the first energy index value for each intermediate audio channel may include some or all of steps S601-10.6030.

In step S6010, if the third array M has not been previously determined and has not already been acquired in any other way, the third array M may be determined. This determination can be performed in the same manner as in step S3010 described above, and the matters described with respect to step S3010 also apply in this step. In step S6020, if the fourth array s has not been previously determined and is not otherwise already earned, the fourth array s may be determined. This determination can be performed in the same manner as in step S3020 described above, and the matters described with respect to step S3020 also apply in this step. Although not shown in FIG. 6, step S2040 is a fifth array r if the fifth array r has not been previously determined and is not otherwise already earned. It may further include determining the array r. This determination can be performed in the same manner as in step S2030 described above with respect to FIG. 3, and the matters described with respect to step S2030 apply similarly to this step.

In step S6030, the power of the entries in the third array M (eg, squared) is multiplied (eg, for each frequency bin) by the power of the absolute values of the corresponding entries in the first array (eg, absolute squares). Can be done. These multiplications can actually determine the first energy index value of the intermediate audio channel, or at least part of this determination. That is, for each frequency bin, this determination is a power (eg, square) of the entries of the third array M (eg, for each frequency bin) and a first array (eg, for each frequency bin). Can include multiplying the power of the absolute value of the corresponding entry of (eg, absolute square) to obtain the product of each of these entries. See their detailed description above for the concepts of the corresponding entries in the first array and the third array.

は、次元Ｎ_outを有する（例えば、それぞれＮ_out個のエントリーを有する）。 The proper product of the entries (the square of the entries in the third array M multiplied by the absolute squares of the entries in the first array) is summed for each entry in the sixth array (in each frequency bin). Can be done. Allocation of the sum of (one or more) products to the entries of the sixth array can be made based on the fifth array r. Each entry in the fifth array r may indicate how many products need to be summed up and assigned to each entry in the sixth array for each entry in the sixth array. When starting from the first entry of the sixth array, the sum of the calculated products is assigned consecutively to the entries of the sixth array in the order in which they were calculated, thereby that of the sixth array. All entries are available. As a result, the first energy index value of all the intermediate audio channels is obtained. Here, the number of terms in each sum is determined by each entry in the fifth array r. Generally speaking, finding the first energy index value is, for each frequency bin, the square of the entry in the third array of each frequency bin and the absolute square of the corresponding entry in the first array. It may further include assigning the sum of one or more products to the entries in the sixth array based on the entries in the fifth array. Here, the fifth array r and the sixth array

_Has dimensions N out (eg, each has N _out entries).

In some exemplary embodiments, the step of finding the first energy index value is to continuously increment the entry index for the sixth array one at a time, and for each such entry index, respectively. It may further include finding the product of the square of the entries of the third array indicated by each entry index of the frequency bin of the first array with the absolute squares of the corresponding entries of the first array. The corresponding entry in the first array can be determined by reference to the fourth array, as described above. For example, the column number of the entry in question can be determined by referring to the fourth array s. The entry in the first array corresponding to the entry in question in the third array can be sought as an entry in the first array having the sought column number as the entry index. Finding the first energy index value may further include assigning the sum of one or more of these products sequentially to the entries in the sixth array. This allocation may be based on the entry of the fifth array. That is, the number of products in the sum assigned to the entries in the sixth array can be determined by the entries in the fifth array that have the same entry index as the entry in question in the sixth array. If the product of the squares of the entries in the third array and the absolute squares of the corresponding entries in the first array is continuously determined, then these calculated products are in the entry of the sixth array, the sixth. Can be added sequentially in those order within the array of (assuming that the entries in the sixth array are pre-initialized to 0). The number of continuously determined products added to each entry in the sixth array can be determined from the corresponding entry in the fifth array. In particular, these steps can be performed on a frequency bin basis. A sixth array of frequency bins within this given processing band may be summed (according to the rules of vector addition) to obtain the first energy index value for a given processing band. It can be said that the first energy index value associated with a given frequency band is the first energy index value of each of the frequency bins in this processing band.

FIG. 7 shows an example according to the embodiment of the present disclosure for obtaining the first energy index value in step S6030 in FIG. That is, obtaining the first energy index value may include some or all of steps S701 to S7900. In particular, the example of FIG. 7 relates to the case of row-direction scanning order, but can be easily adapted as in the case of column-direction scanning order.

This method starts from step S7000. Step S7010, step S7020, and step S7030 can be the same as step S5010, step S5020, and step S5030 in FIG. 5, respectively. The repetitive explanations thereof will be omitted.

If the rows of the downmix matrix M _DMX indicated by the row index row do not contain non-zero entries, the method assigns a (complex) value of 0 to the entries in the sixth array indicated by the row index row, step S7040. ~ S7070 can be skipped.

Step S7040 can be the same as step S5040 in FIG. 5, except for the following points. Instead of the jth entry of the third array M and the corresponding entry of the first array, here the power of the jth entry of the third array M (eg, squared) and of the first array The powers of the absolute values of the corresponding entries (eg, absolute squares) can be multiplied by each other to find their product.

Step S7050 and step S7060 can be the same as step S5050 and step S5060 in FIG. 5, respectively. The repetitive explanations thereof will be omitted.

に割り当てられ得る。 In step S7070, the sum obtained from summing the products obtained for the current value of row index row may be assigned to the entry in the sixth array indicated by row index row. Alternatively, assuming that the problematic entry in the sixth array has been pre-initialized to 0, the product obtained for the current value of the row index row will be the problematic entry in the sixth array. Can be added continuously. In the example of FIG. 4, for low = 1, _{the product of r 1} = 2 ((M ₁ ) ² · | x _S1 | ² and (M ₂ ) ² · | x _S2 | ² )
Can be sought. The sum of them is the first entry in the sixth array.

Can be assigned to.

Again, step S7080 and step S7090 can be identical to step S5080 and step S5090 in FIG. 5, respectively. The repetitive explanations thereof will be omitted.

In particular, the processes of FIGS. 5 and 7 can be integrated into a single process to reduce the computational cost of looping the entries of the first and third arrays and the computational cost of addressing. The new integrated process (method) of performing the actual downmix and determining the first energy index value may include the same steps as the method in FIG. 5, except for step S5040 and step S5070. These steps are replaced by new steps S8040 and S8070, respectively. In particular, this integration means that steps S2030 and S2040 can be performed together.

In step S8040, the integrated processing of steps S5040 and S7040 can be performed. That is, the first product of the jth entry of the third array and the corresponding entry of the first array is obtained (as described with respect to step S5040 above), and at the same time the jth of the third array. A second product of the power of the entry (eg, square) and the power of the absolute value of the corresponding entry in the first array (eg, absolute square) can be obtained (as described with respect to step S7040 above). Note that this requires an exchange of the order of the two addition processes in equation [4].

Next, in step S8070, the integrated processing of step S5070 and step S7070 may be executed. That is, the first product obtained for the current value of the row index row is summed and the resulting sum can be assigned to the index of the second array indicated by the row index row. Alternatively, assuming that the entry in question in the second array has been pre-initialized to 0, the first product found for the current value of the row index row is the problem in the second array. Can be continuously added to the entries to be made. In addition, the second product calculated for the current value of the row index row is summed and the resulting sum can be assigned to the index of the sixth array indicated by the row index row. Alternatively, assuming that the entry in question in the sixth array has been pre-initialized to 0, the second product found for the current value of the row index row is the problem in the sixth array. Can be continuously added to the entries to be made.

The above processing can be executed for each frequency bin. Each sixth array of frequency bins within a given processing band may be summed (according to the rules of vector addition) to obtain a first energy index value per processing band rather than per frequency bin. ..

It is understood that the method of downmixing a plurality of input audio channels according to the present proposal can be carried out by a device, for example, a format converter (format converter block) of an audio decoder or generally an audio decoder. .. Such a device (eg, a format converter or audio decoder) may include a respective unit configured to perform each of the above-mentioned steps. An example of such a device 8000 is schematically shown in FIG. For example, such a device 8000 is configured to perform a frequency coefficient acquisition unit 8010 (eg, a receiving unit or a time-frequency conversion unit) configured to perform the aforementioned step S2020, the aforementioned step S2010. The downmix matrix acquisition unit 8020, the downmixing unit 8030 configured to perform the above-mentioned step S2030, the energy calculation unit 8040 configured to execute the above-mentioned step S2040, and the above-mentioned step S2050 to be executed. It may include a configured tuning unit 8050 and / or a reverse time frequency conversion unit 8060 configured to perform step S2060 described above. Each unit of such a device is embodied by a processor 8100 of a computing device configured to perform the processing performed by that unit, i.e., configured to perform each of the steps described above. It is better understood to get.

In the above description of the embodiments of the present disclosure, the row index row and the entry index j are initialized to 1, and the first index in any given array has a value of 1, i.e., an index value of 1 is the array in question. The first entry of. However, this disclosure should not be construed as being limited to such choices. Instead of the above, these indexes may have an index value of 0. Therefore, it can be said that the value 1 selected above exemplifies the smallest possible index value.

The term "circuitry" as used in this application is as follows: (a) implemented in a hardware-only circuit (analog circuit mechanism and / or implemented in a digital circuit mechanism only). Etc.) and (b) (if applicable) (i) a combination of processors (s) or (ii) working together to cause a device such as a mobile phone or server to perform various functions. Circuits and software (and / or firmware) such as part of a microprocessor (s) / software (including a digital signal processor (s)), software, and memory (s). And (c) microprocessors (s) or microprocessors (s) that require software or firmware to operate even if the software or hardware does not physically exist. Refers to all of the circuit such as a part of). This definition of "circuit mechanism" applies to all uses of this term in this application, including all claims. As a further example, the term "circuit mechanism" used in this application is implemented solely with a processor (or a plurality of processors) or a portion of a processor and its (or their) attached software and / or firmware. Also includes. The term "circuit mechanism" is used, for example, as a baseband integrated circuit or application processor integrated circuit for a mobile phone, or a server, cellular network device, or other network device, where applicable to a particular claim element. (For example, smartphones, cellular phones, personal digital assistants (PDAs), wireless modems, wireless communication devices, laptops, netbooks, tablets or others capable of communicating with FirstNet, E-UTRAN, UTRAN or GERAN compatible devices. Also includes similar integrated circuits in user devices (UEs) such as any device in the world.

It should be noted that the specification and drawings merely illustrate the principles of the methods and devices according to the present proposal. Therefore, a person skilled in the art can embody the principle of the present invention and devise various configurations included in the purpose and scope thereof, although not explicitly explained or illustrated in the present specification. Will be understood. Moreover, all the examples detailed herein are primarily intended to help the reader understand the principles of the methods and devices proposed herein, as well as the concepts by which the inventors contribute to advancing this technique. It is solely intended for educational purposes and should not be construed as being limited to such particularly detailed examples and situations. Moreover, all descriptions herein detailing the principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to include those equivalent.

The methods and devices described in this document can be implemented as software, firmware and / or hardware. Certain components can be implemented, for example, as software running on a digital signal processor or microprocessor. Other components can be implemented, for example, as hardware and / or application-specific integrated circuits. The signals generated by the methods and devices described can be stored in media such as random access memory or optical storage media. These signals can be transferred over networks such as wireless networks, satellite networks, wireless networks or wireline networks (eg, the Internet and / or IP Multimedia Core Network Subsystem (IMS)).

Various changes and modifications to the above-exemplified embodiments can be made apparent to those skilled in the art by considering the above description when the above description is read with the accompanying drawings. All changes are included within the scope of the non-limiting exemplary embodiments. Moreover, other exemplary embodiments of the categories mentioned herein will come to mind to those skilled in the art in which these embodiments relate, benefiting from the teachings presented in the above description and drawings. Will.

Therefore, the present invention can be embodied in any of the forms described herein. For example, the exemplary embodiments (A-EEE) listed below describe some structures, features, and functions of some aspects of the invention.

A-EEE1. A method of downmixing multiple input audio channels,
Acquiring a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels,
For at least one frequency bin, a downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin, and multiple intermediate audio channels for each frequency bin are applied. To obtain a second array formed by the frequency coefficients of
Including
The method is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Including
Applying the downmix matrix to the first array
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Including
The method in which the corresponding entry of the first array is determined based on the fourth array.

A-EEE2. Obtaining the energy index value of each intermediate audio channel for each intermediate audio channel for at least one frequency bin.
Including
Obtaining the energy index value is to obtain the energy index value for the at least one frequency bin.
Multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array.
Including
The corresponding entry in the first array is determined based on the fourth array, the method according to A-EEE1.

A-EEE3. The third array is any of the preceding A-EEEs, which is a linear array containing the non-zero entries of the downmix matrix in their order of appearance in a predetermined scan order across the entries of the downmix matrix. One way.

A-EEE4. A method according to any one of the preceding A-EEEs, wherein each entry in the fourth array indicates the position of each entry in the third array in the downmix matrix in said scan order.

A-EEE5. Each entry in the fourth array, in the case of row-wise scan order, indicates the column of the downmix matrix in which each entry in the third array is located in the downmix matrix and is in column-wise scan order. In the case of a method according to any one of the preceding A-EEEs, where each entry in the third array indicates a row of the downmix matrix located in the downmix matrix.

A-EEE6. Further including finding a fifth array,
The entries in the fifth array indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order and the downmix matrix in the case of column scan order. A method according to either A-EEE3 or one of the preceding A-EEEs subordinate to A-EEE3, which indicates the number of non-zero entries for each column of.

A-EEE7. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assigned to the entry in the array of
The method according to A-EEE6, which is further accompanied by.

A-EEE8. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The entry index for the third array is continuously incremented one at a time, and for each entry index, the entry of the third array indicated by the entry index of each frequency bin and the entry of the third array. Finding the product of the corresponding entry of the first array and
Sequentially assigning the sum of one or more of the products to the entry in the second array,
The allocation is based on the entry of the fifth array, according to A-EEE5 or 6.

A-EEE9. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
For each entry index for the third array, the square of the entry of the third array indicated by the entry index of each frequency bin and the absolute square of the corresponding entry of the first array. To find the product of and as the second product,
The sum of one or more of these second products is sequentially assigned to the entry in the sixth array of each frequency bin, and
Including
The allocation is based on the entry in the fifth array.
The method according to A-EEE8, wherein the sum of the sixth array over the frequency bins in a given processing band gives the energy index value of the frequency bins in the given processing band.

A-EEE10. Finding the frequency bins in the given processing band involves referencing a look-up table, which is an entry indicating the number of frequency bins in each processing band for each processing band. The method according to A-EEE9, which comprises.

A-EEE11. A device that downmixes a plurality of input audio channels, a frequency coefficient acquisition unit configured to acquire a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels, and a frequency coefficient acquisition unit.
With a downmix matrix acquisition unit configured to acquire a downmix matrix,
For at least one frequency bin, the downmix matrix is applied to a first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin to provide multiple intermediate audio in each frequency bin. A downmixing unit configured to acquire a second array formed by said frequency coefficients of the channel, and
With
The down mixing unit is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Is configured to do
The corresponding entry of the first array is determined based on the fourth array.

A-EEE12. The device according to A-EEE11, said third array is a linear array containing the non-zero entries of the downmix matrix in their appearance order in a predetermined scan order across the entries of the downmix matrix.

A-EEE 13. The downmixing unit is further configured to obtain a fifth array.
The entries in the fifth array indicate the number of non-zero entries per row of the downmix matrix in the case of row scan order, and per column in the case of column scan order. A device according to A-EEE12 showing the number of non-zero entries in the downmix matrix of.

A-EEE14. The downmixing unit is for the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assign to said entry in the array of
A device according to A-EEE13, which is further configured as described above.

A-EEE15. A software program configured to run on a processor and, when run on a computing device, to perform a method step of the method according to any one of A-EEE1-10.

As a further example, the exemplary embodiments (B-EEE) listed below describe some structures, features, and functions of some aspects of the invention.

B-EEE1. A method of downmixing multiple input audio channels, for each of the input audio channels, to obtain multiple frequency coefficients in multiple corresponding frequency bins.
For at least one frequency bin, a downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin, and multiple intermediate audio channels for each frequency bin are applied. To obtain a second array formed by the frequency coefficients of
Including
The method is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Including
Applying the downmix matrix to the first array
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
The method, wherein the corresponding entry of the first array is determined based on the fourth array.

B-EEE2. Obtaining the energy index value of each intermediate audio channel for each intermediate audio channel for at least one frequency bin.
Including
Obtaining the energy index value is to obtain the energy index value for the at least one frequency bin.
Multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array.
The method according to B-EEE1, wherein the corresponding entry of the first array is determined based on the fourth array.

B-EEE3. The third array is any of the preceding B-EEEs, which is a linear array containing the non-zero entries of the downmix matrix in their order of appearance in a predetermined scan order across the entries of the downmix matrix. One way.

B-EEE4. A method according to any one of the preceding B-EEEs, wherein each entry in the fourth array indicates the position of each entry in the third array in the downmix matrix in said scan order.

B-EEE5. Each entry in the fourth array, in the case of row-wise scan order, indicates the column of the downmix matrix in which each entry in the third array is located in the downmix matrix and is in column-wise scan order. In the case of a method according to any one of the preceding B-EEEs, where each entry in the third array indicates a row of the downmix matrix located in the downmix matrix.

B-EEE6. Further including finding a fifth array,
The entries in the fifth array indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order, and the downmix matrix in the case of column scan order. A method according to either B-EEE3 or a preceding B-EEE subordinate to B-EEE3, which indicates the number of non-zero entries for each column of.

B-EEE 7. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assigned to the entry in the array of
The method according to B-EEE6, which is further accompanied by.

B-EEE8. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The entry index for the third array is continuously incremented one at a time, and for each entry index, the entry of the third array indicated by the entry index of each frequency bin and the entry of the third array. Finding the product of the corresponding entry of the first array and
Sequentially assigning the sum of one or more of the products to the entry in the second array,
Including
The allocation is based on the entry of the fifth array, according to B-EEE5 or 6.

B-EEE 9. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
For each entry index for the third array, the square of the entry of the third array indicated by the entry index of each frequency bin and the absolute square of the corresponding entry of the first array. To find the product of and as the second product,
The sum of one or more of the second products is sequentially assigned to the entry in the sixth array of each frequency bin.
Including
The allocation is based on the entry in the fifth array.
The method according to B-EEE8, wherein the sum of the sixth array over the frequency bins in a given processing band gives the energy index value of the frequency bins in the given processing band.

B-EEE10. Finding the frequency bins in the given processing band involves referencing a look-up table, which is an entry indicating the number of frequency bins in each processing band for each processing band. The method according to B-EEE9, which comprises.

B-EEE11. A device that downmixes a plurality of input audio channels, a frequency coefficient acquisition unit configured to acquire a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels, and a frequency coefficient acquisition unit.
With a downmix matrix acquisition unit configured to acquire a downmix matrix,
For at least one frequency bin, the downmix matrix is applied to a first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin to provide multiple intermediate audio in each frequency bin. A downmixing unit configured to acquire a second array formed by said frequency coefficients of the channel, and
With
The down mixing unit is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Is configured to do
The corresponding entry of the first array is determined based on the fourth array.

B-EEE12. The device by B-EEE11, wherein the third array is a linear array that includes the non-zero entries of the downmix matrix in their appearance order in a predetermined scanning order across the entries of the downmix matrix.

B-EEE 13. The downmixing unit is further configured to obtain a fifth array.
The entries in the fifth array indicate the number of non-zero entries per row of the downmix matrix in the case of row scan order, and per column in the case of column scan order. A device according to B-EEE12 showing the number of non-zero entries in the downmix matrix of.

B-EEE14. The downmixing unit is for the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assign to said entry in the array of
A device according to B-EEE13, which is further configured as described above.

B-EEE15. A software program configured to run on a processor and, when run on a computing device, to perform a method step of the method according to any one of B-EEE1-10.

As a further example, the exemplary embodiments (C-EEE) listed below describe some structures, features, and functions of some aspects of the invention.

C-EEE1. A method of downmixing multiple input audio channels,
Acquiring a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels,
For at least one frequency bin, a downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin, and multiple intermediate audio channels for each frequency bin are applied. To obtain a second array formed by the frequency coefficients of
Including
The method is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Including
Applying the downmix matrix to the first array
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Including
The method in which the corresponding entry of the first array is determined based on the fourth array.

C-EEE2. Obtaining the energy index value of each intermediate audio channel for each intermediate audio channel for at least one frequency bin.
Including
Obtaining the energy index value is to obtain the energy index value for the at least one frequency bin.
Multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array.
The method according to C-EEE1, wherein the corresponding entry of the first array is determined based on the fourth array.

C-EEE3. The third array is any of the preceding C-EEEs, which is a linear array containing the non-zero entries of the downmix matrix in their order of appearance in a predetermined scan order across the entries of the downmix matrix. One way.

C-EEE4. A method according to any one of the preceding C-EEEs, wherein each entry in the fourth array indicates the position of each entry in the third array in the downmix matrix in said scan order.

C-EEE5. Each entry in the fourth array, in the case of row-wise scan order, indicates the column of the downmix matrix in which each entry in the third array is located in the downmix matrix and is in column-wise scan order. In the case of a method according to any one of the preceding C-EEEs, where each entry in the third array indicates a row of the downmix matrix located in the downmix matrix.

C-EEE6. Further including finding a fifth array,
The entries in the fifth array indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order, and the downmix matrix in the case of column scan order. A method according to either C-EEE3 or a preceding C-EEE subordinate to C-EEE3, which indicates the number of non-zero entries for each column of.

C-EEE7. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assigned to the entry in the array of
The method according to C-EEE6, which is further accompanied by.

C-EEE8. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The entry index for the third array is continuously incremented one at a time, and for each entry index, the entry of the third array indicated by the entry index of each frequency bin and the entry of the third array. Finding the product of the corresponding entry of the first array and
Sequentially assigning the sum of one or more of the products to the entry in the second array,
The allocation is based on the entry of the fifth array, according to the C-EEE 5 or 6.

C-EEE 9. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
For each entry index for the third array, the square of the entry of the third array indicated by the entry index of each frequency bin and the absolute square of the corresponding entry of the first array. To find the product of and as the second product,
The sum of one or more of the second products is sequentially assigned to the entry in the sixth array of each frequency bin.
Including
The allocation is based on the entry in the fifth array.
The method according to C-EEE8, wherein the sum of the sixth array over the frequency bins in a given processing band gives the energy index value of the frequency bins in the given processing band.

C-EEE10. Finding the frequency bins in the given processing band involves referencing a look-up table, which is an entry indicating the number of frequency bins in each processing band for each processing band. The method according to C-EEE9, which comprises.

C-EEE11. A device that downmixes a plurality of input audio channels, a frequency coefficient acquisition unit configured to acquire a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels, and a frequency coefficient acquisition unit.
With a downmix matrix acquisition unit configured to acquire a downmix matrix,
For at least one frequency bin, the downmix matrix is applied to a first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin to provide multiple intermediate audio in each frequency bin. A downmixing unit configured to acquire a second array formed by said frequency coefficients of the channel, and
With
The down mixing unit is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Is configured to do
The corresponding entry of the first array is determined based on the fourth array.

C-EEE12. The third array is a device according to C-EEE11, which is a linear array containing the non-zero entries of the downmix matrix in their appearance order in a predetermined scan order across the entries of the downmix matrix.

C-EEE 13. The downmixing unit is further configured to obtain a fifth array.
The entries in the fifth array indicate the number of non-zero entries per row of the downmix matrix in the case of row scan order, and per column in the case of column scan order. A device according to C-EEE12 showing the number of non-zero entries in the downmix matrix of.

C-EEE14. The downmixing unit is for the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assign to said entry in the array of
A device according to C-EEE13, which is further configured as described above.

C-EEE15. A software program configured to run on a processor and, when run on a computing device, to perform a method step of the method according to any one of C-EEE1-10.

As a further example, the exemplary embodiments (D-EEE) listed below describe some structures, features, and functions of some aspects of the invention.

D-EEE1. A method of downmixing multiple input audio channels,
Acquiring a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels,
For at least one frequency bin, a downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin, and multiple intermediate audio channels for each frequency bin are applied. To obtain a second array formed by the frequency coefficients of
Including
The method is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Including
Applying the downmix matrix to the first array
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Including
The method in which the corresponding entry of the first array is determined based on the fourth array.

D-EEE2. Obtaining the energy index value of each intermediate audio channel for each intermediate audio channel for at least one frequency bin.
Including
Obtaining the energy index value is to obtain the energy index value for the at least one frequency bin.
Multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array.
Including
The corresponding entry in the first array is determined based on the fourth array, the method according to D-EEE1.

D-EEE3. The third array is any of the preceding D-EEEs, which is a linear array containing the non-zero entries of the downmix matrix in their order of appearance in a predetermined scan order across the entries of the downmix matrix. One way.

D-EEE4. A method according to any one of the preceding D-EEEs, wherein each entry in the fourth array indicates the position of each entry in the third array in the downmix matrix in said scan order.

D-EEE5. Each entry in the fourth array, in the case of row-wise scan order, indicates the column of the downmix matrix in which each entry in the third array is located in the downmix matrix and is in column-wise scan order. In the case of a method according to any one of the preceding D-EEEs, where each entry in the third array indicates a row of the downmix matrix located in the downmix matrix.

D-EEE6. Further including finding a fifth array,
The entries in the fifth array indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order, and the downmix matrix in the case of column scan order. A method according to either D-EEE3 or a preceding D-EEE subordinate to D-EEE3, which indicates the number of non-zero entries for each column of.

D-EEE 7. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assigned to the entry in the array of
The method according to D-EEE6, which is further accompanied by.

D-EEE8. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The entry index for the third array is continuously incremented one at a time, and for each entry index, the entry of the third array indicated by the entry index of each frequency bin and the entry of the third array. Finding the product of the corresponding entry of the first array and
Sequentially assigning the sum of one or more of the products to the entry in the second array,
The allocation is based on the entry of the fifth array, according to D-EEE5 or 6.

D-EEE 9. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
For each entry index for the third array, the square of the entry of the third array indicated by the entry index of each frequency bin and the absolute square of the corresponding entry of the first array. To find the product of and as the second product,
The sum of one or more of the second products is sequentially assigned to the entry in the sixth array of each frequency bin.
Including
The allocation is based on the entry in the fifth array.
The method according to D-EEE8, wherein the sum of the sixth array over the frequency bins in a given processing band gives the energy index value of the frequency bins in the given processing band.

D-EEE10. Finding the frequency bins in the given processing band involves referencing a look-up table, which is an entry indicating the number of frequency bins in each processing band for each processing band. The method according to D-EEE9, which comprises.

D-EEE11. A device that downmixes a plurality of input audio channels, a frequency coefficient acquisition unit configured to acquire a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels, and a frequency coefficient acquisition unit.
With a downmix matrix acquisition unit configured to acquire a downmix matrix,
For at least one frequency bin, the downmix matrix is applied to a first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin to provide multiple intermediate audio in each frequency bin. A downmixing unit configured to acquire a second array formed by said frequency coefficients of the channel, and
With
The down mixing unit is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Is configured to do
The corresponding entry of the first array is determined based on the fourth array.

D-EEE12. The third array is a device according to D-EEE11, which is a linear array containing the non-zero entries of the downmix matrix in their appearance order in a predetermined scan order across the entries of the downmix matrix.

D-EEE 13. The downmixing unit is further configured to obtain a fifth array.
The entries in the fifth array indicate the number of non-zero entries per row of the downmix matrix in the case of row scan order, and per column in the case of column scan order. A device according to D-EEE12 showing the number of non-zero entries in the downmix matrix of.

D-EEE14. The downmixing unit is for the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assign to said entry in the array of
A device according to D-EEE13, which is further configured as described above.

D-EEE15. A software program configured to run on a processor and, when run on a computing device, to perform a method step of the method according to any one of D-EEE1-10.

As a further example, the exemplary embodiments (E-EEE) listed below describe some structures, features, and functions of some aspects of the invention.

E-EEE1. A method of downmixing multiple input audio channels,
Acquiring a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels,
For at least one frequency bin, a downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin, and multiple intermediate audio channels for each frequency bin are applied. To obtain a second array formed by the frequency coefficients of
Including
The method is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Including
Applying the downmix matrix to the first array
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
The method, wherein the corresponding entry of the first array is determined based on the fourth array.

E-EEE2. Obtaining the energy index value of each intermediate audio channel for each intermediate audio channel for at least one frequency bin.
Including
Obtaining the energy index value is to obtain the energy index value for the at least one frequency bin.
Multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array.
The method according to E-EEE1, wherein the corresponding entry of the first array is determined based on the fourth array.

E-EEE3. The third array is any of the preceding E-EEEs, which is a linear array containing the non-zero entries of the downmix matrix in their order of appearance in a predetermined scan order across the entries of the downmix matrix. One way.

E-EEE4. A method according to any one of the preceding E-EEEs, wherein each entry in the fourth array indicates the position of each entry in the third array in the downmix matrix in said scan order.

E-EEE 5. Each entry in the fourth array, in the case of row-wise scan order, indicates the column of the downmix matrix in which each entry in the third array is located in the downmix matrix and is in column-wise scan order. In the case of a method according to any one of the preceding E-EEEs, where each entry in the third array indicates a row of the downmix matrix located in the downmix matrix.

E-EEE6. Further including finding a fifth array,
The entries in the fifth array indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order, and the downmix matrix in the case of column scan order. A method according to either E-EEE3 or one of the preceding EEEs subordinate to E-EEE3, which indicates the number of non-zero entries for each column of.

E-EEE 7. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assigned to the entry in the array of
The method according to E-EEE6, which is further accompanied by.

E-EEE 8. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The entry index for the third array is continuously incremented one at a time, and for each entry index, the entry of the third array indicated by the entry index of each frequency bin and the entry of the third array. Finding the product of the corresponding entry of the first array and
Sequentially assigning the sum of one or more of the products to the entry in the second array,
The allocation is based on the entry of the fifth array, according to E-EEE5 or 6.

E-EEE 9. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
For each entry index for the third array, the square of the entry of the third array indicated by the entry index of each frequency bin and the absolute square of the corresponding entry of the first array. To find the product of and as the second product,
The sum of one or more of the second products is sequentially assigned to the entry in the sixth array of each frequency bin.
Including
The allocation is based on the entry in the fifth array.
The method according to E-EEE8, wherein the sum of the sixth array over the frequency bins in a given processing band gives the energy index value of the frequency bins in the given processing band.

E-EEE10. Finding the frequency bins in the given processing band involves referencing a look-up table, which is an entry indicating the number of frequency bins in each processing band for each processing band. The method according to E-EEE9, which comprises.

E-EEE11. A device that downmixes a plurality of input audio channels, a frequency coefficient acquisition unit configured to acquire a plurality of frequency coefficients in a plurality of corresponding frequency bins for each of the input audio channels, and a frequency coefficient acquisition unit.
With a downmix matrix acquisition unit configured to acquire a downmix matrix,
For at least one frequency bin, the downmix matrix is applied to a first array formed by the frequency coefficients of the plurality of input audio channels for each frequency bin to provide multiple intermediate audio in each frequency bin. A downmixing unit configured to acquire a second array formed by said frequency coefficients of the channel, and
With
The down mixing unit is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
The corresponding entry of the first array is obtained based on the fourth array.

E-EEE12. The device according to E-EEE11, said third array is a linear array containing the non-zero entries of the downmix matrix in their appearance order in a predetermined scan order across the entries of the downmix matrix.

E-EEE 13. The downmixing unit is further configured to obtain a fifth array.
The entries in the fifth array indicate the number of non-zero entries per row of the downmix matrix in the case of row scan order, and per column in the case of column scan order. E-EEE12 apparatus showing the number of non-zero entries in the downmix matrix of.

E-EEE14. The downmixing unit is for the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assign to said entry in the array of
A device according to E-EEE13, which is further configured as described above.

E-EEE15. A software program configured to run on a processor and, when run on a computing device, to perform a method step of the method according to any one of E-EEE1-10.

Claims (15)

A method of downmixing multiple input audio channels,
For each of the input audio channels, the method comprising: acquiring a plurality of frequency coefficients corresponding to a plurality of frequency bins,
A downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each of the at least one frequency bin, and the frequencies of the plurality of intermediate audio channels for each frequency bin. To get a second array formed by the coefficients,
Including
The method is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Including
Applying the downmix matrix to the first array
For at least one frequency bin, including multiplying the entry of the third array for each frequency bin by the corresponding entry of the first array.
The corresponding entry of the first array is determined based on the fourth array .
The method is
For each of the intermediate audio channels, the energy index value of each intermediate audio channel is obtained for at least one frequency bin, and
For each of the at least one frequency bin, the adjustment factor is determined based on the energy index value for each intermediate audio channel.
Adjusting the magnitude of the frequency coefficient of each intermediate audio channel in each frequency bin by multiplying the frequency coefficient by each adjustment factor for each intermediate audio channel for the at least one frequency bin. When,
Including
Obtaining the energy index value includes multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array for at least one frequency bin. , The corresponding entry of the first array is sought based on the fourth array.
Method. The method of claim 1 , wherein the third array is a linear array that includes the non-zero entries of the downmix matrix in their order of appearance in a predetermined scan order across the entries of the downmix matrix. .. The method of claim 2 , wherein each entry in the fourth array indicates the position of each entry in the third array in the downmix matrix in said scan order. Each entry in the fourth array, in the case of a row-wise scan order, indicates the column of the downmix matrix in which each entry in the third array is located in the downmix matrix, in columnar scan order. in this case, indicating the row of the downmix matrix, each entry of the third array is positioned in the downmix matrix, the method according to any one of claims 1 to 3. Further including finding a fifth array,
The entries in the fifth array indicate the number of non-zero entries for each row of the downmix matrix in the case of row scan order, and the downmix matrix in the case of columnar scan order. The method of claim 2 or 3 or claim 4 , which is subordinate to claim 2 , indicating the number of non-zero entries for each column of. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. Assigned to the entry in the array of
5. The method of claim 5, further comprising. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
The third incremented by one continuously in temporary et down tree index for the array, each entry index of each frequency bin, and the entry of the third array shown by the respective entry index, the Finding the product of the corresponding entry of the first array and
Sequentially assigning the sum of one or more of the products to the entry in the second array,
Including
The allocation is based on the entry of the fifth array, The method according to claim 5. Applying the downmix matrix to the first array can be applied to the at least one frequency bin.
For each entry index for the third array, the square of the entry of the third array indicated by the entry index of each frequency bin and the absolute square of the corresponding entry of the first array. To find the product of and as the second product,
The sum of one or more of the second products is sequentially assigned to the entry in the sixth array of each frequency bin.
Including
The allocation is based on the entry in the fifth array.
The method of claim 7 , wherein the sum of the sixth array over a frequency bin in a given processing band gives the energy index value of the frequency bin in the given processing band. Finding the frequency bins in the given processing band involves referencing a look-up table, which is an entry indicating the number of frequency bins in each processing band for each processing band. 8. The method of claim 8. A device that downmixes multiple input audio channels.
For each of the input audio channel, and the frequency coefficient acquiring unit configured to acquire a plurality of frequency coefficients corresponding to a plurality of frequency bins,
With a downmix matrix acquisition unit configured to acquire a downmix matrix,
The downmix matrix is applied to the first array formed by the frequency coefficients of the plurality of input audio channels for each of the at least one frequency bin, and the plurality of intermediate audio channels for each frequency bin are described. With a downmixing unit configured to acquire a second array formed by frequency coefficients,
With
The down mixing unit is
Finding a third array containing only non-zero entries in the downmix matrix,
For each entry in the third array, a fourth array containing an entry indicating the position of each entry in the third array in the downmix matrix is determined.
Multiplying the at least one frequency bin by the entry of the third array for each frequency bin and the corresponding entry of the first array.
Is configured to do
The corresponding entry of the first array is determined based on the fourth array .
The device is
An energy determination unit configured to determine the energy index value of each intermediate audio channel for each intermediate audio channel for at least one frequency bin.
It ’s an adjustment unit,
For each of the at least one frequency bin, the adjustment factor is determined for each intermediate audio channel based on the respective energy index value.
For at least one frequency bin, the magnitude of the frequency coefficient of each intermediate audio channel in each frequency bin is adjusted by multiplying the frequency coefficient by each adjustment factor for each intermediate audio channel.
With the adjustment unit configured as
Further prepare
Obtaining the energy index value includes multiplying the power of the entry in the third array by the power of the absolute value of the corresponding entry in the first array for at least one frequency bin. , The corresponding entry of the first array is sought based on the fourth array.
Device. The device of claim 10 , wherein the third array is a linear array that includes the non-zero entries of the downmix matrix in their appearance order in a predetermined scan order across the entries of the downmix matrix. .. The downmixing unit is further configured to obtain a fifth array.
The entries in the fifth array indicate the number of non-zero entries per row of the downmix matrix in the case of row scan order, and per column in the case of column scan order. 11. The apparatus of claim 11 , indicating the number of non-zero entries in the downmix matrix of. The downmixing unit is for the at least one frequency bin.
The sum of one or more products of the entry of the third array and the corresponding entry of the first array for each frequency bin is based on the entry of the fifth array. 12. The device of claim 12 , further configured to be assigned to said entry in the array of. A program of instructions that can be executed by a machine that causes the execution of processing by the method according to any one of claims 1 to 9 . With at least one processor
At least one memory for storing computer programs,
It is a device equipped with
The computer program, the at least one processor to perform the method of any one of at least claims 1-9,
Device.

Images