HK40094574A - Methods for parametric multi-channel encoding - Google Patents

Description

Methods for parametric multichannel coding

This application is a divisional application of the invention patent application with application number 201910673941.4, application date February 21, 2014, and invention title "Method for Parametric Multichannel Coding".

Cross-references to related applications

This application claims priority to U.S. Provisional Patent Application No. 61/767,673, filed February 21, 2013, the entire contents of which are hereby incorporated by reference.

Technical Field

This document relates to audio coding systems. Specifically, it relates to efficient methods and systems for parametric multichannel audio coding.

Background Technology

Parametric multichannel audio coding systems can be used to provide improved listening quality at particularly low data rates. Nevertheless, further improvements to such parametric multichannel audio coding systems are needed, especially for bandwidth efficiency, computational efficiency, and/or robustness.

Summary of the Invention

According to one aspect, an audio coding system configured to generate a bitstream indicative of a downmixer and spatial metadata is described. The spatial metadata can be used by a corresponding decoding system to generate a multichannel upmixer from the downmixer. The downmixer may include m channels, and the multichannel upmixer may include n channels, where n and m are integers, and m < n. In the example, n = 6, m = 2. The spatial metadata enables the corresponding decoding system to generate n channels of the multichannel upmixer from the m channels of the downmixer.

An audio encoding system can be configured to quantize and/or encode the downmix signal and spatial metadata and insert the quantized/encoded data into the bitstream. Specifically, the downmix signal can be encoded using a Dolby Digital Plus encoder, and the bitstream can correspond to a Dolby Digital Plus bitstream. The quantized/encoded spatial metadata can be inserted into the data fields of the Dolby Digital Plus bitstream.

An audio coding system may include a downmixing processing unit configured to generate a downmixed signal from a multichannel input signal. The downmixing processing unit is also referred to herein as a downmixing encoding unit. The multichannel input signal may include n channels, such as a multichannel upmixed signal regenerated based on the downmixed signal. Specifically, the multichannel upmixed signal can provide an approximation of the multichannel input signal. The downmixing unit may include the Dolby Digital Plus encoder mentioned above. The multichannel upmixed signal and the multichannel input signal may be 5.1 or 7.1 signals, and the downmixed signal may be a stereo signal.

Audio coding systems may include parameter processing units configured to determine spatial metadata from multichannel input signals. Specifically, the parameter processing unit (also referred to herein as a parameter encoding unit) may be configured to determine one or more spatial parameters, such as a set of spatial parameters, which may be determined based on different combinations of the channels of the multichannel input signal. The spatial parameters of the set of spatial parameters may indicate the cross-correlation between the different channels of the multichannel input signal. The parameter processing unit may be configured to determine spatial metadata for frames of the multichannel input signal, referred to as spatial metadata frames. Frames of the multichannel input signal typically include a predetermined number (e.g., 1536) samples of the multichannel input signal. Each spatial metadata frame may include one or more sets of spatial parameters.

The audio encoding system may also include a configuration unit configured to determine one or more control settings for the parameter processing unit based on one or more external settings. The one or more external settings may include a target data rate for the bitstream. Alternatively or additionally, the one or more external settings may include one or more of the following: a sampling rate of the multichannel input signal, the number of channels m of the downmix signal, the number of channels n of the multichannel input signal, and/or an update period indicating the time required for the corresponding decoding system to synchronize with the bitstream. The one or more control settings may include a maximum data rate for spatial metadata. In the case of spatial metadata frames, the maximum data rate for spatial metadata may indicate the maximum number of metadata bits in the spatial metadata frame. Alternatively or additionally, the one or more control settings may include one or more of the following: a time resolution setting indicating the number of spatial parameter sets for each spatial metadata frame to be determined; a frequency resolution setting indicating the number of frequency bands for which spatial parameters will be determined; a quantizer setting indicating the type of quantizer to be used to quantize the spatial metadata; and an indication of whether the current frame of the multichannel input signal will be encoded as an independent frame.

The parameter processing unit can be configured to determine whether the number of bits in a spatial metadata frame, determined according to the one or more control settings, exceeds the maximum number of metadata bits. Furthermore, the parameter processing unit can be configured to reduce the number of bits in a particular spatial metadata frame if it is determined that the number of bits in that particular spatial metadata frame exceeds the maximum number of metadata bits. This bit reduction can be performed in a resource-efficient manner. Specifically, this bit reduction can be performed without recalculating the entire spatial metadata frame.

As indicated above, a spatial metadata frame may include one or more sets of spatial parameters. The one or more control settings may include a time resolution setting indicating the number of spatial parameter sets to be determined by the parameter processing unit for each spatial metadata frame. The parameter processing unit may be configured to determine a plurality of spatial parameter sets for the current spatial metadata frame, as indicated by the time resolution setting. Typically, the time resolution setting is a value of 1 or 2. Furthermore, the parameter processing unit may be configured to discard spatial parameter sets from the current spatial metadata frame if the current spatial metadata frame includes multiple sets of spatial parameters, and if the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. The parameter processing unit may be configured to retain at least one set of spatial parameters for each spatial metadata frame. By discarding spatial parameter sets from spatial metadata frames, the number of bits in the spatial metadata frame can be reduced with minimal computational effort and without significantly affecting the perceived listening quality of the multi-channel upmixed signal.

The one or more spatial parameter sets are typically associated with one or more corresponding sampling points. These sampling points can indicate one or more corresponding moments in time. Specifically, a sampling point can indicate the moment when the decoding system should fully utilize the corresponding spatial parameter set. In other words, a sampling point can indicate the moment when the corresponding spatial parameter set has been determined.

The parameter processing unit can be configured to discard a first set of spatial parameters from the current spatial metadata frame if multiple sampling points of the current metadata frame are not associated with transients in the multichannel input signal, wherein the first set of spatial parameters is associated with a first sampling point preceding a second sampling point. Conversely, the parameter processing unit can be configured to discard a second (typically the last) set of spatial parameters from the current spatial metadata frame if multiple sampling points of the current metadata frame are associated with transients in the multichannel input signal. By doing so, the parameter processing unit can be configured to reduce the impact of discarding spatial parameter sets on the listening quality of the multichannel upmixed signal.

The one or more control settings may include quantizer settings indicating a first type of quantizer among a plurality of predetermined types. The plurality of predetermined types of quantizers may each provide different quantizer resolutions. Specifically, the plurality of predetermined types of quantizers may include fine quantization and coarse quantization. The parameter processing unit may be configured to quantize one or more sets of spatial parameters of the current spatial metadata frame according to the first type of quantizer. Furthermore, the parameter processing unit may be configured to, if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits, requantize one, some, or all of the spatial parameters of the one or more sets of spatial parameters according to a second type of quantizer with a resolution lower than that of the first type of quantizer. By doing so, the number of bits in the current spatial metadata frame can be reduced, while only a limited degree of impact on the quality of the upmixed signal, and without significantly increasing the computational complexity of the audio coding system.

The parameter processing unit can be configured to determine the time difference parameter set based on the difference between the current spatial parameter set and the immediately preceding spatial parameter set. Specifically, the time difference parameter can be determined by determining the difference between the parameters of the current spatial parameter set and the corresponding parameters of the immediately preceding spatial parameter set. The spatial parameter set may include, for example, the parameters _α1 , _α2 , _α3 , _β1 , _β2 , _β3 , g, _k1 ^, and _k2 described in this document. Typically, only one of the parameters _k1 and _k2 may need to be transmitted, as these parameters can be correlated using the relation _k1² ⁺ _k2² = 1. For example only, only parameter _k1 can be transmitted; parameter _k2 can be calculated at the receiver. The time difference parameter can be correlated with the difference between the corresponding parameters among those mentioned above.

The parameter processing unit can be configured to encode the set of time difference parameters using entropy coding (e.g., using Huffman codes). Furthermore, the parameter processing unit can be configured to insert the encoded set of time difference parameters into the current spatial metadata frame. Additionally, the parameter processing unit can be configured to reduce the entropy of the set of time difference parameters if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. As a result, the number of bits required for entropy coding of the time difference parameters can be reduced, thereby reducing the number of bits used in the current spatial metadata frame. For example, the parameter processing unit can be configured to set one, some, or all of the time difference parameters in the set of time difference parameters to a value with an increased (e.g., highest) probability among the possible values of the time difference parameters, in order to reduce the entropy of the set of time difference parameters. Specifically, the probability can be increased compared to the probability of the time difference parameters before the setting operation. Typically, the value with the highest probability among the possible values of the time difference parameter corresponds to zero.

It should be noted that time difference coding of spatial parameter sets generally cannot be used for independent frames. Therefore, the parameter processing unit can be configured to verify whether the current spatial metadata frame is an independent frame; time difference coding is only applied if the current spatial metadata frame is not an independent frame. On the other hand, frequency difference coding described below can also be used for independent frames.

The one or more control settings may include a frequency resolution setting, wherein the frequency resolution setting indicates the number of different frequency bands for which respective spatial parameters (referred to as band parameters) will be determined. The parameter processing unit may be configured to determine different corresponding spatial parameters (band parameters) for different frequency bands. Specifically, different parameters _α1 , _α2 , α3, _β1 , _β2 , _β3 , g, _k1 , _k2 may be determined for different frequency bands. The set _of spatial parameters may therefore include corresponding band parameters for different frequency bands. For example, the set of spatial parameters may include T corresponding band parameters for T frequency bands, where T is an integer, for example, T = 7, 9, 12, or 15.

The parameter processing unit can be configured to determine a set of frequency difference parameters based on the difference between one or more band parameters in a first frequency band and corresponding one or more band parameters in an adjacent second frequency band. Furthermore, the parameter processing unit can be configured to encode the set of frequency difference parameters using entropy coding (e.g., based on Huffman codes). Additionally, the parameter processing unit can be configured to insert the encoded set of frequency difference parameters into the current spatial metadata frame. Furthermore, the parameter processing unit can be configured to reduce the entropy of the set of frequency difference parameters if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. Specifically, the parameter processing unit can be configured to set one, some, or all of the frequency difference parameters in the set of frequency difference parameters to a value with an increased probability (e.g., zero) among the possible values of the frequency difference parameters, in order to reduce the entropy of the set of frequency difference parameters. Specifically, the probability can be increased compared to the probability of the frequency difference parameters before the setting operation.

Alternatively or additionally, the parameter processing unit can be configured to reduce the number of frequency bands if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. Furthermore, the parameter processing unit can be configured to use the reduced number of frequency bands to redetermine some or all of the sets of one or more spatial parameters for the current spatial metadata frame. Typically, the change in the number of frequency bands primarily affects higher frequency bands. As a result, the band parameters of one of several frequencies may be unaffected, making it possible for the parameter processing unit to avoid recalculating all band parameters.

As indicated above, the one or more external settings may include an update period, which indicates the time period required for the corresponding decoding system to synchronize with the bitstream. Furthermore, the one or more control settings may include an indication of whether the current spatial metadata frame will be encoded as an independent frame. The parameter processing unit may be configured to determine a spatial metadata frame sequence for the corresponding frame sequence of the multi-channel input signal. The configuration unit may be configured to determine one or more spatial metadata frames to be encoded as independent frames from the spatial metadata frame sequence based on the update period.

Specifically, the one or more independent spatial metadata frames can be determined such that an update period (on average) is satisfied. For this purpose, the configuration unit can be configured to determine whether the current frame of the frame sequence of the multichannel input signal includes samples at times that are integer multiples of the update period (relative to the start point of the multichannel input signal). Furthermore, the configuration unit can be configured to determine that the current spatial metadata frame corresponding to the current frame is an independent frame (because it includes samples at times that are integer multiples of the update period). The parameter processing unit can be configured to encode one or more sets of spatial parameters of the current spatial metadata frame independently of the data included in the previous (and/or future) spatial metadata frames if the current spatial metadata frame will be encoded as an independent frame. Typically, if the current spatial metadata frame will be encoded as an independent frame, all sets of spatial parameters of the current spatial metadata are encoded independently of the data included in the previous (and/or future) spatial metadata frames.

According to another aspect, a parameter processing unit is described, configured to determine a spatial metadata frame for generating a multi-channel upmix signal from a corresponding frame of a downmix signal. The downmix signal may include m channels, and the multi-channel upmix signal may include n channels; n and m are integers, where m < n. As outlined above, the spatial metadata frame may include one or more sets of spatial parameters.

The parameter processing unit may include a transform unit configured to determine multiple spectra from the current frame and the immediately following frame (referred to as the forward frame) of a channel of the multi-channel input signal. The transform unit may use a filter bank, such as a QMF filter bank. The spectra in the multiple spectra may include a predetermined number of transform coefficients in a corresponding predetermined number of frequency bins. The multiple spectra may be associated with corresponding multiple time intervals (or moments). Thus, the transform unit can be configured to provide a time/frequency representation of the current frame and the forward frame. For example, both the current frame and the forward frame may include K samples. The transform unit may be configured to determine twice the number of K/Q spectra, each spectrum including Q transform coefficients.

The parameter processing unit may include a parameter determination unit configured to determine the spatial metadata frame of the current frame for a channel of the multi-channel input signal by weighting the plurality of spectra using a window function. The window function can be used to adjust the influence of the spectra among the plurality of spectra on specific spatial parameters or a specific set of spatial parameters. For example, the window function can take values between 0 and 1.

The window function can depend on one or more of the following: the number of spatial parameter sets included in the spatial metadata frame, the presence of one or more transients in the current or immediately following frame of the multichannel input signal, and/or the timing of the transients. In other words, the window function can be modified based on the properties of the current frame and/or the forward-looking frame. Specifically, the window function used to determine the set of spatial parameters (which is called the set-dependent window function) can depend on the properties of the current frame and/or the forward-looking frame.

Thus, the window function can include set-dependent window functions. Specifically, the window function used to determine the spatial parameters of the spatial metadata frame can include one or more set-dependent window functions (or can be composed of these set-dependent window functions) for one or more sets of spatial parameters respectively. The parameter determination unit can be configured to determine the set of spatial parameters for the current frame (i.e., for the current spatial metadata frame) of the channels of the multi-channel input signal by weighting the plurality of spectra using set-dependent window functions. As outlined above, the set-dependent window function can depend on one or more properties of the current frame. Specifically, the set-dependent window function can depend on whether the set of spatial parameters is associated with transients.

For example, if the spatial parameter set is not associated with a transient, a set-dependent window function can be configured to provide a phase-in gradient of the plurality of spectra from the sampling points of the previous spatial parameter set to the sampling points of the next spatial parameter set. This phase-in can be provided by a window function that transitions from 0 to 1. Alternatively or additionally, if the spatial parameter set is not associated with a transient, the set-dependent window function can include multiple spectra from the sampling points of the spatial parameter set to the spectra preceding the sampling points of the next spatial parameter set (either these spectra can be fully considered, or these spectra can be left unaffected), if the next spatial parameter set is associated with a transient. This can be achieved using a window function with a value of 1. Alternatively or additionally, if the spatial parameter set is not associated with a transient, the set-dependent window function can cancel out the plurality of spectra from the sampling points of the next spatial parameter set (either these spectra can be excluded, or these spectra can be attenuated), if the next spatial parameter set is associated with a transient. This can be achieved using a window function with a value of 0. Alternatively or additionally, if the spatial parameter set is not associated with the transient, a set-dependent window function can phase-out the plurality of spectra from the sampling point of the spatial parameter set up to the spectra preceding the sampling point of the subsequent spatial parameter set, if the subsequent spatial parameter set is not associated with the transient. The phase-out can be provided by a window function transitioning from 1 to 0. On the other hand, if the spatial parameter set is associated with the transient, a set-dependent window function can eliminate (or exclude, or attenuate) the spectra preceding the sampling point of the spatial parameter set. Alternatively or additionally, if the spatial parameter set is associated with the transient, a set-dependent window function can include (or leave unaffected) the spectra from the sampling point of the spatial parameter set up to the spectra preceding the sampling point of the subsequent spatial parameter set, and can eliminate (or exclude, or attenuate) the spectra starting from the sampling point of the subsequent spatial parameter set, if the sampling point of the subsequent spatial parameter set is associated with the transient. Alternatively or additionally, if the set of spatial parameters is associated with a transient, the set-related window function may include the spectrum of the plurality of spectra from the sampling point of the spatial parameter set up to the spectrum of the plurality of spectra at the end of the current frame (or may make these spectra unaffected), and may provide the fading of the spectrum of the plurality of spectra from the beginning of the immediately following frame up to the sampling point of the next set of spatial parameters (or may make these spectra gradually decay), if the next set of spatial parameters is not associated with a transient.

According to another aspect, a parameter processing unit is described, configured to determine a spatial metadata frame for generating a multi-channel upmix signal from a corresponding frame of a downmix signal. The downmix signal may include m channels, and the multi-channel upmix signal may include n channels; n and m are integers, where m < n. As discussed above, the spatial metadata frame may include a set of spatial parameters.

As outlined above, the parameter processing unit may include a transformation unit. The transformation unit may be configured to determine a first plurality of transformation coefficients from frames of the first channel of the multi-channel input signal. Furthermore, the transformation unit may be configured to determine a second plurality of transformation coefficients from corresponding frames of the second channel of the multi-channel input signal. The first channel and the second channel may be different. Thus, the first plurality of transformation coefficients and the second plurality of transformation coefficients respectively provide a first time/frequency representation and a second time/frequency representation of the corresponding frames of the first channel and the second channel. As outlined above, the first time/frequency representation and the second time/frequency representation include a plurality of frequency intervals and a plurality of time intervals.

Furthermore, the parameter processing unit may include a parameter determination unit configured to determine a set of spatial parameters using fixed-point arithmetic based on a first plurality of transform coefficients and a second plurality of transform coefficients. As indicated above, the set of spatial parameters typically includes corresponding band parameters for different frequency bands, wherein the different frequency bands may include different numbers of frequency intervals. Specific band parameters for a specific frequency band can be determined based on transform coefficients from the first plurality of transform coefficients and the second plurality of transform coefficients for that specific frequency band (typically, transform coefficients for other frequency bands are not considered). The parameter determination unit may be configured to determine a shift used by fixed-point arithmetic to determine specific band parameters dependent on the specific frequency band. In particular, the shift used by fixed-point arithmetic to determine specific band parameters for a specific frequency band may depend on the number of frequency intervals included within that specific frequency band. Alternatively or additionally, the shift used by fixed-point arithmetic to determine specific band parameters for a specific frequency band may depend on the number of time intervals considered in determining the specific band parameters.

The parameter determination unit can be configured to determine the shifts required for a specific frequency band to maximize the accuracy of the band-specific parameters. This can be achieved by determining the shifts required for each multiplication and addition operation in the determination process of the band-specific parameters.

The parameter determination unit can be configured to determine a specific band parameter for a specific frequency band p by determining a first energy (or energy estimate) E _{_1,1} (p) based on transform coefficients falling within a specific frequency band p from a first plurality of transform coefficients. Furthermore, a second energy (or energy estimate) E _{_2,2} (p) can be determined based on transform coefficients falling within a specific frequency band p from a second plurality of transform coefficients. Additionally, a cross product or covariance E _1,2(p) can be determined based on transform coefficients falling within a specific frequency band p from the first plurality of transform coefficients and the second plurality of transform coefficients. The parameter determination unit can be configured to determine a shift z_{_p} for the specific frequency band parameter p based on the maximum value among the absolute values of the first energy estimate E_{_1,1}(p), the second energy estimate E _{_2,2}(p), and the covariance E _{_1,2}(p).

According to another aspect, an audio coding system is described, configured to generate a bitstream indicating a frame sequence of a downmixed signal and a corresponding spatial metadata frame sequence, the corresponding spatial metadata frame sequence being used to generate a corresponding frame sequence of a multichannel upmixed signal from the frame sequence of the downmixed signal. The system may include a downmixing processing unit configured to generate the frame sequence of the downmixed signal from the corresponding frame sequence of the multichannel input signal. As indicated above, the downmixed signal may include m channels, and the multichannel input signal may include n channels; n and m are integers, where m < n. Furthermore, the audio coding system may include a parameter processing unit configured to determine the spatial metadata frame sequence from the frame sequence of the multichannel input signal.

Additionally, the audio encoding system may include a bitstream generation unit configured to generate a bitstream comprising a sequence of bitstream frames, wherein the bitstream frames indicate a frame of the downmixed signal corresponding to a first frame of the multichannel input signal and a spatial metadata frame corresponding to a second frame of the multichannel input signal. The second frame may be different from the first frame. Specifically, the first frame may precede the second frame. By doing so, the spatial metadata frame for the current frame can be sent together with the frame of the next frame. This ensures that the spatial metadata frame arrives at the appropriate decoding system only when it is needed. The decoding system typically decodes the current frame of the downmixed signal and generates a decorrelated frame based on the current frame of the downmixed signal. This process introduces an algorithmic delay, and by using the spatial metadata frame delay for the current frame, it ensures that the spatial metadata frame arrives at the decoding system only once the decoded current frame and the decorrelated frame are provided. As a result, the processing power and memory requirements of the decoding system can be reduced.

In other words, an audio encoding system is described, configured to generate a bitstream based on a multichannel input signal. As outlined above, the system may include a downmixing processing unit configured to generate a frame sequence of a downmixed signal from a corresponding first frame sequence of the multichannel input signal. The downmixed signal may include m channels, and the multichannel input signal may include n channels; n and m are integers, where m < n. Furthermore, the audio encoding system may include a parameter processing unit configured to generate a spatial metadata frame sequence from a second frame sequence of the multichannel input signal. The frame sequence of the downmixed signal and the spatial metadata frame sequence can be used by a corresponding decoding system to generate a multichannel upmixed signal comprising n channels.

The audio coding system may also include a bitstream generation unit configured to generate a bitstream comprising a sequence of bitstream frames, wherein the bitstream frames may indicate a frame corresponding to the first frame of a first frame sequence of the multichannel input signal and a spatial metadata frame corresponding to the second frame of a second frame sequence of the multichannel input signal. The second frame may be different from the first frame. In other words, the framing used to determine the spatial metadata frame and the framing used to determine the frame of the downmixer may be different. As outlined above, different framing can be used to ensure that the data is aligned at the corresponding decoding system.

The first and second frames typically consist of the same number of samples (e.g., 1536 samples). Some of the samples in the first frame may precede those in the second frame. Specifically, the first frame may precede the second frame by a predetermined number of samples. This predetermined number of samples may, for example, correspond to a small fraction of the total number of samples in the frame. For instance, the predetermined number of samples may correspond to 50% or more of the total number of samples in the frame. In a particular example, the predetermined number of samples corresponds to 928 samples. As illustrated in this document, this specific number of samples provides minimal total latency and optimal alignment for a particular implementation of the audio encoding and decoding system.

According to another aspect, an audio coding system is described, configured to generate a bitstream based on a multichannel input signal. The system may include a downmixing processing unit configured to determine a clip protection gain (also referred to in this document as clip-gain and/or DRC2 parameters) sequence for a corresponding frame sequence of the multichannel input signal. The current clip protection gain may indicate the current frame to be applied to the multichannel input signal to prevent attenuation of the corresponding current frame clipping of the downmixed signal. Similarly, the clip protection gain sequence may indicate the frames to be applied to the frame sequence of the multichannel input signal to prevent respective attenuation of the corresponding frame clipping of the frame sequence of the downmixed signal.

The downmixing unit can be configured to interpolate the current trim protection gain and the previous trim protection gain of the previous frame of the multichannel input signal to obtain a trim protection gain curve. This can be performed in a manner similar to a trim protection gain sequence. Furthermore, the downmixing unit can be configured to apply the trim protection gain curve to the current frame of the multichannel input signal to obtain the current frame of the attenuated multichannel input signal. Again, this can be performed in a manner similar to a frame sequence of the multichannel input signal. Additionally, the downmixing unit can be configured to generate the current frame of the downmixed signal frame sequence from the current frame of the attenuated multichannel input signal. Similarly, a frame sequence of the downmixed signal can be generated.

The audio processing system may further include a parameter processing unit configured to determine a spatial metadata frame sequence from the multichannel input signal. The downmixed signal frame sequence and the spatial metadata frame sequence can be used to generate a multichannel upmixed signal comprising n channels, such that the multichannel upmixed signal is an approximation of the multichannel input signal. Additionally, the audio processing system may include a bitstream generation unit configured to generate a bitstream indicating a trimmed guard gain sequence, a downmixed signal frame sequence, and a spatial metadata frame sequence, enabling a corresponding decoding system to generate the multichannel upmixed signal.

The trim protection gain curve may include a transition segment and a flat segment. The transition segment provides a smooth transition from the previous trim protection gain to the current trim protection gain, while the flat segment remains flat at the current trim protection gain. The transition segment may extend across a predetermined number of samples of the current frame of the multichannel input signal. The predetermined number of samples may be more than one but less than the total number of samples of the current frame of the multichannel input signal. Specifically, the predetermined number of samples may correspond to a sampling block (where a frame may include multiple blocks) or a frame. In a particular example, a frame may include 1536 samples, and a block may include 256 samples.

According to another aspect, an audio coding system is described, configured to generate a bitstream indicating a downmixed signal and spatial metadata for generating a multichannel upmixed signal from the downmixed signal. The system may include a downmixing processing unit configured to generate the downmixed signal from a multichannel input signal. Furthermore, the system may include a parameter processing unit configured to determine a spatial metadata frame sequence for a corresponding frame sequence of the multichannel input signal.

Furthermore, the audio encoding system may include a configuration unit configured to determine one or more control settings for the parameter processing unit based on one or more external settings. The one or more external settings may include an update period indicating the time period required for the corresponding decoding system to synchronize with the bitstream. The configuration unit may be configured to determine one or more independent spatial metadata frames to be encoded independently from a sequence of spatial metadata frames based on the update period.

According to another aspect, a method for generating a bitstream is described, the bitstream indicating a downmixing signal and spatial metadata for generating a multi-channel upmixing signal from the downmixing signal. The method can generate the downmixing signal from a multi-channel input signal. Furthermore, the method may include determining one or more control settings based on one or more external settings; wherein the one or more external settings include a target data rate for the bitstream, and wherein the one or more control settings include a maximum data rate for the spatial metadata. Additionally, the method may include determining the spatial metadata from the multi-channel input signal according to the one or more control settings.

According to another aspect, a method for determining a spatial metadata frame for generating a multichannel upmix signal from a corresponding frame of an downmix signal is described. The method may include determining multiple spectra from the current frame and the immediately following frame of a channel of the multichannel input signal. Furthermore, the method may include weighting the multiple spectra using a window function to obtain multiple weighted spectra. Additionally, the method may include determining a spatial metadata frame for the current frame of the channel of the multichannel input signal based on the multiple weighted spectra. The window function may depend on one or more of the following: the number of spatial parameter sets included within the spatial metadata frame, the presence of a transient in the current frame or the immediately following frame of the multichannel input signal, and/or the timing of that transient.

According to another aspect, a method for determining a spatial metadata frame for generating a multichannel upmix signal frame from a corresponding frame of an downmix signal is described. The method may include: determining a first plurality of transform coefficients from a frame of a first channel of the multichannel input signal, and determining a second plurality of transform coefficients from a corresponding frame of a second channel of the multichannel input signal. As outlined above, the first plurality of transform coefficients and the second plurality of transform coefficients typically provide a first time/frequency representation and a second time/frequency representation of the corresponding frames of the first and second channels, respectively. The first time/frequency representation and the second time/frequency representation may include a plurality of frequency intervals and a plurality of time intervals. The set of spatial parameters may include corresponding band parameters for different frequency bands comprising different numbers of frequency intervals. The method may further include determining a shift to be applied when determining a specific band parameter for a specific frequency band using fixed-point arithmetic. Furthermore, the shift may be determined based on the number of time intervals to be considered when determining the specific band parameter. Additionally, the method may include determining the specific band parameter using fixed-point arithmetic and the determined shift, based on the first plurality of transform coefficients and the second plurality of transform coefficients falling within the specific frequency band.

A method for generating a bitstream based on a multichannel input signal is described. The method may include generating a frame sequence of a downmixer from a corresponding first frame sequence of the multichannel input signal. Furthermore, the method may include determining a spatial metadata frame sequence from a second frame sequence of the multichannel input signal. The frame sequence of the downmixer and the spatial metadata frame sequence can be used to generate a multichannel upmixer. Additionally, the method may include generating a bitstream comprising a bitstream frame sequence. The bitstream frame may indicate a frame of the downmixer corresponding to a first frame of the first frame sequence of the multichannel input signal and a spatial metadata frame corresponding to a second frame of the second frame sequence of the multichannel input signal. The second frame may be different from the first frame.

According to another aspect, a method for generating a bitstream based on a multichannel input signal is described. The method may include determining a trim protection gain sequence for a corresponding frame sequence of the multichannel input signal. The current trim protection gain may indicate that the current frame of the multichannel input signal will be applied to prevent attenuation of the corresponding current frame of the downmixing signal. The method may continue to interpolate the current trim protection gain and a previous trim protection gain of the previous frame of the multichannel input signal to obtain a trim protection gain curve. Furthermore, the method may include applying the trim protection gain curve to the current frame of the multichannel input signal to obtain a current frame of attenuation of the multichannel input signal. The current frame of the downmixing signal's frame sequence can be generated from the current frame of the attenuation of the multichannel input signal. Additionally, the method may include determining a spatial metadata frame sequence from the multichannel input signal. The frame sequence of the downmixing signal and the spatial metadata frame sequence can be used to generate a multichannel upmixing signal. The bitstream may be generated such that it indicates the trim protection gain sequence, the frame sequence of the downmixing signal, and the spatial metadata frame sequence, enabling the generation of a multichannel upmixing signal based on the bitstream.

According to another aspect, a method for generating a bitstream is described, the bitstream indicating a downmixing signal and spatial metadata, the spatial metadata being used to generate a multi-channel upmixing signal from the downmixing signal. The method may include generating the downmixing signal from a multi-channel input signal. Furthermore, the method may include determining one or more control settings based on one or more external settings, wherein the one or more external settings include an update period indicating a time period required for the decoding system to synchronize with the bitstream. The method may also include determining a spatial metadata frame sequence for a corresponding frame sequence of the multi-channel input signal based on the one or more control settings. Additionally, the method may include encoding one or more spatial metadata frames in the spatial metadata frame sequence as independent frames according to the update period.

According to another aspect, a software program is described. This software program is adapted to execute on a processor, and is adapted to perform the method steps outlined in this document when executed on a processor.

According to another aspect, a storage medium is described. This storage medium may include a software program adapted to execute on a processor and, when executed on a processor, to perform the method steps outlined in this document.

According to another aspect, a computer program product is described. This computer program product may include executable instructions for performing the method steps outlined in this document when executed on a computer.

It should be noted that the methods and systems, including the preferred embodiments outlined in this patent application, can be used independently or in combination with other methods and systems disclosed in this document. Furthermore, all aspects of the methods and systems outlined in this patent application can be combined arbitrarily. Specifically, the features of the claims can be combined with each other in any manner.

Attached Figure Description

The invention will now be described by way of example with reference to the accompanying drawings, wherein...

Figure 1 shows a generalized block diagram of an example audio processing system for performing spatial synthesis;

Figure 2 shows an example of the system in Figure 1;

Figure 3 is similar to the example audio processing system shown in Figure 1 for performing spatial synthesis;

Figure 4 shows an example audio processing system for performing spatial analysis;

Figure 5a shows a block diagram of an example parametric multichannel audio coding system;

Figure 5b shows a block diagram of an example spatial analysis and coding system;

Figure 5c illustrates an example time-frequency representation of the frames of a multichannel audio signal;

Figure 5d illustrates an example time-frequency representation of multiple channels in a multi-channel audio signal;

Figure 5e shows an example windowing application used in the transform unit of the spatial analysis and coding system shown in Figure 5b;

Figure 6 shows a flowchart of an example method for reducing the data rate of spatial metadata;

Figure 7a illustrates an example transition scheme for spatial metadata to be performed at the decoding system;

Figures 7b to 7d illustrate example window functions used to determine spatial metadata;

Figure 8 shows a block diagram of an example processing path for a parametric multichannel codec system;

Figures 9a and 9b show block diagrams of an example parametric multichannel audio coding system configured to perform trim protection and/or dynamic range control.

Figure 10 illustrates an example method for compensating DRC parameters; and

Figure 11 shows an example interpolation curve used for trim protection.

Detailed Implementation

As outlined in the introduction, this document relates to a multichannel audio coding system using a parametric multichannel representation. An example multichannel audio coding and decoding (encoding/decoding) system is described below. Within the context of Figures 1 to 3, the decoder of the audio codec system is described as using the received parametric multichannel representation to generate an n-channel upmix signal Y (typically, n > 2) from a received m-channel downmix signal X (e.g., m = 2). Subsequently, the encoder-related processing of the multichannel audio codec system is described. Specifically, how the parametric multichannel representation and the m-channel downmix signal can be generated from the n-channel input signal is described.

Figure 1 illustrates a block diagram of an example audio processing system 100 configured to generate an overmix signal Y from a downmix signal X and a set of mixing parameters. Specifically, the audio processing system 100 is configured to generate the overmix signal based solely on the downmix signal X and the set of mixing parameters. From the bitstream P, the _audio decoder 140 extracts the downmix signal X = [ _l0r0 ] ^T and the set of mixing parameters. In the illustrated example, the set of mixing parameters includes parameters _α1 , _α2 , _α3 , _β1 , _β2 , _β3 , g, _k1 , and _k2 . The mixing parameters may be included in each mixing parameter data field in the bitstream P in quantized and/or entropy encoded form. The mixing parameters may be referred to as metadata (or spatial metadata), which is transmitted along with the encoded downmix signal X. In some instances of this disclosure, it has been explicitly indicated that some connecting lines are adapted to transmit multichannel signals, wherein these lines are provided with crossover lines adjacent to each number of channels. In the system 100 shown in Figure 1, the downmix signal X includes m = 2 channels, and the upmix signal Y, which will be defined below, includes n = 6 channels (e.g., 5.1 channels).

The upmixing stage 110, whose operation is parameterized to the mixing parameters, receives the downmixing signal. The downmixing modification processor 120 modifies the downmixing signal through nonlinear processing and by forming a linear combination of downmixing channels to obtain the modified downmixing signal D = [ _{d1 d2} ] ^T . The first mixing matrix 130 receives the downmixing signal X and the modified downmixing signal D, and outputs the upmixing signal Y = [l _f l _s r _{f r s} c lfe] ^T by forming the following linear combination:

In the linear combination above, the mixing parameter _α3 controls the contribution of the intermediate-type signal (proportional to _l0 + _r0 ) formed from the downmix to all channels in the upmix. The mixing parameter _β3 controls the contribution of the side-type signal (proportional to _l0 - _r0 ) to all channels in the upmix. Therefore, in practical applications, it is reasonable to expect that the mixing parameters _α3 and _β3 will have different statistical properties, which allows for more efficient encoding. (For comparison, consider a reference parameterization (where independent mixing parameters control the left and right channel contributions of the downmix to the spatial left and right channels in the upmix), note that the statistical observables of such mixing parameters may not be significantly different.)

Returning to the linear combination shown in the equations above, note further that the gain parameters _k1 and _k2 can depend on a single ^, shared mixing parameter in the bitstream P. Furthermore, the gain parameters can be normalized such that _k1² + _k2² = ¹ .

The contributions of the modified downmixer to the spatial left and right channels of the upmixer can be controlled by parameters _β1 (the contribution of the first modified channel to the left channel) and _β2 (the contribution of the second modified channel to the right channel), respectively. Furthermore, the contribution of each channel in the downmixer to its spatially corresponding channel in the upmixer can be individually controlled by changing the independent mixing parameter g. Preferably, the gain parameter g is non-uniformly quantized to avoid large quantization errors.

Referring now to Figure 2, the downmixing modifier 120 can perform the following linear combination (which is crossmixing) of the downmixing channels in the second mixing matrix 121:

As indicated by the formula, the gain filling the second mixing matrix can be parameterized to depend on some of the mixing parameters encoded in the bitstream P. The processing performed by the second mixing matrix 121 yields an intermediate signal Z = [ _{z1 z2} ] ^T , which is supplied to the decorrecorder 122. Figure 1 shows an example of the decorrecorder 122 comprising two sub-decorrecorders 123, 124, which can be configured identically (i.e., providing the same output in response to the same input) or differently. As an alternative, Figure 2 shows an example of all decorrecord-related operations performed by a single unit 122, which outputs a preliminarily modified downmixed signal D'. The downmixing modification processor 120 in Figure 2 may also include an artifact attenuator 125. In the example embodiment, as outlined above, the artifact attenuator 125 is configured to detect tails in the intermediate signal Z and take corrective action by attenuating unwanted artifacts in the signal based on the position of the detected tail. This attenuation generates a modified downmixing signal D, which is output from the downmixing modification processor 120.

Figure 3 illustrates a first mixing matrix 130 of a similar type to that shown in Figure 1, and its associated transform stages 301, 302 and inverse transform stages 311, 312, 313, 314, 315, 316. The transform stages may include, for example, filter banks, such as quadrature mirror filter banks (QMF). Therefore, signals upstream of transform stages 301, 302 are represented in the time domain, just as signals downstream of inverse transform stages 311, 312, 313, 314, 315, 316 are represented in the frequency domain. The time dependencies of other signals can be expressed, for example, as block values or discrete values related to the time blocks to which the signal is segmented. Note that Figure 3 uses alternative notation compared to the matrix equations above; one may, for example, have correspondences such as _XL0 ~ _l0 , _XL0 ~ _r0 , _YL ~ _lf , _YLS ~ _ls , etc. Furthermore, the notation in Figure 3 emphasizes the difference between the time-domain representation of the signal, _XL0 (t), and the frequency-domain representation of the same signal, _XL0 (f). It is understood that the frequency-domain representation is divided into time frames; therefore, it is a function of both time and frequency variables.

Figure 4 illustrates an audio processing system 400 for generating a downmixed signal X and mixing parameters _α1 , _α2 , α3, _β1 , _β2 , _β3 , g, _k1 , _k2 that control the gain applied by the upmixing stage 110. This audio processing system 400 is typically located on the encoder side, for example, in a broadcast or recording device, while the system 100 shown in Figure 1 is typically deployed on _the decoder side, for example, in a playback device. The downmixing stage 410 generates an m-channel signal X based on an n-channel signal Y. Preferably, the downmixing stage 410 operates on the time-domain representation of these signals. The parameter extractor 420 can generate the values of the mixing parameters _α1 , _α2 , _α3 , β1, _β2 , _β3 , g, _k1 , _k2 by analyzing the n-channel signal Y and considering the quantitative and qualitative properties of the downmixing stage ₄₁₀ . The mixing parameters can be a vector of frequency block values, as indicated by the notation in Figure 4, and can be further subdivided into time blocks. In the example implementation, the downmixing stage 410 is time-invariant and/or frequency-invariant. Due to its time and/or frequency invariance, a communication connection is typically not required between the downmixing stage 410 and the parameter extractor 420, but parameter extraction can be performed independently. This provides considerable flexibility for implementation. It also allows for the possibility of reducing the overall system delay, as several processing steps can be executed in parallel. As an example, the Dolby Digital Plus format (or Enhanced AC-3) can be used to encode the downmixed signal X.

The parameter extractor 420 can learn about the quantitative and/or qualitative properties of the downmixing stage 410 by accessing a downmixing specification, which may specify one of the following: a set of gain values, an index identifying a predefined downmixing mode for its predefined gain, etc. The downmixing specification may be data pre-loaded into the memory of each of the downmixing stages 410 and the parameter extractor 420. Alternatively or additionally, the downmixing specification may be transmitted from the downmixing stage 410 to the parameter extractor 420 via a communication line connecting these units. Alternatively, each of the downmixing stages 410 to the parameter extractor 420 may access the downmixing specification from a common data source, such as memory in an audio processing system or in a metadata stream associated with the input signal Y (e.g., the memory of configuration unit 540 shown in FIG. 5a).

Figure 5a illustrates an example multichannel coding system 500 for encoding a multichannel audio input signal Y561 (including n channels) using a downmixed signal X (comprising m channels, where m < n) and a parameterized representation. System 500 includes a downmixing coding unit 510, which includes, for example, the downmixing stage 410 of Figure 4. The downmixing coding unit 510 can be configured to provide an encoded version of the downmixed signal X. The downmixing coding unit 510 can encode the downmixed signal X, for example, using a Dolby Digital Plus encoder. Furthermore, system 500 includes a parameter coding unit 510, which can include the parameter extractor 420 of Figure 4. The parameter coding unit 510 can be configured to quantize and encode the set of mixing parameters _α1 , _α2 , _α3 , _β1 , _β2 , _β3 , g, _k1 (also referred to as spatial parameters) to obtain encoded spatial parameters 562. As indicated above, parameter _k2 can be determined from parameter _k1 . Additionally, system 500 may include a bitstream generation unit 530 configured to generate a bitstream P 564 from an encoded downmixed signal 563 and encoded spatial parameters 562. Bitstream 564 can be encoded according to a predetermined bitstream syntax. Specifically, bitstream 564 can be encoded in a format conforming to Dolby Digital Plus (DD+ or E-AC-3, Enhanced AC-3).

System 500 may include a configuration unit 540 configured to determine one or more control settings 552, 554 for parameter encoding unit 520 and/or downmixing encoding unit 510. The one or more control settings 552, 554 may be determined based on one or more external settings 551 of system 500. For example, the one or more external settings 551 may include the total (maximum or fixed) data rate of bitstream 564. Configuration unit 540 may be configured to determine one or more control settings 552 based on the one or more external settings 551. The one or more control settings 552 for parameter encoding unit 520 may include one or more of the following:

• The maximum data rate for the encoded spatial parameter 562. This control setting is referred to as the metadata data rate setting in this document.

• The maximum and/or specific number of parameter sets determined by parameter encoding unit 520 for each frame of audio signal 561. This control setting is referred to herein as the temporal resolution setting because it allows for the temporal resolution to be affected by the spatial parameters.

The parameter encoding unit 520 determines the number of parameter bands for the spatial parameters. This control setting is referred to herein as the frequency resolution setting because it allows the frequency resolution of the spatial parameters to be affected.

• The resolution of the quantizer used to quantize the spatial parameters. This control setting is referred to as the quantizer setting in this document.

The parameter encoding unit 520 may use one or more of the control settings 552 mentioned above for determining and/or encoding spatial parameters to be included in the bitstream 564. Typically, the input audio signal Y 561 is segmented into a sequence of frames, where each frame includes a predetermined number of samples of the input audio signal Y 561. The metadata data rate setting may indicate the maximum number of bits available for encoding the spatial parameters of the frames of the input audio signal 561. The actual number of bits used to encode the spatial parameters 562 of the frames may be less than the number of bits allocated by the metadata data rate setting. The parameter encoding unit 520 may be configured to notify the configuration unit 540 about the actual number of bits used 553, thereby enabling the configuration unit 540 to determine the number of bits available for encoding the downmix signal X. This number of bits may be transmitted to the downmix encoding unit 510 as a control setting 554. The downmix encoding unit 510 may be configured (e.g., using a multi-channel encoder such as Dolby Digital Plus) to encode the downmix signal X based on the control setting 554. In this way, bits that have not yet been used to encode spatial parameters can be used to encode the downmixed signal.

Figure 5b shows a block diagram of an example parameter encoding unit 520. The parameter encoding unit 520 may include a transformation unit 521 configured to determine the frequency representation of the input signal 561. Specifically, the transformation unit 521 may be configured to transform frames of the input signal 561 into one or more spectra, each spectrum comprising multiple frequency intervals. For example, the transformation unit 521 may be configured to apply a filter bank (e.g., a QMF filter bank) to the input signal 561. The filter bank may be a critical sampling filter bank. The filter bank may include a predetermined number of Q filters (e.g., Q = 64 filters). Thus, the transformation unit 521 may be configured to determine Q sub-band signals from the input signal 561, wherein each sub-band signal is associated with a corresponding frequency interval 571. For example, K sampled frames of the input signal 561 may be transformed into Q sub-band signals, wherein each sub-band signal has K/Q frequency coefficients. In other words, K sampled frames of the input signal 561 are transformed into K/Q spectra, wherein each spectrum comprises Q frequency intervals. In a specific example, the frame length is K = 1536, the number of frequency ranges is Q = 64, and the number of spectrums is K/Q = 24.

The parameter encoding unit 520 may include a banding unit 522 configured to group one or more frequency ranges 571 into frequency bands 572. The grouping of frequency ranges 571 to frequency bands 572 may depend on a frequency resolution setting 552. Table 1 illustrates example mappings of frequency ranges 571 to frequency bands 572, which may be applied by the banding unit 522 based on the frequency resolution setting 552. In the illustrated example, the frequency resolution setting 552 may indicate banding of frequency ranges 571 to 7, 9, 12, or 15 frequency bands. Banding typically models the psychoacoustic behavior of the human ear. As a result, the number of frequency ranges 571 for each frequency band 572 typically increases with increasing frequency.

Table 1

The parameter determination unit 523 of the parameter encoding unit 520 (and specifically, the parameter extractor 420) can be configured to determine one or more sets of mixing parameters _α1 , _α2 , α3, _β1 , _β2 , _β3 , g, _{k1 , k2 for} each frequency band 572. Therefore, frequency band 572 can also be referred to as a parameter band. The mixing parameters _α1 , _α2 , α3, _β1 , _β2 , _β3 , g, _k1 , _k2 for frequency band 572 can be referred to as band parameters. _Thus , the entire set of mixing parameters typically includes band parameters for each frequency band 572. The band parameters can be applied to the mixing matrix 130 of FIG. 3 to determine the sub-band version of the decoded upmixed signal.

The number of mixed parameter sets for each frame determined by the parameter determination unit 523 can be indicated by the temporal resolution setting 552. For example, the temporal resolution setting 552 can indicate that one or two mixed parameter sets will be determined for each frame.

Figure 5c illustrates the determination of a set of mixed parameters including band parameters for multiple frequency bands 572. Figure 5c illustrates an example set of transform coefficients 580 derived from frames of input signal 561. Transform coefficients 580 correspond to specific times 582 and specific frequency intervals 571. Frequency bands 572 may include multiple transform coefficients 580 from one or more frequency intervals 571. As can be seen from Figure 5c, the transform of time-domain sampling of input signal 561 provides a time-frequency representation of frames of input signal 561.

It should be noted that the set of hybrid parameters for the current frame can be determined based on the transform coefficients 580 of the current frame and possibly also on the transform coefficients 580 of the immediately following frame (which is also referred to as the look-ahead frame).

The parameter determination unit 523 can be configured to determine _{the mixing parameters α1, α2, α3, β1, β2, β3 , g , k1 , and k2} for each frequency band 572. If the time resolution setting is set to 1, all transform coefficients 580 of a specific frequency band 572 (for the current frame and the forward-looking frame) can be considered to determine the mixing parameters for that specific frequency band 572. Alternatively, the parameter determination unit 523 can be configured to determine two sets of mixing parameters for each frequency band 572 (e.g., when the time resolution setting is set to 2). In this case, the first half-time of the transform coefficients 580 of the specific frequency band 572 (corresponding to, for example, the transform coefficients 580 of the current frame) can be used to determine the first set of mixing parameters, while the second half-time of the transform coefficients 580 of the specific frequency band 572 (corresponding to, for example, the transform coefficients 580 of the forward-looking frame) can be considered to determine the second set of mixing parameters.

Generally, the parameter determination unit 523 can be configured to determine one or more sets of mixed parameters based on the transform coefficients 580 of the current frame and the forward-looking frame. A window function can be used to limit the effect of the transform coefficients 580 on the one or more sets of mixed parameters. The shape of the window function can depend on the number of sets of mixed parameters for each frequency band 572 and/or the nature of the current frame and/or the forward-looking frame (e.g., the presence of one or more transients). An example window function will be described in the context of Figures 5e and 7b-7d.

It should be noted that the above applies to cases where the frame of input signal 561 does not include a transient signal portion. System 500 (e.g., parameter determination unit 523) can be configured to perform transient detection based on input signal 561. In the event that one or more transients are detected, one or more transient indicators 583, 584 can be set, wherein transient indicators 583, 584 can identify the moment 582 of the corresponding transient. Transient indicators 583, 584 can also be referred to as sampling points for each set of mixed parameters. In the case of a transient, parameter determination unit 523 can be configured to determine the set of mixed parameters based on the transform coefficients 580 starting from the moment of the transient (this is illustrated by the areas with different shaded lines in Figure 5c). Alternatively, transform coefficients 580 prior to the moment of the transient can be ignored, thereby ensuring that the set of mixed parameters reflects the multi-channel situation after the transient.

Figure 5c illustrates the transform coefficients 580 of the channels of the multi-channel input signal Y 561. The parameter encoding unit 520 is typically configured to determine the transform coefficients 580 for the multiple channels of the multi-channel input signal 561. Figure 5d shows example transform coefficients for the first 561-1 channel and the second 561-2 channel of the input signal 561. The frequency band p 572 comprises frequency intervals 571 ranging from frequency index i to j. The transform coefficient 580 of the first channel 561-1 at time (or in the spectrum) q, in frequency interval i, can be referred to as a _q,i . Similarly, the transform coefficient 580 of the second channel 561-2 at time (or in the spectrum) q, in frequency interval i, can be referred to as b _q,i . The transform coefficients 580 can be complex numbers. The determination of the mixing parameters for the frequency band p can involve determining the energy and/or covariance of the first channel 561-1 and the second channel 561-2 based on the transform coefficients 580. For example, the covariance of the transform coefficients 580 of the first channel 561-1 and the second channel 561-2 in the frequency band p with respect to the time interval [q,v] can be determined as follows:

The energy estimate of the transform coefficients 580 for the first channel 561-1 in frequency band p with respect to the time interval [q, v] can be determined as follows:

The energy estimate _E2,2 (p) of the transformation coefficients 580 for the second channel 561-2 in frequency band p for time interval [q,v] can be determined in a similar manner.

Thus, the parameter determination unit 523 can be configured to determine one or more band parameter sets 573 for different frequency bands 572. The number of frequency bands 572 typically depends on the frequency resolution setting 552, while the number of mixed parameter sets for each frame typically depends on the temporal resolution setting 552. For example, the frequency resolution setting 552 may indicate the use of 15 frequency bands 572, while the temporal resolution setting 552 may indicate the use of 2 mixed parameter sets. In this case, the parameter determination unit 523 can be configured to determine two temporally different mixed parameter sets, wherein each mixed parameter set includes 15 band parameter sets 573 (i.e., mixed parameters for different frequency bands 572).

As indicated above, the mixing parameters for the current frame can be determined based on the transform coefficients 580 of the current frame and the transform coefficients 580 of the following preceding frame. The parameter determination unit 523 may apply a window to the transform coefficients 580 to ensure a smooth transition between the mixing parameters of consecutive frames in the frame sequence, and/or to account for disruptive portions (e.g., transients) within the input signal 561. This is illustrated in Figure 5e, which shows the K/Q spectra 589 of the current frame 585 and the immediately following frame 590 of the input audio signal 561 at corresponding K/Q consecutive moments 582. Furthermore, Figure 5e shows an example window 586 used by the parameter determination unit 523. Window 586 reflects the effect of the K/Q spectra 589 of the current frame 585 and the immediately following frame 590 (which is referred to as the preceding frame) on the mixing parameters. As will be summarized in more detail below, window 586 reflects the case where the current frame 585 and the preceding frame 590 do not include any transients. In this case, window 586 ensures smooth inflation and deflation of the spectra 589 of the current frame 585 and the preceding frame 590, respectively, thereby allowing for smooth evolution of spatial parameters. Furthermore, Figure 5e shows example windows 587 and 588. The dashed window 587 reflects the effect of the K/Q spectra 589 of the current frame 585 on the mixing parameters of the previous frame. Additionally, the dashed window 588 reflects the effect of the K/Q spectra 589 of the immediately following frame 590 on the mixing parameters of the immediately following frame 590 (in the case of smooth interpolation).

The one or more sets of mixed parameters can then be quantized and encoded using the encoding unit 524 of the parameter encoding unit 520. The encoding unit 524 can apply various encoding schemes. For example, the encoding unit 524 can be configured to perform differential encoding of the mixed parameters. Differential encoding can be based on a time difference (for the same frequency band 572, the time difference between the current mixed parameter and the corresponding previous mixed parameter) or a frequency difference (the frequency difference between the current mixed parameter of the first frequency band 572 and the corresponding current mixed parameter of the adjacent second frequency band 572).

Furthermore, encoding unit 524 can be configured to quantize the set of mixed parameters and/or the time difference or frequency difference of the mixed parameters. The quantization of the mixed parameters can depend on quantizer setting 552. For example, quantizer setting 552 can take two values, a first value indicating fine quantization and a second value indicating coarse quantization. Thus, encoding unit 524 can be configured to perform fine quantization (with relatively low quantization error) or coarse quantization (with relatively increased quantization error) based on the quantization type indicated by quantizer setting 552. The quantized parameters or parameter differences can then be encoded using entropy-based codes (such as Huffman codes). As a result, encoded spatial parameters 562 are obtained. The number of bits 553 used for encoding spatial parameters 562 can be transmitted to configuration unit 540.

In an embodiment, encoding unit 524 can be configured to first quantize different mixing parameters (within consideration of quantizer setting 552) to obtain quantized mixing parameters. The quantized mixing parameters can then be entropy-coded (using, for example, Huffman codes). Entropy coding can then encode the quantized mixing parameters of a frame (regardless of previous frames), the frequency difference of the quantized mixing parameters, or the time difference of the quantized mixing parameters. Encoding the time difference may not be used in the case of so-called independent frames, which are encoded independently of previous frames.

Therefore, parameter encoding unit 520 can use a combination of differential coding and Huffman coding to determine the encoded spatial parameters 562. As outlined above, the encoded spatial parameters 562 can be included as metadata (also referred to as spatial metadata) in the bitstream 564 along with the encoded downmix signal 563. Differential coding and Huffman coding can be used for transmitting spatial metadata to reduce redundancy and thus increase the spare bit rate available for encoding the downmix signal 563. Because Huffman codes are variable-length codes, the size of the spatial metadata can vary considerably depending on the statistics of the encoded spatial parameters 562 to be transmitted. The data rate required to transmit the spatial metadata is subtracted from the data rate available to the core codec (e.g., Dolby Digital Plus) for encoding the stereo downmix signal. To avoid compromising the audio quality of the downmix signal, the number of bytes that may be spent transmitting spatial metadata for each frame is generally limited. This limit can be subject to encoder tuning considerations, which can be taken into account by configuration unit 540. However, due to the variable-length nature of the basic differential/Huffman coding of the spatial parameters, it is generally not guaranteed that the data rate limit (as reflected in the metadata data rate setting 552) will not be exceeded without any further means.

This document describes a method for post-processing encoded spatial parameters 562 and/or spatial metadata including encoded spatial parameters 562. The method 600 for post-processing spatial metadata is described in the context of Figure 6. Method 600 can be applied when the total size of a frame of spatial metadata exceeds a predefined limit indicated, for example, by metadata data rate setting 552. Method 600 aims to progressively reduce the amount of metadata. Reducing the size of spatial metadata typically also reduces the accuracy of the spatial metadata and thus compromises the quality of the spatial image of the reproduced audio signal. However, method 600 generally guarantees that the total amount of spatial metadata does not exceed a predefined limit, thus allowing for the determination of a trade-off between the spatial metadata (used to regenerate the m-channel multichannel signal) and the audio codec metadata (used to decode the encoded downmix signal 563) in terms of overall audio quality. Furthermore, method 600 for post-processing spatial metadata can be implemented with relatively low computational complexity (compared to completely recalculating the encoded spatial parameters with modified control settings 552).

Method 600 for post-processing spatial metadata may include one or more of the following steps. As outlined above, each spatial metadata frame may include multiple (e.g., one or two) parameter sets, wherein the use of additional parameter sets allows for increased temporal resolution of the mixed parameters. The use of multiple parameter sets per frame can improve audio quality, especially in the case of attack-rich (i.e., transient) signals. Even in the case of audio signals with relatively slowly changing spatial images, spatial parameter updates with a grid twice the size of the sampling points can improve audio quality. However, sending multiple parameter sets per frame results in an approximately 2x increase in data rate. Therefore, if it is determined that the data rate of the spatial metadata exceeds the metadata data rate setting 552 (step 601), it can be checked whether the spatial metadata frame includes more than one mixed parameter set. Specifically, it can be checked whether the metadata frame includes two mixed parameter sets that should be sent (step 602). If it is determined that the spatial metadata includes multiple mixed parameter sets, one or more of the sets exceeding a single mixed parameter set can be discarded (step 603). As a result, the data rate of spatial metadata can be significantly reduced (typically by half in the case of two mixed parameter sets), while only relatively little loss to audio quality.

The decision of which of two (or more) mixed parameter sets to discard can depend on whether the encoding system 500 detects a transient location (“attack”) in the portion of the input signal 561 covered by the current frame: if multiple transients exist in the current frame, earlier transients are generally more significant than later transients due to the psychoacoustic backmasking effect of each individual attack. Therefore, if a transient exists, it may be recommended to discard the later mixed parameter set (e.g., the second of two). On the other hand, if no attack exists, the earlier mixed parameter set (e.g., the first of two) may be discarded. This may be due to the windowing used when calculating spatial parameters (as shown in Figure 5e). The window 586 used to window out the portion of the input signal 561 used to calculate the spatial parameters for the second mixed parameter set typically has its greatest influence at the time point when the sampling points for parameter reconstruction are placed in the upper mixing stage 130 (i.e., at the end of the current frame). On the other hand, the first mixed parameter set typically receives an offset of half the frame at that time point. Therefore, the error resulting from discarding the first set of mixed parameters is most likely lower than the error resulting from discarding the second set of mixed parameters. This is illustrated in Figure 5e, where it can be seen that the second half of the spectrum 589 of the current frame 585 used to determine the second set of mixed parameters is more influenced by the sampling of the current frame 585 than the first half of the spectrum 589 of the current frame 585 (for the first half, the value of the window function 586 is lower than the value for the second half of the spectrum 589).

The spatial cue (i.e., the mixing parameters) computed in the encoding system 500 is sent to the corresponding decoder 100 via a bitstream 562 (which may be part of a bitstream 564 in which the encoded stereo downmix signal 563 is delivered). Between the computation of the spatial cue and its representation in the bitstream 562, the encoding unit 524 typically applies a two-step encoding approach: the first step, quantization, is a lossy step because it adds error to the spatial cue; the second step, differential/Huffman coding, is a lossless step. As outlined above, the encoder 500 can choose between different types of quantization (e.g., two types of quantization): a high-resolution quantization scheme, which adds relatively little error but results in a larger potential quantization index, thus requiring a larger Huffman codeword; and a low-resolution quantization scheme, which adds relatively more error but results in a lower amount of quantization index, thus not requiring such a large Huffman codeword. It should be noted that different types of quantization can be applied to some or all of the mixing parameters. For example, different types of quantization can be applied to the mixed parameters _α1 , _α2 , _α3 , _β1 , _β2 , _β3 , _k1 . On the other hand, the gain g can be quantized using a fixed type of quantization.

Method 600 may include a step 604 of verifying which type of quantization has been used to quantize the spatial parameters. If it is determined that a relatively fine quantization resolution was used, the encoding unit 524 may be configured to reduce the quantization resolution to a lower type of quantization 605. As a result, the spatial parameters are quantized again. However, this does not significantly increase computational overhead (compared to redetermining the spatial parameters using different control settings 552). It should be noted that different types of quantization can be used for different spatial parameters _α1 , _α2 , _α3 , _β1 , _β2 , _β3 , g, _k1 . Therefore, the encoding unit 524 may be configured to select the quantizer resolution individually for each type of spatial parameter, thereby adjusting the data rate of the spatial metadata.

Method 600 may include a step of reducing the frequency resolution of the spatial parameters (not shown in Figure 6). As outlined above, the set of mixed parameters for a frame is typically clustered into frequency bands or parameter bands 572. Each parameter band represents a frequency range, and for each band, a separate set of spatial clues is determined. The number of parameter bands 572 (e.g., 7, 9, 12, or 15 bands) can be progressively changed depending on the data rate available for transmitting spatial metadata. The number of parameter bands 572 is approximately linearly related to the data rate, and therefore a reduction in frequency resolution can significantly reduce the data rate of spatial metadata while only moderately affecting audio quality. However, such a reduction in frequency resolution typically requires recalculating the set of mixed parameters using the changed frequency resolution, thus increasing computational complexity.

As outlined above, encoding unit 524 can use differential coding of (quantized) spatial parameters. Configuration unit 551 can be configured to apply direct coding of the spatial parameters of the frames of the input audio signal 561 to ensure that transmission errors do not propagate over an infinite number of frames and to allow the decoder to synchronize with the received bitstream 562 at intermediate moments. Thus, a small portion of a frame can be used without differential coding along the timeline. Such frames without differential coding can be referred to as independent frames. Method 600 may include a step 606 to verify whether the current frame is an independent frame and/or whether the independent frame is a forced independent frame. The coding of the spatial parameters may depend on the result of step 606.

As outlined above, differential coding is typically designed to compute differences between temporal successors or between adjacent frequency bands of quantized spatial cues. In both cases, the statistics of spatial cues mean that smaller differences occur more frequently than larger differences, and therefore, smaller differences are represented by shorter Huffman codewords compared to larger differences. This document proposes performing smoothing (temporally or frequency-wise) of the quantized spatial parameters. Smoothing spatial parameters temporally or frequency-wise generally results in smaller differences and thus a reduction in data rate. Due to psychoacoustic considerations, temporal smoothing is generally preferred over frequency-wise smoothing. If it is determined that the current frame is not a forced independent frame, method 600 can proceed with temporal differential coding (step 607), possibly combined with temporal smoothing. On the other hand, if the current frame is determined to be an independent frame, method 600 can proceed with frequency differential coding (step 608), possibly with frequency smoothing.

The differential coding in step 607 can be submitted to temporal smoothing to reduce the data rate. The degree of smoothing can be varied depending on the amount by which the data rate will be reduced. The most severe type of temporal "smoothing" corresponds to keeping the previous set of mixed parameters unchanged, which corresponds to sending only incremental values equal to zero. Temporal smoothing of differential coding can be performed on one or more (e.g., all) of the spatial parameters.

Frequency smoothing can be performed in a similar manner to temporal smoothing. In its most extreme form, frequency smoothing corresponds to sending the same quantized spatial parameters over the entire frequency range of the input signal 561. While ensuring that the limits set by the metadata data rate settings are not exceeded, frequency smoothing can have a relatively high impact on the quality of the spatial image that can be reproduced using spatial metadata. Therefore, it may be preferable to apply frequency smoothing only when temporal smoothing is not permitted (e.g., if the current frame is a forced independent frame for which temporal differential coding of the previous frame is not available).

As outlined above, system 500 can operate subject to one or more external settings 551, such as the overall target data rate of bitstream 564 or the sampling rate of input audio signal 561. Typically, there is no single optimal operating point for all combinations of external settings. Configuration unit 540 can be configured to map valid combinations of external settings 551 to combinations of control settings 552, 554. For example, configuration unit 540 can rely on the results of psychoacoustic listening tests. Specifically, configuration unit 540 can be configured to determine combinations of control settings 552, 554 that ensure (on average) optimal psychoacoustic coding results for a particular combination of external settings 551.

As outlined above, the decoding system 100 should be able to synchronize with the received bitstream 564 within a given time period. To ensure this, the encoding system 500 can periodically encode so-called independent frames (i.e., frames that do not depend on knowledge about their predecessors). The average distance between two independent frames can be given by the ratio between the maximum latency for synchronization and the duration of a frame. This ratio does not necessarily have to be an integer, where the distance between two independent frames is always an integer.

Encoding system 500 (e.g., configuration unit 540) can be configured to receive the maximum latency or desired update time period for synchronization as an external setting 551. Furthermore, encoding system 500 (e.g., configuration unit 540) may include a timer module configured to track the absolute amount of time elapsed since the first encoded frame of bitstream 564. The first encoded frame of bitstream 564 is defined as an independent frame. Encoding system 500 (e.g., configuration unit 540) can be configured to determine whether the next encoded frame includes a sample corresponding to a time point that is an integer multiple of the desired update time period. Whenever the next encoded frame includes a sample at a time point that is an integer multiple of the desired update time period, encoding system 500 (e.g., configuration unit 540) can be configured to ensure that the next encoded frame is encoded as an independent frame. By doing so, it is ensured that the desired update time period is maintained even if the ratio of the desired update time period to the frame length is not an integer.

As outlined above, the parameter determination unit 523 is configured to calculate spatial cues based on the time/frequency representation of the multichannel input signal 561. Spatial metadata frames can be determined based on K/Q (e.g., 24) spectra 589 (e.g., QMF spectra) of the current frame and/or K/Q (e.g., 24) spectra 589 (e.g., QMF spectra) of the forward-looking frame, where each spectrum 589 can have a frequency resolution of Q (e.g., 64) frequency intervals 571. Depending on whether the encoding system 500 detects transients in the input signal 561, the time length of the signal portion used to calculate a single set of spatial cues can include a different number of spectra 589 (e.g., from 1 spectrum up to twice the number of K/Q spectra). As shown in Figure 5c, each spectrum 589 is divided into a certain number of frequency bands 572 (e.g., 7, 9, 12, or 15 bands), which, due to psychoacoustic considerations, include a different number of frequency intervals 571 (e.g., from 1 frequency interval up to 41 frequencies). Different frequency bands p 572 and different time segments [q,v] define a grid on the time/frequency representation of the current frame and the forward-looking frame of the input signal 561. For different “boxes” within this grid, different sets of spatial cues can be calculated based on estimates of the energy and/or covariance of at least some of the input channels within each “box.” As outlined above, the energy estimate and/or covariance can be calculated by summing the squares of the transform coefficients 580 of a channel and/or by summing the products of the transform coefficients 580 of different channels, respectively (as indicated by the formulas provided above). The different transform coefficients 580 can be weighted according to a window function 586 used to determine the spatial parameters.

The calculation of energy estimates E _{_1,1} (p), E _{_2,2} (p), and/or covariance E _{_1,2}(p) can be performed using fixed-point arithmetic. In this case, the different sizes of the “boxes” of the time/frequency grid can affect the arithmetic accuracy of the values determined for the spatial parameters. As outlined above, the number of frequency intervals (j-i+1) 571 for each band 572 and/or the length of the time interval [q,v] of the “boxes” of the time/frequency grid can vary significantly (e.g., between 1×1×2 and 48×41×2 transform coefficients 580 (e.g., the real and complex parts of complex QMF coefficients)). As a result, the number of products Re{ _at,f }Re{b _,f } and Im{at,f}Im{ _b,f } that need to be summed to determine the energy E _{_1,1 (p} )/covariance E_1,2(p) can vary significantly. To prevent the calculated result from exceeding the range of quantities that can be represented by fixed-point arithmetic, the signal can be scaled down by the maximum number of bits (e.g., scaled down by 6 bits since 2 ^{^6} * 2^ ⁶ = 4096 ≥ 48 * 41 * 2). However, for smaller “boxes” and/or for “boxes” that only include relatively low signal energy, this method results in a significant reduction in arithmetic accuracy.

This document proposes using a separate scale for each “box” of the time/frequency grid. The separate scale can depend on the number of transform coefficients 580 included within the “box” of the time/frequency grid. Typically, the spatial parameters for a particular “box” of the time/frequency grid (i.e., for a specific frequency band 572 and for a specific time interval [q,v]) are determined solely based on the transform coefficients 580 from that particular “box” (and not depending on the transform coefficients 580 from other “boxes”). Furthermore, the spatial parameters are typically determined solely based on the energy estimate and/or covariance ratio (and are generally unaffected by the absolute energy estimate and/or covariance). In other words, a single spatial line typically does not use the energy estimate and/or cross-channel product from a single time/frequency “box.” Moreover, the spatial line is generally unaffected by the absolute energy estimate/covariance, but only by the energy estimate/covariance ratio. Therefore, a separate scale can be used for each individual “box.” This scale should be matched to the channels that contribute to a particular spatial line.

For frequency band p572 and for time interval [q, v], the energy estimates _E1,1 (p) and _E2,2 (p) of the first channel 561-1 and the second channel 561-2, and the covariance _E1,2 (p) between the first channel 561-1 and the second channel 561-2, can be determined, for example, as indicated by the formulas above. The energy estimates and covariance can be scaled by a scaling factor _sp to provide scaled energy and covariance: _sp · _E1,1 (p), _sp · _E2,2 (p), and _sp · _E1,2 (p). The spatial parameter P(p) derived from the energy estimates _E1,1 (p), _E2,2 (p) and covariance _E1,2 (p) typically depends on the ratio of energy and/or covariance such that the value of the spatial parameter P(p) is independent of the scaling factor _sp . As a result, different scaling factors _sp , sp ₊₁ , and _sp+2 can be used for different frequency bands p, p+1, and p+2.

It should be noted that one or more of the spatial parameters may depend on more than two different input channels (e.g., three different channels). In this case, the one or more spatial parameters can be derived based on the energy estimates _E1,1 (p), _E2,2 (p), ... for the different channels, and based on the covariances between different pairs of channels (i.e., _E1,2 (p), _E1,3 (p), _E2,3 (p), etc.). Furthermore, in this case, the values of the one or more spatial parameters are independent of the scaling factors applied to the energy estimates and/or covariances.

Specifically, the scaling factor _sp = 2 ^{- zp} for a specific frequency band p (where _zp is a positive integer indicating the shift in fixed-point arithmetic) can be determined such that

0.5＜s _p ·max{|E _1,1 (p)|, |E _2,2 (p)|, |E _1,2 (p)|}≤1.0

And this minimizes the shift _zp . By ensuring this individually for each frequency band p and/or for each time interval [q, v] that determines the mixing parameters, increased (e.g., maximum) precision in fixed-point arithmetic can be achieved while ensuring a valid range of values.

For example, individual scaling can be achieved by checking whether the result of each individual MAC (multiply-accumulate) operation exceeds +/- 1. Only if this is the case can the individual scaling for the "box" be increased by one bit. Once this is done for all channels, the maximum scaling for each "box" can be determined, and all deviations in scaling for the "box" can be adjusted accordingly.

As outlined above, spatial metadata may include one or more (e.g., two) sets of spatial parameters per frame. Thus, encoding system 500 may send one or more sets of spatial parameters per frame to the corresponding decoding system 100. Each of these sets of spatial parameters corresponds to a specific spectrum in the K/Q time-series of the spectrum 289 of the spatial metadata frame. This specific spectrum corresponds to a specific moment, which may be referred to as a sampling point. Figure 5c shows two example sampling points 583 and 584 for the two sets of spatial parameters. Sampling points 583 and 584 may be associated with specific events included within the input audio signal 561. Alternatively, the sampling points may be predetermined.

Sampling points 583 and 584 indicate the moments when the corresponding spatial parameters should be fully utilized by the decoding system 100. In other words, the decoding system 100 can be configured to update the spatial parameters at sampling points 583 and 584 based on the transmitted set of spatial parameters. Furthermore, the decoding system 100 can be configured to interpolate spatial parameters between two subsequent sampling points. Spatial metadata can indicate the type of transition to be performed between successive sets of spatial parameters. Examples of transition types are “smooth” and “steep” transitions between spatial parameters, meaning that the spatial parameters can be interpolated smoothly (e.g., linearly) or updated abruptly, respectively.

In the case of a “smooth” transition, the sampling point can be fixed (i.e., predetermined) and therefore does not need to be transmitted as a signal in the bitstream 564. If the spatial metadata frame delivers a single set of spatial parameters, the predetermined sampling point can be the position at the very end of the frame, i.e., the sampling point can correspond to the (K/Q)th spectrum 589. If the spatial metadata frame delivers two sets of spatial parameters, the first sampling point can correspond to the (K/2Q)th spectrum 589, and the second sampling point can correspond to the (K/Q)th spectrum 589.

In the case of a "steep" transition, sampling points 583 and 584 can be variable and can be transmitted as signals in bitstream 562. The portion of bitstream 562 carrying the following information can be referred to as the "framing" portion of bitstream 562: information about the number of spatial parameter sets used in a frame, information about the choice between "smooth" and "steep" transitions, and information about the position of the sampling points in the case of a "steep" transition. Figure 7a shows an example transition scheme that can be applied by decoding system 100 based on the framing information included in the received bitstream 562.

For example, the framing information for a particular frame may indicate a “smooth” transition and a single spatial parameter set 711. In this case, the decoding system 100 (e.g., the first mixing matrix 130) may assume that the sampling points of the spatial parameter set 711 correspond to the last spectrum of the particular frame. Furthermore, the decoding system 100 may be configured to perform (e.g., linear) interpolation 701 between the last received spatial parameter set 710 for the immediately preceding frame and the spatial parameter set 711 for the particular frame. In another example, the framing information for a particular frame may indicate a “smooth” transition and two spatial parameter sets 711, 712. In this case, the decoding system 100 (e.g., the first mixing matrix 130) may assume that the sampling points of the first spatial parameter set 711 correspond to the last spectrum of the first half of the particular frame, and the sampling points of the second spatial parameter set 712 correspond to the last spectrum of the second half of the particular frame. Furthermore, the decoding system 100 can be configured to perform (e.g., linear) interpolation 702 between the last received spatial parameter set 710 for the preceding frame and the first spatial parameter set 711, and between the first spatial parameter set 711 and the second spatial parameter set 712.

In another example, the framing information for a particular frame may indicate a "steep" transition, a single spatial parameter set 711, and a sampling point 583 for that single spatial parameter set 711. In this case, the decoding system 100 (e.g., the first mixing matrix 130) may be configured to apply the last received spatial parameter set 710 to the immediately preceding frame up to sampling point 583, and to apply spatial parameter set 711 starting from sampling point 583 (as shown by curve 703). In another example, the framing information for a particular frame may indicate a "steep" transition, two spatial parameter sets 711, 712, and two corresponding sampling points 583, 584 for the two spatial parameter sets 711, 712, respectively. In this configuration, the decoding system 100 (e.g., the first mixing matrix 130) can be configured to apply the last received set of spatial parameters 710 to the immediately preceding frame up to the first sampling point 583, apply the first set of spatial parameters 711 from the first sampling point 583 up to the second sampling point 584, and apply the second set of spatial parameters 712 (as shown in curve 704) from the second sampling point 584 at least until the end of the particular frame.

The encoding system 500 should ensure that the framing information matches the signal characteristics, and that an appropriate portion of the input signal 561 is selected to calculate the one or more sets of spatial parameters 711, 712. For this purpose, the encoding system 500 may include a detector configured to detect signal locations in one or more channels where signal energy suddenly increases. If at least one such signal location is found, the encoding system 500 can be configured to switch from a “smooth” transition to a “steep” transition; otherwise, the encoding system 500 can continue with a “smooth” transition.

As outlined above, the encoding system 500 (e.g., parameter determination unit 523) can be configured to calculate spatial parameters for the current frame based on multiple frames 585, 590 of the input audio signal 561 (e.g., based on the current frame 585 and the immediately following frame 590 (i.e., the so-called forward frame)). Thus, the parameter determination unit 523 can be configured to determine the spatial parameters based on twice the number of K/Q spectra 589 (as shown in Figure 5e). As shown in Figure 5e, the spectra 589 can be windowed using window 586. In this document, it is proposed to adjust the window 586 based on the number of spatial parameter sets 711, 712 to be determined, based on the transition type, and/or based on the position of sampling points 583, 584. By doing so, it can be ensured that the framing information matches the signal characteristics, and that appropriate portions of the input signal 561 are selected to calculate the one or more spatial parameter sets 711, 712.

The following describes example window functions for different encoder/signal scenarios:

a) Case: Single spatial parameter set 711, smooth transition, no transients in forward-looking frame 590;

Window function 586: Between the last spectrum of the previous frame and the (K/Q)th spectrum 589, window function 586 can linearly increase from 0 to 1. Between the (K/Q)th spectrum 589 and the 48th spectrum 589, window function 586 can linearly decrease from 1 to 0 (see Figure 5e).

b) Case: A single set of spatial parameters 711, smooth transition, transient exists in the Nth spectrum (N>K/Q), that is, transient exists in the forward-looking frame 590;

As shown in Figure 7b, window function 721 linearly increases from 0 to 1 between the last spectrum of the previous frame and the (K/Q)th spectrum. Between the (K/Q)th spectrum and the (N-1)th spectrum, window function 721 remains constant at 1. Between the Nth spectrum and the (2*K/Q)th spectrum, window function remains constant at 0. The transient at the Nth spectrum is represented by transient point 724 (which corresponds to the sampling point for the set of spatial parameters immediately following frame 590). Furthermore, Figure 7b shows complementary window function 722 (applied to the spectrum of the current frame 585 when determining the set of one or more spatial parameters for the previous frame) and window function 723 (applied to the spectrum of the next frame 590 when determining the set of one or more spatial parameters for the next frame). In general, window function 721 ensures that, in the case of one or more transients in the forward-looking frame 590, the spectrum of the forward-looking frame preceding the first transient point 724 is fully considered for determining the set of spatial parameters 711 for the current frame 585. On the other hand, the spectrum of the forward-looking frame 590 following the transient point 724 is ignored.

c) Case: Single spatial parameter set 711, steep transition, transient in the Nth spectrum (N<=K/Q), no transient in subsequent frame 590.

As shown in Figure 7c, window function 731 remains constant at 0 between the 1st and (N-1)th spectra. Between the Nth and (K/Q)th spectra, window function 731 remains constant at 1. Between the (K/Q)th and (2*K/Q)th spectra, window function 731 linearly decreases from 1 to 0. Figure 7c indicates the transient point 734 at the Nth spectra (which corresponds to a sampling point of a single spatial parameter set 711). Furthermore, Figure 7c shows window functions 732 and 733, where window function 732 is applied to the spectrum of the current frame 585 when determining the one or more spatial parameter sets for the previous frame, and window function 733 is applied to the spectrum of the next frame 590 when determining the one or more spatial parameter sets for the next frame.

d) Cases: Single set of spatial parameters, steep transition, transients in the Nth and Mth spectra (N <= K/Q, M > K/Q);

Window function 741 in Figure 7d: Between the 1st spectrum and the (N-1)th spectrum, window function 741 remains constant at 0. Between the Nth spectrum and the (M-1)th spectrum, window function 741 remains constant at 1. Between the Mth spectrum and the 48th spectrum, window function 741 remains constant at 0. Figure 7d indicates the transient point 744 (i.e., the sampling point of the spatial parameter set) at the Nth spectrum and the transient point 745 at the Mth spectrum. Furthermore, Figure 7d shows window functions 742 and 743, where window function 742 is applied to the spectrum of the current frame 585 when determining the one or more spatial parameter sets for the previous frame, and window function 743 is applied to the spectrum of the next frame 590 when determining the one or more spatial parameter sets for the next frame.

e) Case: Two sets of spatial parameters, smooth transition, no transients in subsequent frames;

Window function:

i.) First set of spatial parameters: Between the last spectrum of the previous frame and the (K/2Q)th spectrum, the window linearly increases from 0 to 1. Between the (K/2Q)th spectrum and the (K/Q)th spectrum, the window linearly decreases from 1 to 0. Between the (K/Q)th spectrum and the (2*K/Q)th spectrum, the window remains constant at 0.

ii.) The second set of spatial parameters: Between the first and (K/2Q)th spectra, the window remains constant at 0. Between the (K/2Q)th and (K/Q)th spectra, the window linearly increases from 0 to 1. Between the (K/Q)th and (3*K/2Q)th spectra, the window linearly decreases from 1 to 0. Between the (3*K/2Q)th and (2*K/Q)th spectra, the window remains constant at 0.

f) Case: Two sets of spatial parameters, smooth transition, transient in the Nth spectrum (N>K/Q);

Window function:

i.) First set of spatial parameters: Between the last spectrum of the previous frame and the (K/2Q)th spectrum, the window linearly increases from 0 to 1. Between the (K/2Q)th spectrum and the (K/Q)th spectrum, the window linearly decreases from 1 to 0. Between the (K/Q)th spectrum and the (2*K/Q)th spectrum, the window remains constant at 0.

ii.) The second set of spatial parameters: Between the first and (K/2Q)th spectra, the window remains constant at 0. Between the (K/2Q)th and (K/Q)th spectra, the window linearly increases from 0 to 1. Between the (K/Q)th and (N-1)th spectra, the window remains constant at 1. Between the Nth and (2*K/Q)th spectra, the window remains constant at 0.

g) Cases: Two sets of spatial parameters, steep transition, transients in the Nth and Mth spectra (N<M<=K/Q), and no transients in subsequent frames;

Window function:

i.) The first set of spatial parameters: Between the first and (N-1)th spectra, the window remains constant at 0. Between the Nth and (M-1)th spectra, the window remains constant at 1. Between the Mth and (2*K/Q)th spectra, the window remains constant at 0.

ii.) The second set of spatial parameters: Between the first and (M-1)th spectra, the window remains constant at 0. Between the Mth and (K/Q)th spectra, the window remains constant at 1. Between the (K/Q)th and (2*K/Q)th spectra, the window linearly decreases from 1 to 0.

h) Case: Two sets of spatial parameters, steep transition, transients in the Nth, Mth and Oth spectra (N<M<=K/Q, O>K/Q);

Window function:

i.) The first set of spatial parameters: Between the first and (N-1)th spectra, the window remains constant at 0. Between the Nth and (M-1)th spectra, the window remains constant at 1. Between the Mth and (2*K/Q)th spectra, the window remains constant at 0.

ii.) The second set of spatial parameters: Between the first and (M-1)th spectra, the window remains constant at 0. Between the Mth and (O-1)th spectra, the window remains constant at 1. Between the Oth and (2*K/Q)th spectra, the window remains constant at 0.

In general, the following example rules can be specified for the window function used to determine the current set of spatial parameters:

• If the current set of spatial parameters is not associated with transients,

- Window functions provide sampling points from the previous set of spatial parameters up to the current set of spatial parameters.

The smooth, gradual increase in the spectrum of the sampling points of the number set;

- The window function provides a smooth decay of the spectrum from the sampling points of the current spatial parameter set to the sampling points of the next spatial parameter set, if the next spatial parameter set is similar to...

If the transients are unrelated;

- The window function fully considers the spectrum from the sampling points of the current spatial parameter set to the sampling points of the next spatial parameter set, and eliminates the spectrum starting from the sampling points of the next spatial parameter set if the next spatial parameter set is transient.

Related words;

If the current set of spatial parameters is associated with transients,

- Window functions eliminate the spectrum preceding the sampling points of the current spatial parameter set;

- The window function fully considers the spectrum from the sampling points of the current spatial parameter set to the sampling points of the next spatial parameter set, and eliminates the spectrum starting from the sampling points of the next spatial parameter set.

If it is related to transients;

- The window function fully considers the spectrum from the sampling points of the current spatial parameter set to the end of the current frame, and provides a smooth fading of the spectrum from the beginning of the previous frame to the sampling points of the next spatial parameter set, if the next spatial parameter set is not associated with the transient.

The following describes a method for reducing latency in a parameterized multichannel codec system including an encoding system 500 and a decoding system 100. As outlined above, the encoding system 500 includes several processing paths, such as downmixing signal generation and encoding, and parameter determination and encoding. The decoding system 100 typically performs decoding of the encoded downmixing signal and generation of the decorrelated downmixing signal. Furthermore, the decoding system 100 performs decoding of the encoded spatial metadata. Subsequently, the decoded spatial metadata is applied to the decoded downmixing signal and the decorrelated downmixing signal to generate an upmixing signal in a first upmixing matrix 130.

It is desirable to provide an encoding system 500 configured to provide a bitstream 564 that enables the decoding system 100 to generate the upmixed signal Y with reduced latency and/or reduced buffer memory. As outlined above, the encoding system 500 includes several distinct paths that can be aligned such that the encoded data provided to the decoding system 100 within the bitstream 564 matches correctly during decoding. As outlined above, the encoding system 500 performs downmixing encoding of the PCM signal 561. Furthermore, the encoding system 500 determines spatial metadata from the PCM signal 561. Additionally, the encoding system 500 can be configured to determine one or more trimming gains (typically one trimming gain per frame). The trimming gain indicates a trimming prevention gain applied to the downmixed signal X to ensure that the downmixed signal X is not trimmed. The one or more trimming gains can be transmitted within the bitstream 564 (typically within spatial metadata frames) to enable the decoding system 100 to regenerate the upmixed signal Y. Additionally, the encoding system 500 can be configured to determine one or more Dynamic Range Control (DRC) values (e.g., one or more DRC values per frame). The one or more DRC values can be used by the decoding system 100 to perform dynamic range control of the upmix signal Y. Specifically, the one or more DRC values can ensure that the DRC performance of the parametric multichannel codec system described in this document is similar to (or equal to) the DRC performance of older multichannel codec systems such as Dolby Digital Plus. The one or more DRC values can be transmitted within the downmix audio frame (e.g., within an appropriate field of the Dolby Digital Plus bitstream).

Thus, the encoding system 500 may include at least four signal processing paths. To align these four paths, the encoding system 500 may also take into account the latency introduced into the system by different processing components not directly related to the encoding system 500, such as core encoder latency, core decoder latency, spatial metadata decoder latency, LFE filter latency (used for filtering LFE channels), and/or QMF analysis latency.

To align different paths, the delay of the DRC processing path can be considered. The DRC processing delay can typically only be aligned to the frame level, rather than based on time-samples. Thus, the DRC processing delay usually depends only on the core encoder delay that can be rounded up to the next frame for alignment, i.e., DRC processing delay = round up(core encoder delay / frame size). Based on this, the downmixing processing delay used to generate the downmixed signal can be determined, since the downmixing processing delay can be delayed based on time samples, i.e., downmixing processing delay = DRC delay * frame size - core encoder delay. As shown in Figure 8, the remaining delays can be calculated by summing the individual delay lines and ensuring that the delays match at the decoder level.

By considering the different processing delays when writing 564 bitstreams, delaying the resulting spatial metadata by one frame (reducing memory by the number of input channels * 1536 * 4 bytes - 245 bytes) instead of delaying the encoded PCM data by 1536 samples reduces the processing power at the decoding system by 100 (reducing the number of input channels by 1 * 1536) and memory. As a result of the delay, all signal paths are accurately aligned by time sampling, not just roughly matched.

As outlined above, Figure 8 illustrates different delays induced by the example encoding system 500. The numbers in parentheses in Figure 8 indicate example delays based on the number of samples of the input signal 561. The encoding system 500 typically includes a delay 801 caused by filtering the LFE channels of the multi-channel input signal 561. Furthermore, a delay 802 (referred to as “clipgainpcmdelayline”) can be caused by determining a clipping gain (i.e., the DRC2 parameter below) that will be applied to the input signal 561 to prevent clipping of the downmixed signal. Specifically, this delay 802 can be introduced to synchronize the application of the clipping gain in the encoding system 500 with the application of the clipping gain in the decoding system 100. For this purpose, the input delay of the downmixing calculation (performed by the downmixing processing unit 510) can be made equal to the delay 811 (referred to as “coredecdelay”) of the decoder 140 of the downmixed signal. This means that, in the illustrated example, clipgainpcmdelayline = coredecdelay = 288 samples.

The downmixing unit 510 (which includes, for example, a Dolby Digital Plus encoder) delays the processing path of the audio data (e.g., the downmix signal), but it does not delay the processing path of the spatial metadata or the processing path for the DRC/trimmed gain data. Therefore, the downmixing unit 510 should delay the calculated DRC gain, trimmed gain, and spatial metadata. For the DRC gain, this delay typically needs to be a multiple of one frame. The delay 807 of the DRC delay line (referred to as "drcdelayline") can be calculated as drcdelayline = ceil((corencdelay + clipgainpcmdelayline) / frame_size) = 2 frames; where "corencdelay" refers to the delay 810 of the encoder of the downmix signal.

The delay of the DRC gain can typically only be a multiple of the frame size. Therefore, an additional delay may need to be added in the downmixing path to compensate for this and round up to the next multiple of the frame size. The additional downmixing delay 806 (which is called "dmxdelayline") can be determined by dmxdelayline + coreencdelay + clipgainpcmdelayline = drcdelayline * frame_size; and dmxdelayline = drcdelayline * frame_size - coreencdelay - clipgainpcmdelayline, such that dmxdelayline = 100.

When spatial parameters are applied in the frequency domain (e.g., in the QMF domain) on the decoder side, the spatial parameters should be synchronized with the downmixing signal. To compensate for the fact that the encoder of the downmixing signal delays the downmixing processing path instead of the spatial metadata frame, the input of parameter extractor 420 should be delayed such that the following condition applies: dmxdelayline + coreencdelay + coredecdelay + aspdecanadelay = aspdelayline + qmfanadelay + framingdelay. In the above formula, "qmfanadelay" specifies the delay 804 caused by transform unit 521, and "framingdelay" specifies the delay 805 caused by windowing of transform coefficients 580 and determination of spatial parameters. As outlined above, the framing calculation uses two frames (the current frame and the look-forward frame) as input. Due to the look-forward, framing introduces a delay 805 of exactly one frame length. Furthermore, a delay of 804 is known, such that the additional delay to be applied to the processing path used to determine spatial metadata is aspdelayline = dmxdelayline + coreencdelay + coredecdelay + aspdecanadelay - qmfanadelay - framingdelay = 1856. Because this delay is greater than a frame, the memory size of the delay line can be reduced by delaying the calculated bitstream instead of the input PCM data, thus providing aspbsdelayline = floor(aspdelayline/frame_size) = 1 frame (delay 809) and asppcmdelayline = aspdelayline - aspbsdelayline * frame_size = 320 (delay 803).

After calculating the one or more trimming gains, the one or more trimming gains are provided to the bitstream generation unit 530. Therefore, the one or more trimming gains undergo a delay applied to the final bitstream by aspbsdelayline 809. Thus, the additional delay 808 for the trimming gains should be: clipgainbsdelayline + aspbsdelayline = dmxdelayline + coreencdelay + core decdelay, which provides: clipgainbsdelayline = dmxdelayline + coreencdelay + coredecdelay - aspbsddelayline = 1 frame. In other words, it should be ensured that the one or more trimming gains are provided to the decoding system 500 immediately after the corresponding frame of the downmixed signal is decoded, so that the one or more trimming gains can be applied to the downmixed signal before upmixing is performed in the upmixing stage 130.

Figure 8 illustrates further delays induced at the decoding system 100, such as delay 812 (referred to as “aspdecanadelay”) caused by time-domain to frequency-domain transformations 301, 302 of the decoding system 100, delay 813 (referred to as “aspdecsyndelay”) caused by frequency-domain to time-domain transformations 311 to 316, and further delay 814.

As can be seen from Figure 8, the different processing paths of the encoding/decoding system include processing-related delays or alignment delays, which ensure that different output data from different processing paths are available at the decoding system 100 when needed. Alignment delays (e.g., delays 803, 809, 807, 808, 806) are provided within the encoding system 500, thereby reducing the processing power and memory required at the decoding system 100. The total delay for the different processing paths (excluding the LFE filter delay 801 which can be applied to all processing paths) is as follows:

• Downmixing processing path: The sum of delays 802, 806, and 810 = 3072, that is, two frames;

• DRC processing path: delay 807 = 3072, that is, two frames;

• Trimmed gain processing path: The sum of delays 808, 809, and 802 = 3360, which, in addition to the 811 delay of the decoder for the downmixed signal, also corresponds to the delay of the downmixing processing path;

• Spatial metadata processing path: The sum of delays 802, 803, 804, 805, and 809 = 4000, which corresponds to the delay of the downmixing processing path in addition to the delay 811 of the decoder of the downmixed signal and the delay 812 caused by the time-to-frequency transformation stages 301 and 302.

Therefore, it is ensured that DRC data is available at decoding system 100 at time 821, trimmed gain data is available at time 822, and spatial metadata is available at time 823.

Furthermore, as shown in Figure 8, the bitstream generation unit 530 can combine encoded audio data and spatial metadata that may be associated with different segments of the input audio signal 561. Specifically, it can be seen that the downmixing processing path, the DRC processing path, and the trimmed gain processing path have a delay of exactly two frames (3072 samples) up to the output of the encoding system 500 (indicated by interfaces 831, 832, 833) (when delay 801 is ignored). The encoded downmixed signal is provided by interface 831, the DRC gain data by interface 832, and the spatial metadata and trimmed gain data by interface 833. Typically, the encoded downmixed signal and DRC gain data are provided in a conventional Dolby Digital Plus frame, while the trimmed gain data and spatial metadata can be provided in a spatial metadata frame (e.g., in an auxiliary field of the Dolby Digital Plus frame).

It can be seen that the spatial metadata processing path at interface 833 has a delay of 4000 samples (when delay 801 is ignored), which differs from the delay of other processing paths (3072 samples). This means that the spatial metadata frame may be associated with a segment of the input signal 561 that is different from the downmixed signal. Specifically, it can be seen that, in order to ensure alignment at decoding system 100, bitstream generation unit 530 should be configured to generate a bitstream 564 comprising a sequence of bitstream frames, wherein the bitstream frames indicate the frame of the downmixed signal corresponding to the first frame of the multichannel input signal 561 and the spatial metadata frame corresponding to the second frame of the multichannel input signal 561. The first and second frames of the multichannel input signal 561 may include the same number of samples. Nevertheless, the first and second frames of the multichannel input signal 561 may be different from each other. Specifically, the first and second frames may correspond to different segments of the multichannel input signal 561. More specifically, the first frame may include samples preceding the samples of the second frame. For example, the first frame may include samples of the multi-channel input signal 561 such that these samples precede the samples of the second frame of the multi-channel input signal 561 by a predetermined number of samples (e.g., 928 samples).

As outlined above, the encoding system 500 can be configured to determine dynamic range control (DRC) and/or trim gain data. Specifically, the encoding system 500 can be configured to ensure that the downmixed signal X is not trimmed. Furthermore, the encoding system 500 can be configured to provide dynamic range control (DRC) parameters that ensure the DRC behavior of a multichannel signal Y encoded using the aforementioned parameterized encoding scheme is similar to or equal to the DRC behavior of a multichannel signal Y encoded using a reference multichannel encoding system (such as Dolby Digital Plus).

Figure 9a shows a block diagram of an example dual-mode coding system 900. It should be noted that portions 930 and 931 of the dual-mode coding system 900 are typically provided separately. An n-channel input signal Y 561 is provided to each of the upper portion 930 and the lower portion 931, the upper portion 930 being valid at least in the multi-channel coding mode of the coding system 900, and the lower portion 931 being valid at least in the parametric coding mode of the system 900. The lower portion 931 of the coding system 900 may correspond to or may include, for example, coding system 500. The upper portion 930 may correspond to a reference multi-channel encoder (such as a Dolby Digital Plus encoder). The upper portion 930 generally includes a discrete-mode DRC analyzer 910 arranged in parallel with an encoder 911, both of which receive the audio signal Y 561 as input. Based on the input signal 561, encoder 911 outputs the encoded n-channel signal, while DRC analyzer 910 outputs one or more post-processing DRC parameters DRC1 applied to the decoder-side DRC. DRC parameter DRC1 can be a "compr" gain (compressor gain) and/or a "dynrng" gain (dynamic range gain) parameter. The parallel outputs of the two units 910 and 911 are acquired by discrete mode multiplexer 912, which outputs a bitstream P. Bitstream P can have a predetermined syntax, such as Dolby DigitalPlus syntax.

The lower portion 931 of the encoding system 900 includes a parametric analysis stage 922 arranged in parallel with a parametric mode DRC analyzer 921, which, like the parametric analysis stage 922, receives an n-channel input signal Y. The parametric analysis stage 922 may include a parameter extractor 420. Based on the n-channel audio signal Y, the parametric analysis stage 922 outputs one or more mixing parameters (as outlined above) (represented collectively as a in Figures 9a and 9b) and an m-channel (1 < m < n) downmixed signal X. This m-channel downmixed signal X is then processed by a core signal encoder 923 (e.g., a Dolby Digital Plus encoder), which, based on this output, encodes the downmixed signal. The parametric analysis stage 922 influences the dynamic range limits of the input signal in time blocks or frames, where necessary. Possible conditions for controlling when to apply dynamic range limiting can be "untrimmed conditions" or "within-range conditions," which imply that in time blocks or frame segments where the undermixed signal has high amplitude, the signal is processed to fit within a defined range. This condition can be implemented based on a single time block or a time frame comprising several time blocks. For example, the frames of the input signal 561 may include a predetermined number (e.g., 6) of blocks. Preferably, this condition is implemented by applying wide-spectrum gain reduction, rather than simply truncating peaks, or using a similar method.

Figure 9b illustrates a possible implementation of the parametric analysis stage 922, which includes a preprocessor 927 and a parametric analysis processor 928. The preprocessor 927 is responsible for performing dynamic range limiting on the n-channel input signal 561, thereby outputting a dynamically range-limited n-channel signal, which is supplied to the parametric analysis processor 928. The preprocessor 927 further outputs block-by-block or frame-by-frame values of the preprocessed DRC parameter DRC2. The parameter DRC2 is included in the output of the parametric analysis stage 922 along with the m-channel downmixing signal X and mixing parameter α from the parametric analysis processor 928.

The parameter DRC2 can also be referred to as the trimming gain. The parameter DRC2 indicates the gain applied to the multichannel input signal 561 to ensure that the downmix signal X is not trimmed. The one or more channels of the downmix signal X can be determined from the channels of the input signal Y by determining some or all of the linear combinations of the channels of the input signal Y. For example, the input signal Y can be a 5.1 multichannel signal, and the downmix signal can be a stereo signal. Samples of the left and right channels of the downmix signal can be generated based on different linear combinations of samples of the 5.1 multichannel input signal.

The DRC2 parameter can be determined such that the maximum amplitude of the downmixed signal channels does not exceed a predetermined threshold. This can be ensured block-by-block or frame-by-frame. A single gain (trimmed gain) for each block or frame can be applied to the channels of the multichannel input signal Y to ensure that the conditions mentioned above are met. The DRC2 parameter can indicate this gain (e.g., the inverse of the gain).

Referring to Figure 9a, note that the discrete-mode DRC analyzer 910 operates similarly to the parametric-mode DRC analyzer 921 because it outputs one or more post-processed DRC parameters DRC1 to which the decoder-side DRC will be quantized. Thus, the parametric-mode DRC analyzer 921 can be configured to simulate the DRC processing performed by the reference multichannel encoder 930. The parameters DRC1 provided by the parametric-mode DRC analyzer 921 are typically not included in the bitstream P in parametric coding mode, but are compensated so that the dynamic range limitations implemented by the parametric analysis level 922 are taken into account. For this purpose, the DRC up-compensator 924 receives the post-processed DRC parameters DRC1 and the pre-processed DRC parameters DRC2. For each block or frame, the DRC up-compensator 924 derives the value of one or more compensated post-processed DRC parameters DRC3, such that the combined effect of the compensated post-processed DRC parameters DRC3 and the pre-processed DRC parameters DRC2 is quantitatively equivalent to the DRC quantized by the post-processed DRC parameters DRC1. In other words, the DRC compensator 924 is configured to reduce the share (if any) of the post-processed DRC parameters output by the DRC analyzer 921, which has already been implemented by the parameterized analysis stage 922. This is the compensated post-processed DRC parameter DRC3 that can be included in the bitstream P.

Referring to the lower portion 931 of system 900, parameterized mode multiplexer 925 collects compensated post-processing DRC parameters DRC3, pre-processing DRC parameters DRC2, mixing parameters α, and encoded downmixed signal X, and forms a bitstream P based thereon. Thus, parameterized mode multiplexer 925 may include or correspond to bitstream generation unit 530. In a possible implementation, compensated post-processing DRC parameters DRC3 and pre-processing DRC parameters DRC2 may be encoded in logarithmic form as dB values that affect amplitude amplification or reduction on the decoder side. Compensated post-processing DRC parameters DRC3 may have any sign. However, pre-processing DRC parameters DRC2, derived from implementing "unpruned conditions," will generally always be represented by non-negative dB values.

Figure 10 illustrates an example process that can be performed, for example, in the parameterized mode DRC analyzer 921 and the DRC compensator 924 to determine the modified DRC parameters DRC3 (e.g., the modified "dynrng gain" and/or "compr gain" parameters).

The DRC2 and DRC3 parameters can be used to ensure that the decoding system 100 plays back different audio bitstreams at a consistent loudness level. Furthermore, it can be ensured that the bitstream generated by the parameterized coding system 500 has a consistent loudness level relative to the bitstream generated by legacy and/or reference coding systems (such as Dolby Digital Plus). As outlined above, this can be ensured by generating an untrimmed downmix at the coding system 500 (using the DRC2 parameter) and by providing the DRC2 parameter within the bitstream (e.g., the inverse of the attenuation applied to prevent downmix trimming) so that the decoding system 100 can recreate the original loudness (when an upmix is generated).

As outlined above, the downmix signal is typically generated based on a linear combination of some or all of the channels of the multichannel input signal 561. Thus, the scaling factor (or attenuation) applied to the channels of the multichannel input signal 561 can depend on all channels of the multichannel input signal 561 that contribute to the downmix signal. Specifically, the one or more channels of the downmix signal can be determined based on the LFE channels of the multichannel input signal 561. Therefore, the scaling factor (or attenuation) applied to trim protection should also take into account the LFE channels. This differs from other multichannel coding systems (such as Dolby Digital Plus) where LFE channels are typically not considered for trim protection. By considering the LFE channels and/or all channels that contribute to the downmix signal, the quality of trim protection can be improved.

Thus, the one or more DRC2 parameters provided to the corresponding decoding system 100 can depend on all channels of the input signal 561 that contribute to the downmix signal; specifically, the DRC2 parameters can depend on the LFE channel. By doing so, the quality of trim protection can be improved.

It should be noted that the dialnorm parameter may not be considered in calculating the scaling factor and/or the DRC2 parameter (as shown in Figure 10).

As outlined above, encoding system 500 can be configured to write a so-called "trimmed gain" (i.e., DRC2 parameter) into a spatial metadata frame indicating which gains have been applied to the input signal 561 to prevent trimming in the undermixed signal. The corresponding decoding system 100 can be configured to accurately invert the trimmed gain applied in encoding system 500. However, only the sample points of the trimmed gain are transmitted in the bitstream. In other words, the trimmed gain parameter is typically determined only frame-by-frame or block-by-block. Decoding system 100 can be configured to interpolate the trimmed gain value (i.e., the received DRC2 parameter) between sample points from adjacent sample points.

Figure 11 illustrates example interpolation curves for interpolating DRC2 parameters for adjacent frames. Specifically, Figure 11 shows a first DRC2 parameter 953 for the first frame and a second DRC2 parameter 954 for the subsequent second frame 950. The decoding system 100 can be configured to perform interpolation between the first DRC2 parameter 953 and the second DRC2 parameter 954. Interpolation can be performed within a sample subset 951 of the second frame 950 (e.g., within the first block 951 of the second frame 950) (as shown by interpolation curve 952). Interpolation of the DRC2 parameters ensures a smooth transition between adjacent audio frames and thus avoids audible artifacts that may be caused by the difference between the subsequent DRC2 parameters 953 and 954.

Encoding system 500 (specifically, downmixing processing unit 510) can be configured to apply a corresponding trimmed gain interpolation to the DRC2 interpolation 952 performed by decoding system 500 when a downmixed signal is generated. This ensures that the trimmed gain protection of the downmixed signal is consistently removed when an upmixed signal is generated. In other words, encoding system 500 can be configured to simulate the curve of the DRC2 value derived from the DRC2 interpolation 952 applied by decoding system 100. Furthermore, encoding system 500 can be configured to accurately (i.e., sample-by-sample) inversely apply the curve of this DRC2 value to the multichannel input signal 561 when a downmixed signal is generated.

The methods and systems described in this document can be implemented as software, firmware, and/or hardware. Some components can be implemented, for example, as software running on a digital signal processor or microprocessor. Other components can be implemented, for example, as hardware or application-specific integrated circuits (ASICs). Signals encountered in the described methods and systems can be stored on media such as random access memory or optical storage media. They can be transmitted via networks such as radio networks, satellite networks, wireless networks, or wired networks (e.g., the Internet). Typical devices using the methods and systems described in this document are portable electronic devices or other consumer devices for storing and/or presenting audio signals.

Claims (15)

1. A method comprising: The audio processor receives multi-channel input audio signals. A first dynamic range control (DRC) value set is determined, the first DRC value set being configured to control the dynamic range of the output audio signal; A second set of DRC values is determined, the second set of DRC values being configured to prevent the multi-channel input audio signal from being trimmed during downmixing by the audio processor; The second set of DRC values is applied to the multi-channel input audio signal to obtain a decayed multi-channel input audio signal; The attenuated multi-channel input audio signal is down-mixed to obtain a down-mixed signal; and The output audio signal is generated from the first set of DRC values and the downmixing signal. 2. An apparatus comprising: One or more processors; A memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including the following: By receiving multi-channel input audio signals; A first dynamic range control (DRC) value set is determined, the first DRC value set being configured to control the dynamic range of the output audio signal; A second set of DRC values is determined, the second set of DRC values being configured to prevent the multi-channel input audio signal from being trimmed during downmixing by the device; The second set of DRC values is applied to the multi-channel input audio signal to obtain a decayed multi-channel input audio signal; The attenuated multi-channel input audio signal is down-mixed to obtain a down-mixed signal; and The output audio signal is generated from the first set of DRC values and the downmixing signal. 3. The apparatus of claim 2, wherein generating the output audio signal includes applying the first set of DRC values to the downmixed signal. 4. The apparatus of claim 2, wherein the first set of DRC values and/or the second set of DRC values are represented as dB values in logarithmic form. 5. The apparatus of claim 2, wherein the multichannel input audio signal is divided into a frame sequence of samples of the multichannel audio signal, and determining the first DRC value set and/or the second DRC value set includes determining the DRC value of each sample of each frame in the frame sequence. 6. The apparatus of claim 5, wherein determining the DRC value of each sample of the frame includes interpolation between the DRC value of the frame and the DRC value of the previous frame. 7. The method according to claim 6, wherein the interpolation is spline interpolation. 8. The apparatus according to claim 2, wherein the downmixing signal is a stereo signal. 9. The apparatus of claim 2, wherein the left and right channels of the downmix signal are generated based on different linear combinations of the channels of the multichannel input audio signal. 10. A parameter processing unit (520) configured to determine a spatial metadata frame for generating a multi-channel upmix signal from a corresponding frame of a downmix signal; wherein the downmix signal includes m channels, and wherein the multi-channel upmix signal includes n channels; n and m are integers, where m < n; wherein the spatial metadata frame includes one or more sets of spatial parameters (711, 712); the parameter processing unit (520) includes: - A transformation unit (521) configured to determine a plurality of spectra (589) from the current frame (585) and the immediately following frame (590) of the channels of the multi-channel input signal (561); and - Parameter determination unit (523), the parameter determination unit (523) is configured to determine the spatial metadata frame of the current frame of the channel for the multi-channel input signal (561) by weighting the plurality of spectra (589) using a window function (586); The window function (586) depends on one or more of the following: the number of spatial parameter sets (711, 712) included in the spatial metadata frame, the presence of one or more transients in the current frame or the immediately following frame of the multi-channel input signal (561), and/or the timing of the transients. 11. A parameter processing unit (520) configured to determine a spatial metadata frame for generating a multi-channel upmix signal from a corresponding frame of a downmix signal; wherein the downmix signal includes m channels, and wherein the multi-channel upmix signal includes n channels; n and m are integers, where m < n; wherein the spatial metadata frame includes a set of spatial parameters (711); the parameter processing unit (520) includes: - A transformation unit (521) configured to: determine a first plurality of transformation coefficients (580) from frames (585) of a first channel (561-1) of a multi-channel input signal (561), and determine a second plurality of transformation coefficients (580) from corresponding frames of a second channel (561-2) of the multi-channel input signal (561); wherein the first channel (561-1) and the second channel (561-2) are different; wherein the first plurality of transformation coefficients (580) and the second plurality of transformation coefficients (580) respectively provide a first time/frequency representation and a second time/frequency representation of frames (585) of the first channel and the second channel; wherein the first time/frequency representation and the second time/frequency representation include a plurality of frequency intervals (571) and a plurality of time intervals (582); and - A parameter determination unit (523) configured to determine the spatial parameter set (711) using fixed-point arithmetic based on the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580); wherein the spatial parameter set (711) includes corresponding band parameters for different frequency bands (572) including different numbers of frequency intervals (571); wherein a specific band parameter for the specific frequency band (572) is determined based on the transform coefficients (580) of the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580) from the specific frequency band (572); and wherein the shift used by the fixed-point arithmetic to determine the specific band parameter depends on the specific frequency band (572). 12. An audio encoding system (500) configured to generate a bitstream (564) based on a multi-channel input signal (561); the system (500) comprising: - A downmixing processing unit (510) configured to generate a frame sequence of a downmixed signal from a corresponding first frame sequence of the multichannel input signal (561); wherein the downmixed signal includes m channels, and wherein the multichannel input signal (561) includes n channels; n and m are integers, where m < n; - A parameter processing unit (520) configured to determine a spatial metadata frame sequence from a second frame sequence of the multichannel input signal (561); wherein the frame sequence of the downmixed signal and the spatial metadata frame sequence are used to generate a multichannel upmixed signal comprising n channels; and - A bitstream generation unit (503) configured to generate a bitstream (564) including a bitstream frame sequence, wherein the bitstream frame indicates a frame of the downmixed signal corresponding to a first frame of a first frame sequence of the multichannel input signal (561) and a spatial metadata frame corresponding to a second frame of a second frame sequence of the multichannel input signal (561); wherein the second frame is different from the first frame. 13. A method for determining a spatial metadata frame, the spatial metadata frame being used to generate a frame of a multichannel upmix signal from a corresponding frame of a downmix signal; wherein the downmix signal comprises m channels, and wherein the multichannel upmix signal comprises n channels; n and m are integers, where m < n; wherein the spatial metadata frame comprises one or more sets of spatial parameters (711, 712); the method comprising: - Determine multiple spectra (589) from the current frame (585) and the immediately following frame (590) of the channels of the multi-channel input signal (561); - The plurality of spectra (589) are weighted using a window function (586) to obtain a plurality of weighted spectra; and - Determine the spatial metadata frame of the current frame of the channel for the multi-channel input signal (561) based on the plurality of weighted spectra; The window function (586) depends on one or more of the following: the number of spatial parameter sets (711, 712) included in the spatial metadata frame, the presence of one or more transients in the current frame or the immediately following frame of the multi-channel input signal (561), and/or the timing of the transients. 14. A method for determining a spatial metadata frame, the spatial metadata frame being used to generate a frame of a multichannel upmix signal from a corresponding frame of a downmix signal; wherein the downmix signal comprises m channels, and wherein the multichannel upmix signal comprises n channels; n and m are integers, where m < n; wherein the spatial metadata frame comprises a set of spatial parameters (711); the method comprising: - Determine the first plurality of transform coefficients (580) from the frame (585) of the first channel (561-1) of the multi-channel input signal (561); - Determine a second plurality of transformation coefficients (580) from the corresponding frame of the second (561-2) channel of the multi-channel input signal (561); wherein the first channel (561-1) and the second channel (561-2) are different; The first plurality of transformation coefficients (580) and the second plurality of transformation coefficients (580) respectively provide a first time/frequency representation and a second time/frequency representation of the frames (585) of the first channel and the second channel; wherein the first time/frequency representation and the second time/frequency representation include a plurality of frequency intervals (571) and a plurality of time intervals (582); wherein the spatial parameter set (711) includes corresponding band parameters for different frequency bands (572) that include different numbers of frequency intervals (571); - Determine the shift to be applied when using fixed-point arithmetic to determine the specific band parameters for a specific frequency band (572); wherein the shift is determined based on the specific frequency band (572); and - The specific band parameters are determined using fixed-point arithmetic and the determined shift, based on the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580) falling in the specific band (572). 15. A method for generating a bitstream (564) based on a multi-channel input signal (561); the method comprising: - A frame sequence for generating a downmixing signal from the corresponding first frame sequence of the multichannel input signal (561); wherein the downmixing signal comprises m channels, and wherein the multichannel input signal (561) comprises n channels; n and m are integers, where m < n; - Determine a spatial metadata frame sequence from the second frame sequence of the multichannel input signal (561); wherein the frame sequence of the downmixing signal and the spatial metadata frame sequence are used to generate a multichannel upmixing signal comprising n channels; and - Generates a bitstream (564) that includes a sequence of bitstream frames; The bitstream frame indicates the frame corresponding to the first frame of the first frame sequence of the multi-channel input signal (561) and the spatial metadata frame corresponding to the second frame of the second frame sequence of the multi-channel input signal (561); wherein the second frame is different from the first frame.