RU2815754C2 - Systems and methods for smoothing covariance - Google Patents

Abstract

FIELD: audio signal processing.

SUBSTANCE: invention relates to audio signal processing. Technical result is achieved by calculating the forcing factor for the frequency band as the ratio of the effective amount to the required amount; and for each frequency band of a plurality of frequency bands, generating a corresponding smoothed covariance matrix for a specific band for a current signal frame using a previously generated value of the corresponding smoothed covariance matrix for a specific band for a previous signal frame with respect to the current signal frame and a loss factor; and generating an audio signal for an audio service using frequency-domain representations of a sequence of signal frames in the time domain and further use of corresponding sets of corresponding smoothed covariance matrices for a specific band, wherein comparison, calculation and formation are performed by a system comprising one or more computer processors.

EFFECT: preventing excessive smoothing of values of audio signals and thereby preventing the addition of undesirable artefacts due to smoothing of audio signals.

24 cl, 7 dwg

Description

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62/881,825, filed August 1, 2019, and U.S. Provisional Patent Application No. 63/057,533, filed July 28, 2020, both of which are incorporated herein by reference.

TECHNICAL FIELD

[0002] The present disclosure relates to improvements for signal processing. In particular, the present disclosure relates to processing audio signals to improve covariance smoothing for improved processing.

BACKGROUND OF THE ART

[0003] The audio signal processing aspect involves presenting multi-channel audio to a listener such that the listener can determine virtual spatial locations for the audio, providing an immersive experience for the listener. An early implementation of this is stereo, in which the listener can determine the "direction" in space from which the sound is coming.

[0004] More recent developments in this technology utilize the inter-channel dependencies of a multi-channel system to provide a more immersive audio experience. This may involve using the audio channel covariance matrix.

SUMMARY OF THE INVENTION

[0005] Various signal processing systems and methods are disclosed herein. Some such systems and methods may include smoothing covariance values for a frequency band over successive frames.

[0006] In some examples, a system and method for smoothing a covariance matrix estimate for a sequence of signal frames in a frequency band, the method comprises: for signal frames, comparing an effective number of cells in the frequency band with a required number of cells for the frequency band; calculating the forgetting coefficient for the strip as the ratio of the effective amount to the required amount; and if the effective number of cells in the frequency band is less than the required number, generating a current estimate of the covariance matrix value for the current frame using the previously generated estimate of the covariance matrix value for the previous frame relative to the current frame and a forgetting rate; wherein the comparison, calculation and generation are performed by a system comprising one or more computer processors. The smoothed covariance matrix can be used to improve further signal processing by reducing artifacts caused by rough transitions in the matrix.

[0007] In some such examples, the system and method may include calculating an effective number of cells as the sum of the response values of a filter bank for a frequency band.

[0008] In some such examples, the shaping uses a first order filter.

[0009] In some such examples, the first order filter includes the difference between the value for the current frame and a previously generated estimate for the previous frame, the difference weighted by a forgetting factor.

[0010] In some such examples, the system and method also include comparing a forgetting rate with a maximum forgetting rate; and setting the forgetting rate to the minimum of the calculated forgetting rate and the maximum allowed forgetting rate; wherein the comparison and setting are performed before generating a current estimate of the value of the covariance matrix for the current frame.

[0011] In some such examples, the system and method also include detecting whether a transient occurs in a monitored frame; and resetting the smoothing in response to detecting that a transient occurs by setting the current estimate of the covariance matrix value for the tracked frame to the original covariance matrix value for the tracked frame without using a forgetting factor for the tracked frame. Detection can be performed by using aspects of the muting decorrelator.

[0012] In some such examples, the system and method also include storing a current estimate of the covariance matrix of the present frame; and converting the stored current estimate samples into frequency bands, converting from one band allocation structure to another band allocation structure.

[0013] The system and method may be part of an audio decoder.

[0014] Some or all of the methods described herein may be performed by one or more devices in accordance with instructions (eg, software) stored on one or more non-transitory storage media. Such non-transitory storage media may include storage devices such as those described herein, including, but not limited to, random access memory (RAM), read only memory (ROM), etc. Accordingly, various innovative aspects of the subject matter described in the present disclosure may be implemented in a non-transitory storage medium having software stored thereon. The software, for example, may be executed by one or more control system components, such as those disclosed herein. The software, for example, may include instructions for performing one or more methods disclosed herein.

[0015] At least some aspects of the present disclosure may be implemented through a device or devices. For example, one or more devices may be configured to perform (at least in part) the methods disclosed herein. In some implementations, the device may include an interface system and a control system. The interface system may include one or more network interfaces, one or more interfaces between a control system and a memory system, one or more interfaces between a control system and another device, and/or one or more external device interfaces. The control system may include at least one component of a general purpose processor on one or more chips, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete or transistor logic or discrete hardware components. Accordingly, in some implementations, the control system may include one or more processors and one or more non-transitory storage media operably coupled to the one or more processors.

[0016] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and in the description below. Other features, aspects and advantages will become apparent from the description, drawings and claims. It should be noted that the relative sizes of the following figures may not be to scale. The same reference numbers and symbols in different drawings generally indicate the same elements, but different reference numbers do not necessarily indicate different elements between different drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG. 1 illustrates an example of a signal channel divided into samples, frames, cells and stripes.

[0018] FIG. 2 illustrates a flowchart for an exemplary method of smoothing a signal covariance matrix using a smoothing algorithm.

[0019] FIG. 3 illustrates a flowchart (partial) for an exemplary method of smoothing a signal covariance matrix using a maximum forgetting rate smoothing algorithm.

[0020] FIG. 4 illustrates a flowchart (partial) for an exemplary method of smoothing a signal covariance matrix using a transient detection smoothing algorithm.

[0021] FIG. 5 illustrates an example of an encoder/decoder system with an anti-aliasing algorithm in the encoder.

[0022] FIG. 6 illustrates an example of an encoder/decoder system with an anti-aliasing algorithm in the decoder.

[0023] FIG. 7 illustrates an example of an encoder/decoder system with an anti-aliasing algorithm as part of the DTX.

DETAILED DESCRIPTION

[0024] The present disclosure describes methods and systems for smoothing the values of an estimation matrix (e.g., a covariance matrix) over time, including cases in which the estimation is calculated based on a small number of samples, whereby unsmoothed variation of these values can cause problems such as noticeable artifacts in the output (eg audio).

[0025] The present disclosure also describes (among other things) the use of transient detection to prevent over-smoothing of values and thereby preventing the addition of unwanted artifacts due to dithering.

[0026] The term "smoothing" as used herein refers to reducing the variability in the value of successive frames of a signal. For example, to reduce the variability of covariance values over successive frames.

[0027] The term "covariance" as used herein refers to a measure of the joint variability of two signals. For example, covariance of signal energies.

[0028] The term "covariance matrix" as used herein refers to the covariance matrix of multiple signals. The term “banded covariance matrix” refers to a covariance matrix in which several adjacent frequency cells of the covariance matrix for each cell are combined into a band, with the result that the covariance can be represented by a significantly smaller number (equal to the number of bands). For example, input data with 960 frequency bins could become 12 frequency bands with 960 bins distributed evenly or unevenly across the 12 bands. For the examples presented herein, the “covariance matrix” is assumed to be a banded covariance matrix.

[0029] The term "Estimation Matrix" as used herein refers to a sample covariance matrix. As used herein, "covariance matrix" and "smoothing matrix" include estimation matrices.

[0030] The term "complex advanced coupling" (CACPL) as used herein refers to a stereo encoding method whereby the left and right channel signals are combined into a mid-signal (L+R)/2 and a side signal. (L-R)/2, and then the side signal is predicted based on the mid signal.

[0031] The term "smoothing algorithm" as used herein refers to smoothing a covariance matrix in accordance with the present disclosure, particularly smoothing based on the effective number of stripe cells. The term "smoothed matrix" as used herein refers to a covariance matrix modified by a smoothing algorithm.

[0032] The term "effective number of cells" as used herein refers to either the actual number of cells in a stripe or an estimate of the number of effective cells in a stripe. An example of estimating the number of effective cells in a band involves summing the response values of a bank of filters for that band.

[0033] The term "forgetting factor" as used herein refers to a weighting value that modifies the significance of a portion of a feature.

[0034] The term "DTX" as used herein refers to an encoder module that processes discontinuous transmission events.

[0035] A solution to the problem of smoothing the covariance of input channels is described herein. This improves signal processing technology, for example, for voice and audio services, for example, providing users with an immersive experience (eg, virtual reality, augmented reality).

[0036] In digital signal processing, as shown in FIG. 1, the input signal (channel) is divided in the time domain into frames 101. A series of consecutive frames may be grouped into a frame sequence 102. In the frequency domain, the signal can be divided into frequency bins called cells 103. A given cell in a given frame is called a sample 106. Cells can be grouped together in a frequency band 104/105. This allows statistical averaging to be performed over a given stripe for a given windowed frame/sequence of frames (a "window" is a weighting function applied to a frame or sequence of frames). One application of this averaging is to estimate the covariance between audio channels.

[0037] Not all stripes necessarily contain the same number of cells. For example, lower frequency bands 104 may have fewer cells per band than higher frequency bands 105. This means that statistical analysis for lower frequency bands requires a larger window (more frames) per cell to obtain meaningful statistical results than for higher frequency bands. stripes. A given cell in a given frame may have signal energy associated with it, and multiple cell values may be averaged across a stripe and/or window.

Example of a covariance matrix

[0038] Taking complex extended link (CACPL) as an example, the parameters α (complex prediction coefficient) and β (real decorrelation coefficient) can be calculated based on the elements of the mid-side (2×2) covariance matrix . This matrix is calculated from a series of analytical bands 1≤b≤B.

Equation 1

where are the members in the form represent the covariance of signals X and Y , calculated over frequency band b. The superscript * denotes the complex conjugate transpose of the value. The example above is a 2 x 2 covariance matrix. In various implementations, any number of input channels is possible (for example, a four-channel system with a 4x4 covariance matrix). The above example is for mid signals (M) and side signals (S), but one skilled in the art can extrapolate the above for other signal types.

Smoothing function and forgetting coefficient

[0039] Typically, the smoothed matrix can be calculated using a low-pass filter designed according to the smoothing requirements. In the case of a smoothing algorithm, the smoothing requirements are such that previous estimates are used to artificially increase the number of frequency samples (cells) used to generate the current estimate of the covariance matrix. In some embodiments, computing the smoothed matrix based on input covariance matrix over a sequence of frames uses a first-order autoregressive low-pass filter that uses a weighted sum of the estimated values of a matrix of past and current frames:

Equation 2

Where - forgetting coefficient or update rate, i.e. how much attention is given to the previous evaluation data, and n is the frame number. In some embodiments, this only matters for frames after the first frame, since there is no meaning for . In some embodiments, A [0] is equal to 0, resulting in A [1] being smoothed. In some embodiments, A [0] is assigned the value of A [1], resulting in no smoothing of A [1]. The algorithm in Equation 2 is an example of an algorithm that can be used for the smoothing algorithm, but other equations can also be used. For example, while Equation 2 represents a first-order low-pass filter, a higher-order filter design can also be used. Important factors of the algorithm (smoothing function) are the retrospective aspect of using previously smoothed results and the forgetting factor for assigning weight to the influence of these results.

Forgetting rate

[0040] The effect of the forgetting factor is that as smoothing is applied to successive frames, previous frames have less and less influence on the smoothing of the frame being smoothed (corrected).

[0041] When the forgetting coefficient in Equation 1 is one ( ), no smoothing occurs, and it effectively acts as a filter for all frequencies. When , the equation acts as a low-pass filter. Lower pays more attention to older covariance data, while higher takes the new covariance into account more. The forgetting rate is greater than one (for example, ) is implemented as a high-pass filter. In some embodiments, a maximum allowable forgetting rate is implemented . This maximum value will determine the behavior of the algorithm once the cell/bar values become large. In some embodiments will always implement some smoothing in each band no matter what the calculated forgetting rate is; And will only apply the smoothing function to stripes with fewer cells than the required number , leaving large stripes unsmoothed.

[0042] In some of those embodiments, the band-specific forgetting ratio calculated as a minimum from the maximum allowed forgetting rate and the ratio of the effective number of cells in a strip and minimum number of cells , which are determined to give a good statistical estimate based on window size.

Equation 3

[0043] In some embodiments is the actual number of cells for the frequency band. In some embodiments can be calculated based on the sum of the frequency response of a particular band, for example if the frequency response of the band is , effective number of cells . In some embodiments , such as , remains within a reasonable range, e.g. . This means that smoothing is applied proportionally to small sample estimates, and no smoothing is applied at all to large sample estimates. In some embodiments , which causes large stripes to be smoothed to a certain extent regardless of their size (e.g. ). In some embodiments can be selected based on the available data that gives the best subjective results. In some embodiments can be chosen based on how much initial smoothing is desired (the first subsequent frame after the initial frame of a given window).

[0044] In an example using an analysis filter bank with narrower low frequency bands (i.e., fewer cells, more frames needed for good statistical analysis) and wider high frequency bands (i.e., more cells, with fewer frames required for good statistical analysis) this will lead to an increase in the degree of smoothing in the low frequency bands and a decrease in the amount of smoothing (or no smoothing at all if ) in high frequency bands.

[0045] An example flowchart for this method is shown in FIG. 2. The FFT (Fast Fourier Transform) input signal 201 provides, for a given band in the input signal, a corresponding windowed covariance matrix. An effective number of cells for this stripe is retrieved 202. For example, it can be calculated using the response values of a bank of bandpass filters. The required number of cells is determined 203, for example, by subjectively analyzing how many cells will be needed to provide a good statistical analysis for the window. The forgetting rate is calculated 204 by calculating the ratio of the calculated number of cells to the required number of cells. For a given frame (except the first frame), a new covariance matrix value is calculated 205 based on the new covariance value calculated for the previous frame, the original value for the current frame, and the forgetting rate. The new (smoothed) matrix formed by these new values is used in further signal processing 206.

[0046] In FIG. Figure 3 shows an example of modifying the sequence of operations for the maximum allowed forgetting rate. As in FIG. 2, the forgetting coefficient is calculated for the strip. In addition, the maximum allowed forgetting rate is determined 302. The values are compared 303 and if the calculated coefficient is less than the maximum allowed coefficient, then the calculated coefficient is used in smoothing 305. If the calculated coefficient is greater than the maximum allowed coefficient, the maximum allowed coefficient is used 304 in smoothing 305. The example shows the calculated coefficient that is used , if the coefficients are equal (not greater), but an equivalent sequence of operations can be provided that uses the minimum value if they are equal.

Reset smoothing

[0047] In some embodiments, it may be desirable to avoid smoothing transients (sudden changes in signal magnitude) as this may result in unwanted signal distortion/artifacts at the output. In these embodiments, the smoothing may be “reset” at points where transients are detected in the signal.

[0048] The estimation matrix of the previous time frame may be stored to allow a smoothed value to be calculated for the current frame. If a transient is detected in the input signals during this frame, the smoothing function can be set to reinitialize.

[0049] When a transient is detected, the past matrix estimate is reset to the current estimate, causing the post-transient anti-aliasing filter output to be the estimate itself (no changes are applied). In other words, for a reset frame After a reset frame, the smoothing function can be applied again in subsequent frames until the next reset.

[0050] In FIG. 4 shows an exemplary modification of the process flow for transient detection. A determination is made 401 whether a transient is detected for a given frame. If yes 403, then the new matrix value remains the same as the input value. If there is no 402, the normal anti-aliasing algorithm is used for that frame. A combination (matrix) of smoothed and unsmoothed (transient) frame values is used to process the 404 signals.

[0051] In some embodiments, dithering is reset when a transient is detected on any channel. For example, if there are N channels, N transient detectors (one per channel) can be used, and if any of them detect a transient, the dithering is reset, either the end of the signal or the end of the dithering (dithering is turned off).

[0052] For an example stereo input signal, the channels may be defined as separate (or perhaps quite different), such that accounting for transients in the left channel may mean that an important transient in the right channel may not be properly smoothed (and vice versa). Therefore, two transient detectors are used (left and right), and either of them can reset the smoothing of the entire 2x2 matrix.

[0053] In some embodiments, smoothing is only reset on transients for some channels. For example, if N channels exist, only M (less than N, possibly 1) detectors are used.

[0054] For an example of a First Order Ambisonics input signal (FOA), the first channel (W) can be determined to be the most important compared to the other three channels (X, Y, Z), and given the spatial relationships between the FOA signals, it is likely that the transients in the last three channels will be reflected in the W channel one way or another. Thus, the system can be configured with a transient detector on the W channel only, triggering a reset of the entire 4x4 covariance matrix when it detects a transient on the W channel.

[0055] In some embodiments, only covariance elements that have experienced a transient are reset. This would mean that a transient in the nth channel would reset the values in only the nth row and nth column of the covariance matrix (the entire row and the entire column). This can be accomplished by separately tracking transients on each channel, and a detected transient on any given channel triggers a reset of the matrix positions that correspond to that channel's covariance with the other channel (and vice versa, and also trivially with itself).

[0056] In some embodiments, the reset occurs only for a majority or a threshold number of channels detecting the transient. For example, in a four-channel system, a threshold can be set to trigger a reset only if at least two channels report a transient in the same frame.

Transient detection example

[0057] An example of a transient detector is the use of the mute property of a mute decorrelator, which can be used to determine whether a transient has occurred in one input channel. See, for example, US Pat. No. 9,747,909, the contents of which are incorporated herein by reference. With transients, there is a sudden decrease in the damping factor. The change in damping factor can be compared to a threshold value to determine whether a transient has occurred. The threshold value can be determined by a subjective assessment of what accurately detects the transient.

[0058] An exemplary transient detector operates as follows.

1. Input signals are high-pass filtered to enhance transients.

2. The absolute value of the high-pass filter output is then low-pass filtered to determine the signal envelope.

3. Two envelopes are formed, fast and slow , using two different low-pass filters with slightly different cutoff frequencies, .

4. For each count in the time domain of signal envelopes in a particular frame, where , If

Equation 4

then the muting coefficient decreases to

Equation 5

otherwise the coefficient can slowly increase to 1 as follows:

Equation 6

Where is a time constant that determines how quickly the damping factor returns to .

5. If there is a significant change in the mute factor compared to the current frame, a transient is considered to have occurred: i.e.

Equation 7

[0059] In some embodiments, a method for determining the occurrence of a transient based on a damping ratio is to count the number/fraction of time domain samples in each frame in which the above condition is met. If this value exceeds a certain threshold, the frame is determined to contain a transient.

[0060] Although a damping factor can be used to detect transients for the dithering algorithm, any transient detection method known in the art can be used.

Conversion of samples (transfer of band distribution)

[0061] In some embodiments, a band undergoing smoothing may be converted to a new band for smoothing when the channel band allocation changes. If the input signal's banding distribution changes from one frame to the next in terms of number and/or width of bands (for example, due to switching the filter bank in the encoder), the stored (previous) matrix estimate can be converted into a new banding structure to continue continuous operation smoothing mechanism. The transformation matrix between band allocation structures can be calculated by using a set of window functions in the frequency domain, i.e. ideal magnitudes of the filter bank responses for the previous band allocation scheme (i.e., the previous filter bank) as a matrix.

[0062] As an example:

is an array , containing strings of previous ideal filter bank responses , And is an array containing the new ideal filter bank responses . - number of frequency cells.

[0063] Transformation matrix elements , which transform the strip from into the strip from :

Equation 8

creating a matrix , which can be used to convert previous band estimates into a new band allocation scheme.

Example of smoothing with reset

[0064] An example of one strip is given 1x1 covariance matrix , where the strip consists of 3 cells ( ), but it is determined (subjectively) that good statistical analysis requires at least 8 cells ( ). This gives the band forgetting rate .

[0065] If the covariance matrix of the initial band distribution (first frame and transients shown in bold) is:

and a transient detector with the following output indicating that the first frame and the seventh frame contain transients:

The output with smoothed covariance will look like this:

[0066] The first frame remains the same in what he was like , since anti-aliasing does not start until the second frame due to the transient specified in the first. If there is no transient in the first frame, the first frame may be smoothed by taking the previous frame as having a value of zero (in one embodiment) or as having a value equal to itself (in another embodiment); the situation depends on how the algorithm is configured. In some embodiments, the first frame is always marked as having a transient, regardless of whether any actual detection occurs. Seventh meaning matches the value due to anti-aliasing reset.

[0067] The second frame uses its own value (0.05), the "smoothed" value of the previous frame (0.1), and the forgetting factor (3/8) to calculate the smoothed value (approximately 0.813). The formula in this example (first order filter):

=0.1 + (3/8) (0.05-0.1) = 0.8125

[0068] This is repeated for subsequent frames until frame 7 is reached, which is designated as having a transient. For this= 0.9. Subsequent frames are then calculated again using the smoothing function (as if frame 7 were the starting frame).

[0069] In some embodiments, a covariance matrix smoothing system may be integrated into the encoder. An example of the system is shown in Fig. 5. An exemplary encoder 501 generates 502 one or more covariance matrices for the signal 504, which the smoothing algorithm 503 modifies before sending the signal 504 to the decoder 505.

[0070] In some embodiments, a covariance matrix smoothing system may be integrated into the decoder. An example of a decoder with a smoothing algorithm is shown in FIG. 6. Encoder 601 sends signal 602 (eg, audio channels) to decoder 603. Decoder 603 applies anti-aliasing algorithm 604 before performing subsequent processing 605 (processing may also be performed prior to anti-aliasing).

[0071] In some embodiments, covariance smoothing may also be used in the encoder in the case of discontinuous transmission (DTX) frames to stabilize the spatial comfort noise parameterization. In this case, covariance smoothing will not be reset during DTX frames due to transients, but will instead be reset upon entering/exiting the codec's DTX mode. An example of an encoder with a smoothing algorithm is shown in FIG. 7, where encoder 701 has a DTX module 702 that includes an anti-aliasing algorithm 703.

[0072] A number of embodiments of the invention have been described. However, it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. Accordingly, other embodiments are included within the scope of the following claims.

[0073] A computing device implementing covariance smoothing techniques may have the following exemplary architecture. Other architectures are possible, including architectures with more or fewer components. In some implementations, an exemplary architecture includes one or more processors (e.g., dual-core Intel® Xeon® processors), one or more output devices (e.g., a liquid crystal display (LCD)), one or more network interfaces, one or more input devices ( e.g., mouse, keyboard, touch display) and one or more computer-readable media (e.g., random access memory (RAM), read-only memory (ROM), synchronous dynamic RAM (SDRAM), hard disk, optical disk, flash -memory, etc.). These components may communicate and exchange data over one or more communication channels (eg, a bus), which may use various hardware and software to enable the transfer of data and control signals between the components.

[0074] The term "computer readable medium" refers to a medium that is involved in providing instructions to a processor for execution, including, but not limited to, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memories), and media transfers. Transmission media include, but are not limited to, coaxial cables, copper wire and fiber optics.

[0075] The computer-readable medium may further comprise an operating system (eg, a Linux® operating system), a network communication module, an audio interface manager, an audio processing manager, and a live content distributor. The operating system can be multi-user, multi-processor, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks including, but not limited to: recognizing input and providing output to network interfaces 706 and/or devices 708; tracking and managing files and directories on machine-readable media (such as memory or mass storage); peripheral device management; and managing traffic over one or more communication channels. The network communication module includes various components for establishing and maintaining network connections (for example, software for implementing communication protocols such as TCP/IP, HTTP, etc.).

[0076] The architecture may be implemented in a parallel processing or peer-to-peer infrastructure or on a single device with one or more processors. The software may include multiple software components or may be a single code.

[0077] The described functions may be implemented advantageously in one or more computer programs that execute on a programmable system including at least one programmable processor coupled to receive data and commands and to transmit data and commands to a data storage system, according to at least one input device and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, by a computer to perform a specific activity or achieve a specific result. A computer program may be written in any programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it may be deployed in any form, including as a standalone program or as a module, component, subroutine, web a browser-based application or other object suitable for use in a computing environment.

[0078] Suitable processors for executing the instruction program include, by way of example, general purpose and special purpose microprocessors and a single processor or one of multiple processors or computer cores of any kind. Typically, the processor receives instructions and data from read only memory and/or random access memory. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, the computer also includes or may be operatively coupled to communicate with one or more mass storage devices for storing data files; such devices include magnetic disks such as internal hard drives and removable drives; magneto-optical disks; and optical discs. Storage devices suitable for physically embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices; magnetic disks such as internal hard drives and removable drives; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory can be extended or built into ASICs (application specific integrated circuits).

[0079] To provide user interaction, the functions may be implemented on a computer having a display device such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), monitor, or retinal display for displaying information to the user. A computer may have a touch surface input device (such as a touch screen) or a keyboard and a pointing device, such as a mouse or trackball, that allows the user to enter data into the computer. The computer may have a voice input device for receiving voice commands from the user.

[0080] The functions may be implemented in a computer system that includes an internal component, such as a data server, or that includes a middleware component, such as an application server or Internet server, or that includes an external component, such as a client computer having a graphical user interface or an Internet browser, or any combination thereof. The system components can be connected using any kind of digital data transmission medium, such as a communications network. Examples of communication networks include, for example, a local area network, a wide area network, and the computers and networks that make up the Internet.

[0081] The computing system may include clients and servers. The client and server are typically remote from each other and usually communicate through a communications network. The client-server relationship occurs through computer programs running on the respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (eg, an HTML page) to the client device (eg, for the purposes of displaying data and receiving user input from the user interacting with the client device). Data generated on the client device (for example, as a result of user interaction) can be retrieved from the client device to the server.

[0082] A system of one or more computers may be configured to perform certain actions by having software, firmware, hardware, or a combination thereof installed on the system that, when operated, causes the system to perform the actions. One or more computer programs may be configured to perform certain actions by including instructions that, when executed by a processing device, cause the device to perform the actions.

[0083] Although this description contains many detailed information about a particular implementation, they should not be construed as limitations on the scope of any inventions or claimed subject matter, but as descriptions of features specified for specific embodiments of specific inventions. Certain features that are described in this specification in the context of individual embodiments may also be implemented in combination in a single embodiment. On the other hand, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. In addition, although features may be described above as operating in certain combinations, and even initially stated as such, one or more features from a claimed combination may in some cases be removed from the combination, and the claimed combination may be directed into subcombinations or a variation of a subcombination.

[0084] Likewise, although operations are depicted in the drawings in a particular order, this should not be construed as a requirement that such operations must be performed in the particular order shown or in a sequential order, or that all illustrated operations must be performed to achieve the desired results. Under certain circumstances, multitasking and parallel processing may be beneficial. In addition, the separation of various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it is understood that the described program components and the system as a whole may be combined together into a single software product or packaged into multiple software products. products.

[0085] Thus, specific embodiments of the subject matter of the invention have been described. Other embodiments are within the scope of the following claims. In some cases, the steps stated in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or sequential order to achieve the desired results. Some implementations may benefit from multitasking and parallel processing.

[0086] Several implementations of the invention have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of the invention.

Claims (50)

1. A method for providing an audio service, comprising the steps of: for a time domain sequence of signal frames, comparing the effective number of cells in the frequency band with the required number of cells for the frequency band; calculating the forgetting coefficient for the frequency band as the ratio of the effective amount to the required amount; And for each frequency band of the plurality of frequency bands, generating a corresponding band-specific smoothed covariance matrix for the current signal frame using a previously generated value of the corresponding band-specific smoothed covariance matrix for the previous signal frame relative to the current signal frame and a forgetting factor; And generating an audio signal for an audio service using frequency domain representations of a sequence of signal frames in the time domain, and further using corresponding sets of corresponding smoothed covariance matrices for a particular band, wherein the comparison, calculation and generation are performed by a system comprising one or more computer processors. 2. The method of claim 1, wherein the effective number of cells for each frequency band of the plurality of frequency bands is calculated as the sum of the values of the filter bank for the frequency band. 3. Method according to any one of paragraphs. 1.2, in which an autoregressive first-order low-pass filter is used to form the corresponding smoothed covariance matrix for a specific band. 4. The method of claim 3, wherein using the first order autoregressive low-pass filter includes calculating the difference between the input covariance matrix for the current frame and a previously generated value of the corresponding smoothed band-specific covariance matrix for the previous frame, the difference being weighted by a forgetting factor. 5. Method according to any one of paragraphs. 1-4, also containing the steps of: setting the forgetting rate to a smaller value selected from a pair of values including the calculated forgetting rate and the maximum allowed forgetting rate; wherein the setting is performed before generating the corresponding smoothed covariance matrix for a particular band for the current frame. 6. Method according to any one of paragraphs. 1-5, also containing steps in which: detecting whether a transient occurs in the tracked frame; And reset the low-pass filter in response to the detected transient. 7. The method of claim 6, wherein the detection is performed by using aspects of the muting decorrelator. 8. The method according to claim 6, in which the detection is performed on all channels. 9. The method of claim 8, wherein the detection detects that a transient occurs if any transient is detected on any channel for the tracked frame. 10. The method of claim 8, wherein the detection detects that a transient occurs if the transient occurs only on a specific channel. 11. Method according to any one of paragraphs. 1-10, also containing the steps of: storing the corresponding smoothed covariance matrix for the particular band; And transforming samples of the stored corresponding smoothed covariance matrix for a particular band across a plurality of frequency bands. 12. A system for providing audio services, containing: one or more computer processors; And a machine-readable non-transitory storage medium storing instructions that, when executed by one or more computer processors, cause one or more computer processors to perform the operations of the method of claim 1. 13. A machine-readable non-transitory storage medium storing instructions that, when executed by one or more computer processors, cause one or more computer processors to perform the operations of the method of claim 1. 14. An encoder containing the system according to clause 12. 15. A decoder containing the system according to claim 12. 16. A method for providing an audio service, comprising the steps of: receiving an input signal representing N audio channels, where N>1; dividing the input signal into a sequence of frames in the time domain; converting each frame of the sequence of time domain frames into a corresponding frequency domain representation having a plurality of frequency bands, each frequency band of the plurality of frequency bands including a corresponding plurality of frequency cells; for each frame of a sequence of time domain frames, calculate, based on the frequency domain representation of the frame, a corresponding set of band-specific N×N covariance matrices, each of the band-specific N×N covariance matrices having N audio channel covariances corresponding to one of a plurality of frequency stripes; for each frequency band of the plurality of frequency bands, calculating a corresponding band-specific smoothed covariance matrix by applying a low-pass filter to a sequence of band-specific N×N covariance matrices corresponding to the sequence of time domain frames and also corresponding to the frequency band; And generating an audio signal for the audio service using appropriate frequency domain representations of the sequence of frames in the time domain, and further using appropriate sets of corresponding smoothed covariance matrices for the particular band. 17. The method of claim 16, wherein the low-pass filter is a first-order autoregressive low-pass filter. 18. The method of claim 16, wherein the application includes calculating the difference between the value of the input covariance matrix for the current frame and the previously generated value of the smoothed covariance matrix for the previous frame, the difference being weighted by a forgetting factor. 19. The method according to claim 16, also containing the steps of: detecting whether a transient occurs in the tracked frame; And reset the low-pass filter in response to the detected transient. 20. The method of claim 16, wherein the plurality of frequency bands includes a first frequency band and a second frequency band; wherein the first frequency band has fewer frequency cells than the second frequency band; And wherein the second frequency band has higher frequencies than the first frequency band. 21. Method according to any one of paragraphs. 16-20, in which a plurality of frequency bands are ordered according to increasing frequency; And in this case, the forgetting factor is set depending on the band so that the degree of smoothing increases in lower bands and decreases in higher bands. 22. Method according to any one of paragraphs. 16-20, in which the forgetting factor is set in accordance with the audio service mode such that the smoothing amount increases in modes having a lower bit rate and decreases in higher bit rates. 23. The method according to claim 18, also containing the steps of: set the forgetting coefficient to a smaller value selected from a pair of values including the calculated forgetting coefficient and the maximum permissible forgetting coefficient, wherein the setting is performed before generating the corresponding smoothed covariance matrix for a particular band for the current frame. 24. The method of claim 19, wherein the detection detects that a transient occurs if the transient occurs only on a specific channel.

Images