KR101360456B1 - Providing a Time Warp Activation Signal and Encoding an Audio Signal Therewith - Google Patents

Abstract

The audio encoder includes a window function controller 504, a windower 502, a time warper 506 having a final quality check function, a time / frequency converter 508, a TNS stage 510, or a quantizer encoder 512, And includes a window function controller 504, a windower 502, a time warper 506 with a final quality check function, a time / frequency converter 508, a TNS stage 510, or an additional noise-filling analyzer 524 Is controlled by the time warp analyzer 516 or the signal analysis result obtained by the signal classifier 520. [ The decoder also applies a noise filling operation using the manipulated noise-filling estimates according to the harmonic or speech characteristics of the audio signal.

Description

Providing a time warp activation signal and encoding an audio signal using the same provide a time warp activation signal and an audio signal.

The present invention relates to the encoding / decoding of audio signals having harmonic or speech content, which can be provided for audio encoding and decoding, especially time warping processing.

In the following, a brief introduction to the time-warped audio encoding arts will be given, the concept of which can be applied in conjunction with various embodiments of the present invention.

Recently, techniques have been developed for converting audio signals into frequency domain representations, and efficiently encoding such frequency domain representations, for example, considering perceptual masking thresholds. This concept of audio signal encoding is particularly applicable when only a small number of spectral coefficients with a long block length, over which a series of encoded spectral coefficients are transmitted, well exceed the global masking threshold and a large number of spectral coefficients, And is negligible (or coded with a minimum code length) near or below the masking threshold.

For example, cosine based or sign-based modulated lap transforms can often be used for applications for source coding due to their energy compaction nature. That is, for harmonic tones with constant fundamental frequencies (pitch), these transformations concentrate the signal energy into a small number of spectral components (sub-bands), which leads to efficient signal representation.

In general, the (fundamental) pitch of the signal will be understood as the lowest dominant frequency that is distinguishable from the spectrum of the signal. In a typical speech model, the pitch is the frequency of the excitation signal modulated by the human neck. If there is only a single fundamental frequency, the spectrum will be extremely simple, including only the fundamental frequency and overtones. This spectrum can be encoded very efficiently. However, for a signal with a varying pitch, the energy corresponding to each harmonic component is distributed across the various transform coefficients, thus reducing the coding efficiency.

To overcome this reduction in coding efficiency, the audio signal to be encoded is efficiently resampled on a non-uniform time grid. In subsequent processing, sample locations obtained by non-uniform resampling are treated as if they represent values on a uniform time grid. This behavior is often indicated by the term " time warping ". The sample times are advantageously chosen according to the temporal variation of the pitch such that the pitch variation in the time warped version of the audio signal may be less than the pitch variation in the original version of the audio signal (before time warping). This pitch variation can also be expressed in terms of a "time warp contour ". After time warping of the audio signal, the time warped version of the audio signal is transformed into the frequency domain. Pitch-dependent time warping has the effect that the frequency domain representation of a time warped audio signal routinely exhibits energy compaction with a much smaller number of spectral components than the frequency domain representation of the original (non-time warped) audio signal .

On the decoder side, the frequency-domain representation of the time-warped audio signal is converted back to the time domain so that time-domain transformation of the time-warped audio signal is possible on the decoder side. However, in the time-domain representation of the reconstructed time-warped audio signal on the decoder side, the original pitch variation of the input-side audio signal on the encoder-side is not included. Thus, another time warping by re-sampling of the reconstructed time-domain representation of the decoder-side of the time-warped audio signal is applied. In order to obtain a good reconstruction of the input audio signal on the encoder side from the decoder side, it is desirable that the time warping on the decoder side is at least approximately the opposite of the time warping on the encoder side. In order to obtain an appropriate time warping, it is desirable to have valid information on the decoder side that takes account of adjustment of the time warping of the decoder-side.

As the transmission of this information from the audio signal encoder to the audio signal decoder is typically required, it is desirable to keep the bitrate required for such transmission low while still allowing reliable reconstruction of the time warp information requested at the decoder side.

In view of the foregoing discussion, it is desirable to create a concept that takes into account the bit rate efficient application of a time warp in an audio encoder.

The present invention aims to create a concept that improves the sense of hearing provided by an audio signal encoded based on information available in a time warping audio signal encoder or a time warping audio signal decoder.

This object comprises a time warp active signal provider for providing a time warp active signal based on a representation of an audio signal according to claim 1, an audio signal encoder for encoding an input audio signal according to claim 12, a time warp active signal according to claim 14. A method for providing a method, a method for providing an encoded representation of an input audio signal according to claim 15, or a computer program according to claim 16.

It is yet another object of the present invention to provide an improved audio encoding / decoding method that provides higher quality or lower bit rates.

This object is achieved by an audio encoder according to claim 17, 26, 32, 37, an audio decoder according to claim 20, an audio encoding method according to claim 23, 30, 35 or 37, a decoding method according to claim 24, 25, By a computer program according to 31, 36 or 43.

Embodiments according to the present invention relate to a method for time warped MDCT transcoder. Some embodiments relate to an encoder-only means. However, other embodiments also relate to decoder means.

One embodiment of the invention creates a time warp active signal provider that provides a time warp active signal based on a representation of the audio signal. The time warp active signal provider comprises an energy compaction information provider configured to provide energy compaction information that describes energy compaction into a time warp transformed spectral representation of the audio signal. The time warp active signal provider also includes a comparator configured to compare the energy collapse information with a reference value and to provide a time warp active signal according to the result of the comparison.

This embodiment is particularly advantageous if the time warp transformed spectral representation of the audio signal includes a sufficiently compact energy distribution in which the energy is concentrated in one or more spectral regions (or spectral lines) It is based on the discovery that the use of the time warping function in the audio signal encoder usually results in improvement. This allows successful time warping to transform a smeared spectrum of an audio frame, for example, into a spectrum having one or more identifiable peaks, and thus a higher energy compromise than the spectrum of the original (non-time-warped) The effect of reducing the bit rate is brought about.

With regard to this issue, it should be understood that an audio signal frame includes a smear spectrum while the pitch of the audio signal changes significantly. The time-varying pitch of the audio signal has the effect of deriving a smear distribution of the signal energy on the frequency, especially in the higher frequency region, the time-domain versus frequency-domain transformation performed on the audio signal frame. Thus, the spectral representation of this original (non-temporally warped) audio signal includes low energy compaction, and typically does not show spectral peaks in the higher frequency portion of the spectrum, or is relatively high in the higher frequency portion of the spectrum Only small spectral peaks are shown. In contrast, if time warping is successful (in terms of providing an improvement in encoding efficiency), the temporal warping of the original audio signal is time warped with relatively high and cleaner peaks (especially in the higher frequency portion of the spectrum) Lt; / RTI > This is because an audio signal having a time varying pitch is converted into a time warped audio signal having a smaller pitch variation or a roughly constant pitch. Thus, the spectral representation of a time-warped audio signal (which may be considered as a time-warped transformed spectral representation of an audio signal) includes one or more clean spectral peaks. In other words, the smearing of the spectrum of the original audio signal (with a temporally variable pitch) is reduced by a successful time warping operation, and the time warp transformed spectral representation of the audio signal is higher than the spectrum of the original audio signal Compaction. Nonetheless, time warping is not always successful in improving coding efficiency. For example, if the input audio signal contains large noise components, or if the extracted time warp contour is inaccurate, time warping does not improve coding efficiency.

From this contextual point of view, the energy compaction information provided by the energy compaction information provider is a useful indicator for determining if the time warp is successful in terms of reducing the bit rate.

One embodiment of the invention creates a time warp active signal provider that provides a time warp active signal based on a representation of the audio signal. The time warp active signal provider includes two time warp representation providers configured to provide two time warp representations of the same audio signal using different time warp contour information. Thus, the temporal warp representation provider can be constructed (structurally and / or functionally) in the same way and use the same audio signal, but other time warp contour information. The time warp active signal provider may also include two energy compaction information providers configured to provide first energy compaction information based on a first time warp representation and second energy compaction information based on a second time warp representation . The energy compaction information provider may be constructed in the same way, but may be configured to use different time warp representations. In addition, the time warp active signal provider includes a comparator that compares two different energy compaction information and provides a time warp activation signal according to the result of the comparison.

In a preferred embodiment, the energy compaction information provider is configured to provide a spectral flatness measure indicative of a time warp transformed spectral representation of the audio signal as energy compaction information. It has been found that time warping is successful in reducing the bit rate if it converts the spectrum of the input audio signal to a less flat time warp spectrum representing a time warped version of the input audio signal. Thus, a measure of spectral flatness can be used to determine whether a time warp should be activated or deactivated, without performing an overall spectral encoding process.

In one preferred embodiment, the energy compaction information provider is configured to calculate an arithmetic mean of the time warped transformed power spectrum and a quotient of the geometric mean of the time warped transformed power spectrum, and to obtain a measure of spectral flatness . This index was found to be a well-applied spectral flatness measure to describe possible bitrate savings obtainable by time warping.

In another preferred embodiment, the energy compaction information provider is configured to highlight the high-frequency portion of the time warp-transformed spectrum by comparing the low frequency portion of the time warp-transformed spectrum to obtain energy compaction information. This concept is based on the fact that the time warp typically has a much greater influence over the higher frequency range than the lower frequency range. Thus, in order to determine the effectiveness of the time warp using the spectral flatness measure, a dominant estimate in the higher frequency range is appropriate. In addition, typical audio signals exhibit harmonic content (including fundamental frequency harmonics) whose intensity is attenuated as the frequency increases. The emphasis of the higher frequency portion of the time warp distorted spectral representation over the lower frequency portion of the time warped transformed spectral representation also helps compensate for this typical attenuation of the spectral lines as the frequency increases. In summary, an emphasis on the higher portion of the spectrum leads to increased reliability of the energy compaction information, thereby allowing more reliable provisioning of time warped active signals.

In another preferred embodiment, the energy compaction information provider is configured to provide a plurality of band-like measures of spectral flatness and to calculate an average of a plurality of band-like measures of spectral flatness. It has been found that the consideration of a plurality of band-wise measures of spectral flatness leads to particularly reliable information regarding whether the time warp is efficient in reducing the bit rate of the encoded audio signal. First, the encoding of the time-warped transformed spectral representation is typically performed in a band-wise fashion so that a combination of band-like measures of spectral flatness is well-suited to encoding, and thus an improvement in bit rate that can be obtained with good accuracy . In addition, the band-wise computation of the spectral flatness measures substantially eliminates the dependence of energy compaction information from the distribution of harmonics. For example, even though the higher frequency band includes relatively less energy (smaller than the energies of the lower frequency bands), the higher frequency band may still be perceptually related. However, if the spectral flatness measure is not calculated in a band-wise fashion, the positive effect of a time warp on this higher frequency band (due to the small energy of only the higher frequency band, a reduction in the smearing of the spectral lines ) Will be considered small. By applying band-wise operations in contrast, the positive impact of the time warp can be considered with appropriate weights, since band-wise spectral flatness measures are dependent on absolute energy in the individual frequency bands.

In another preferred embodiment, the time warp active signal provider comprises a reference value calculator configured to calculate a measure of spectral flatness representing a non-time-warped spectral representation of the audio signal to obtain a reference value. Accordingly, the time warp enable signal is provided based on a comparison of the spectral flatness of the non-time-warped (or "warped") version of the input audio signal and the spectral flatness of the time warped version of the input audio signal .

In another preferred embodiment, the energy compaction information provider is configured to provide a measure of a perceptual entropy representing a time warp transformed spectral representation of the audio signal as energy compaction information. This concept is based on the fact that the time warp transformed spectral representation is a good estimate of the number of bits (or bit rate) needed to encode the time warp transformed spectrum. Thus, a measure of the perceptual entropy of a time warp distorted spectral representation is a good measure of whether a decrease in bit rate can be predicted by temporal warping, in terms of the fact that additional time warp information should be encoded if time warping is used Scale.

In another preferred embodiment, the energy compaction information provider is configured to provide an autocorrelation measure indicative of autocorrelation of the time warped representation of the audio signal as energy compaction information. This concept is based on the fact that the efficiency of the time warp (in terms of reducing the bit rate) can be measured (or at least estimated) based on time warped (or non-uniformly resampled) time domain signals. It has been found that time warping is efficient if the time warped time domain signal has a relatively high degree of periodicity as reflected by the autocorrelation measure. Conversely, if the time warped time domain signal does not contain significant periodicity, it can be concluded that time warping is not efficient.

This finding is based on the fact that efficient time warping transforms a portion of the sine wave signal of varying frequencies (not including the periodicity) into a portion of the sine wave signal of a substantially constant frequency (including a high degree of periodicity) do. Conversely, if time warping does not have the capability to provide a time domain signal with a high degree of periodicity, time warping can also be expected to fail to provide significant bit rate savings to justify the application of time warping.

In a preferred embodiment, the energy compaction information provider is configured to determine (sum over a plurality of lag values) the sum of the absolute values of the normalized autocorrelation function of the time warped representation of the audio signal to obtain energy compaction information. It has been found that the computationally complex determination of autocorrelation peaks is not required to estimate the efficiency of time warping. Rather, it has been found that the summation of autocorrelation over the (broad) range of autocorrelation lag values also yields very reliable results. This is due to the fact that the time warp transforms a plurality of signal components (e.g., a fundamental frequency and its harmonics) at a frequency at which it substantially changes into periodic signal components. Hence, autocorrelation of this time warped signal represents peaks of a plurality of autocorrelation lag values. Thus, the summation-form is a computationally efficient way to extract energy compaction information from autocorrelation.

In another preferred embodiment, the time warp active signal provider comprises a reference value calculator configured to calculate a reference value based on a non-time-warped spectral representation or based on an empty time-domain representation of the audio signal. In this case, the comparator is typically configured to form a ratio value using the energy collapse information and the reference value, which indicate compaction of the energy in the time warped representation of the audio signal. The comparator is also configured to compare the ratio value to one or more thresholds to obtain a time warp active signal. It has been found that the ratio between the energy compaction information in the non-time-warped case and the energy compaction information in the time warping case is computationally efficient and still allows a sufficiently reliable generation of the time warp active signal.

Another preferred embodiment of the present invention creates an audio signal encoder that encodes an input audio signal to obtain an encoded representation of the input audio signal. The audio signal encoder includes a time warp converter configured to provide a time warp transformed spectral representation based on the input audio signal using a time warp contour. The audio signal encoder also includes a time warp active signal provider as previously described. The time warp active signal provider is configured to receive an input audio signal and to provide a time warp enable signal such that the energy compaction information describes energy compaction in a time warp transformed spectral representation of the input audio signal. The audio signal encoder may additionally include a set of non-uniform (varying) time warp contour information or time warping information, or standard uniform (non-changing) time warp contour information or time warping information, And a controller that selectively provides the time warping converter. In this manner, it is possible to selectively accept or reject set non-uniform time warping contour portions in the derivation of the audio signal representation encoded from the input audio signal.

This concept is based on the discovery that it is not always efficient to introduce time warping information into the encoded representation of the input audio signal, since a tremendous number of bits are needed to encode the time warping information. Further, the energy collapse information calculated by the time warp activation signal provider may be provided to the time warp converter, such as non-uniform (varying) time warp contour information, or standard (non-changing, uniform time warp contour information) It is a computationally efficient measure of whether or not it is advantageous to do so. It should be noted that when the time warp transformer includes a nested transform, the set temporal warp contour portion can be used in the calculation of two or more consecutive transform blocks. In particular, in order to determine whether time warping is saved in terms of bit rate, a version of the time warped transformed spectral representation of the input audio signal using a newly set time warping contour portion and a standard (non-changing) It has been found that it is unnecessary to encode all versions of the time warped transformed spectral representation of the input audio signal using the contour portion. Rather, it has been found that the evaluation of the energy compaction of the time-warped transformed spectral representation of the input audio signal forms a reliable basis for the decision. Therefore, the required bit rate can be kept small.

In another preferred embodiment, the audio signal encoder comprises an output interface configured to selectively include in the encoded representation of the audio signal time warp contour information indicative of a set varying time warp contour, in accordance with a time warp active signal. Thus, regardless of whether the input signal is highly suitable for time warping, a high efficiency of the audio signal encoding can be obtained.

Yet another embodiment in accordance with the present invention creates a method of providing a time warp active signal based on an audio signal. The method performs the function of a time warp active signal provider and can be supplemented by any of the features and functions described herein for the time warp active signal provider.

Yet another embodiment in accordance with the present invention creates a method of encoding an input audio signal to obtain an encoded representation of the input audio signal. This method may be supplemented by any of the features and functions described herein with respect to an audio signal encoder.

Yet another embodiment in accordance with the present invention creates a computer program that executes the methods described above.

According to a first aspect of the present invention, an analysis of whether an audio signal has a harmonic or speech characteristic can be advantageously used to control the noise-filling processing at the encoder side and / or at the decoder side. The audio signal analysis is readily obtainable in a system in which the time warp function is used, which typically distinguishes between the pitch tracker and / or the speech on one side and the music on the other side and / or the voiced speech and unvoiced speech As shown in FIG.

Since this information is available at no additional cost, such available information can be advantageously used to control noise filling characteristics, and in particular for speech signals, the noise filling between harmonic lines is reduced, or especially for speech signals It can even be removed. Even if strong harmonic content is obtained, but the speech is not directly detected by the speech detector, a reduction in noise filling will nevertheless result in a higher perceived quality. When a particular signal analyzer is to be inserted into the system because these properties are particularly useful in systems in which harmonic / speech analysis is performed, and thus such information is available at no additional cost, but quality is improved without increasing the bit rate, In other words, even if the bit rate is reduced without loss in quality because the bits needed to encode the noise fill level are reduced when the noise fill level that can be transmitted from the encoder to the decoder itself is reduced, It is additionally useful to control the noise filling method based on signal analysis as to whether or not it has speech characteristics.

In a further aspect of the invention, the signal analysis result, i. E. Whether the signal is a harmonic or speech signal, is used to control the window function processing of the audio encoder. It has been found that in the situation where a speech or harmonic signal is initiated, the direct encoder will switch from a long window to a short window. However, this short window will have a correspondingly reduced frequency resolution, but conversely, it will reduce the coding gain for strongly harmonized signals, thereby increasing the number of bits needed to code such signal portions. In this regard, the invention defined in this aspect uses a window that is longer than a short window when a speech or harmonic signal onset is detected. Alternatively, the windows are selected to be substantially similar to the long window, but with a shorter overlap, to effectively reduce the pre-echo. In general, the signal characteristics, whether the time frame of the audio signal has harmonic or speech characteristics, is used to select the window function for this time frame.

According to a further aspect of the present invention, TNS (temporal noise shaping) means are controlled based on whether the underlying signal is based on a time warping operation or in a linear domain. Typically, the signal processed by the time warping operation will have a strong harmonic content. Otherwise, the pitch tracker associated with the time warping stage will not output a valid pitch contour, and without such a valid pitch contour, the time warping function will be deactivated during this time frame of the audio signal. However, it is generally not appropriate to apply TNS processing to harmonic signals. TNS processing is particularly useful when the signal processed by the TNS stage has a fairly flat spectrum and results in significant gains in bit rate / quality. However, if the form of the signal is tonal, i.e. non-flat, as in the case of spectrums with harmonic or voiced components, the gain at the quality / bit rate provided by the TNS means will decrease. Therefore, without significant variations of the TNS means, the time-warped portions will typically not be TNS processed and will be processed without TNS filtering. Conversely, the noise shaping characteristics of the TNS nonetheless provide particularly improved quality in situations where the signal varies in amplitude / power. Where there is an onset of the harmonic or speech signal and where block switching characteristics are implemented such that longer windows or windows longer than at least a short window are maintained instead of such onsets the activation of the temporal noise shaping property Will result in a concentration of noise around the speech onset that effectively reduces the pre-echoes that can occur prior to the onset of speech due to the quantization of frames occurring in subsequent encoder processing.

According to a further aspect of the present invention, a variable number of lines introduced from frame to frame by performing a time warping operation using a variable time warping property / warping contour to account for variable bandwidth, And processed by a quantizer / entropy encoder in the device. When a time warping operation results in a situation where the time of a frame included in a time warping frame (in a linear fashion) is reduced, a number of frequency lines for a single frequency line bandwidth, and a constant overall bandwidth, . ≪ / RTI > On the other hand, when the time warping operation results in the fact that the actual time of the audio signal in the time warping region is reduced for the blow length of the audio signal in the linear region, the frequency bandwidth of the single frequency line is reduced, and therefore, reduced To avoid bandwidth fluctuations or, optimally, bandwidth fluctuations, the number of lines processed by the source encoder should be reduced for non-time-warping situations.

According to the present invention, which provides an audio signal encoded based on information valid in a time warping audio signal encoder or a time warping audio signal decoder, it not only improves the audibility but also provides a higher quality or lower bit rate.

Figure 1 shows a block schematic diagram of a time warp active signal provider, in accordance with an embodiment of the invention.
Figure 2a shows a block schematic diagram of an audio signal encoder, in accordance with an embodiment of the present invention.
Figure 2B shows another block schematic diagram of a time warp active signal provider, in accordance with an embodiment of the present invention.
3A shows a graphical representation of the spectrum of a non-time-warped version of an audio signal.
Figure 3B shows a graphical representation of the spectrum of the time warped version of the audio signal.
Figure 3c shows a graphical representation of the individual operations of the spectral flatness measure for several frequency bands.
FIG. 3D shows a graphical representation of a spectral flatness scale operation that considers only the high frequency portion of the spectrum.
3E shows a graphical representation of a spectral flatness scale operation using a spectral representation with a higher frequency portion highlighted than the lower frequency portion.
FIG. 3F shows a block schematic diagram of an energy compaction information provider according to another embodiment of the present invention.
Figure 3g shows a graphical representation of an audio signal having a temporally variable pitch in the time domain.
Figure 3h shows a graphical representation of a time warped (non-uniformly resampled) version of the audio signal of Figure 3g.
Figure 3i shows a graphical representation of the autocorrelation function of the audio signal according to Figure 3g.
Figure 3j shows a graphical representation of the autocorrelation function of the audio signal according to Figure 3h.
FIG. 3K shows a block schematic diagram of an energy compaction information provider according to another embodiment of the present invention.
4A shows a flowchart of a method of providing a time warp activation signal based on an audio signal.
4B shows a flowchart of a method of encoding an encoded input audio signal to obtain an encoded representation of the input audio signal, in accordance with an embodiment of the present invention.
5A shows a preferred embodiment of an audio encoder having aspects of the invention.
Figure 5b illustrates a preferred embodiment of an audio decoder having aspects of the invention.
6A shows a preferred embodiment of the noise filling aspect of the present invention.
6B shows a table defining the control operation performed by the noise filling level operator.
Figure 7A illustrates a preferred embodiment of performing time warp-based block switching in accordance with the present invention.
Figure 7b shows another alternative embodiment that affects the window function.
Figure 7c shows yet another alternative embodiment showing a window function based on time warping information.
7D shows a window sequence of normal AAC operation at the onset of the voiced sound.
Figure 7E illustrates another window sequence obtained in accordance with a preferred embodiment of the present invention.
Figure 8A shows a preferred embodiment of a time warp-based control of TNS (temporal noise shaping) means.
FIG. 8B shows a table defining the control procedures executed in the threshold control signal generator of FIG. 8A.
Figures 9A-9E illustrate the corresponding effects on the bandwidth of the audio signal that occur following various time warping characteristics and decoder-side temporal inverse dewarping operation.
10A shows a preferred embodiment of a controller for controlling the number of lines in the encoding processor.
FIG. 10B shows the relationship of the number of lines to be discarded / added to the sampling rate.
11 shows a comparison between a linear time scale and a warped time scale.
Figure 12A illustrates an implementation in terms of bandwidth extension.
Figure 12B shows a table showing the relationship between the local sampling rate and the control of the spectral coefficients in the time warped region.

Hereinafter, preferred embodiments will be described in order with reference to the accompanying drawings.

Figure 1 shows a block schematic diagram of a time warp active signal provider, in accordance with an embodiment of the invention. The time warp active signal provider 100 is configured to receive a representation 110 of an audio signal and provide a time warp activation signal 112 based thereon. The time warp active signal provider 100 comprises an energy compaction information provider 120 configured to provide energy compaction information 122 indicative of energy compaction of a time warped transformed spectral representation of an audio signal. The time warp enable signal provider 100 further includes a comparator 130 configured to compare the reference value 132 with the energy compaction information 122 and to provide a time warp activation signal 112 according to the result of the comparison .

As described above, it has been found that the energy collapse information provides a computationally efficient estimation as to whether the time warp leads to a bit saving or not. Bite saving is closely related to the question of whether time warp leads to energy compaction.

Figure 2a shows a block schematic diagram of an audio signal encoder 200, in accordance with an embodiment of the invention. The audio signal encoder 200 is configured to receive an input audio signal 210 (also represented as a (t)) and provide an encoded representation 212 of the input audio signal 210 based thereon. The audio signal encoder 200 is configured to receive an input audio signal 210 (which may be represented in time domain) and to provide a time warp transformed spectral representation 222 of the input audio signal 210 based thereon And a time warp converter 220. The audio signal encoder 200 also includes a time warp analyzer (e.g., a time warp analyzer) configured to analyze the input audio signal 210 and provide time warp contour information (e.g., absolute or relative time warp contour information) 284).

The audio signal encoder 200 may also be configured to determine whether the time warping contour information 286 set in the further processing is used or the standard time warping contour information 288 is used, , And a switching mechanism. Thus, the switching mechanism 240 may set the set time warp contour information 286 or standard time warp contour information 288 to new time warp contour information 242 for subsequent processing in accordance with the time warp activity information, For example, to a time warp converter 220. The temporal warp transformer 220 may be configured to transform the temporal distortion of the audio frame, for example, to a new temporal warp contour information 242 (e.g., a new temporal contour portion) One or more previously obtained time warp contour portions). Selective spectral post processing may include, for example, temporal noise shaping and / or noise filling analysis. The audio signal encoder 200 also includes a quantizer / encoder (not shown) that receives and quantizes the spectral representation 222 and the transformed spectral representation 222 (which is optionally processed by the spectral post processing 250) 260). To this end, a quantizer / encoder 260 is coupled with a perceptual model 270 to take perceptual masking into account for perceptual masking and to control the quantization accuracy in various frequency bins, Related information 272. [0156] FIG. The audio signal encoder 200 further includes an output interface 280 configured to provide an encoded representation of the audio signal based on the quantized and encoded spectral representation 262 provided by the quantizer / do.

The audio signal encoder 200 also includes a time warp enable signal provider 230 configured to provide a time warp enable signal 232. The time warp enable signal 232 is used to determine whether the newly set time warp contour information 286 in the additional processing steps (e.g., time warp converter 220) is to be used or standard time warp contour information 288 is to be used For example, to control the switching mechanism 240. [0035] Additionally, the time warp activity information 232 may be used to determine whether the selected new time warp contour information 242 (selected from the newly set time warp contour information 286 and standard time warp contour information 288) May be used in the switch 280 to determine whether it should be included in the encoded representation. Typically the time warp contour information is only included in the encoded representation 212 of the audio signal if the selected time warp contour information represents a non-uniform (varying) time warp contour. In addition, the time warp activity information 232 may itself be included in the encoded representation 212, e.g., in the form of a one-bit flag indicating activation or deactivation of a time warp.

For purposes of understanding, the time warp converter 220 typically includes an analysis windower 220a, a resampler or "time warper" 220b, and a spectral region converter (or time / frequency converter) It should be understood. However, depending on the implementation, the time warper 220b may be located before the analysis windower 220a - in the signal processing direction. However, in some embodiments, time warping and time domain to spectral domain transformation may be combined in a single unit.

In the following, details relating to the time warp enable signal provider 230 will be described. The time warp enable signal provider 230 may be equal to the time warp enable signal provider 100.

The time warp enable signal provider 230 preferably receives the time-domain audio signal representation 210 (also represented as a (t)), the newly set time warp contour information 286 and the standard time warp contour information 288 . The time warping enable signal provider 230 also uses the time-domain audio signal 210 to generate a new set of time warp contour information 286, standard time warp contour information 288 and a newly set time warp contour information 286 And provides a time warp activation signal 232 based on the energy compaction information.

FIG. 2B shows another block schematic diagram of a time warp enable signal provider 234, in accordance with an embodiment of the present invention. The time warp enable signal provider 234 may serve as the time warp enable signal provider 230 in some embodiments. The time warp enable signal provider 234 is configured to receive the input audio signal 210 and the two time warp contour information 286 and 288 and to provide a time warp enable signal 234p based thereon. The time warp enable signal 234p may serve as a time warp enable signal 232. The time warp active signal provider includes two identical temporal warp presentation providers 234a and 234g that receive input audio signal 210 and time warp contour information 286 and 288, respectively, Time warped representations 234e and 234k. The time warp active signal provider 234 also includes two identical energy compaction information providers 234f and 234l that each include time warped representations 234e and 234k, (234m and 234n), respectively. The time warp active signal provider also includes a comparator 234o configured to receive energy compaction information 234m and 234n and to provide a time warp activation signal 234p based thereon.

For the sake of clarity, the temporal warp expressors 234a and 234g are typically (optionally) the same analysis windowers 234b and 234h, the same resampler or time warpers 234c and 234i, ) And the same spectral region converters 234d and 234j.

In the following, several concepts for obtaining energy compaction information will be discussed. First, a description of the effect of time warping on a typical audio signal will be presented.

Hereinafter, with reference to FIGS. 3A and 3B, a description of the effect of time warping on a typical audio signal will be introduced. 3A shows a graphical representation of the spectrum of a non-time-warped version of an audio signal. The horizontal axis 301 represents the frequency and the vertical axis 302 represents the intensity of the audio signal. Curve 303 represents a non-time-warped audio signal as a function of frequency f.

Figure 3B shows a graphical representation of the spectrum of the time warped version of the audio signal. Again, the horizontal axis 306 represents the frequency and the vertical axis 307 represents the strength of the warped version of the audio signal. Curve 303 represents the intensity of the time-warped version of the audio signal on the frequency. As can be seen from a comparison of the graphical representations of Figures 3a and 3b, the non-time-warped ("warped") version of the audio signal includes a smear spectrum, especially in the high frequency domain. Conversely, the time warped version of the historical audio signal includes a spectrum with clearly distinguishable spectral peaks, even in the high frequency domain. Additionally, the sharpness of some of the spectral peaks may be absorbed even in the low spectral region of the time warped version of the input audio signal.

The time warped version of the spectrum of the input audio signal shown in FIG. 3B is quantized by a quantizer / encoder 260, for example, at a lower bit rate than the spectrum of the unaffected input audio signal shown in FIG. 3A Lt; / RTI > This is because while the "less planar" spectrum as shown in Figure 3 typically includes a large number of spectral coefficients quantized to zero or a small value, the smear spectrum is typically a large number of perceptually related spectral (I. E., Spectral coefficients quantized to a relatively small number, 0 or a small value). A large number of spectral coefficients quantized to zero or a small value may be encoded using fewer bits than quantized spectral values with higher values so that the spectrum of Figure 3b can be encoded using fewer bits than the spectrum of Figure 3a .

Nevertheless, it should be noted that the use of a time warp does not always result in a significant improvement in the coding efficiency of the time warped signal. Thus, in some cases, the cost in terms of bit rate required to encode temporal warp information (e.g., a time warp contour) may outweigh the bit rate savings (non-temporal warp transformed spectra Compared to encoding). In this case, it is desirable to provide a representation of the encoded audio signal using a standard (non-changing) time warp contour to control the time warp transformation. As a result, transmission of any time warp information (i.e., time warp contour information) can be omitted (except for flag indicating inactivation of time warping), and the bit rate can thereby be kept low.

In the following, several concepts for a reliable and computationally efficient operation of the time warp enable signal 112, 232, 234p will be described with reference to Figures 3c-3k. Prior to that, however, the background of the concept of the present invention will be briefly summarized.

The basic assumption is that applying time warping to a harmonic signal with a varying pitch will make the pitch constant, and making the pitch constant will cause the harmonics to be smoothed over several spectral bins (see Figure 3a) (See FIG. 3B) because only a limited number of significant lines remain (see FIG. 3B), thereby improving the coding of the spectrum obtained by subsequent time-frequency transforms. However, even if a pitch variation is detected, an improvement in the coding gain (i. E. The amount of bits saved) is negligible (e. G., When there is strong noise inherent in the harmonic signal, Or the amount of bits needed to transmit the time warping contour to the decoder, or simply be wrong. In these cases, an efficient 1-bit signaling is used to reject the varying time warp contour (e.g., 286) generated by the time warp contour encoder and instead signal a standard (bar-changing) time warp contour .

The scope of the present invention includes the generation of a method for determining whether the obtained time warp contour portion provides sufficient coding gain (e.g., coding gain sufficient to compensate for overhead required for encoding for a time warp contour).

As described above, the most important aspect of time warping is the compaction of the spectral energy to a smaller number of lines (see FIGS. 3A and 3B). It can also be seen that the compaction of the energy corresponds to a more "non-flat" spectrum, as the differences between the peaks and the valleys of the spectrum are increased. The energies are concentrated on fewer lines with lines with less energy in between than before.

Figures 3a and 3b show an illustrative example having a strong harmonics and an empty spectrum of a frame with pitch variation (Figure 3a), and a spectrum of a time warped version of the same frame (Figure 3b).

From this contextual point of view, it has been found advantageous to use the spectral flatness measure as a possible measure for the effectiveness of time warping.

The spectral flatness can be calculated, for example, by dividing the geometric mean of the power spectrum by the arithmetic mean of the power spectrum. For example, spectral flatness (also referred to briefly as "flatness") can be calculated according to the following equation:

Flatness =

In the above, x (n) represents the size of the bin number n. Also, N represents the total number of spectral bins considered in the calculation of the spectral flatness measure.

In one embodiment of the present invention, the calculation of the aforementioned "flatness ", which may be used as energy compaction information, is performed using time warp transformed spectral representations 234e and 234k, have.

In this case, N is equal to the number of spectral lines provided by spectral region converter 234d, 234j, and | X | _tw (n) are time warp transformed spectral representations 234e and 234k.

One disadvantage of the spectral flatness measure, even though the spectral scale is a useful amount for providing a time warp active signal, is that it does not provide a high energy when applied to the entire spectrum, such as a signal-to-noise-ratio (SNR) The emphasis is on the parts that we have. Generally, harmonic spectra have a specific spectral tilt, because most of the energy is concentrated on the first few tones and decreases with increasing frequency, resulting in insufficient representations of higher parts of the scale it means. In some embodiments, this phenomenon is undesirable because the high portions are desired to be maximally smeared (Fig. 3A) and to improve the quality of these high portions. In the following, some optional concepts for improving the relevance of spectral flatness will be discussed.

In one embodiment in accordance with the present invention, an approach similar to the so-called "segmental SNR" measure is chosen, leading to a band-wise flatness measure. The calculation of the spectral flatness measure is performed within several bands (e.g., individually) and the main (or average) is selected. Multiple bands may have the same bandwidth. However, the bandwidths may well follow the perceptual scale, such as the critical bands, or may correspond to the scale factor bands of "enhanced audio coding" also known as AAC for example.

The above-mentioned concept will be briefly described below with reference to FIG. 3C which shows a graphical representation of the individual operations of the spectral flatness measure for several frequency bands. As can be seen, the spectrum may be divided into several frequency bands 311, 312, and 313 that may have the same bandwidth or have different bandwidths. For example, a first spectral flatness measure may be calculated for the first frequency band 311 using, for example, the equation for "flatness" given earlier. In this calculation, the frequency bins of the first frequency band can be taken into account (the operating variable n can take on the frequency bin indices of the frequency bins of the first frequency band) and the width of the first frequency band 311 can be taken into account (Variable N may take the width in terms of frequency bins of the first frequency band). Thereby, a flatness measure of the first frequency band 311 is obtained. Similarly, a flatness measure may be calculated for the second frequency band 312, taking into account the widths of the second frequency bands 312 and also the second frequency band. Also, the flatness measures of additional frequency bands, such as the third frequency band 313, can be calculated in the same manner.

Thereafter, an average of the flatness measures for the various frequency bands 311, 312, and 313 can be calculated, and this average can be utilized as the energy compaction information.

Another approach (for improving temporal warp signal derivation) is to apply a spectral flatness measure only on a specific frequency. This approach is illustrated in Figure 3b. As can be seen, only frequency bins in the upper frequency portions 316 of the spectra can be considered in the calculation of the spectral flatness measure. The lower frequency part of the spectrum is ignored in the calculation of the spectral flatness scale. The high frequency portion 316 may be considered in a frequency-band manner for the calculation of the spectral flatness measure. Alternatively, the entire high frequency portion 316 may be regarded as all for calculation of the spectral flatness measure.

Summarizing the above, it can be mentioned that a reduction in spectral flatness (caused by the application of time warping) can be considered as the first measure for the efficiency of time warping.

For example, the temporal warp enable signal provider 100, 230, 234 (or its comparator 130, 234o) may use a spectral flatness measure of a time warp transformed spectral representation 234e using standard time warp contour information With the spectral flatness measure of the time warp transformed spectral representation 234k and determine based on the comparison whether the time warp active signal should be activated or deactivated. For example, if the time warping leads to a sufficient reduction in the spectral flatness measure as compared to the case without time warping, the time warping is activated by an appropriate setting of the time warping active signal.

In addition to the methods described above, higher frequency portions of the spectrum may be emphasized (e.g., by appropriate scaling) relative to the lower frequency portion for calculation of the spectral flatness measure. Figure 3c shows a graphical representation of a time warp transformed spectrum in which the high frequency portions of the spectrum are emphasized relative to the low frequency portion for calculation of the spectral flatness measure. Accordingly, an insufficient representation of high portions is compensated. Thus, as shown in Figure 3E, the flatness measure can be calculated on a fully scaled spectrum where high frequency bins are emphasized compared to low frequency bins.

In the bit-saving dimension, a typical measure of coding efficiency can be perceptual entropy, which is described in 3GPP TS 26.403 V7.0.0 (Technical Specification Group Services and System Aspects, aacPlus general audio codec can be very well correlated with the actual number of bits needed to encode a specific spectrum as described in Encoder specification AAC part: Section 5.6.1.1.3 Relation between bit demand and perceptual entropy . ≪ / RTI > As a result, the reduction of perceptual entropy will be another measure of the effectiveness of time warping.

Figure 3f shows an energy compaction information provider 325 that may replace the energy compaction information provider 120, 234f, 234l and may be used in the time warp activation signal provider 100, 290, 234. The energy compaction information provider 325, for example, warped transformed spectral representations 234e and 234k, also indicated as _tw, as shown in FIG. The energy compaction information provider 325 is also configured to provide perceptual entropy information 326 that may replace the energy compaction information 122, 234m, 234n.

The energy compaction information provider 325 includes a format factor calculator 327 configured to receive the time warp transformed spectral representations 234e and 234k and to provide format factor information 328 that may be related to the frequency band based thereon, . The energy compaction information provider 325 also includes a frequency band energy calculator 329 configured to compute frequency band energy information en (n) 330 based on the time warp transformed spectral representations 234e and 234k. The energy compaction information provider 325 also includes a line number estimator 331 configured to provide predicted line number information nl 332 for a frequency band having index n. The energy compaction information provider 325 includes a perceptual entropy operator 333 configured to calculate perceptual entropy information 326 based on the frequency band energy information 330 and the predicted line number information 332 . For example, the format parameter calculator 327 calculates,

(1)

(One)

, ≪ / RTI >

In the above equation, ffac (n) represents a format factor for a frequency band having a frequency band index n. k indicates an operating variable operating on the spectral bin indices of the scale factor band (or frequency band) n. X (k) indicates a spectral value (e.g., an energy value or a magnitude value) of a spectral bin (frequency bin) having a spectral bin index (or frequency bin index) k.

The line number estimator is configured to estimate the number of nonzero lines represented by nl according to the following equation:

(2)

(2)

In the above equation, en (n) represents the energy in the frequency band or the scale factor band having the index n. kOffset (n + 1) - kOffset (n) indicates the width of the frequency band or the scale factor band of index n in terms of the frequency bin.

In addition, the perceptual entropy operator 332 may be configured to calculate the perceptual entropy information sfbPe according to the following equation.

(3)

(3)

In the above, the following relation holds.

(4)

(4)

The overall perceptual entropy pe can be computed as the sum of the perceptual entropy of multiple frequency bands or scale factor bands.

As noted above, perceptual entropy information 326 may be used as energy compaction information.

For further details concerning the calculation of perceptual entropy, reference is made to section 5.6.1.1.3 of the international standard 3GPP TS 26.403 V7.0.0 (2006-06).

Hereinafter, the concept of calculation of the energy collapse information in the time domain will be described.

Another idea of looking at the time-warped modified discrete cosine transform (TW-MDCT) is the basic idea for changing the signal in such a way that the signal has a constant or nearly constant pitch in one block. If a constant pitch is obtained, this means that the maximum value of the autocorrelation of one process block is increased. Since it is not easy to find the corresponding maximum values of autocorrelation for time warped and non-time warped cases, the sum of absolute values for normalized autocorrelation can be used as a measure for improvement. This increase in total corresponds to an increase in energy compaction.

Referring to Figures 3g, 3h, 3j, 3j and 3k, this concept will be described in more detail.

Figure 3g shows a graphical representation of the non-time-warped signal in the time domain. The horizontal axis 350 represents the time, and the vertical axis 351 represents the level a (t) of the non-time-warped time signal. Curve 352 represents the temporal evolution of the non-time-warped time signal. As shown in FIG. 3G, it is assumed that the frequency of the non-time-warped time signal exhibited by curve 352 increases over time.

Figure 3h shows a graphical representation of a time warped (non-uniformly resampled) version of the time signal of Figure 3g. Warped version a (t _w ) of the signal a (t), while the horizontal axis 355 is warped (e.g., in normalized form) and the vertical axis 356 represents the level of the time-warped version a (t _w ) of the signal a As shown in FIG. 3h, the time-warped version a (t _w ) of the non-time-warped time signal a (t) includes a frequency that is temporally constant (at least roughly) in the warped time domain.

In other words, FIG. 3h shows the fact that the time signal of a time varying frequency is converted into a time signal of a temporally constant frequency by an appropriate time warped operation, which may include time-warping re-sampling.

를 그리고 세로 축(361)은 자기상관 함수의 크기를 나타낸다. 마크들(362)은 자기상관 래그

의 함수로의 자기상관 함수

의 전개(evolution)를 나타낸다. 도 3i에서 볼 수 있는 바와 같이, 워핑되지 않은 시간 신호 a(t)의 자기상관 함수 R_uw 는

= 0 에 대한 피크(신호 a(t)의 에너지를 반영하는) 및

≠ 0 에 대한 작은 값들을 포함한다.Figure 3i shows a graphical representation of the autocorrelation function of the non-warped time signal a (t). The transverse axis 360 has an autocorrelation lag (lag)

And the vertical axis 361 represents the magnitude of the autocorrelation function. Marks 362 may be < RTI ID = 0.0 >

Autocorrelation function as a function of

And the like. As can be seen in Figure 3i, the autocorrelation function R _uw of the non-warped time signal a (t)

= 0 (reflecting the energy of the signal a (t)) and

RTI ID = 0.0 > 0 < / RTI >

= 0 에 대한 피크(신호 a(t)의 에너지를 반영하는) 및 자기상관 래그

의 다른 값들

에 대한 피크들을 또한 포함한다.

에 대한 이러한 추가적인 피크들은 시간 워핑된 시간 신호 a(t_w)의 주기성을 증가시키기 위해 시간 워프의 효과에 의해 얻어질 수 있다. 이러한 주기성은 자기상관 함수

와 비교했을 때, 자기상관 함수

의 추가적인 피크들에 의해 반영된다. 따라서, 시간 워핑된 오디오 신호의 자기상관 함수의 추가적인 피크들(또는 피크들의 증가된 강도)의 존재는 원래 오디오 신호의 자기상관 함수와 비교했을 때 시간 워프의 효율성(비트레이트 감소 측면에서의)의 지시자로서 나타낼 수 있다.3J shows a graphical representation of the autocorrelation function R _tw of the time warped time signal a (t _w ). As shown in Fig. 3j, the autocorrelation function R _tw The

= 0 (reflecting the energy of the signal a (t)) and the autocorrelation lag

Other values of

Lt; / RTI >

These additional peaks for the time warped time signal a (t _w ) can be obtained by the effect of the time warp to increase the periodicity of the time warped time signal a (t _w ). This periodicity is called an autocorrelation function

, The autocorrelation function

≪ / RTI > Thus, the presence of additional peaks (or increased intensities of peaks) of the autocorrelation function of a time-warped audio signal results in an improvement in the efficiency of the time warp (in terms of bitrate reduction) compared to the autocorrelation function of the original audio signal. As an indicator.

의 기 설정된 범위 상에서 시간 워핑된 신호 a(t_w)의 자기상관 함수

를 계산하도록 구성된 자기상관 연산기(371)를 포함한다. 에너지 다짐 정보 제공기(370)는 또한 자기상관 함수

(예를 들어, 이산 값들

의 기 설정된 범위 상에서)의 복수의 값들을 합산하고 얻어진 합계를 에너지 다짐 정보(122, 234m, 234n)로서 제공하도록 구성된 자기상관 합산기(372)를 포함한다. 3K illustrates a time warped time-domain representation of the audio signal, e.g., time warped signals 234e and 234k (spectral region transforms 234d and 234j and optionally analysis windows 234b and 234h) And provides energy compaction information 374 that can take on the role of energy compaction information 372 on the basis of which energy compaction information 374 is provided. The energy compaction information provider 370 of FIG.

Of the time-warped signal a (t _w ) over a predetermined range of the autocorrelation function

And an autocorrelation operator 371 configured to calculate the autocorrelation function. The energy compaction information provider 370 may also include an autocorrelation function

(E. G., Discrete values

, And an autocorrelation adder 372 configured to sum up the plurality of values of the energy summation information (in a predetermined range of the energy collision information 122, 234m, 234n).

Thus, the energy compaction information provider 370 allows the provision of reliable information indicative of the efficiency of the time warp without substantially performing spectral region transform of the time warped time domain version of the input audio signal 210 . Therefore, only when the temporal warp is found to result in substantially improved encoding efficiency, it is possible to reduce the amount of compression of the input audio signal 310 based on the energy compaction information 122, 234m, 234n provided by the energy compaction information provider 370, It is possible to perform a time warped version of the spectral region transform.

In summary, the embodiments of the present invention generate concepts for final quality checking. The resulting pitch contour is evaluated and accepted or rejected in terms of its coding gain. Several measures related to the sparsity or coding gain of the spectrum may be considered for such determinations, such as spectral flatness measure, band-wise segmental spectral flatness measure, and / or perceptual entropy.

The use of multiple spectral compaction information, for example, the use of spectral flatness measures, the use of perceptual entropy measures, and the use of time-domain autocorrelation measures have been discussed. Nevertheless, there are other measures that show the compaction of energy in a time-warped spectrum.

All these measures can be used. Preferably, for all these scales, a ratio between the scales for the unwarped and time warped spectra is defined, and a threshold for this ratio in the encoder is used to determine whether the obtained time warp contour is advantageous for encoding Respectively.

Only the third part of the pitch contour is new (here, for example, three parts of the pitch contour are related to the whole frame), or preferably this new part is only for the part of the acquired signal, , All of these measures may be used for the entire frame, with a transformation using a low overlapping window centered on the (individual) signal portion.

Of course, a single measure or combination of the aforementioned measures may be used as desired.

4A shows a flowchart of a method of providing a time warp activation signal based on an audio signal. The method 400 of FIG. 4A includes providing 410 energy compaction information indicative of energy compaction in a time-warped transformed spectral representation of the audio signal. Step 400 further includes comparing (step 420) energy compaction information to a reference value. Step 400 further includes providing (430) a time warp activation signal according to the result of the comparison.

The method 400 may be supplemented by any of the features and functions described herein in connection with the provision of the time warp activation signal.

Figure 4B shows a flowchart of a method of encoding an input audio signal to obtain an encoded representation of an input audio signal, in accordance with an embodiment of the invention. The method 450 optionally includes providing (460) a time warp-transformed spectral representation based on the input audio signal. The method 450 further includes providing (470) providing a time warp activation signal. Step 470 may include, for example, the functionality of method 400. Thus, the energy compaction information is provided so that the energy compaction information represents energy compaction in the time warped transformed spectral representation of the input audio signal. The method 450 may also include, for inclusion in an encoded representation of the input audio signal, a description or standard of a time warped transformed spectral representation of the input audio signal using the newly set time warp contour information (480) selectively providing a description of a non-time-warp-transformed spectral representation of the input audio signal using temporal warp contour information according to the temporal warp activation signal.

The method 450 may be supplemented by any of the features and functions discussed herein in connection with encoding an input audio signal.

Figure 5 illustrates a preferred embodiment of an audio encoder according to the present invention in which various aspects of the invention are implemented. An audio signal is provided to the encoder input 500. Such an audio signal will typically be a discrete audio signal derived from an analog audio signal using a sampling rate that is also referred to as the normal sampling rate. This normal sampling rate is different from the local sampling rate generated in the time warping operation and the normal sampling rate of the audio signal at the input 500 is a constant sampling rate that derives audio samples separated by a constant time portion. This signal is input to the analysis window language 502, which in this embodiment is connected to the window function controller 504. [ This analysis window word 502 is connected to a time warper 506. However, depending on the implementation, the time warper 506 may be located before the analysis window word 502 - in a single processing direction. This implementation is desirable when a time warping property is required at the analysis windowing at block 502 and the time warping operation is to be performed on time warped samples rather than unwarped samples. In particular, this is the aspect of MDCT-based time warping as described in International Patent Application PCT / EP2009 / 002118 entitled " Tiime Warped MDCT " by Bernd Edler et al. For other time warping applications, such as PCT / EP2006 / 010246, November 2005, L. Villemoes, entitled " Time Warped Transform Coding of Audio Signals ", the arrangement between time warper 506 and analysis windower 502 It can be set as desired. In addition, a time / frequency converter 508 is provided for performing time / frequency conversion to the spectral representation of the time warped audio signal. The spectral representation may be input to a temporal noise shaping (TNS) stage 510 that provides TNS information as output 510a and spectrum residual values as output 510b. Output 510b is controlled by a perceptual model 514 that quantizes the signal and is coupled to a quantizer and coder block 512 where the quantization noise can be hidden below the perceptual masking threshold of the audio signal.

In addition, the encoder shown in FIG. 5A includes a time warp analyzer 516, which may be implemented as a pitch tracker that provides time warping information at output 518. The signal on line 518 may include information regarding whether the signal analyzed by the time warping characteristic, pitch characteristic, pitch contour or time warping analyzer is a harmonic or non-harmonic signal. The time warp analyzer may also implement a function to distinguish between voiced speech and unvoiced speech. Depending on the implementation, however, depending on whether or not the signal classifier 520 is implemented, a voiced / unvoiced decision can also be made by the signal classifier 520. In this case, the time warp analyzer does not necessarily have to perform the same function. The time warp analyzer output 518 includes at least one of a group of functions including a window function controller 504, a time warper 506, a TNS stage 510, a quantizer and a coder 512 and an output interface 522 And preferably one or more functions.

Similarly, the output 522 of the signal classifier 520 is coupled to a window function controller 504, a time warper 506, a TNS stage 510, a quantizer and coder 512, and a function including an output interface 522 At least one and preferably one or more functions in the group of < / RTI > Additionally, the time warp analyzer output 518 may also be coupled to the noise-filling analyzer 524.

Although FIGURE 5a illustrates the situation where the audio signal on the analysis window word input 500 is input to the time warp analyzer 516 and the signal classifier 520, And may be taken from the output of the time warper 506, the time / frequency converter 508, or the output of the TNS stage 510, to the signal classifier.

In addition to the signal output by the quantizer / coder 512, indicated at 526, the output interface 522 includes TNS side information 510a, perceptual model side information (e.g., 528, time warp indication data for more advanced forms of time warp side information, such as pitch contour on line 518, and signal classification information on line 522. [ Additionally, the noise-filling analyzer 524 also outputs noise-filling data on the output 530 to the output interface 522. Output interface 522 is configured to generate encoded audio output data on line 532 for transmission to a decoder or storage in a storage device such as a memory device. Depending on the implementation, the output data 532 may include all of the inputs to the output to the output interface 522, or because a corresponding decoder with reduced functionality does not need that information, Lt; / RTI > may contain less information because the information in the decoder is already valid due to transmission over the < RTI ID = 0.0 >

The encoder shown in FIG. 5A is implemented by a window function controller 504, a noise-filling analyzer 524, a quantizer encoder 512 and a TNS stage 510, which have improved functions as compared to the MPEG-4 standard. May be implemented as defined in detail in the MPEG-4 standard separately from the additional functions shown in the inventive encoder of FIG. Additional descriptions are provided in the AAC standard (International Standard 13818-7) or in the 3GPP TS 26.403 V7.0.0 (technical specification group services and system aspects; enhanced AAC plus general audio codec) .

5b, which illustrates one preferred embodiment of an audio decoder for decoding an encoded audio signal received via input 540, is discussed. An input interface 540 is operative to process the encoded audio signal such that various information items of information are extracted from the signal on line 540. This information includes signal classification information 541, time warp information 542, noise filling data 543, scale factors 544, TNS data 545 and encoded spectral information 546. The encoded spectral information is input to an entropy decoder 547 that includes a Huffman decoder or an arithmetic decoder if the encoder function in block 512 of Figure 5a is provided as a corresponding encoder, such as a Huffman encoder or an arithmetic encoder . The decoded spectral information is input to a re-quantizer 550 connected to a noise filter 552. The output of the noise filler 552 is additionally input to an inverse TNS stage 554 that receives TNS data on line 545. Depending on the implementation, the noise filler 552 and the TNS stage 554 are applied in a different order so that the noise filler 552 can operate on the TNS stage 554 output data rather than being applied to the TNS input data. In addition, a frequency / time converter 556 is provided that provides it to the time reversal warper 558. At the output of the signal processing chain, the synthesis window that performs the overlap / summation process is preferably applied as indicated by 560. [ The order of the time reversal warper 558 and synthesis stage 560 is changeable, but in a preferred embodiment, implementing an MDCT-based encoding / decoding algorithm as defined in the AAC standard (AAC = advanced audio coding) desirable. And, the inherent cross-fade operation from one block to the next block due to the overlap / summation procedure is used as the last operation in the processing chain, effectively preventing all block artifacts.

Additionally, as is the case, a noise filling analyzer (not shown) configured to control the noise filler 552 and receiving as input the time warping information 542 and / or the signal classification information 541 and the information on the re-quantized spectrum (562).

Preferably, all of the functions described below apply together in an improved audio encoder / decoder scheme. Nevertheless, the functions described hereinafter may be applied independently of each other, i.e. only one or a group may be implemented in a particular encoder / decoder scheme, rather than all functions.

Next, the noise filling aspects of the present invention will be described in detail.

In one embodiment, the side information provided by the time warping / pitch contour means in FIG. 5A may be provided by other codec means and, in particular, by a noise filling analyzer 524 at the encoder end and / And is usefully used to control the noise filling means implemented by the noise filler analyzer 562 and the noise filler 552 at the stage.

Various encoder means within the AAC framework, such as noise filling means, are controlled by information collected by pitch contour analysis and / or additional knowledge of the signal classification provided by the signal classifier 520.

The identified pitch contour will indicate signal segments having a clear harmonic structure and thus noise filling between harmonic lines will reduce the perceived quality, especially the quality perceived for the speech signal, and therefore, if a pitch contour is found, . Otherwise, there will be noise between the sub-tones, which has the same effect as the increased quantization noise for the smear spectrum. In addition, the amount of noise level reduction can also be further improved using signal classifier information, so there will be no noise peeling for speech signals and normal noise filling will be applied to generic signals having a strong harmonic structure.

Generally, the noise filler 552 is used to insert spectral lines into the decoded spectrum, where zeros are transmitted from the encoder to the decoder, i.e., when the quantizer 512 of FIG. 5A quantizes the spectral lines to zero useful. Of course, quantizing the spectral lines to zero reduces the bit rate of the transmitted signal, and if these spectral lines are below the perceptual masking threshold determined by the perceptual model 514, ) Removal of spectral lines is inaudible. Nonetheless, it has been found that these spectral holes, including a number of adjacent spectral lines, produce a significantly unnatural sound. Therefore, a noise filling means is provided for inserting spectral lines into positions where the lines are quantized to zero by encoder-side quantization. These spectral lines may have random amplitude or phase, and such decoder-side synthesized spectral lines may be generated using a noise-filling measure determined at the encoder-side as shown in FIG. 5A, Is scaled by the selection block 562 according to the scale determined on the - side. The noise-filling analyzer 524 of FIG. 5A is therefore configured to estimate the noise-filling measure of the energy of audio values quantized to zero for the time frame of the audio signal.

In one embodiment of the present invention, an audio encoder for encoding an audio signal on line 500 includes a quantizer 512 configured to quantize audio values, wherein the quantizer 512 also includes an audio < RTI ID = 0.0 > And to quantize the values to zero. This quantization threshold may be the first step of the step-based quantizer, which determines whether an audio value is quantized to 0, i. E. To quantization index 0, or 1, i. E. Audio value is quantized index 1 Lt; / RTI > may be used to determine whether or not to be quantized. Although the quantizer of FIG. 5A is shown to perform quantization of frequency domain values, the quantizer may also be used to quantize time domain values in yet another implementation where noise filling is performed in a time domain rather than a frequency domain.

The noise-filling analyzer 524 is implemented by a quantizer 512 as a noise-filling operator that estimates a noise-filling measure of the energy of audio values quantized to zero for a time frame of the audio signal. Additionally, the audio encoder includes the audio signal analyzer 600 shown in FIG. 6A configured to analyze whether the time frame of audio data has harmonic or speech characteristics. The audio signal analyzer 600 may include, for example, block 516 of FIG. 5A or block 520 of FIG. 5A and may include any other device that analyzes whether the signal is a harmonic or a speech signal . Since the temporal warp analyzer 516 is always configured to find the pitch contour and the presence of the pitch contour indicates the harmonic composition of the signal, the signal analyzer 600 of FIG. 6A may be used to determine the time warping contour of the pitch tracker or time warp analyzer, Lt; / RTI >

The audio encoder additionally includes a noise filling level manipulator 602 as shown in FIG. 6A, which outputs a manipulated noise filling measure / level to the output interface 522 indicated by 530 in FIG. 5A. The noise filling scale operator 602 is configured to manipulate a noise filling measure according to the harmonic or speech characteristics of the audio signal. The audio encoder further includes an output interface 522 for generating an encoded signal comprising the manipulated noise filling measure 530 output by block 602 on line 530 for transmission or storage. This value corresponds to the value output by block 562 on the decoder-side shown in Fig. 5B.

As indicated in FIGS. 5A and 5B, the noise fill level manipulation may be implemented in an encoder, implemented in a decoder, or implemented together in both devices. In a decoder-side implementation, a decoder that decodes the encoded audio signal is used to decode the encoded audio data 546 and the encoded audio data 546 on the noise-filling measure 543, i. E., On the line 540 to obtain noise- And an input interface 539 for processing signals. The decoder further includes a decoder 547 and a re-quantizer 550 for generating re-quantized data.

In addition, the decoder includes a signal analyzer 600 (FIG. 6A) that may be implemented in the noise-filling analyzer 562 of FIG. 5B to reproduce information about the time frame of audio data having harmonic or speech characteristics.

In addition, a noise filler 552 is provided for generating noise-filling audio data, and the noise filler 552 is provided by a noise-filling measure transmitted by the encoded signal and generated by the input interface on line 543, Side item 562 via processing and interpretation of the time warp information 542, which is defined by the signal analyzer 516 and / or 550 of the time-warping process 516 and / To generate noise filling data in response to a harmonic or speech characteristic of the audio data as defined by the audio data.

Additionally, the decoder includes a processor for processing re-quantized data and noise-filling audio data to obtain a decoded audio signal. The processor may include the items 554, 556, 558, 560 of Figure 5B, which may be the case in this case. Additionally, in accordance with certain implementations of the encoder / decoder algorithm, the processor may include other processing blocks provided in a time domain encoder, e.g., an AMR WB + encoder or other speech coders.

Thus, the noise filling operation of the present invention can be achieved by simply calculating the direct noise measure, manipulating this noise measure based on the harmonic / speech information, and then transmitting an already-correctly-operated noise fill measure that can be applied by the decoder in a direct manner , Decoder side. The non-manipulated noise filling measure may also be transmitted from the encoder to the decoder, and then the decoder will analyze whether the actual time frame of the audio signal is time warped, i. E. Have a harmonic or speech characteristic, A substantial operation will occur at the decoder side.

Thereafter, Fig. 6B will be discussed to illustrate the preferred embodiments for manipulating the noise level estimation.

In the first embodiment, a normal noise level is applied if the signal does not have a harmonic or speech characteristic. This is the case where no time warp is applied. If a signal classifier is additionally provided, a signal classifier that distinguishes speech from non-speech will indicate that the time warp is inactive, i.e., non-speech for the situation where no pitch contour is found.

However, if a time warp is active, that is, a pitch contour representing a harmonic component is found, the noise fill level will be manipulated lower than normal. If an additional signal classifier is provided, this signal classifier represents speech, and at the same time a lower or zero noise fill level is signaled if the time warp information indicates a pitch contour. Thus, the noise filling level manipulator 602 of FIG. 6A will reduce the manipulated noise level to zero or at least to a value lower than the low value shown in FIG. 6B. Preferably, the signal classifier further has a voiced / unvoiced detector as shown in the left side of FIG. 6B. For voiced speech, a very low or zero noise filling level is signaled / applied. However, in the case of unvoiced speech, in which the time warp indication does not direct time warp processing due to the fact that no pitch is found, and the signal classifier signals the speech component, the noise fill measure is not manipulated and the normal noise filling level Is applied.

Preferably, the audio signal analyzer includes a pitch tracker that generates an indication of a pitch, such as the absolute pitch or pitch contour of the time frame of the audio signal.

The manipulator is then configured to reduce the noise filling measure if a pitch is found and not to reduce the noise fill measure if no pitch is found.

As shown in FIG. 6A, when applied to the decoder side, the signal analyzer 600 does not perform any substantial signal analysis, such as a pitch tracker or a voiced / unvoiced sound detector, and the signal analyzer is used to extract time warp information or signal classification information And parses the encoded audio signal. Therefore, the signal analyzer 600 may be implemented within the input interface 539 of the decoder of Figure 5b.

Additional embodiments of the present invention will now be discussed with reference to Figures 7a through 7e.

For the onset of speech where the speech portion of the voiced sound starts after a relatively quiet signal portion, the block switching algorithm may classify it as an attack and for this particular frame, loss of coding gain on the signal segment with a clear harmonic structure May be selected. Therefore, the voiced / unvoiced classification of the pitch tracker is used to detect the voices of the voiced sound and prevents the block switching algorithm from exhibiting a transient attack near the discovered onset. This characteristic can also be combined with the signal classifier to prevent block switching on the speech signal and to allow it for all other signals. Further, more sophisticated control of block switching can be implemented not only by allowing or disallowing detection of an attack, but also by using a varying threshold for attack detection based on onset of voiced sounds and signal classification information. This information can also be used to detect attacks such as the aforementioned voiced sounds' onsets. Instead of switching to short blocks, a long window with short overlapping can be used, pre) and post echoes can occur. FIG. 7D shows typical operation without adjustment, and FIG. 7E shows two different possibilities of adjustment (inhibition and low overlapping windows).

An audio encoder in accordance with one embodiment of the present invention operates to generate an audio signal, such as a signal output, output by the output interface 522 from FIG. 5A. The audio encoder includes an audio signal analyzer such as the time warp analyzer 516 and the signal classifier 520 of Figure 5A. Generally, an audio signal analyzer analyzes whether a time frame of an audio signal has a harmonic or speech characteristic. To this end, the signal analyzer 520 of FIG. 5A may include a voiced / unvoiced detector 520a or a speech / non-speech detector 520b. Although not shown in FIG. 7A, a time warp analyzer, such as the time warp analyzer 516 of FIG. 5A, which may include a pitch tracker, may be provided instead of or in addition to items 520a and 520b . Additionally, the audio encoder includes a window function controller 504 that selects a window function according to the harmonic or speech characteristics of the audio signal as determined by the audio signal analyzer. Windower 502 then window-wraps the audio signal, or, in accordance with certain implementations, uses the selected window function to obtain the windowed frame. This window frame is then further processed by the processor to obtain the encoded audio signal. The processor may include a time domain including LPC filters such as the items 508, 510, 512 shown in FIG. 5A or speech-based audio encoders or speech coders, and in particular speech coders implemented in accordance with the AMR- Lt; RTI ID = 0.0 > audio encoders. ≪ / RTI >

In a preferred embodiment, the window function controller 504 comprises an oversamper detector 700 for detecting a transient of an audio signal, wherein the window function controller determines if a transient is detected and a harmonic or speech characteristic is detected by the audio signal analyzer If not, it is configured to switch from a window function for a long block to a window function for a short block. However, when the transient is detected and a harmonic or speech characteristic is found by the audio signal analyzer, the window function controller 504 does not switch to the window function for the short block. Window function outputs showing a long window when the overflow portion is not obtained and a short window when there is an overflow portion detected by the overflow detector are shown as 701 and 702 in FIG. 7A. This normal procedure, as performed by the known AAC encoder, is shown in Figure 7d. At the location of the speech onset, transient detector 700 detects an increase in energy from one frame to the next and thus switches from a long window 710 to a short window 712. To accommodate such a switch, a first point on the time axis represented by the first overlapping portion 714a, a non-aliasing portion 714b, a second short overlapping portion 714c, and 2048 samples, A long stop window 714 having a zero portion is used. A short window sequence, denoted 712, is then performed and then terminated by a long start window 718 having a long overlap portion 718a that overlaps the next long window, not shown in Figure 7d. This window also has a zero portion that extends between the non-aliasing portion 718b, the short overlap portion 718c, and points 720 on the time axis up to 2048 points. This part is the zero part.

Typically, switching to a short window is useful to avoid pre-echoes that can occur within the onset of a voiced sound or in the frame before the transient event, which is typically the beginning of a signal with the beginning of a speech or a harmonic component. In general, if the pitch tracker determines that the signal has a pitch, the signal has a harmonic component. There are also other harmonicity measures, such as tonality measures above a certain minimum level, along with the characteristic that the prominent peaks are harmonic to each other. There are a number of additional technical factors that determine whether the signal is harmonic or not.

The disadvantage of short windows is that the frequency resolution decreases because of the increased temporal resolution. For high quality encoding of speech and especially speech portions of voiced speech or portions having strong harmonic components, good frequency resolution is required. Therefore, the audio signal analyzer shown in 516, 520, or 520a, 520b operates to output an inactivation signal to the oversampling detector 700 so that when a voiced speech segment or a signal segment having strong harmonic characteristics is detected, Switching to a short window is prevented. This ensures that high frequency resolution is maintained to code these signal portions. This is a tradeoff between a high-resolution encoding of the pitch for a pre-echo, on the other hand, a pitch for a harmonic non-speech signal or of a pitch for a speech signal. It has been found that it is much more complicated when the harmonic spectra are not encoded correctly compared to some pre-echoes which are likely to occur. In order to further reduce the pre-echos, TNS processing is preferred for the situation to be discussed with respect to Figures 8A and 8B.

In the alternative embodiment shown in FIG. 7B, the audio signal analyzer includes voiced / unvoiced and / or speech / non-speech detectors 520a and 520b. However, the transient detector 700 included in the window function controller is not fully activated / deactivated as in FIG. 7A, and the thresholds included in the transient detector are controlled using the threshold control signal 704. In this embodiment, transient detector 700 is configured to determine a quantitative characteristic of the audio signal and to compare the quantitative characteristic with a controllable threshold, wherein when the quantitative characteristic has a predetermined relevance to the controllable threshold, do. The quantitative characteristic may be a number indicating an increase in energy from one block to the next block, and the threshold may be a specific threshold energy increase. If the energy increase from one block to the next block is higher than the threshold energy increase, an overflow is detected, in which case the predetermined relationship is a "larger" relationship. In other embodiments, for example, when the quantitative characteristic is an inverted energy increase, the predetermined relationship may also be a "lower" relationship. In the embodiment of FIG. 7B, the controllable threshold is controlled such that if the audio signal analyzer finds a harmonic or speech characteristic, the probability of switching to a window function for a short block is reduced. In an energy-increasing embodiment, the threshold control signal 704 will cause an increase in the threshold and a switch to a short block will occur only if the energy increase from one block to the next is particularly high.

In another embodiment, the output signal from the voiced / unvoiced detector 520a or the speech / non-speech detector 520b may also be used for switching from a speech onset to a short block instead of a window function for a short block, May be used to control the window function controller 504 in a manner in which the window function controller 504 is performed. This window function guarantees a higher frequency resolution than the short window function, but has a shorter length than the long window function, so a good compromise between one pre-echo and a sufficient frequency resolution of the other is achieved. In another alternative embodiment, a switch to a long window function with a smaller overlap may be performed as indicated by the hatched lines in 706 in Figures 7E. Window function 706 has a length of 2048 samples as a long block but this window has a zero portion 708 and a nonalloying portion 710 and has a length from the window 706 to the corresponding window 707 A short overlap length 712 is obtained. The window function 707 again has a non-aileronizing portion for the zero portion left of the region 712 and the right of the region 712, similar to the window function 710. [ This low-overlapping embodiment has a shorter time length to reduce the pre-echoes due to the zero (0) portion of the windows 706 and 707, but on the other hand the overlap portion 714 and the non- Gt; 710 < / RTI > effectively, resulting in a sufficiently satisfactory frequency resolution.

Maintaining a particular overlay in a preferred MDCT implementation as implemented by an AAC encoder provides the additional advantage that overlay / summation processing can be performed, which means that some kind of cross-fading between blocks on the decoder side is performed. This effectively avoids blocking artifacts. In addition, this superposition / summation feature provides cross-fading characteristics without an increase in bit rate, in other words, a critically sampled cross-fade is obtained. In typical long windows or short windows, the overlapping portion is 50% overlap as indicated by overlap portion 714. [ In an embodiment where the window function is 2048 samples long, the overlap portion is 50%, or 1024 samples. The window function with short overlap, which will be used to effectively window on the speech-onset or harmonic signal, is preferably less than 50%, and in the embodiment of FIG. 7e, only 128 samples are 1/16 of the total window length . Preferably, overlapping portions between 1/4 and 1/32 of the total window function length are used.

FIG. 7C illustrates an exemplary voiced / unvoiced detector 520a having a window function controller 504 for selecting a window shape having a short overlap as indicated by 749 or a window shape having a long overlap as indicated by 750 ≪ / RTI > this embodiment of controlling an included window shape selector. When voiced / unvoiced detector 520a issues a detected signal of voiced sound to 751, one of the two shapes is selected and the audio signal used for the analysis is the audio signal at input 500 of FIG. Processed audio signal such as a time-warped audio signal or an audio signal that has been processed by some other pre-processing function. Preferably, when an overflow detector included in the window function controller detects the overflow and commands to switch from a long window function to a short window function as discussed in connection with FIG. 7A, the window function controller 504 The window shape selector 504 of FIG. 7C included in the window only uses the signal 751 only.

Preferably, the window function switching embodiment is combined with the temporal noise shaping embodiment discussed with respect to Figures 8A and 8B. However, a temporal noise shaping (TNS) embodiment may also be implemented without a block switching embodiment.

The spectral energy compaction property of time warped MDCT also affects temporal noise shaping (TNS) means, since the TNS gain shows a tendency to decrease for time warped frames especially for certain speech signals. For example, if block switching is not desired, although it may be desirable to activate the TNS to reduce pre-echos for onsets or offsets of voiced sounds (see block switching application), the temporal envelope of the speech signal It still shows rapid change. Typically, the encoder uses some measure to see if the application of the TNS is useful for a particular frame, for example the prediction gain of the TNS filter when applied to the spectrum. Thus, a lower effective TNS gain threshold is desirable for segments with active pitch contours, which ensures that the TNS is more active for these critical signal portions, such as the voiced onsets. As with other measures, this can also be supplemented by considering signal classification.

An audio encoder according to this embodiment for generating an audio signal includes a controllable time warper such as a controllable time warper 506 that warps the audio signal to obtain a time warped audio signal. In addition, a time / frequency converter 508 is provided for converting at least a portion of the time warped audio signal into a spectral representation. Preferably, the time / frequency converter 508 implements an MDCT transform as known from the AAC encoder, but the time / frequency transformer can also perform any kind of transform, such as a DCT, DST, DFT, FFT or MDST transform And may include a filter bank such as a QMF filter bank.

Additionally, the encoder includes a temporal noise shaping stage 510 that performs predictive filtering on the frequency of the spectral representation in accordance with the temporal noise shaping control indication, and predictive filtering is not performed if there is no temporal noise shaping control indication .

In addition, the encoder includes a temporal noise shaping controller that generates a temporal noise shaping control indication based on the spectral representation.

In particular, the temporal noise shaping controller may increase the likelihood of performing predictive filtering on the frequency if the spectral representation is based on a time-warped audio signal, or if the spectral representation is not based on a time warped audio signal , And to reduce the likelihood of performing predictive filtering on the frequency. The details of the temporal noise shaping controller are discussed with respect to FIG.

The audio encoder further includes a processor for additionally processing the result of the prediction filtering on the frequency to obtain the encoded audio signal. In one embodiment, the processor includes the quantizer encoder stage 512 of FIG. 5A.

The TNS stage 510 shown in FIG. 5A is described in detail in FIG. The temporal noise shaping controller preferably included in stage 510 includes a TNS gain calculator 800 that is successively connected to TNS determinator 802 and critical control signal generator 804. The threshold control signal generator 804 outputs the threshold control signal 806 to the TNS determiner in accordance with the signal from the time warp analyzer 516 or the signal classifier 520 or both. The TNS determiner 802 has a controllable threshold that is increased or decreased in accordance with the threshold control signal 806. [ The threshold of the TNS determiner 802 is, in this embodiment, the TNS gain threshold. If the substantially calculated TNS gain output by block 800 exceeds the threshold, then the TNS control indication requires TNS processing as an output, whereas in other cases where the TNS gain is below the TNS gain threshold, the TNS indication is not output Or a signal indicating that TNS processing is not useful and will not be performed in this particular time frame is output.

The TNS gain calculator 800 receives the spectral representation derived from the time warped signal as an input. Typically, the time warped signal will have a lower TNS gain, but on the other hand the TNS processing due to the temporal noise shaping characteristics in the time domain may be performed in a specific situation, such as a voiced / harmonic signal that has been processed by a time warping operation . TNS processing, on the other hand, is not useful in situations where the TNS gain is low, which means that the TNS residual signal in line 510b has the same or higher energy as the signal before TNS stage 510. [ In a situation where the energy of the TNS residual signal on line 510d is slightly lower than the energy prior to TNS stage 510, TNS processing may also not be beneficial because it is efficiently used by quantizer / entropy encoder stage 512 Since the bit reduction due to the slightly smaller energy in the received signal is less than the bit increase introduced by the necessary transmission of the TNS side information indicated by 510a in Figure 5a. Although an embodiment would automatically switch on TNS processing for all frames, where the time warped signal is the input indicated by the pitch information from block 516 or the signal classifier information from block 520, One embodiment also maintains the possibility of deactivating TNS processing only when the gain is really low or at least lower than normal, not when the harmonic / speech signal is not being processed.

8B illustrates an embodiment in which three different threshold settings are implemented by the threshold control signal generator 804 / TNS determinator 802. [ If no pitch contour is present, and the signal classifier does not show speech or speech or speech at all, the TNS decision threshold is set to a steady state requiring a relatively high TNS gain activating the TNS. However, if a pitch contour is detected but the signal classifier does not show any speech, or if the voiced / unvoiced detector detects speech of unvoiced tones, the TNS decision threshold is set to a lower level, which results in a relatively low TNS gain, If not calculated by the block, it means that TNS processing is nevertheless activated.

In situations where an active pitch contour is detected and voiced speech is found, the TNS decision threshold is set to the same lower value or even set to a lower state so that even a small TNS gain is sufficient to activate TNS processing.

In one embodiment, when the audio signal is subject to predictive filtering on frequency, the TNS gain controller 800 is configured to predict the bit rate or quality gain. TNS determiner 802 may determine that this predetermined relationship can be a "larger" relationship when the estimated gain is in a predetermined relationship to the decision threshold, but for example, "≪ / RTI > the estimated gain is compared to the decision threshold. As discussed, the temporal noise shaping controller is also preferably configured to change the decision threshold using the threshold control signal 806 such that, for the same estimated gain, the spectral representation is based on the time warped audio signal, If not activated, predictive filtering is activated and is not activated if the spectral representation is not based on a time warped audio signal.

Generally speaking, voiced speech will show a pitch contour, and unvoiced speech such as fricatives or sibilants will not show a pitch contour. However, even though the speech detector does not detect speech, there is a non-speech signal having a strong harmonic component and thus a pitch contour. Additionally, there are music signals superior to music, superior to speech, determined to have a harmonic component by the audio signal analyzer (e. G., 516 in FIG. 5A) but not detected by the signal classifier 520 as a speech signal. do. In some cases, all processing operations may be applied to voiced speech signals, and will also yield beneficial results.

Subsequently, a further preferred embodiment of the invention in connection with an audio encoder for encoding an audio signal is described. This audio encoder is particularly useful in terms of bandwidth expansion, and is also useful in stand-alone encoder applications where an audio encoder is configured to code a certain number of lines to obtain a specific bandwidth limiting / low-band filtering operation. In non-time-warped applications, this bandwidth limitation will yield a constant bandwidth by selecting a predetermined number of lines, since the sampling frequency of the audio signal is constant. However, in situations in which time warping processing is performed as in block 506 of FIG. 5A, an encoder that relies on a fixed number of lines is not only recognizable by trained listeners, but also to untrained listeners Will also result in varying bandwidth representing strong artifacts that are also recognizable.

The AAC coder typically codes a fixed number of lines by setting all others that are beyond the maximum line to zero. In the unwarped case this leads to a low-band effect with a constant cut-off frequency and hence a constant bandwidth of the decoded AAC signal. In the time warped case, the bandwidth varies due to the variation of the local sampling frequency, which causes audible artifacts, and the function of the local time warping contour. Artifacts can be reduced by adaptively selecting the number of lines to be coded in the core coder according to the local sampling frequency as a function of the local temporal warping contour and the obtained average sampling rate, A certain average bandwidth is obtained after warping. An additional benefit is bit saving in the encoder.

The audio encoder according to the present embodiment includes a time warper 506 that time warps an audio signal using a varying time warping characteristic. In addition, a time / frequency converter 508 is provided that converts the time warped audio signal to a spectral representation with some spectral coefficients. Also, a processor is used to process a variable number of spectral coefficients to produce an encoded audio signal, wherein such a processor, including the quantizer / coder block 512 of FIG. 5A, is based on time warping characteristics for the frame To set the number of spectral coefficients for a frame of the audio signal such that the bandwidth variation represented by the processed number of frequency coefficients per frame is reduced or eliminated.

The processor implemented by block 512 includes a controller 1000 that controls the number of lines and the result of the controller 1000 is related to the number of lines set for when the time frame is encoded without any time warping A certain variable number of lines are added or discarded at the upper end of the spectrum. Depending on the implementation, the controller 1000 may receive pitch contour information in a particular frame 1001 and / or a local average sampling frequency in a frame denoted 1002.

9A to 9E, the right figure shows a specific bandwidth situation for a particular pitch contour on a frame, and the pitch contours on the frame are shown in separate left figures for the time warps, and a time warp The latter is shown in the middle figures. This is the purpose of the time warping function after the time warping, where the pitch characteristic is as constant as possible.

The bandwidth 900 is used when the number of lines output by the time / frequency converter 508 or output by the TNS stage 510 of FIG. 5A is selected and the time warping operation is not performed, (506) is deactivated, as indicated by the hatched line (507). However, if a non-constant time warp contour is obtained and such a time warp contour results in a high pitch that includes a sampling rate increase (Figures 9 (a), (c)), the bandwidth of the spectrum is usually non- Which is reduced compared to the time warped situation. This means that the number of lines to be transmitted for this frame should be increased in order to balance this loss of bandwidth.

Alternatively, bringing the pitch to the lower constant pitch shown in FIG. 9 (b) or FIG. 9 (d) results in a reduction in the sampling rate. The reduction of the sampling rate leads to an increase in the bandwidth of the spectrum of this frame for a linear scale and this bandwidth increase should be balanced by eliminating or discarding a certain number of lines for the value of the number of lines for normal non-time- do.

Figure 9 (e) shows a special case in which the pitch contour falls to an intermediate level so that the average sampling rate in one frame becomes equal to the sampling frequency in the case where there is no time warping, instead of performing a time warping operation. Thus, even if the time warping operation is performed, the bandwidth of the signal is not affected, and the direct line number to be used for the normal case without time warping can be handled. From Fig. 9, it is clear that performing the time warping operation does not necessarily affect the bandwidth, and that the influence of the bandwidth depends on the pitch contour and how the time warp is performed within the frame. Therefore, it is preferable to use a local or average sampling rate as the control value. The determination of this local sampling rate is shown in FIG. The upper stage of FIG. 11 shows a time portion having sampling values of a certain distance. The frame is, for example, T _n in the top plot It includes seven sampling values indicated by. The bottom plot shows the result of a time warping operation where an overall increase in sampling rate occurs. This means that the time length of the time warped frame is less than the time length of the non-time warped frame. However, since the time length of the time warped frame to be introduced into the time / frequency converter is fixed, an additional portion of the time signal that does not belong to the frame indicated by Tn in the case of a sampling rate increase is indicated by lines 1100 Resulting in the phenomenon of being introduced into the time-warped frame as shown in FIG. Thus, a time warped frame is T _lin longer than time T _n Covers the time portion of the audio signal indicated by. In such a respect, the effective distance between two frequency lines or the frequency bandwidth of a single line (which is the inverse of the resolution) in the linear domain is reduced, and when it is multiplied by the reduced frequency distance result, The number N _n of lines set for N, causes a smaller bandwidth, i.e., a bandwidth reduction.

f 를 정상 경우에 대한 라인들의 개수 N_N 으로 곱함으로써, 감소된 주파수 해상도/두 인접한 주파수 계수들간의 증가된 주파수 거리로 인한 증가된 대역폭을 도출할 것이다. In another case not shown in FIG. 11 where the sampling rate reduction is performed by the time warper, the effective time length of the frame in the time warped region is smaller than the time length of the non-time warped region, The distance between the two frequency lines is increased. Now,

By multiplying f by the number of lines N _N for the steady state, we will derive the increased bandwidth due to the reduced frequency resolution / increased frequency distance between the two adjacent frequency coefficients.

11 further shows the average sampling rate f _SR Shows how this is calculated. To this end, the time distance between the two time warped samples is determined, and the inverse value is taken, which will be defined as the local sampling rate between the two time warped samples. This value can be computed between each pair of adjacent samples, and an arithmetic average value is calculated, which ultimately leads to an average local sampling rate, which is preferably used for input to the controller 1000 of FIG. 10A Is used.

Figure 10B shows a plot showing how many lines should be added or discarded according to the local sampling frequency, where the number of lines set for the non-time-warped case, N _n , for non-time- The sampling frequency f _N defines the intended bandwidth, which must be kept constant for a sequence of time-warped and non-time-warped frames or for a sequence of time warped frames.

FIG. 12B shows the dependency between the various parameters discussed in connection with FIGS. 9, 10B and 11. FIG. Basically, if the sampling rate, i.e. the average sampling rate f _SR decreases for non-time warped, the lines should be controlled and the normal sampling rate f _N for the case where the sampling rate is non-time warped Lines should be added when increasing with respect to which the bandwidth variation between frames is reduced or preferably eliminated as much as possible.

The bandwidth derived by the number of lines N _N and the number of sampling rates f _N preferably includes a cross-over frequency 1200 for an audio coder with a bandwidth extension encoder (BWE encoder), in addition to the source core audio encoder define. As is known in the art, bandwidth extension encoders only code spectra with high bit rates up to the cross-over frequency and have high bands, ie cross-over frequency and frequency f _MAX. The spectrum between is encoded using a low bit rate, which is typically even lower than 1/10 or less than the bit rate required for low bands between frequency 0 and cross-over frequency 1200. Figure 12A also shows the bandwidth BW _AAC of the direct AAC audio encoder. The lines may then be added as well as destroyed. Also, bandwidth variations for a certain number of lines in accordance with the local sampling rate f _SR are also shown. Preferably, the number of lines to be added or removed is set for the line number for the normal case so that each frame of the AAC encoded data has a maximum frequency that is as close as possible to the cross-over frequency 1200. Thus, on the one hand, spectral holes due to overhead due to bandwidth reduction or transmission of information on frequencies above the cross-over frequency in low-band encoded frames are avoided. This on the one hand increases the quality of the decoded audio signal and on the other hand reduces the bit rate.

Substantial addition of the line in relation to the set number of lines or the number of lines associated with the set number of lines may be performed before quantizing the lines, i.e. at the input of block 512, or following quantization, But may also be performed after entropy coding.

Additionally, while it is desirable to bring the bandwidth variation to the lowest level and even eliminate the bandwidth variation, in other embodiments it may be desirable to reduce the bandwidth variation by determining the number of lines in accordance with the time warping characteristic, Even increase the audio quality and reduce the required bit rate, compared to situations where a certain number of lines are applied.

While various aspects have been described in terms of devices, it should be understood that these aspects are also indicative of how the blocks or devices correspond to the features of the method steps or method steps. Similarly, aspects described in terms of method steps also represent the corresponding block or item description or the characteristics of the corresponding device.

Depending on the specific implementation requirements, embodiments of the invention may be implemented in hardware or software. An implementation may be implemented using a digital recording medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or FLASH memory, on which an electronically readable control signal is stored, Cooperate (or collaborate) with a programmable computer system. Some embodiments include a data carrier having an electronically readable control signal capable of cooperating with a programmable computer system, wherein one of the methods is performed thereon. In general, embodiments of the present invention may be implemented as a computer program product having program code that is operative to perform one of the methods when the computer program product is running on a computer. The program code may be stored on, for example, a machine readable carrier. Other embodiments include a computer program for performing one of the methods described herein, stored on a machine readable carrier. In other words, one embodiment of the method of the present invention is therefore a computer program having program code for performing one of the methods described herein when the computer program runs on a computer. Therefore, a further embodiment of the inventive method is a data carrier (or digital storage medium or computer-readable medium) having recorded thereon a computer program for performing one of the methods described herein. A further embodiment of the invention is therefore a sequence or data stream of signals representing a computer program that performs one of the methods described herein. A sequence of data streams or signals may be implemented to be communicated, for example, over a data communication connection, e.g., the Internet. Additional embodiments include processing means, e.g., a computer, or programmable logic device, adapted or configured to perform one of the methods described herein. Additional embodiments include a computer on which a computer program that performs one of the methods described herein is installed. In some embodiments, a programmable logic device may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can cooperate with the microprocessor to perform one of the methods described herein.

Claims (43)

An audio encoder for encoding an audio signal,
Time warper 506;
A time-frequency converter 508 for performing time / frequency conversion in a spectral representation of the time-warped audio signal;
A quantizer 512 for quantizing audio values, configured to quantize to zero audio values below a quantization threshold;
A noise filling calculator 524 that estimates a measure of the energy of zero quantized audio values over a time frame of the audio signal to obtain a noise filling measure;
An audio signal analyzer (516, 520) for analyzing whether the time frame of the audio signal has harmonic or speech characteristics;
A manipulator 602 for manipulating a noise filling measure in accordance with the harmonic or speech characteristics of the audio signal to obtain an engineered noise filling measure; And
An output interface 522 for generating an encoded signal comprising the manipulated noise filling measure 530 for transmission or storage,
The manipulator 602 applies a normal noise level when the audio signal has no harmonic or speech characteristics and no time warping is applied, and a pitch contour, which indicates a harmonic content, has been found and the time warping is active. When configured, the audio encoder is configured to manipulate the noise filling level to be lower than normal. The method according to claim 1,
The audio signal analyzer 516, 520 includes a pitch tracker for generating an indicator of the pitch when a pitch is found in the time frame of the audio signal,
The manipulator (602) is configured to reduce the noise filling measure when a pitch is found. The method according to claim 1,
The audio signal analyzer includes a voiced / unvoiced detector 520 that detects whether at least a portion of the time frame is voiced,
The manipulator 602 is configured to reduce the noise filling scale or to zero the noise filling scale when the portion is detected to be voiced,
The manipulator (602) is configured to not manipulate the noise filling scale when the portion is detected as unvoiced or to manipulate the noise filling scale to a lesser extent than when the portion is detected as voiced. A decoder for decoding an encoded audio signal,
An input interface 539 for processing the encoded audio signal to obtain encoded audio data 546 and noise filling measure 543;
Decoder / re-quantizers 547 and 550 for generating re-quantized data;
A signal analyzer 600 for restoring information on whether a time frame of audio data has harmonic or speech characteristics;
A noise filler 552 that generates noise filling audio data, the noise filler 552 configured to generate noise filling data in response to the harmonic or speech characteristics of the audio data and the noise filling measure. ; And
A processor 556, 558, 560 for processing the re-quantized data and noise filling audio data to obtain a decoded audio signal 564,
The encoded audio signal comprises data 542, 541 indicating whether a time frame of audio data has a harmonic or speech characteristic,
The signal analyzer 600 is configured to analyze the encoded audio signal to recover data indicating whether a time frame of audio data has a harmonic or speech characteristic,
The data is an indicator that the time portion has been processed by time warping processing,
The processor includes a time dewarper (558) for time dewarping audio signals derived from noise filling data and re-quantized data. CLAIMS 1. A method of encoding an audio signal,
Time warping 506 the audio signal;
Performing time / frequency conversion of the time-warped audio signal into a spectral representation (508);
Quantizing the audio values such that the values below the quantization threshold are quantized to zero (512);
Estimating 524 a measure of the energy of audio values quantized to zero over a time frame of the audio signal;
Analyzing (516, 520) whether the time frame of the audio signal has harmonic or speech characteristics;
Manipulating the noise filling scale according to the harmonic or speech characteristics of the audio signal to obtain an engineered noise filling scale (602), wherein the audio signal has no harmonic or speech characteristics and no time warping is to be applied. Manipulating the noise filling scale so that a normal noise level is applied so that a pitch contour representing the harmonic content is found and the noise filling level is manipulated lower than normal when time warping is active; And
Generating (522) an encoded signal comprising the manipulated noise filling measure (530) for transmission or storage. A method of decoding an encoded audio signal comprising data 542, 541 indicating whether a time frame of audio data has harmonic or speech characteristics, the method comprising:
Processing (539) the encoded audio signal to obtain encoded audio data (546) and a noise filling measure (543);
Analyzing the encoded audio signal to recover data indicating whether a time frame of audio data has harmonic or speech characteristics, the data being an indication of whether the time portion has undergone time warping processing; Analyzing;
Generating re-quantized data (547, 550);
Restoring information 600 on whether a time frame of the audio data has a harmonic or speech characteristic;
Generating (552) noise filling audio data in response to the harmonic or speech characteristics of the audio data and the noise filling measure; And
Processing (556, 558, 560) the re-quantized data and noise filling audio data to obtain a decoded audio signal (564), wherein the steps (556, 558, 560) comprise noise filling data and reconstruction. Time dewarping the audio signal derived from the quantized data. The method (556, 558, 560) comprising the step of decoding the encoded audio signal. A computer-readable recording medium storing a computer program having a program code for executing the method according to claim 5 or the method according to claim 6 when operating on a computer. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images