KR20070003594A - Method of clipping sound restoration for multi-channel audio signal - Google Patents

Abstract

An apparatus and a method for decoding an audio signal, and an apparatus and a method for encoding the audio signal are provided to effectively prevent the clipping problem occurring while downmixing a multi-channel audio signal by determining whether clipping is prevented or performed after testing the occurrence of clipping and the degree of clipping. For decoding an audio signal, a downmix audio signal is separated from a bit stream of the audio signal(602). After decoding the downmix audio signal, the occurrence of clipping and the degree of clipping are tested during a decoding process(605). A clipped portion of the downmix audio signal is restored(606). The restoring of the clipped portion is capable of being performed through soft clipping. The clipped portion is capable of being restored by using a filter.

Description

Restoration of Clipped Signals in Multichannel Audio Signals {METHOD OF CLIPPING SOUND RESTORATION FOR MULTI-CHANNEL AUDIO SIGNAL}

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram illustrating a method for a human to recognize spatial information about an audio signal in the present invention.

2 is a diagram illustrating a clipping process.

3 is a diagram of a decoding method for restoring a clipped portion generated during a decoding process of a downmix audio signal according to the present invention;

4 is a diagram illustrating a decoding method for recovering a clipped portion generated during a decoding process of a multichannel audio signal according to the present invention.

5 is a diagram illustrating an encoding method for restoring a clipped portion of a downmix process of a multichannel audio signal according to the present invention.

6 is a flowchart illustrating a decoding method for recovering a clipped portion generated during a decoding process of a downmix audio signal according to the present invention.

7 is a flowchart illustrating a decoding method for reconstructing a clipped portion generated during a decoding process of a multichannel audio signal according to the present invention.

8 is a flowchart illustrating an encoding method for restoring a clipped portion of a downmix process of a multichannel audio signal according to the present invention.

* Explanation of symbols for main parts of the drawings

101.Remote sound source 102.Direct sound wave

104.Reflected sound waves 301.Bitstream

302 Bitstream decoder 303 Downmix audio signal

304 Spatial information signal 305 Core codec decoder

307.Clipping Detector 308.Clipping Process

309.Multichannel generator 311.Multichannel audio signal

502. Downmix unit 503. Spatial information generator

504. Clipping detection unit 505. Clipping prevention unit

506.Bitstream Formatter

The present invention relates to an encoding-decoding method for spatial information of a multichannel audio signal, and more particularly to a multichannel audio signal having a clipping recovery method for a downmix audio signal or a multichannel audio signal. The present invention relates to an encoding-decoding method.

Recently, various coding techniques and methods for digital audio signals have been developed, and related products have been produced. In addition, coding methods for multi-channel audio signals have been developed using a psychoacoustic model, and standardization thereof has been conducted. The psychoacoustic model is an unnecessary part of the coding process by using a method of recognizing a sound, for example, a small sound following a loud sound, and only a sound corresponding to a frequency of 20 Hz to 20000 Hz. By eliminating the audio signal for, the amount of data needed can be effectively reduced.

Currently, audio standard technologies such as MPEG-1 Audio (MEPG-1 Layer III), MPEG-4 Advanced Audio Coding (AAC), and MPEG-4 High-Efficiency AAC (HE-AAC) have been developed and commercialized. In addition, a method of coding a multichannel audio signal using spatial information has been developed. The multi-channel audio signal coding method uses a compressed audio signal (e.g., mono or stereo audio signal) and a low bit-rate side information (e.g., spatial information) channel. Therefore, the transmission efficiency of the multichannel audio signal is greatly improved.

However, in configuring the bitstream of the multichannel audio signal, a conventional clipping problem occurs when downmixing the multichannel to a mono or stereo audio signal. In particular, since the coded signal should be limited in size to 16 bits or the like, the coded signal continues clipping even after core codec encoding. In addition, a clipping problem occurs in the decoding process of the downmix audio signal or the multichannel audio signal. The clipping also affects the output of the audio signal, and has been a cause of sound quality degradation.

Accordingly, the present invention proposed to solve the above problems, in decoding the multi-channel audio signal, to restore the clipped portion generated during the decoding process of the downmix audio signal, or clipping generated during the decoding process of the multi-channel audio signal It is an object of the present invention to provide a method and apparatus for solving a clipping problem occurring in a multichannel audio signal by reconstructing a portion of an image. Another object of the present invention is to provide a method and apparatus for recovering a clipped portion generated during a downmixing process of a multichannel audio signal.

In order to achieve the above object, the present invention comprises the steps of: separating the downmix audio signal from the bitstream of the audio signal; And after the decoding of the downmix audio signal, checking whether clipping occurs and the degree of clipping during the decoding process.

In addition, to achieve the above object, the present invention comprises the steps of: separating the downmix audio signal and the spatial information signal from the bitstream; Converting the downmix audio signal into a multichannel signal using the spatial information signal; And a step of checking whether clipping occurs and the degree of clipping in the conversion process.

In addition, to achieve the above object, the present invention comprises the steps of downmixing the audio signal to generate a downmix audio signal; And determining whether to prevent clipping or to take the clipping of the downmix audio signal generated during the downmixing process.

In addition, in order to achieve the above object, the present invention is a bitstream decoder for separating the downmix audio signal from the bitstream; A clipping detector to check whether clipping has occurred in the decoding process after decoding the downmix audio signal; And a clipping processor for restoring a clipped portion of the downmix audio signal.

In addition, in order to achieve the above object, the present invention is a bitstream decoder for separating the downmix audio signal and spatial information signal from the bitstream; A multichannel generator converting the downmix audio signal into a multichannel audio signal using the spatial information signal; A clipping detector to check whether clipping has occurred in the conversion process; And a clipping processing unit for restoring a clipped portion of the multichannel audio signal.

In addition, to achieve the above object, the present invention provides a downmix unit for downmixing an audio signal to generate a downmix audio signal; A clipping detector to determine a clipping frequency and a clipping degree of the downmix audio signal; And an anti-clipping decision unit to determine whether to prevent clipping of the downmix audio signal or to take the clipping.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described.

1 shows a method for a human to recognize spatial information about an audio signal in the present invention. The coding method for a multichannel audio signal is based on the fact that a human perceives the audio signal as a three-dimensional space. I use it. Such parameters, called "spatial parameters" for indicating spatial information of a multichannel audio signal, include ICLD (Inter Channel level differences), ICC (Inter Channel Coherences), ICTD (Inter Channel Time Difference), and the like. The ICLD means an energy difference between two channels, the ICC means a correlation between two channels, and the ICTD means a time difference between two channels.

How a human perceives an audio signal spatially and how the concept of the spatial parameter is generated is shown in FIG. 1. Direct sound wave 103 from the remote source 105 arrives at the human left ear 107, and another direct sound wave 102 is diffracted around the head to the right ear 106. Will be reached. The two sound waves 102 and 103 show a difference in arrival time and energy level, and this difference generates the CLD, CPC and CTD parameters.

Also, if the reflected sound waves 104 and 105 reach both ears, or if the sound source 105 is dispersed, sound waves that do not correlate with each other will reach both ears, which will generate the ICC parameter. Spatial parameters generated on the principle as described above are known to enable a strong number of bits in transmitting a multichannel audio signal as a mono or stereo signal and then outputting the multichannel audio signal back to the multichannel. According to an embodiment of the present invention, a clipping phenomenon occurs during downmixing and coding of multichannels, decoding downmixed audio signals, or decoding multichannel audio signals in a multichannel audio signal using the spatial information. It suggests ways to prevent this.

2 shows a clipping process. Clipping occurs mainly for two reasons. The first occurs when the sound level of the original signal is high. The second occurs when the number of input channels is large during the downmix process. For example, clipping occurs more often when downmixing seven channels to one channel than when three channels are downmixed. The clipping generation process of FIG. 2 illustrates a case of downmixing five channels into one channel, but the present invention is not limited thereto.

2 (a) shows the volume of the original signal consisting of five channels. Each channel can use almost the entire range of limited size (eg 16 bits). 2B illustrates a downmix audio signal generated by downmixing the five channels. As shown, the downmix audio signal can have many clipping points. FIG. 2C illustrates an audio signal obtained by encoding / decoding the downmix audio signal using a core codec (eg, an AAC codec). Since the audio signal encoded / decoded using the core codec is also represented in a limited size (eg, 16 bits), clipping can be continued. The clipping also affects the output from the reproduction unit of the multi-channel audio signal and may cause sound quality degradation.

3 illustrates a decoding method for restoring a clipped portion generated during a decoding process of a downmix audio signal according to the present invention. As shown, the bitstream decoder 302 receives a bitstream 301 that includes a downmix audio signal 303 and a spatial information signal 304, and the downmix audio signal 303 and the spatial information signal. Separate 304. The downmix audio signal 303 includes a mono or stereo audio signal and may be a multichannel audio signal. The downmix audio signal 303 is decoded in the core codec decoder 305. The demixed downmix audio signal may be in PCM data form 306. Then, the clipping detector 307 may check whether clipping occurs and the degree of clipping influence in the decoding process of the downmix audio signal. The clipping processor 308 outputs the original signal as it is without restoring the clipped portion by comparing the inspection result with an appropriate reference, or multi-channel audio signal 311 after restoring the clipped portion in an appropriate manner. Can be printed as If the multi-channel audio signal 311 cannot be output, the mono or stereo downmix audio signal 310 may be output.

As the restoration method, first, a soft-clipping method may be used. "Softclipping" refers to a method of preventing clipping by scaling the data value to a small value without cutting the data of the clipped portion. For example, if a specific threshold is set, and the data value of the audio signal is greater than or equal to the threshold value, it is scaled to a small value by applying an appropriate soft clipping function. In this case, the scaled value must correspond between the threshold value and the maximum value at which clipping does not occur. That is, the clipped portion may be restored by applying the soft clipping function to the clipped portion of the downmix audio signal. Secondly, a method of showing a smoothing effect may be used as the restoration method. The clipped portion has a straight shape with the top cut off, and "smoothing" refers to smoothing the straight portion in a curved form. For the smoothing, a filter such as a sink function or a polynomial function may be used. Third, the time or frequency information around the clipped portion may be analyzed by the reconstruction method, and a method of resynthesizing the clipped portion may be used. For example, the clipped portion may be reconstructed by using quantized values of time envelope or frequency envelope information of the downmix audio signal over time.

4 illustrates a decoding method for reconstructing a clipped portion generated during a decoding process of a multichannel audio signal according to the present invention. As shown, the bitstream decoder 402 receives a bitstream 401 comprising a downmix audio signal 403 and a spatial information signal 404, and the downmix audio signal 403 and the spatial information signal. 404 is separated. The downmix audio signal 403 is decoded in the core codec decoder 405. The decoded downmix audio signal may be in mono or stereo PCM data form 406 and may also be in multichannel PCM data form. Thereafter, the multichannel generator 407 may convert the downmix audio signal 406 in the PCM data form into a multichannel audio signal using the spatial information obtained by decoding the spatial information signal 404. The converted multichannel audio signal may be in multichannel PCM data form 408. The clipping detector 409 may check whether clipping occurs and the influence of clipping on the multichannel audio signal in the PCM data format. The clipping processor 410 outputs the original signal as it is without performing restoration of the clipped portion by comparing the inspection result with an appropriate reference, or multi-channel audio signal 412 after restoring the clipped portion in an appropriate manner. Can be printed as If the multichannel audio signal 412 cannot be output, the mono or stereo downmix audio signal 411 may be output.

As the restoration method, i) a soft-clipping method, i) a method of exhibiting a smoothing effect, i) a filter such as a sink function or a polynomial function, or the like. Alternatively, or iii) time or frequency information around the clipped portion may be analyzed and used. In addition, the clipped portion may be restored by using a function inside the multi-channel generator 407.

As a method of using the internal function, first, after adjusting gains of QMF (Qaudrature Mirror Filter) or Hybrid Filters, the filters are applied. The QMF refers to a filter used to convert an audio signal in a time domain into a QMF domain or vice versa. The hybrid filter refers to a filter used to convert an audio signal of a QMF region into a hybrid domain or vice versa. When applying the filters to an audio signal, the gain value can be adjusted to restore clipping. Second, after adjusting the overall gain, it can be applied. For example, clipping restoration may be performed by adjusting a gain value applicable to all channels of the multichannel audio signal. Third, in the process of converting the downmix audio signal into a multi-channel, the downmix audio signal is subjected to a de-correlation process, which is a matrix (hereinafter, referred to as “first”). 1 matrix "or post-matrix (hereinafter, referred to as" second matrix ") used after the decoration or adjusting both the first matrix value and the second matrix value. Later, this can be applied. Fourth, after adjusting the decoration function used in the decoration process, it can be applied. Fifth, a time envelope tool can be applied to preserve the temporal structure of the multi-channel audio signal that is output. Adjust a function for the time envelope tool, and apply the time envelope tool. can do. The temporal envelope tool can operate in the frequency domain or can operate in the time domain. Sixth, when summation of the audio signal after the decoration and the original signal, the respective gains may be adjusted and applied.

5 illustrates an encoding method for restoring a clipped portion of a downmix process of a multichannel audio signal according to the present invention. As shown, after the multichannel audio signal 501 is input to the encoder 508, it is downmixed by the downmix unit 502 to generate a downmix audio signal. In addition, the spatial information generator 503 extracts spatial information from the multichannel audio signal to generate a spatial information signal. Then, the clipping detector 504 determines the frequency and influence of clipping (eg, deterioration in sound quality) generated during downmixing the multichannel audio signal. The anti-clipping determination unit 505 including the clipping detection unit 504 adjusts a gain with respect to the downmix audio signal in which clipping has occurred, to prevent clipping or to take clipping without changing the gain. You can judge. If it is determined that it is advantageous to prevent clipping, the gain may be adjusted to generate a downmix audio signal in which clipping does not occur. If it is determined that taking a clipping is advantageous, use a downmix audio signal in which clipping has occurred without adjusting gain. The bitstream formatter 506 then generates and transmits a bitstream 507 comprising the downmix audio signal and the spatial information signal.

6 is a flowchart illustrating a decoding method for recovering a clipped portion generated during the decoding process of a downmix audio signal according to the present invention. First, a bitstream including a downmix audio signal and a spatial information signal is received (601), and a downmix audio signal and spatial information signal are extracted (602 and 603) from the bitstream. After decoding the downmix audio signal (604), it is checked whether clipping occurs and the degree of clipping influence in the decoding process (605). The test result is compared with an appropriate criterion to either output the original signal or restore 606 the clipped portion. The downmixed audio signal is then converted into a multichannel audio signal 607 using the spatial information signal.

7 is a flowchart illustrating a decoding method for recovering a clipped portion generated during the decoding process of a multichannel audio signal according to the present invention. First, a bitstream including a downmix audio signal and a spatial information signal is received (701), and a downmix audio signal and spatial information signal are extracted (702 and 703) from the bitstream. After decoding the downmix audio signal (604), the downmix audio signal is converted into a multichannel audio signal using the spatial information signal (705). Thereafter, in operation 706, the clipping occurs and the degree of the influence of clipping in the process of converting the multi-channel audio signal. The inspection result is compared with an appropriate standard, and the original signal is output as it is or the clipped part is restored (706) and output.

8 is a flowchart illustrating an encoding method for restoring a clipped portion of a downmix process of a multichannel audio signal according to the present invention. First, the multichannel audio signal 801 is downmixed to generate a downmix audio signal (802). In addition, spatial information is extracted from the multi-channel audio signal (803), and the spatial information signal is generated (806) using the extracted spatial information. Then, after determining the clipping frequency and the degree of clipping that occur during downmixing the multichannel audio signal (804), it is determined whether to adjust the gain to prevent clipping or to take the clipping without adjusting the gain (805). do. Information about the determination result may be included in the spatial information signal. The bitstream including the downmix audio signal and the spatial information signal is then transmitted 807.

Although the present invention has been described in detail with reference to some embodiments, the above embodiments are presented for the purpose of understanding the present invention, and the scope of the present invention is not limited to the above embodiments. Those skilled in the art will understand that various modifications are possible without departing from the scope of the technical idea of the present invention, and the scope of the present invention should be interpreted by the appended claims.

As described above, in coding the multi-channel audio signal according to the present invention, the clipping portion generated in the process of decoding the downmix audio signal is restored, or the downmix audio signal is stored using a spatial information signal. By restoring the clipped portion generated during the conversion to the multichannel audio signal in an appropriate manner, the clipping problem for the multichannel audio signal can be effectively prevented.

In addition, after checking the clipping frequency and the degree of clipping generated during the downmixing of the multichannel audio signal, a step of determining whether to prevent clipping or take clipping is performed. The clipping problem can be effectively prevented.

Claims (26)

Separating the downmix audio signal from the bitstream of the audio signal; And After decoding the downmix audio signal, checking whether clipping occurs and the degree of clipping in the decoding process. The method of claim 1, The decoding method, Restoring a clipped portion of the downmix audio signal. The method of claim 2, And restoring said clipped portion by a method by softclipping. The method of claim 2, And recovering the clipped portion using a filter. The method of claim 2, Recovering the clipped portion using a method that exhibits a smoothing effect. The method of claim 2, And restoring the clipped portion using time or frequency information around the portion where the clipping occurred. The method of claim 1, The decoding method, And outputting the original audio signal without restoring the clipped portion of the downmix audio signal. Separating the downmix audio signal and the spatial information signal from the bitstream; Converting the downmix audio signal into a multichannel signal using the spatial information signal; And And checking whether clipping occurs and the degree of clipping during the conversion process. The method of claim 8, The decoding method, Restoring a clipped portion of the multichannel audio signal. The method of claim 9, And restoring said clipped portion by a method by softclipping. The method of claim 9, And recovering the clipped portion using a filter. The method of claim 9, Recovering the clipped portion using a method that exhibits a smoothing effect. The method of claim 9, And restoring the clipped portion using time or frequency information around the portion where the clipping occurred. The method of claim 9, And restoring the clipped portion using an internal function of a multichannel audio signal generator for converting the downmix audio signal into a multichannel audio signal. The method of claim 14, And adjusting the gain of the QMF filter or the hybrid filter of the internal function, and restoring the clipped portion by applying the QMF filter or the hybrid filter. The method of claim 14, And restoring the clipped portion by adjusting the overall gain of the internal function and applying it. The method of claim 14, And reconstructing the clipped portion by adjusting a value of one or more of a first matrix or a second matrix of the internal functions, and applying the same. The method of claim 14, And reconstructing the clipped portion by adjusting a decorrelation function of the internal function and applying the same. The method of claim 14, And restoring the clipped portion by adjusting a temporal envelope function of the inner function and applying it. The method of claim 14, And reconstructing the clipped portion by adjusting each gain when applying the decoded audio signal of the internal function and the original audio signal. The method of claim 14, The decoding method, And outputting the multichannel audio signal without restoring a clipped portion of the multichannel audio signal. Downmixing the audio signal to produce a downmix audio signal; And And determining whether to prevent clipping of the downmix audio signal generated during the downmixing process or to take clipping. The method of claim 22, The encoding method is And determining a clipping frequency and a clipping degree of the downmix audio signal. A bitstream decoder for separating the downmix audio signal from the bitstream; A clipping detector to check whether clipping has occurred in the decoding process after decoding the downmix audio signal; And And a clipping processor for restoring a clipped portion of the downmix audio signal. A bitstream decoder for separating the downmix audio signal and the spatial information signal from the bitstream; A multichannel generator converting the downmix audio signal into a multichannel audio signal using the spatial information signal; A clipping detector to check whether clipping has occurred in the conversion process; And And a clipping processor for restoring a clipped portion of the multi-channel audio signal. A downmix unit which downmixes the audio signal to generate a downmix audio signal; A clipping detector to determine a clipping frequency and a clipping degree of the downmix audio signal; And And an anti-clipping decision unit to determine whether to prevent clipping of the downmix audio signal or to take clipping.

Images