CN115410584A

CN115410584A - Method and apparatus for encoding multi-channel audio signal

Info

Publication number: CN115410584A
Application number: CN202110595367.2A
Authority: CN
Inventors: 王智; 王喆
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-05-28
Filing date: 2021-05-28
Publication date: 2022-11-29
Also published as: WO2022247651A1; EP4336494A1

Abstract

The application provides a method and a device for coding a multi-channel audio signal. The method for encoding a multi-channel audio signal includes: acquiring a first audio frame to be coded, wherein the first audio frame comprises at least five channel signals; acquiring the sum of correlation values of all channel pairs in a target channel pair set, wherein the target channel pair set comprises at least one channel pair, one channel pair comprises two channel signals in at least five channel signals, one channel pair has one correlation value, and the correlation value is used for representing the correlation between the two channel signals of one channel pair; when the sum of the correlation values is larger than a preset threshold value, performing energy equalization processing on at least two channel signals in at least five channel signals to obtain at least two equalized channel signals; and encoding the at least two equalized sound channel signals to obtain an encoded code stream. The method and the device can improve the coding efficiency of the audio frame.

Description

Method and apparatus for encoding multi-channel audio signal

Technical Field

The present application relates to audio processing technologies, and in particular, to a method and an apparatus for encoding a multi-channel audio signal.

Background

The codec of multi-channel audio is a technique of encoding or decoding audio including more than two channels. Common multichannel audio is 5.1 channel audio, 7.1 channel audio, 7.1.4 channel audio, 22.2 channel audio, and the like.

The MPEG Surround (MPS) standard specifies joint coding for four channels, but there is still a need for a codec that can address the various multi-channel audio signals described above.

Disclosure of Invention

The application provides a method and a device for coding a multi-channel audio signal so as to improve the coding efficiency of an audio frame.

In a first aspect, the present application provides a method for encoding a multi-channel audio signal, including: acquiring a first audio frame to be encoded, wherein the first audio frame comprises at least five channel signals; obtaining a sum of correlation values of all channel pairs in a target channel pair set, wherein the target channel pair set comprises at least one channel pair, one channel pair comprises two channel signals in the at least five channel signals, the one channel pair has a correlation value, and the correlation value is used for representing the correlation between the two channel signals of the one channel pair; when the sum of the correlation values is larger than a preset threshold value, performing energy equalization processing on at least two channel signals in the at least five channel signals to obtain at least two equalized channel signals; and coding the at least two equalization sound channel signals to obtain a coded code stream.

In this embodiment, at least five channel signals included in the audio frame are paired to obtain a target channel pair set with the purpose of obtaining the sum of the maximum correlation values, and when the sum of the correlation values of the target channel pair set is greater than a preset threshold, energy equalization processing is performed on at least two channel signals of the at least five channel signals, so as to perform coding, thereby improving the coding efficiency of the audio frame.

In one possible implementation, the method further includes: and when the sum of the correlation values is less than or equal to the preset threshold value, encoding the at least five sound channel signals to obtain an encoded code stream.

In this embodiment, if the sum of the correlation values is less than or equal to the preset threshold, it indicates that the correlation between two channel signals in the channel pairs in the target channel pair set is low, there is no need for group encoding, and further there is no need to perform energy equalization on at least five channel signals, where the encoded object is the at least five channel signals, but is not the equalized channel signals.

In one possible implementation manner, the performing energy equalization processing on at least two channel signals of the at least five channel signals to obtain at least two equalized channel signals includes: acquiring fluctuation interval values of the at least five sound channel signals; determining an energy balance mode according to the fluctuation interval values of the at least five sound channel signals; and respectively carrying out energy equalization processing on the at least two channel signals according to the energy equalization mode to obtain the at least two equalized channel signals.

The fluctuation interval value is used to indicate the magnitude of the difference in energy or amplitude between at least five channel signals. The energy equalization modes include a first energy equalization mode and a second energy equalization mode, wherein the first energy equalization mode obtains two equalized channel signals corresponding to one channel pair by using two channel signals in the one channel pair. The second energy equalization mode obtains two equalized channel signals corresponding to one channel pair by using two channel signals in one channel pair and at least one channel signal in one channel pair.

In one possible implementation manner, the determining an energy equalization mode according to the fluctuation interval values of the at least five channel signals includes: when the fluctuation interval value meets a preset condition, determining that the energy balance mode is a first energy balance mode; or when the fluctuation interval value does not meet the preset condition, determining that the energy balance mode is a second energy balance mode.

In one possible implementation, the fluctuation interval value includes an energy flatness of the first audio frame; the fluctuation interval value meets a preset condition that the energy flatness is smaller than a first threshold value; or the fluctuation interval value comprises the amplitude flatness of the first audio frame; the fluctuation interval value meeting the preset condition means that the amplitude flatness is smaller than a second threshold value; or, the fluctuation interval value includes an energy deviation degree of the first audio frame; the fluctuation interval value meets a preset condition, namely the energy deviation degree is not in a first preset range; or, the fluctuation interval value includes a degree of amplitude deviation of the first audio frame; the fluctuation interval value meets the preset condition, namely that the amplitude deviation degree is not in a second preset range.

In a possible implementation manner, when the energy equalization mode is the first energy equalization mode, the performing energy equalization processing on at least two channel signals of the at least five channel signals to obtain at least two equalized channel signals includes: and performing energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals.

In one possible implementation manner, the performing energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals includes: and aiming at a current sound channel pair in the target sound channel pair set, calculating an average value of energy values or amplitude values of two sound channel signals contained in the current sound channel pair, and respectively carrying out energy equalization processing on the two sound channel signals contained in the current sound channel pair according to the average value to obtain two corresponding equalized sound channel signals.

Therefore, when the fluctuation interval value of at least five channel signals is larger, energy equalization can be performed only between the two related channel signals, so that the bit allocation during stereo processing is more consistent with the fluctuation interval value of the channel signals, and the problem that coding noise of a channel pair with large energy is possibly far larger than that of a channel pair with small energy due to insufficient bits and bits of a channel pair with small energy have redundancy in a coding environment with low code rate is avoided.

In a possible implementation manner, when the energy equalization mode is the second energy equalization mode, the performing energy equalization processing on at least two channel signals of the at least five channel signals to obtain at least two equalized channel signals includes: and calculating the average value of the energy values or amplitude values of the at least five sound channel signals, and respectively carrying out energy equalization processing on the at least five sound channel signals according to the average value to obtain the at least five equalization sound channel signals.

In a possible implementation manner, before determining the energy equalization mode according to the fluctuation interval values of the at least five channel signals, the method further includes: judging whether the coding code rate corresponding to the first audio frame is greater than a code rate threshold value or not; when the coding code rate is larger than the code rate threshold, determining that the energy balance mode is a second energy balance mode; and when the coding code rate is smaller than or equal to the code rate threshold value, determining the energy balance mode according to the fluctuation interval value.

In one possible implementation, the method further includes: and coding the channel signals which are not subjected to the energy equalization processing in the at least five channel signals.

In a second aspect, the present application provides an encoding apparatus comprising: the device comprises an acquisition module, a coding module and a decoding module, wherein the acquisition module is used for acquiring a first audio frame to be coded, and the first audio frame comprises at least five channel signals; obtaining a sum of correlation values of all channel pairs in a target channel pair set, wherein the target channel pair set comprises at least one channel pair, one channel pair comprises two channel signals in the at least five channel signals, the one channel pair has a correlation value, and the correlation value is used for representing the correlation between the two channel signals of the one channel pair; the processing module is used for performing energy equalization processing on at least two channel signals in the at least five channel signals to obtain at least two equalized channel signals when the sum of the correlation values is greater than a preset threshold value; and the coding module is used for coding the at least two equalization sound channel signals to obtain a coded code stream.

In a possible implementation manner, the encoding module is further configured to encode the at least five channel signals to obtain an encoded code stream when the sum of the correlation values is less than or equal to the preset threshold.

In a possible implementation manner, the processing module is specifically configured to obtain fluctuation interval values of the at least five channel signals; determining an energy balance mode according to the fluctuation interval values of the at least five sound channel signals; and respectively carrying out energy equalization processing on the at least two channel signals according to the energy equalization mode to obtain the at least two equalized channel signals.

In a possible implementation manner, the processing module is specifically configured to determine that the energy balance mode is a first energy balance mode when the fluctuation interval value meets a preset condition; or when the fluctuation interval value does not meet the preset condition, determining that the energy balance mode is a second energy balance mode.

In one possible implementation, the fluctuation interval value includes an energy flatness of the first audio frame; the fluctuation interval value meeting the preset condition means that the energy flatness is smaller than a first threshold value; or the fluctuation interval value comprises the amplitude flatness of the first audio frame; the fluctuation interval value meeting the preset condition means that the amplitude flatness is smaller than a second threshold value; or the fluctuation interval value comprises the energy deviation degree of the first audio frame; the fluctuation interval value meets a preset condition, namely the energy deviation degree is not in a first preset range; or, the fluctuation interval value includes a degree of amplitude deviation of the first audio frame; the fluctuation interval value meets the preset condition, namely that the amplitude deviation degree is not in a second preset range.

In a possible implementation manner, when the energy equalization mode is the first energy equalization mode, the processing module is specifically configured to perform energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals.

In a possible implementation manner, the processing module is specifically configured to, for a current channel pair in the target channel pair set, calculate an average value of energy values or amplitude values of two channel signals included in the current channel pair, and perform energy equalization processing on the two channel signals included in the current channel pair respectively according to the average value to obtain two corresponding equalized channel signals.

In a possible implementation manner, when the energy equalization mode is the second energy equalization mode, the processing module is specifically configured to calculate an average value of energy values or amplitude values of the at least five channel signals, and perform energy equalization processing on the at least five channel signals respectively according to the average value to obtain the at least five equalization channel signals.

In a possible implementation manner, the processing module is further configured to determine whether a coding rate corresponding to the first audio frame is greater than a rate threshold; when the coding code rate is larger than the code rate threshold, determining that the energy balance mode is a second energy balance mode; and when the coding code rate is smaller than or equal to the code rate threshold value, determining the energy balance mode according to the fluctuation interval value.

In a possible implementation manner, the encoding module is further configured to encode the channel signal that is not subjected to the energy equalization process in the at least five channel signals.

In a third aspect, the present application provides an apparatus comprising: one or more processors; a memory for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement the method of any one of the first aspects as described above.

In a fourth aspect, the present application provides a computer readable storage medium comprising a computer program which, when executed on a computer, causes the computer to perform the method of any of the first aspects above.

In a fifth aspect, the present application provides a computer-readable storage medium comprising an encoded bitstream obtained according to the method for encoding a multi-channel audio signal as described in any one of the above first aspects.

Drawings

Fig. 1 schematically shows a block diagram of an audio decoding system 10 to which the present application applies;

fig. 2 schematically shows a block diagram of an audio decoding device 200 to which the present application applies;

FIG. 3 is a flow chart of an exemplary embodiment of a method of encoding a multi-channel audio signal provided herein;

fig. 4a is a block diagram of an exemplary encoding apparatus to which the encoding method of a multi-channel audio signal provided in the present application is applied;

FIG. 4b is a block diagram of an exemplary multi-channel adaptive pairing block;

FIG. 4c is an exemplary block diagram of a pairing process module;

fig. 5 is a block diagram of an exemplary decoding apparatus to which a multi-channel audio decoding method provided in the present application is applied;

FIG. 6 is a schematic structural diagram of an embodiment of an encoding apparatus according to the present application;

fig. 7 is a schematic structural diagram of an embodiment of the apparatus of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely with reference to the accompanying drawings in the present application, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

The terms "first," "second," and the like in the description examples and claims of this application and in the drawings are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or order. Furthermore, the terms "comprises" and "comprising," as well as any variations thereof, are intended to cover a non-exclusive inclusion, such as a list of steps or elements. A method, system, article, or apparatus is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, system, article, or apparatus.

It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

The present application refers to the interpretation of the terms:

audio frame: the audio data is streamed, and in practical applications, for convenience of audio processing and transmission, the amount of audio data within a time duration, referred to as "sampling time", may be determined according to the requirements of the codec and the specific application, for example, the time duration is 2.5ms to 60ms, and ms is millisecond.

Audio signal: the audio signal is a regular sound wave frequency and amplitude variation information carrier with voice, music and sound effects. Audio is a continuously varying analog signal that can be represented by a continuous curve called a sound wave. The audio signal is the digital signal generated by analog-to-digital conversion or computer. There are three important parameters of sound waves: frequency, amplitude and phase, which also determine the characteristics of the audio signal.

Sound channel signal: refers to mutually independent audio signals acquired or played back at different spatial positions when sound is recorded or played. The number of channels is therefore the number of sources when recording or the number of loudspeakers when playing back sound.

The following is the system architecture to which the present application applies.

Fig. 1 schematically shows a block diagram of an audio decoding system 10 to which the present application applies. As shown in fig. 1, audio coding system 10 may include a source device 12 and a destination device 14, source device 12 generating an encoded codestream, and thus source device 12 may be referred to as an audio encoding apparatus. Destination device 14 may decode the encoded codestream generated by source device 12, and thus destination device 14 may be referred to as an audio decoding apparatus.

Source device 12 includes an encoder 20, which may optionally include an audio source 16, an audio preprocessor 18, and a communication interface 22.

Audio source 16 may include or may be any type of audio capture device for capturing real-world speech, music, sound effects, and the like, and/or any type of audio generation device, such as an audio processor or device for generating speech, music, and sound effects. The audio source may be any type of memory or storage that stores the audio described above.

The audio preprocessor 18 is configured to receive (raw) audio data 17 and preprocess the audio data 17 to obtain preprocessed audio data 19. For example, the pre-processing performed by audio pre-processor 18 may include pruning or denoising. It will be appreciated that the audio pre-processing unit 18 may be an optional component.

The encoder 20 is arranged to receive pre-processed audio data 19 and to provide encoded audio data 21.

The communication interface 22 in the source device 12 may be used to receive the encoded audio data 21 and transmit the encoded audio data 21 to the destination device 14 over the communication channel 13 for storage or direct reconstruction.

The destination device 14 includes a decoder 30, and optionally, a communication interface 28, an audio post-processor 32, and a playback device 34.

Communication interface 28 in destination device 14 is operable to receive encoded audio data 21 directly from source device 12 and provide encoded audio data 21 to decoder 30.

The communication interface 22 and the communication interface 28 may be used to send or receive the encoded audio data 21 over a direct communication link, such as a direct wired or wireless connection, etc., between the source device 12 and the destination device 14, or over any type of network, such as a wired network, a wireless network, or any combination thereof, any type of private and public networks, or any type of combination thereof.

For example, communication interface 22 may be used to encapsulate encoded audio data 21 into a suitable format such as a message and/or process encoded audio data 21 using any type of transport encoding or processing for transmission over a communication link or communication network.

Communication interface 28 corresponds to communication interface 22, and may be used, for example, to receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and/or decapsulation to obtain encoded audio data 21.

Both the communication interface 22 and the communication interface 28 may be configured as a one-way communication interface, as indicated by the arrows pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14 in fig. 1, or a two-way communication interface, and may be used to send and receive messages, etc., to establish a connection, acknowledge and exchange any other information related to a communication link and/or data transmission, such as encoded audio data, etc.

The decoder 30 is arranged to receive encoded audio data 21 and to provide decoded audio data 31.

The audio post-processor 32 is configured to perform post-processing on the decoded audio data 31 to obtain post-processed audio data 33. Post-processing performed by the audio post-processor 32 may include, for example, pruning or resampling, etc.

The playback device 34 is used to receive post-processed audio data 33 for playing back audio to a user or listener. The playback device 34 may be or include any type of player for playing the reconstructed audio, such as an integrated or external speaker. For example, the speakers may include speakers, and the like.

Fig. 2 schematically shows a block schematic diagram of an audio decoding device 200 to which the present application applies. In one embodiment, the audio coding apparatus 200 may be an audio decoder (e.g., the decoder 30 of fig. 1) or an audio encoder (e.g., the encoder 20 of fig. 1).

The audio decoding apparatus 200 includes: an ingress port 210 and a reception unit (Rx) 220 for receiving data, a processor, logic unit or central processor 230 for processing data, a transmission unit (Tx) 240 and an egress port 250 for transmitting data, and a memory 260 for storing data. The audio decoding device 200 may further comprise an optical-to-electrical conversion component and an electrical-to-optical (EO) component coupled with the ingress port 210, the reception unit 220, the transmission unit 240 and the egress port 250 for egress or ingress of optical or electrical signals.

The processor 230 is implemented by hardware and software. Processor 230 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. Processor 230 is in communication with ingress port 210, receiving unit 220, transmitting unit 240, egress port 250, and memory 260. The processor 230 includes a decode module 270 (e.g., an encode module or a decode module). The decoding module 270 implements the embodiments disclosed in the present application to implement the encoding method of the multi-channel audio signal provided in the present application. For example, the decode module 270 implements, processes, or provides various encoding operations. Thus, a substantial improvement is provided for the functionality of the audio decoding apparatus 200 by the decoding module 270 and the transition of the audio decoding apparatus 200 to different states is influenced. Alternatively, decode module 270 may be implemented as instructions stored in memory 260 and executed by processor 230.

Memory 260, which may include one or more disks, tape drives, and solid state drives, may be used as an over-flow data storage device for storing programs when such programs are selectively executed, and for storing instructions and data that are read during program execution. The memory 260 may be volatile and/or nonvolatile, and may be Read Only Memory (ROM), random Access Memory (RAM), random access memory (TCAM), and/or Static Random Access Memory (SRAM).

Based on the description of the above embodiments, the present application provides an encoding method of a multi-channel audio signal.

Fig. 3 is a flowchart of an exemplary embodiment of a method for encoding a multi-channel audio signal provided in the present application. The process 300 may be performed by the source device 12 or the audio decoding device 200 in the audio decoding system 10. Process 300 is described as a series of steps or operations, it being understood that process 300 may be performed in various orders and/or concurrently, and is not limited to the order of execution shown in FIG. 3. As shown in fig. 3, the method includes:

step 301, a first audio frame to be encoded is obtained.

The first audio frame of the present embodiment may be any one of frames in multi-channel audio to be encoded, and the first audio frame includes five or more channel signals. For example, the 5.1 channels include six channel signals of a center channel (C), a front left channel (left, L), a front right channel (right, R), a rear left surround channel (LS), a rear right surround channel (RS), and a 0.1 channel Low Frequency Effects (LFE). The 7.1 channels include eight channel signals of C, L, R, LS, RS, LB, RB and LFE, where LFE is an audio channel from 3-120Hz that is typically sent to a speaker specifically designed for low tones.

Step 302, obtaining the sum of correlation values of all channel pairs in the target channel pair set.

The target channel pair set is obtained with a view to obtaining a sum of maximum correlation values, the target channel pair set including at least one channel pair including two channel signals of at least five channel signals. One channel pair has a correlation value representing a correlation between two channel signals of the one channel pair.

The two channel signals with higher correlation are coded together to reduce redundancy and improve coding efficiency, so that the embodiment determines according to the correlation value between the two channel signals when pairing. In order to find the pairing mode with the highest correlation as much as possible, correlation values between every two of at least five channel signals in the first audio frame may be calculated to obtain a correlation value set of the first audio frame. For example, a total of five channel signals may constitute 10 channel pairs, and correspondingly, the correlation value set may include 10 correlation values.

Alternatively, the correlation values may be normalized, so that the correlation values of all channel pairs are limited within a specific range, so as to set a uniform judgment criterion for the correlation values, for example, a group pairing threshold, which may be set to a value greater than or equal to 0.2 and less than or equal to 1, for example, 0.3, so that as long as the normalized correlation values of two channel signals are less than the group pairing threshold, the correlation of the two channel signals is considered to be poor, and no group pairing coding is required.

In one possible implementation, the correlation value between two channel signals (e.g., ch1 and ch 2) may be calculated using the following formula:

where corr (ch 1, ch 2) represents a normalized correlation value between the channel signal ch1 and the channel signal ch2, spec _ ch1 (i) represents a frequency domain coefficient of the ith frequency bin of the channel signal ch1, spec _ ch2 (i) is a frequency domain coefficient of the ith frequency bin of the channel signal ch2, and N represents an integer value not exceeding the total number of frequency points of one audio frame.

It should be noted that other algorithms or formulas may also be used to calculate the correlation value between the two channel signals, which is not specifically limited in this application.

The method for acquiring the group pair set of the target channel pair set comprises the following steps: and selecting channel pairs from the channel pairs corresponding to at least five channel signals to add into the target channel pair set by taking the sum of the maximum correlation values as an aim. The sum of the correlation values of the target channel pair set is the sum of the correlation values of all channel pairs in the target channel pair set obtained by pairing at least five channel signals according to the pairing mode. The pairing method of the embodiment may include the following two implementation manners:

(1) The largest M correlation values are selected from the set of correlation values, which must be greater than or equal to the group pairing threshold value, because correlation values smaller than the group pairing threshold value indicate that the correlation between the two channel signals in the channel pair to which they correspond is low, and no group coding is necessary. In order to improve the coding efficiency, all correlation values greater than or equal to the group pairing threshold value do not need to be selected, so an upper limit N of M is set, that is, at most N correlation values are selected.

N may be an integer greater than or equal to 2, and the maximum value of N may not exceed the number of all channel pairs corresponding to all channel signals of the first audio frame. The larger the value of N, the more computation involved, while the smaller the value of N, the more likely the set of channel pairs is lost, thereby reducing the coding efficiency.

Alternatively, N may be set to the maximum channel logarithm plus one, i.e.

CH represents the number of channel signals included in the first audio frame. For example, if the 5.1 channels include five channel signals, N =3;7.1 channels contain seven channel signals, N =4.

Then, M channel pair sets are obtained according to the M correlation values, each channel pair set at least includes one of the M channel pairs corresponding to the M correlation values, and when the channel pair set includes more than two channel pairs, the more than two channel pairs do not include the same channel signal. For example, 5.1 channels, the 3 channel pairs corresponding to the maximum correlation value selected from the correlation value set are (L, R), (R, C), and (LS, RS), where the correlation value of (LS, RS) is smaller than the group pair threshold, thus excluding the remaining two channel pairs (L, R) and (R, C) to obtain two channel pair sets, one of which includes (L, R) and the other includes (R, C).

Taking any one of M channel pairs (e.g., a first channel pair) corresponding to M correlation values greater than or equal to the group pair comparison threshold as an example, the method for acquiring M channel pair sets in this embodiment may include: adding a first channel pair into a target channel pair set, wherein the M channel pair sets comprise the target channel pair set, when other channel pairs except the associated channel pair in the plurality of channel pairs comprise channel pairs with correlation values larger than a pair-pairing threshold value, selecting a channel pair with the maximum correlation value from the other channel pairs to be added into the target channel pair set, and the associated channel pair comprises any one of channel signals included in the channel pair added into the target channel pair set.

The above process is an iterative process except for the step of adding the first channel pair to the target channel pair set. Namely, it is

a. And judging whether the other channel pairs except the associated channel pair in the plurality of channel pairs comprise channel pairs with correlation values larger than a group pair threshold value.

b. And if the channel pairs with the correlation values larger than the pair pairing threshold value are included, selecting one channel pair with the maximum correlation value from other channel pairs to be added into the target channel pair set.

At this time, the step b may be performed iteratively as long as the other channel pairs include a channel pair having a correlation value greater than the pair threshold.

Optionally, in order to reduce the amount of computation, the correlation values smaller than the group pairing threshold may be deleted from the correlation value set, so that the number of channel pairs may be reduced, and the number of iterations may be reduced.

(2) Acquiring all channel pair sets corresponding to at least five channel signals according to the plurality of channel pairs, acquiring the sum of correlation values of all channel pairs contained in any one channel pair set in all channel pair sets according to the correlation value set, and determining the channel pair set corresponding to the largest sum of correlation values in all channel pair sets as a target channel pair set.

The correlation value set includes correlation values of a plurality of channel pairs of at least five channel signals of the first audio frame, and the channel pairs are regularly combined (that is, the channel pairs in the same channel pair set cannot contain the same channel signal), so that a plurality of channel pair sets corresponding to the at least five channel signals can be obtained.

In one possible implementation, when the number of channel signals is odd, the number of all channel pair sets may be calculated by using the following formula:

in one possible implementation, when the number of channel signals is even, the number of all channel pair sets can be calculated by using the following formula:

wherein, pair _ num represents the number of all channel Pair sets, and CH represents the number of channel signals participating in multi-channel processing in the first audio frame, which is the result of multi-channel mask screening.

Optionally, in order to reduce the amount of computation, after obtaining the correlation value set, a plurality of channel pair sets may be obtained according to channel pairs other than the correlated channel pair in the plurality of channel pairs, where a correlation value of the uncorrelated channel pair is smaller than the set-to-set threshold, so that the number of channel pairs participating in computation may be reduced when obtaining the channel pair set, and further the number of channel pair sets may be reduced, and in subsequent steps, the amount of computation of the sum of correlation values may also be reduced.

And step 303, when the sum of the correlation values is greater than a preset threshold, performing energy equalization processing on at least two channel signals in the at least five channel signals to obtain at least two equalized channel signals.

In a possible implementation manner, the fluctuation interval values of at least five channel signals may be obtained first, then the energy equalization mode is determined according to the fluctuation interval values of at least five channel signals, and then the energy equalization processing is performed on the at least five channel signals according to the energy equalization mode to obtain at least five equalized channel signals.

The fluctuation interval value is used to indicate the magnitude of the difference in energy or amplitude between at least five channel signals.

The energy equalization modes include a first energy equalization mode and a second energy equalization mode, wherein the first energy equalization mode obtains two equalized channel signals corresponding to one channel pair by using two channel signals in the one channel pair. The second energy equalization mode obtains two equalized channel signals corresponding to one channel pair by using two channel signals in one channel pair and at least one channel signal in one channel pair.

Determining the energy equalization mode according to the fluctuation interval values of the at least five channel signals may include: when the fluctuation interval value meets the preset condition, determining that the energy balance mode is a first energy balance mode; and when the fluctuation interval value does not meet the preset condition, determining the energy balance mode as a second energy balance mode.

The fluctuation interval value comprises the energy flatness of the first audio frame, and the fluctuation interval value meeting the preset condition means that the energy flatness is smaller than a first threshold value; or the fluctuation interval value comprises the amplitude flatness of the first audio frame, and the fluctuation interval value meeting the preset condition means that the amplitude flatness is smaller than a second threshold value; or the fluctuation interval value comprises the energy deviation degree of the first audio frame, and the fluctuation interval value meeting the preset condition means that the energy deviation degree is not in a first preset range; or, the fluctuation interval value includes an amplitude deviation degree of the first audio frame, and the fluctuation interval value meeting the preset condition means that the amplitude deviation degree is not within a second preset range.

In the embodiment of the invention, the energy flatness represents the fluctuation of the frame energy after the energy of the current frame frequency domain coefficient of a plurality of sound channels screened by the multi-channel screening unit is normalized, and can be measured by a flatness calculation formula. When the energy of all the sound channels of the current frame is the same, the energy flatness of the current frame is 1; when the energy of a certain channel of the current frame is 0, the energy flatness of the current frame is 0, and thus the value range of the energy flatness between channels is [0,1]. The greater the fluctuation of the inter-channel energy, the smaller the value of its energy flatness. In one embodiment, a uniform first threshold may be set for all channel formats (e.g., 5.1, 7.1, 9.1, 11.1), which may be, for example, 0.483,0.492, or 0.504, etc. In another embodiment, different first thresholds are set for different channel formats. For example, the first threshold value for the 5.1 channel format is 0.511, the first threshold value for the 7.1 channel format is 0.563, the first threshold value for the 9.1 channel format is 0.608, and the first threshold value for the 11.1 channel format is 0.654.

The amplitude flatness represents the fluctuation of the frame amplitude after the amplitude normalization of the current frame frequency domain coefficient of the multiple sound channels screened by the multi-channel screening unit, and can be measured by a flatness calculation formula. When the frame amplitudes of all the sound channels are the same, the flatness is 1; when the frame amplitude of one of the channels is 0, the flatness thereof is 0. The amplitude flatness thus ranges between 0, 1. The greater the fluctuation of the inter-channel amplitude, the smaller the value of the flatness thereof. In one embodiment, a uniform second threshold may be set for all channel formats (e.g., 5.1, 7.1, 9.1, 11.1), such as 0.695,0.701, or 0.710, etc. In another embodiment, different second threshold values may be given for different channel formats, for example, the second threshold value for the 5.1 channel format may be 0.715, the second threshold value for the 7.1 channel format may be 0.753, the second threshold value for the 9.1 channel format may be 0.784, and the second threshold value for the 11.1 channel format may be 0.809.

Since there is a square relationship between the amplitude and the energy, there is also a square relationship between the amplitude flatness and the energy flatness, i.e. the fluctuation of the inter-channel frame amplitude corresponding to the square of the amplitude flatness is approximately equal to the fluctuation of the inter-channel frame energy corresponding to the energy flatness.

The present embodiment may determine the energy equalization pattern including the energy flatness, the amplitude flatness, the energy deviation, or the amplitude deviation, from the above-described kinds of information representing the fluctuation interval values of the at least five channel signals.

(1) Calculating energy values of at least five sound channel signals, acquiring the energy flatness of a first audio frame according to the energy values of the at least five sound channel signals, and determining the energy equalization mode as a first energy equalization mode when the energy flatness of the first audio frame is smaller than a first threshold value; when the energy flatness of the first audio frame is greater than or equal to a first threshold, determining the energy equalization mode as a second energy equalization mode.

(2) Calculating amplitude values of at least five sound channel signals, acquiring amplitude flatness of a first audio frame according to the amplitude values of the at least five sound channel signals, and determining an energy balance mode as a first energy balance mode when the amplitude flatness of the first audio frame is smaller than a second threshold; and when the amplitude flatness of the first audio frame is larger than or equal to a second threshold value, determining the energy equalization mode as a second energy equalization mode.

(3) Calculating energy values of at least five sound channel signals, acquiring energy deviation degrees of a first audio frame according to the energy values of the at least five sound channel signals, and determining an energy equalization mode as a first energy equalization mode when the energy deviation degrees of the first audio frame are not in a first preset range; and when the energy deviation degree of the first audio frame is within a first preset range, determining the energy balance mode as a second energy balance mode.

(4) Calculating amplitude values of at least five sound channel signals, acquiring amplitude deviation degrees of a first audio frame according to the amplitude values of the at least five sound channel signals, and determining an energy equalization mode as a first energy equalization mode when the amplitude deviation degrees of the first audio frame are not in a second preset range; and when the amplitude deviation degree of the first audio frame is within a second preset range, determining the energy balance mode as a second energy balance mode.

It should be noted that the present application may also adopt other energy balancing modes, which is not particularly limited.

In a possible implementation manner, before determining the energy balance mode according to the fluctuation interval value of at least five channel signals, the energy balance mode may also be determined according to the coding rate corresponding to the first audio frame, that is, whether the coding rate is greater than a rate threshold is determined, and when the coding rate is greater than the rate threshold, the energy balance mode is determined to be the second energy balance mode; and when the coding code rate is smaller than or equal to the code rate threshold value, determining an energy balance mode according to the fluctuation interval values of at least five sound channel signals.

When the energy equalization mode is the first energy equalization mode, the average value of the energy or amplitude values of the two channel signals included in the current channel pair can be calculated for the current channel pair in the target channel pair set corresponding to the group pairing mode, and the two channel signals are respectively subjected to energy equalization processing according to the average value to obtain the corresponding two equalized channel signals.

When the energy equalization mode is the second energy equalization mode, an average value of the energy or amplitude values of the at least five channel signals may be calculated, and the at least five channel signals are respectively subjected to energy equalization processing according to the average value to obtain at least five equalized channel signals.

It should be noted that, in step 303, at least two channel signals of the at least five channel signals are mainly subjected to energy equalization processing to obtain at least two equalized channel signals, where the at least two channel signals are already paired channel signals in the target channel pair set, and the remaining channel signals that are not paired are directly subjected to encoding processing except for already paired channel signals in the target channel pair set.

Step 304, encoding at least two equalization channel signals to obtain encoded code streams.

And 305, when the sum of the correlation values is less than or equal to a preset threshold value, encoding at least five sound channel signals to obtain an encoded code stream.

In this embodiment, at least five channel signals included in the audio frame are paired to obtain a target channel pair set with the purpose of obtaining the sum of the maximum correlation values, and when the sum of the correlation values of the target channel pair set is greater than a preset threshold, energy equalization processing is performed on the at least five channel signals, so as to perform coding, thereby improving the coding efficiency of the audio frame.

The following describes how to determine the grouping mode and the energy balance mode in the embodiment of the method shown in fig. 3 by two specific embodiments. Taking the 5.1 channels as an example, the 5.1 channels include a center channel (C), a front left channel (left, L), a front right channel (right, R), a rear left surround channel (LS), a rear right surround channel (RS), and a 0.1 channel Low Frequency Effect (LFE), and the six channel signals are provided with channel indexes as shown in table 1, for example.

TABLE 1

Sound channel index	Sound channel signal
		0	L
1	R
		2	LS
3	RS
		4	C
5	LFE

Fig. 4a is a block diagram of an exemplary encoding apparatus applied in the method for encoding a multi-channel audio signal provided in the present application, where the encoding apparatus may be the encoder 20 of the source device 12 in the audio decoding system 10, or may be the decoding module 270 in the audio decoding device 200. The encoding apparatus may include a multi-channel adaptive pair pairing module, a channel encoding module, and a code stream multiplexing interface, wherein,

the input of the multi-channel adaptive pairwise module comprises six channel signals (L, R, C, LS, RS, LFE) of 5.1 channels and a multi-channel processing indicator (MultiProcFlag), and the output comprises six channel signals (M1, S1, M2, S2, C, LFE) after a pairwise grouping, wherein M1 and S1 are a pair of channel pairs resulting from the pairwise grouping, M2 and S2 are a pair of channel pairs resulting from the pairwise grouping, and multi-channel side information (sidelnfo) comprising a set of channel pairs.

The channel coding module uses a single-channel coding unit (or a single-channel box or a single-channel tool) to code the channel signals (M1, S1, M2, S2, C and LFE) output by the multi-channel adaptive group module and output corresponding coded channel signals (E1-E6). In the process of encoding the channel signals, the monaural encoding unit assigns a larger number of bits to channel signals having higher energy (or higher amplitude) and assigns a smaller number of bits to channel signals having lower energy (or lower amplitude). Optionally, the channel encoding module may also employ a stereo encoding unit, such as a parametric stereo encoder or a lossy stereo encoder, to encode the processed channel signal output by the multi-channel processing module.

It should be noted that the channel signals of unpaired pair (e.g., C and LFE) can be directly input into the channel encoding module to obtain the encoded channel signals E5 and E6.

The code stream multiplexing interface generates an encoded multi-channel signal which comprises encoded channel signals (E1-E6) output by the channel encoding module and side information (comprising multi-channel side information). Optionally, the code stream multiplexing interface may process the encoded multi-channel signal into a serial signal or a serial bit stream.

Fig. 4b is a block diagram of an exemplary multi-channel adaptive pairing block, as shown in fig. 4b, including: the device comprises a multi-channel screening unit, a global correlation value statistic unit, a multi-channel energy balance selection module and a group pairing processing module.

A multi-channel filtering unit filters five channel signals (L, R, C, LS, RS) participating in multi-channel processing from six channel signals (L, R, C, LS, LFE) according to a multi-channel processing indicator (MultiProcFlag).

The global correlation value statistical unit calculates the normalized correlation value between any two channel signals of the channel signals participating in the multi-channel processing, namely L, R, C, LS and RS. The present application can calculate a correlation value between two channel signals (e.g., the channel signal ch1 and the channel signal ch 2) using the following formula:

wherein, corr (ch 1, ch 2) represents a normalized correlation value between the channel signal ch1 and the channel signal ch2, spec _ ch1 (i) represents a frequency domain coefficient of the ith frequency bin of the channel signal ch1, spec _ ch2 (i) represents a frequency domain coefficient of the ith frequency bin of the channel signal ch2, and N represents an integer value not exceeding the total frequency point number of one audio frame. Then, in all channel pair sets corresponding to the channel signals participating in the multi-channel processing, a maximum correlation value sum (i.e., a sum of correlation values of all channel pairs included in the channel pair set) and a channel pair set corresponding to the maximum correlation value sum (regarded as a target channel pair set) are determined according to the normalized correlation value between any two channel signals. And finally, outputting global correlation value side information, wherein the global correlation value side information comprises the sum corr _ sum _ max of the maximum correlation values and the target channel pair set. Assuming that the target channel pair set includes (R, C) and (LS, RS), the sum of the maximum correlation values corr _ sum _ max = corr (L, R) + corr (LS, RS).

It should be noted that, after obtaining the normalized correlation value between any two channel signals, the global correlation value statistics unit may screen the correlation value according to a set group pairing threshold value, that is, the correlation value greater than or equal to the group pairing threshold value is retained, and the correlation value smaller than the group pairing threshold value is deleted, or the value thereof is set to 0. This can reduce the amount of calculation.

And the multi-channel energy balance selection module determines whether the energy balance processing needs to be carried out on the five-channel signals according to the coding code rate and the five-channel signals. The grouping manner of the five channel signals is a global grouping manner, which aims to obtain the sum of the maximum correlation values, and the description of step 302 can be specifically referred to. When the sum of the correlation values of the target sound channel pair set is greater than a preset threshold value, determining that energy equalization processing needs to be carried out on five sound channel signals; when the sum of the correlation values of the target channel pair set is less than or equal to a preset threshold, it is determined that the energy equalization processing is not required for the five channel signals. When it is determined that the energy equalization processing needs to be performed on the five channel signals, an energy equalization mode is determined.

Fig. 4c is a diagram illustrating an exemplary configuration of a group pairing process module, which includes a group pairing decision device, an energy equalization unit, and a stereo process box, as shown in fig. 4 c.

The group pair decision device calculates the energy or amplitude value of each channel signal, and the following formula can be used to calculate the energy or amplitude value of the channel signal (ch):

wherein energy (ch) represents the energy or amplitude value of the channel signal ch, sepc _ coeff (ch, i) represents the frequency domain coefficient of the ith frequency point of the channel signal ch, and N represents an integer value not exceeding the total frequency point number of an audio frame.

Then, normalized energy or amplitude values of the respective channel signals are calculated, and the application may calculate the normalized energy or amplitude values of the channel signals (ch) using the following formula:

wherein, energy _ uniform (ch) represents a normalized energy or amplitude value of the channel signal ch, and energy _ max represents the maximum one of the energy or amplitude values of the five channel signals (i.e., energy (L), energy (R), energy (C), energy (LS), and energy (RS)). If energy _ max =0, energy _ uniform (ch) is 0.

Then, the fluctuation interval value of the five channel signals is calculated, optionally, the fluctuation interval value may refer to energy flatness, and the following formula may be adopted in the present application to calculate the energy flatness of the five channel signals:

where efm represents the energy flatness of five channel signals, and the channel indexes of L, R, C, LS, and RS are shown in table 1.

Optionally, the fluctuation interval value may also refer to an energy deviation degree, and based on the normalized energy or amplitude value energy _ uniform (ch) obtained through the above calculation, the following formula may be used in the present application to calculate an average energy or amplitude value of the five channel signals:

wherein avg _ energy _ uniform represents the average energy or amplitude value of the five channel signals, and the channel indexes of L, R, C, LS, RS are shown in table 1.

The energy deviation degree of the channel signal (ch) is calculated using the following formula:

here, the deviation (ch) represents the energy deviation of the channel signal ch. The maximum one of the energy deviations of L, R, C, LS, RS is determined as the energy deviation of the five channel signals.

Optionally, the fluctuation interval value may also refer to an amplitude value or an amplitude deviation, and the principle thereof is similar to the above energy-related value, and is not described herein again.

As described above, the energy equalization mode of the present application includes two implementation manners, where the Pair energy equalization mode is to use two channel signals in one channel Pair to obtain two equalized channel signals corresponding to the one channel Pair for each channel Pair in the target channel Pair set corresponding to the Pair pairing manner determined by the module selecting unit. The overall energy equalization mode is to use two channel signals in one channel pair and at least one channel signal in one channel pair to obtain two equalized channel signals corresponding to the one channel pair. For the channel signals without pair pairing, the corresponding equalized channel signal is the channel signal itself.

The group judgment device determines an energy balance mode according to the fluctuation interval value, and the method comprises the following two judgment modes:

(1) When efm is smaller than a first threshold value, the energy balance mode is a Pair energy balance mode; when efm is greater than or equal to the first threshold, the energy balance mode is the global energy balance mode.

(2) When the devision is within a value interval [ threshold,1/threshold ], the energy balance mode is an overall energy balance mode; when the devision is not within the value interval [ threshold,1/threshold ], the energy balance mode is a Pair energy balance mode. the value range of threshold may be (0, 1).

The deviation may be represented as a ratio of the frequency domain amplitude of each channel of the current frame to an average value of the frequency domain amplitudes of each channel of the current frame, i.e., an amplitude deviation. When the proportional relationship between the frequency domain amplitude of the current channel of the current frame and the average value of the frequency domain amplitudes of the channels of the current frame is less than 5 (corresponding to threshold = 0.2), two cases can be distinguished: 1. the frequency domain amplitude of the current channel is less than or equal to the average value of the frequency domain amplitudes of the channels of the current frame, and the "frequency domain amplitude of the current channel/average value of the frequency domain amplitudes of the channels of the current frame" satisfying the condition is between (0.2, 1), that is, between (threshold, 1); secondly, the frequency domain amplitude of the current sound channel is larger than the average value of the frequency domain amplitudes of all the sound channels of the current frame, and the average value of the frequency domain amplitudes of the current sound channel/all the sound channels of the current frame, which meets the condition, is between (1 and 5); in summary of the above two cases, when the proportional relationship between the frequency domain amplitude of the current channel and the average value of the frequency domain amplitude of each channel of the current frame is less than 5, the range of "the frequency domain amplitude of the current channel/the average value of the frequency domain amplitude of each channel of the current frame" satisfying the condition is between (0.2, 5), that is, (1/threshold) is the second preset range mentioned above, wherein the value of threshold may be between (0, 1), the smaller the value of threshold is, the larger the fluctuation of the frequency domain amplitude of the current channel with respect to the average value of the frequency domain amplitude of each channel of the current frame is, the larger the value of threshold is, the smaller the fluctuation of the frequency domain amplitude of the current channel with respect to the average value of the frequency domain amplitude of each channel of the current frame is, wherein the value of threshold may be 0.2,0.15,0.125,0.11, or 0.1, and so on.

The definition may also indicate a ratio of the frequency domain energy of each channel to the average of the frequency domain energy of each channel, i.e., an energy deviation. When the proportional relationship between the frequency domain energy of the current channel of the current frame and the average value of the frequency domain energy of each channel is less than 25 (threshold = 0.04), two cases can be classified: 1. the frequency domain energy of the current channel is less than or equal to the average value of the frequency domain energy of each channel of the current frame, the condition that the frequency domain energy of the current channel/the average value of the frequency domain energy of each channel of the current frame is greater than the average value of the frequency domain energy of each channel of the current frame, the condition that the frequency domain energy of the current channel/the average value of the frequency domain energy of each channel of the current frame is less than 25 is satisfied, the sum of the above two cases, the condition that the range of the frequency domain energy of the current channel/the average value of the frequency domain energy of each channel of the current frame is between the frequency domains (0.04, 25), i.e., between (Threshold, 1/Threshold), (Threshold, 1/Threshold) is the first preset range mentioned above when the ratio of the frequency domain energy of the current channel to the average value of the frequency domain energy of each channel of the current frame is less than 25, wherein the Threshold may be between (0, 1), the value of the Threshold represents the energy of the channel energy of the current channel is greater relative to the frequency domain energy of the frequency domain of the current channel, the frequency domain is represented by 0.0120, 5.

Since there is a square relationship between the amplitude and the energy, there is also a square relationship between the amplitude deviation and the energy deviation, i.e. the fluctuation of the inter-channel frame amplitude corresponding to the square of the amplitude deviation is approximately equal to the fluctuation of the inter-channel frame energy corresponding to the energy deviation.

In another embodiment, the first predetermined range may be extended to (0, 1/threshold), where the range of the interval for Pair energy equalization is [1/threshold, + ∞ ], and this indicates that Pair energy equalization is performed when the frequency-domain energy of the current channel is greater than the average of the frequency-domain energy of each channel of the current frame, and "the frequency-domain energy of the current channel/the average of the frequency-domain energy of each channel of the current frame" is greater than 1/threshold.

In another embodiment, the second preset range may be extended to (0, 1/threshold), where the range of the interval for Pair amplitude equalization is [1/threshold, + ∞ ], and this time it indicates that Pair amplitude equalization is performed when the frequency domain amplitude of the current channel is greater than the average value of the frequency domain amplitudes of the channels of the current frame, and "the frequency domain amplitude of the current channel/the average value of the frequency domain amplitudes of the channels of the current frame" is greater than 1/threshold.

It should be noted that the group pairing decision device may calculate normalized energy or amplitude values according to the five channel signals, so as to obtain energy flatness or energy deviation, may also calculate normalized energy or amplitude values only according to the successful channel signals of the group pairing, so as to obtain energy flatness or energy deviation, and may also calculate normalized energy or amplitude values according to some of the five channel signals, so as to obtain energy flatness or energy deviation. This is not particularly limited in this application.

The stereo processing unit may employ prediction-based or Karhunen-Loeve Transform (KLT) -based processing, i.e., the input two channel signals are rotated (e.g., via a 2 × 2 rotation matrix) to maximize energy compression, thereby concentrating the signal energy within one channel.

The stereo processing unit processes the input two channel signals and outputs processed channel signals (P1-P4) corresponding to the two channel signals and multi-channel side information, wherein the multi-channel side information comprises a sum of correlation values and a target channel pair set.

Fig. 5 is a schematic diagram of an exemplary decoding apparatus applied in the method for decoding multi-channel audio provided in the present application, where the decoding apparatus may be the decoder 30 of the destination device 14 in the audio decoding system 10, or may be the decoding module 270 in the audio decoding device 200. The decoding device can comprise a code stream demultiplexing interface, a sound channel decoding module and a multi-channel processing module, wherein,

the code stream demultiplexing interface receives an encoded multi-channel signal (such as a serial bit stream bitstream) from an encoding device, and an encoded channel signal (E) and multi-channel parameters (SIDE _ PAIR) are obtained after demultiplexing. E.g. E1, E2, E3, E4, \ 8230;, ei1, ei, and SIDE _ PAIR1, SIDE _ PAIR2, \ 8230;, SIDE _ PAIRM.

And the sound channel decoding module decodes the coded sound channel signal output by the code stream demultiplexing interface by using a single sound channel decoding unit (or a single sound channel box or a single sound channel tool) and outputs a decoded sound channel signal (D). For example, E1, E2, E3, E4, \ 8230, ei1 and Ei are respectively decoded by a single-sound-channel decoding unit to obtain E1, D2, D3, D4, \ 8230and Di1 and Di.

The multi-channel processing module comprises a plurality of stereo processing units which may employ prediction-based or KLT-based processing, i.e. the input two channel signals are de-rotated (e.g. via a 2 x 2 rotation matrix) to transform the signals into the original signal direction.

The decoding sound channel signal output by the sound channel decoding module can identify which two decoding sound channel signal pairs are paired by the multi-channel parameter, the paired decoding sound channel signals are input into the stereo processing unit, and the stereo processing unit processes the two input decoding sound channel signals and outputs the sound channel signals (CH) corresponding to the two decoding sound channel signals. For example, stereo processing unit 1 processes D1 and D2 according to SIDE _ PAIR1 to obtain CH1 and CH2, stereo processing unit 2 processes D3 and D4 according to SIDE _ PAIR2 to obtain CH3 and CH4, \ 8230, and stereo processing unit m processes Di-1 and Di according to SIDE _ PAIRm to obtain CHi-1 and CHi.

It should be noted that, for unpaired channel signals (e.g., CHj) do not need to be processed by the stereo processing unit in the multi-channel processing module, and can be decoded and directly output.

Fig. 6 is a schematic structural diagram of an embodiment of the encoding apparatus of the present application, as shown in fig. 6, the apparatus can be applied to the source device 12 or the audio decoding device 200 in the above embodiments. The encoding device of the present embodiment may include: an acquisition module 601, a processing module 602 and an encoding module 603. Wherein the content of the first and second substances,

an obtaining module 601, configured to obtain a first audio frame to be encoded, where the first audio frame includes at least five channel signals; obtaining a sum of correlation values of a target channel pair set, the target channel pair set being obtained by obtaining a maximum sum of correlation values, the target channel pair set including at least one channel pair, one channel pair including two channel signals of the at least five channel signals, the one channel pair having a correlation value, the correlation value being used for representing a correlation between the two channel signals of the one channel pair; a processing module 602, configured to perform energy equalization processing on the at least five channel signals to obtain at least five equalized channel signals when the sum of the correlation values is greater than a preset threshold; an encoding module 603 configured to encode the at least five equalized channel signals.

In a possible implementation manner, the encoding module 603 is further configured to encode the at least five channel signals when the sum of the correlation values is smaller than or equal to the preset threshold.

In a possible implementation manner, the processing module 602 is specifically configured to obtain fluctuation interval values of the at least five channel signals; determining an energy balance mode according to the fluctuation interval values of the at least five sound channel signals; and respectively carrying out energy equalization processing on the at least five channel signals according to the energy equalization mode to obtain the at least five equalized channel signals.

In a possible implementation manner, the processing module 602 is specifically configured to determine that the energy balance mode is a first energy balance mode when the fluctuation interval value meets a preset condition; or when the fluctuation interval value does not meet the preset condition, determining that the energy balance mode is a second energy balance mode.

In one possible implementation, the fluctuation interval value includes an energy flatness of the first audio frame; the fluctuation interval value meeting the preset condition means that the energy flatness is smaller than a first threshold value; or the fluctuation interval value comprises the amplitude flatness of the first audio frame; the fluctuation interval value meets the preset condition that the amplitude flatness is smaller than a second threshold value; or, the fluctuation interval value includes an energy deviation degree of the first audio frame; the fluctuation interval value meets a preset condition, namely the energy deviation degree is not in a first preset range; or, the fluctuation interval value includes a degree of amplitude deviation of the first audio frame; the fluctuation interval value meets the preset condition, namely that the amplitude deviation degree is not in a second preset range.

In a possible implementation manner, when the energy equalization mode is the first energy equalization mode, the processing module 602 is specifically configured to calculate, for a current channel pair in the target channel pair set, an average value of energy or amplitude values of two channel signals included in the current channel pair, and perform energy equalization processing on the two channel signals respectively according to the average value to obtain two corresponding equalized channel signals.

In a possible implementation manner, when the energy equalization mode is the second energy equalization mode, the processing module 602 is specifically configured to calculate an average value of energy or amplitude values of the at least five channel signals, and perform energy equalization processing on the at least five channel signals according to the average value to obtain the at least five equalized channel signals.

In a possible implementation manner, the processing module 602 is further configured to determine whether a coding rate corresponding to the first audio frame is greater than a rate threshold; when the coding code rate is larger than the code rate threshold, determining that the energy balance mode is a second energy balance mode; and when the coding code rate is less than or equal to the code rate threshold, determining the energy balance mode according to the fluctuation interval value.

The apparatus of this embodiment may be configured to implement the technical solution of the method embodiment shown in fig. 3, and the implementation principle and the technical effect are similar, which are not described herein again.

Fig. 7 is a schematic structural diagram of an embodiment of the apparatus of the present application, and as shown in fig. 7, the apparatus may be an encoding apparatus in the above embodiment. The apparatus of this embodiment may include: a processor 701 and memory 702, memory 702 to store one or more programs; when the one or more programs are executed by the processor 701, the processor 701 may implement the technical solution of the method embodiment shown in fig. 3.

In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The processor may be a general purpose processor, a Digital Signal Processor (DSP), an application-specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in the present application may be directly implemented by a hardware encoding processor, or implemented by a combination of hardware and software modules in the encoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

The memory referred to in the various embodiments above may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SLDRAM (synchronous DRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, or portions thereof, which substantially or partly contribute to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of encoding a multi-channel audio signal, comprising:

acquiring a first audio frame to be encoded, wherein the first audio frame comprises at least five channel signals;

obtaining a sum of correlation values of all channel pairs in a target channel pair set, wherein the target channel pair set comprises at least one channel pair, one channel pair comprises two channel signals in the at least five channel signals, the one channel pair has a correlation value, and the correlation value is used for representing the correlation between the two channel signals of the one channel pair;

when the sum of the correlation values is larger than a preset threshold value, performing energy equalization processing on at least two channel signals in the at least five channel signals to obtain at least two equalized channel signals;

and coding the at least two equalization sound channel signals to obtain a coded code stream.

2. The method of claim 1, further comprising:

and when the sum of the correlation values is smaller than or equal to the preset threshold value, encoding the at least five sound channel signals to obtain an encoded code stream.

3. The method according to claim 1 or 2, wherein said energy equalizing at least two of the at least five channel signals to obtain at least two equalized channel signals comprises:

acquiring fluctuation interval values of the at least five sound channel signals;

determining an energy balance mode according to the fluctuation interval values of the at least five sound channel signals;

and respectively carrying out energy equalization processing on the at least two channel signals according to the energy equalization mode to obtain the at least two equalized channel signals.

4. The method according to claim 3, wherein the determining an energy equalization mode according to the fluctuation interval values of the at least five channel signals comprises:

when the fluctuation interval value meets a preset condition, determining that the energy balance mode is a first energy balance mode; alternatively, the first and second liquid crystal display panels may be,

and when the fluctuation interval value does not meet the preset condition, determining that the energy balance mode is a second energy balance mode.

5. The method of claim 4, wherein the fluctuation interval value comprises an energy flatness of the first audio frame; the fluctuation interval value meets a preset condition that the energy flatness is smaller than a first threshold value; alternatively, the first and second electrodes may be,

the fluctuation interval value comprises an amplitude flatness of the first audio frame; the fluctuation interval value meeting the preset condition means that the amplitude flatness is smaller than a second threshold value; alternatively, the first and second liquid crystal display panels may be,

the fluctuation interval value comprises an energy deviation degree of the first audio frame; the fluctuation interval value meets a preset condition, namely the energy deviation degree is not in a first preset range; alternatively, the first and second electrodes may be,

the fluctuation interval value includes a magnitude deviation of the first audio frame; the fluctuation interval value meets the preset condition, namely that the amplitude deviation degree is not in a second preset range.

6. The method according to claim 4 or 5, wherein said performing energy equalization processing on at least two of the at least five channel signals to obtain at least two equalized channel signals when the energy equalization mode is the first energy equalization mode comprises:

and performing energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals.

7. The method according to claim 6, wherein said energy equalizing the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals comprises:

and aiming at the current sound channel pair in the target sound channel pair set, calculating the average value of the energy values or the amplitude values of the two sound channel signals contained in the current sound channel pair, and respectively carrying out energy equalization processing on the two sound channel signals contained in the current sound channel pair according to the average value to obtain two equalized sound channel signals.

8. The method according to claim 4 or 5, wherein said energy equalizing at least two of the at least five channel signals to obtain at least two equalized channel signals when the energy equalization mode is the second energy equalization mode comprises:

and calculating the average value of the energy values or amplitude values of the at least five sound channel signals, and respectively carrying out energy equalization processing on the at least five sound channel signals according to the average value to obtain the at least five equalization sound channel signals.

9. The method according to any of claims 3-8, wherein before determining the energy equalization mode based on the fluctuation interval values of the at least five channel signals, further comprising:

judging whether the coding code rate corresponding to the first audio frame is greater than a code rate threshold value or not;

when the coding code rate is larger than the code rate threshold, determining that the energy balance mode is a second energy balance mode;

and when the coding code rate is less than or equal to the code rate threshold, determining the energy balance mode according to the fluctuation interval value.

10. The method according to any one of claims 1-9, further comprising:

and coding the channel signals which are not subjected to the energy equalization processing in the at least five channel signals.

11. An encoding apparatus, comprising:

the device comprises an acquisition module, a coding module and a decoding module, wherein the acquisition module is used for acquiring a first audio frame to be coded, and the first audio frame comprises at least five channel signals; obtaining a sum of correlation values of all channel pairs in a target channel pair set, wherein the target channel pair set comprises at least one channel pair, one channel pair comprises two channel signals in the at least five channel signals, the one channel pair has a correlation value, and the correlation value is used for representing the correlation between the two channel signals of the one channel pair;

the processing module is used for performing energy equalization processing on at least two channel signals in the at least five channel signals to obtain at least two equalized channel signals when the sum of the correlation values is greater than a preset threshold value;

and the coding module is used for coding the at least two equalization sound channel signals to obtain a coded code stream.

12. The apparatus of claim 11, wherein the encoding module is further configured to encode the at least five channel signals to obtain an encoded code stream when the sum of the correlation values is smaller than or equal to the preset threshold.

13. The apparatus according to claim 11 or 12, wherein the processing module is specifically configured to obtain fluctuation interval values of the at least five channel signals; determining an energy balance mode according to the fluctuation interval values of the at least five sound channel signals; and respectively carrying out energy equalization processing on the at least two channel signals according to the energy equalization mode to obtain the at least two equalized channel signals.

14. The apparatus according to claim 13, wherein the processing module is specifically configured to determine that the energy balance mode is a first energy balance mode when the fluctuation interval value meets a preset condition; or when the fluctuation interval value does not meet the preset condition, determining that the energy balance mode is a second energy balance mode.

15. The apparatus of claim 14, wherein the fluctuation interval value comprises an energy flatness of the first audio frame; the fluctuation interval value meets a preset condition that the energy flatness is smaller than a first threshold value; alternatively, the first and second electrodes may be,

the fluctuation interval value comprises an energy deviation degree of the first audio frame; the fluctuation interval value meets a preset condition, namely the energy deviation degree is not in a first preset range; alternatively, the first and second liquid crystal display panels may be,

the fluctuation interval value includes a magnitude deviation of the first audio frame; the fluctuation interval value meeting the preset condition means that the amplitude deviation degree is not in a second preset range.

16. The apparatus according to claim 14 or 15, wherein, when the energy equalization mode is the first energy equalization mode, the processing module is specifically configured to perform energy equalization processing on the channel signals corresponding to the target channel pair set to obtain the at least two equalized channel signals.

17. The apparatus according to claim 16, wherein the processing module is specifically configured to calculate, for a current channel pair in the target channel pair set, an average value of energy values or amplitude values of two channel signals included in the current channel pair, and perform energy equalization processing on the two channel signals included in the current channel pair respectively according to the average value to obtain two corresponding equalized channel signals.

18. The apparatus according to claim 14 or 15, wherein when the energy equalization mode is the second energy equalization mode, the processing module is specifically configured to calculate an average value of energy values or amplitude values of the at least five channel signals, and perform energy equalization processing on the at least five channel signals respectively according to the average value to obtain the at least five equalized channel signals.

19. The apparatus of any of claims 13-18, wherein the processing module is further configured to determine whether a coding rate corresponding to the first audio frame is greater than a rate threshold; when the coding code rate is larger than the code rate threshold, determining that the energy balance mode is a second energy balance mode; and when the coding code rate is less than or equal to the code rate threshold, determining the energy balance mode according to the fluctuation interval value.

20. The apparatus according to any of claims 11-19, wherein the encoding module is further configured to encode the channel signals of the at least five channel signals that are not energy-equalized.

21. An apparatus, comprising:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.

22. A computer-readable storage medium, comprising a computer program which, when executed on a computer, causes the computer to perform the method of any one of claims 1-10.

23. A computer-readable storage medium comprising a codestream obtained according to the method for encoding a multi-channel audio signal according to any one of claims 1 to 10.